.. _models:

================
torchtune.models
================

.. currentmodule:: torchtune.models

llama3.3
--------

Text-only models from the 3.3 version of `Llama3 family <https://llama.meta.com/llama3/>`_.

Important: You need to request access on `Hugging Face <https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct>`__ before downloading it.

To download the Llama-3.3-70B-Instruct model:

.. code-block:: bash

    tune download meta-llama/Llama-3.3-70B-Instruct --ignore-patterns "original/consolidated.00.pth" --hf-token <HF_TOKEN>

.. autosummary::
    :toctree: generated/
    :nosignatures:

    llama3_3.llama3_3_70b
    llama3_3.lora_llama3_3_70b
    llama3_3.qlora_llama3_3_70b

.. note::

    The Llama3.3 tokenizer reuses the :class:`~torchtune.models.llama3.llama3_tokenizer` class.

llama3.2
--------

Text-only models from the 3.2 version of `Llama3 family <https://llama.meta.com/llama3/>`_.

Important: You need to request access on `Hugging Face <https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct>`__ before downloading it.

To download the Llama-3.2-1B-Instruct model:

.. code-block:: bash

    tune download meta-llama/Llama-3.2-1B-Instruct --output-dir /tmp/Llama-3.2-1B-Instruct --ignore-patterns "original/consolidated.00.pth" --hf-token <HF_TOKEN>

To download the Llama-3.2-3B-Instruct model:

.. code-block:: bash

    tune download meta-llama/Llama-3.2-3B-Instruct --output-dir /tmp/Llama-3.2-3B-Instruct --ignore-patterns "original/consolidated*" --hf-token <HF_TOKEN>

.. autosummary::
    :toctree: generated/
    :nosignatures:

    llama3_2.llama3_2_1b
    llama3_2.llama3_2_3b
    llama3_2.lora_llama3_2_1b
    llama3_2.lora_llama3_2_3b
    llama3_2.qlora_llama3_2_1b
    llama3_2.qlora_llama3_2_3b

.. note::

    The Llama3.2 tokenizer reuses the :class:`~torchtune.models.llama3.llama3_tokenizer` class.

llama3.2 Vision
---------------

Vision-Language Models from the 3.2 version of `Llama3 family <https://llama.meta.com/llama3/>`_.

Important: You need to request access on `Hugging Face <https://huggingface.co/meta-llama/Llama-3.2-11B-Vision-Instruct>`__ before downloading it.

To download the Llama-3.2-11B-Instruct model:

.. code-block:: bash

    tune download meta-llama/Llama-3.2-11B-Vision-Instruct --output-dir /tmp/Llama-3.2-11B-Vision-Instruct --hf-token <HF_TOKEN>

.. autosummary::
    :toctree: generated/
    :nosignatures:

    llama3_2_vision.llama3_2_vision_11b
    llama3_2_vision.llama3_2_vision_transform
    llama3_2_vision.lora_llama3_2_vision_11b
    llama3_2_vision.qlora_llama3_2_vision_11b
    llama3_2_vision.llama3_2_vision_decoder
    llama3_2_vision.llama3_2_vision_encoder
    llama3_2_vision.lora_llama3_2_vision_decoder
    llama3_2_vision.lora_llama3_2_vision_encoder
    llama3_2_vision.Llama3VisionEncoder
    llama3_2_vision.Llama3VisionProjectionHead
    llama3_2_vision.Llama3VisionTransform

.. note::

    The Llama3.2 tokenizer reuses the :class:`~torchtune.models.llama3.llama3_tokenizer` class.

llama3 & llama3.1
-----------------

Models 3 and 3.1 from the `Llama3 family <https://llama.meta.com/llama3/>`_.

Important: You need to request access on `Hugging Face <https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct>`__ before downloading it.

To download the Llama3.1-8B-Instruct model:

.. code-block:: bash

    tune download meta-llama/Meta-Llama-3.1-8B-Instruct --output-dir /tmp/Meta-Llama-3.1-8B-Instruct --ignore-patterns "original/consolidated.00.pth" --hf-token <HF_TOKEN>

To download the Llama3.1-70B-Instruct model:

.. code-block:: bash

    tune download meta-llama/Meta-Llama-3.1-70B-Instruct --output-dir /tmp/Meta-Llama-3.1-70B-Instruct --ignore-patterns "original/consolidated*" --hf-token <HF_TOKEN>

To download the Llama3.1-405B-Instruct model:

.. code-block:: bash

    tune download meta-llama/Meta-Llama-3.1-405B-Instruct --ignore-patterns "original/consolidated*" --hf-token <HF_TOKEN>

To download the Llama3 weights of the above models, you can instead download from `Meta-Llama-3-8B-Instruct` and
`Meta-Llama-3-70B-Instruct`, and remove the ignore patterns flag.

.. autosummary::
    :toctree: generated/
    :nosignatures:

    llama3.llama3
    llama3.lora_llama3
    llama3.llama3_8b
    llama3.lora_llama3_8b
    llama3.qlora_llama3_8b
    llama3.llama3_70b
    llama3.lora_llama3_70b
    llama3.qlora_llama3_70b
    llama3.llama3_tokenizer

    |

    llama3_1.llama3_1
    llama3_1.lora_llama3_1
    llama3_1.llama3_1_8b
    llama3_1.lora_llama3_1_8b
    llama3_1.qlora_llama3_1_8b
    llama3_1.llama3_1_70b
    llama3_1.lora_llama3_1_70b
    llama3_1.qlora_llama3_1_70b
    llama3_1.llama3_1_405b
    llama3_1.lora_llama3_1_405b
    llama3_1.qlora_llama3_1_405b


.. note::

    The Llama3.1 tokenizer reuses the `llama3.llama3_tokenizer` builder class.

llama2
------

All models from the `Llama2 family <https://llama.meta.com/llama2/>`_.

Important: You need to request access on `Hugging Face <https://huggingface.co/meta-llama/Llama-2-7b-hf>`__ before downloading it.

To download the Llama2-7B model:

.. code-block:: bash

   tune download meta-llama/Llama-2-7b-hf --output-dir /tmp/Llama-2-7b-hf --hf-token <HF_TOKEN>

To download the Llama2-13B model:

.. code-block:: bash

    tune download meta-llama/Llama-2-13b-hf --output-dir /tmp/Llama-2-13b-hf --hf-token <HF_TOKEN>

To download the Llama2-70B model:

.. code-block:: bash

    tune download meta-llama/Llama-2-70b-hf --output-dir /tmp/Llama-2-70b-hf --hf-token <HF_TOKEN>

.. autosummary::
    :toctree: generated/
    :nosignatures:

    llama2.llama2
    llama2.lora_llama2
    llama2.llama2_7b
    llama2.lora_llama2_7b
    llama2.qlora_llama2_7b
    llama2.llama2_13b
    llama2.lora_llama2_13b
    llama2.qlora_llama2_13b
    llama2.llama2_70b
    llama2.lora_llama2_70b
    llama2.qlora_llama2_70b
    llama2.llama2_tokenizer
    llama2.llama2_reward_7b
    llama2.lora_llama2_reward_7b
    llama2.qlora_llama2_reward_7b
    llama2.Llama2ChatTemplate


code llama
----------

Models from the `Code Llama family <https://arxiv.org/pdf/2308.12950>`_.

Important: You need to request access on `Hugging Face <https://huggingface.co/meta-llama/CodeLlama-7b-hf>`__ before downloading it.

To download the CodeLlama-7B model:

.. code-block:: bash

    tune download meta-llama/CodeLlama-7b-hf --output-dir /tmp/CodeLlama-7b-hf --hf-token <HF_TOKEN>

.. autosummary::
    :toctree: generated/
    :nosignatures:

    code_llama2.code_llama2_7b
    code_llama2.lora_code_llama2_7b
    code_llama2.qlora_code_llama2_7b
    code_llama2.code_llama2_13b
    code_llama2.lora_code_llama2_13b
    code_llama2.qlora_code_llama2_13b
    code_llama2.code_llama2_70b
    code_llama2.lora_code_llama2_70b
    code_llama2.qlora_code_llama2_70b

qwen-2.5
--------

Models of size 0.5B, 1.5B, 3B, 7B, 14B, 32B, 72B from the `Qwen2.5 family <https://huggingface.co/collections/Qwen/qwen25-66e81a666513e518adb90d9e>`_.

To download the Qwen2.5 1.5B model, for example:

.. code-block:: bash

    tune download Qwen/Qwen2.5-1.5B-Instruct --output-dir /tmp/Qwen2_5-1_5B-Instruct

.. autosummary::
    :toctree: generated/
    :nosignatures:

    qwen2_5.qwen2_5_0_5b
    qwen2_5.lora_qwen2_5_0_5b
    qwen2_5.qwen2_5_1_5b_base
    qwen2_5.qwen2_5_1_5b_instruct
    qwen2_5.lora_qwen2_5_1_5b_base
    qwen2_5.lora_qwen2_5_1_5b_instruct
    qwen2_5.qwen2_5_3b
    qwen2_5.lora_qwen2_5_3b
    qwen2_5.qwen2_5_7b_base
    qwen2_5.qwen2_5_7b_instruct
    qwen2_5.lora_qwen2_5_7b_base
    qwen2_5.lora_qwen2_5_7b_instruct
    qwen2_5.qwen2_5_14b_base
    qwen2_5.qwen2_5_14b_instruct
    qwen2_5.lora_qwen2_5_14b_base
    qwen2_5.lora_qwen2_5_14b_instruct
    qwen2_5.qwen2_5_32b_base
    qwen2_5.qwen2_5_32b_instruct
    qwen2_5.lora_qwen2_5_32b_base
    qwen2_5.lora_qwen2_5_32b_instruct
    qwen2_5.qwen2_5_72b_base
    qwen2_5.qwen2_5_72b_instruct
    qwen2_5.lora_qwen2_5_72b_base
    qwen2_5.lora_qwen2_5_72b_instruct
    qwen2_5.qwen2_5_tokenizer

qwen-2
------

Models of size 0.5B, 1.5B, and 7B from the `Qwen2 family <https://huggingface.co/collections/Qwen/qwen2-6659360b33528ced941e557f>`_.

To download the Qwen2 1.5B model, for example:

.. code-block:: bash

    tune download Qwen/Qwen2-1.5B-Instruct --output-dir /tmp/Qwen2-1.5B-Instruct

.. autosummary::
    :toctree: generated/
    :nosignatures:

    qwen2.qwen2
    qwen2.lora_qwen2
    qwen2.qwen2_0_5b
    qwen2.lora_qwen2_0_5b
    qwen2.qwen2_1_5b
    qwen2.lora_qwen2_1_5b
    qwen2.qwen2_7b
    qwen2.lora_qwen2_7b
    qwen2.qwen2_tokenizer

phi-4
-----

Models from the `Phi-4 family <https://arxiv.org/abs/2412.08905>`_.

To download the Phi-4 instruct model:

.. code-block:: bash

    tune download microsoft/phi-4 --hf-token <HF_TOKEN>

.. autosummary::
    :toctree: generated/
    :nosignatures:

    phi4.phi4_14b
    phi4.lora_phi4_14b
    phi4.qlora_phi4_14b
    phi4.phi4_tokenizer

phi-3
-----

Models from the `Phi-3 mini family <https://news.microsoft.com/source/features/ai/the-phi-3-small-language-models-with-big-potential/>`_.

To download the Phi-3 Mini 4k instruct model:

.. code-block:: bash

    tune download microsoft/Phi-3-mini-4k-instruct --output-dir /tmp/Phi-3-mini-4k-instruct --hf-token <HF_TOKEN>

.. autosummary::
    :toctree: generated/
    :nosignatures:

    phi3.phi3
    phi3.lora_phi3
    phi3.phi3_mini
    phi3.lora_phi3_mini
    phi3.qlora_phi3_mini
    phi3.phi3_mini_tokenizer

mistral
-------

All models from `Mistral AI family <https://mistral.ai/technology/#models>`_.

Important: You need to request access on `Hugging Face <https://huggingface.co/mistralai/Mistral-7B-v0.1>`__ to download this model.

To download the Mistral 7B v0.1 model:

.. code-block:: bash

    tune download mistralai/Mistral-7B-v0.1 --output-dir /tmp/Mistral-7B-v0.1 --ignore-patterns "*.safetensors" --hf-token <HF_TOKEN>

.. autosummary::
    :toctree: generated/
    :nosignatures:

    mistral.mistral
    mistral.lora_mistral
    mistral.mistral_classifier
    mistral.lora_mistral_classifier
    mistral.mistral_7b
    mistral.lora_mistral_7b
    mistral.qlora_mistral_7b
    mistral.mistral_reward_7b
    mistral.lora_mistral_reward_7b
    mistral.qlora_mistral_reward_7b
    mistral.mistral_tokenizer
    mistral.MistralChatTemplate


gemma
-----

Models of size 2B and 7B from the `Gemma family <https://blog.google/technology/developers/gemma-open-models/>`_.

Important: You need to request access on `Hugging Face <https://huggingface.co/google/gemma-2b>`__ to use this model.

To download the Gemma 2B model (not Gemma2):

.. code-block:: bash

    tune download google/gemma-2b --ignore-patterns "gemma-2b.gguf"  --hf-token <HF_TOKEN>

To download the Gemma 7B model:

.. code-block:: bash

    tune download google/gemma-7b --ignore-patterns "gemma-7b.gguf"  --hf-token <HF_TOKEN>

.. autosummary::
    :toctree: generated/
    :nosignatures:

    gemma.gemma
    gemma.lora_gemma
    gemma.gemma_2b
    gemma.lora_gemma_2b
    gemma.qlora_gemma_2b
    gemma.gemma_7b
    gemma.lora_gemma_7b
    gemma.qlora_gemma_7b
    gemma.gemma_tokenizer


gemma2 :
--------

Models of size 2B, 9B, 27B from the `Gemma family <https://blog.google/technology/developers/gemma-open-models/>`_.

Important: You need to request access on `Hugging Face <https://huggingface.co/google/gemma-2-2b>`__ to use this model.

To download the Gemma2 2B, 9B, 27B models :

.. code-block:: bash

    tune download google/gemma-2-<MODEL_SIZE>b --ignore-patterns "gemma-2-<MODEL_SIZE>b.gguf"  --hf-token <HF_TOKEN>


.. autosummary::
    :toctree: generated/
    :nosignatures:

    gemma2.gemma2
    gemma2.lora_gemma2
    gemma2.gemma2_2b
    gemma2.lora_gemma2_2b
    gemma2.qlora_gemma2_2b
    gemma2.gemma2_9b
    gemma2.lora_gemma2_9b
    gemma2.qlora_gemma2_9b
    gemma2.gemma2_27b
    gemma2.lora_gemma2_27b
    gemma2.qlora_gemma2_27b
    gemma.gemma_tokenizer

clip
----

Vision components to support multimodality using `CLIP encoder <https://arxiv.org/abs/2103.00020>`_.

.. autosummary::
    :toctree: generated/
    :nosignatures:

    clip.clip_vision_encoder
    clip.TokenPositionalEmbedding
    clip.TiledTokenPositionalEmbedding
    clip.TilePositionalEmbedding