.. _sphx_glr_recipes_recipes:

PyTorch Recipes
---------------------------------------------
1. defining_a_neural_network.py
	 Defining a Neural Network in PyTorch
         https://pytorch.org/tutorials/recipes/recipes/defining_a_neural_network.html

2. what_is_state_dict.py
	 What is a state_dict in PyTorch
         https://pytorch.org/tutorials/recipes/recipes/what_is_state_dict.html

3. saving_and_loading_models_for_inference.py
	 Saving and loading models for inference in PyTorch
         https://pytorch.org/tutorials/recipes/recipes/saving_and_loading_models_for_inference.html

4. custom_dataset_transforms_loader.py
	 Developing Custom PyTorch Dataloaders
         https://pytorch.org/tutorials/recipes/recipes/custom_dataset_transforms_loader.html


5. Captum_Recipe.py
	 Model Interpretability using Captum
         https://pytorch.org/tutorials/recipes/recipes/Captum_Recipe.html

6. dynamic_quantization.py
         Dynamic Quantization
         https://pytorch.org/tutorials/recipes/recipes/dynamic_quantization.html

7. warmstarting_model_using_parameters_from_a_different_model.py
         Warmstarting models using parameters from different model
         https://pytorch.org/tutorials/recipes/recipes/warmstarting_model_using_parameters_from_a_different_model.html

8. zeroing_out_gradients.py
         Zeroing out gradients
         https://pytorch.org/tutorials/recipes/recipes/zeroing_out_gradients.html

9. mobile_perf.py
         PyTorch Mobile Performance Recipes
         https://pytorch.org/tutorials/recipes/mobile_perf.html

10. amp_recipe.py
         Automatic Mixed Precision
         https://pytorch.org/tutorials/recipes/amp_recipe.html

11. regional_compilation.py
	Reducing torch.compile cold start compilation time with regional compilation
         https://pytorch.org/tutorials/recipes/regional_compilation.html



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Performance Tuning Guide is a set of optimizations and best practices which can accelerate training and inference of deep learning models in PyTorch. Presented techniques often can be implemented by changing only a few lines of code and can be applied to a wide range of deep learning models across all domains.">

.. only:: html

  .. image:: /recipes/recipes/images/thumb/sphx_glr_tuning_guide_thumb.png
    :alt:

  :ref:`sphx_glr_recipes_recipes_tuning_guide.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Performance Tuning Guide</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="It is common practice to write PyTorch code in a device-agnostic way, and then switch between CPU and CUDA depending on what hardware is available. Typically, to do this you might have used if-statements and cuda() calls to do this:">

.. only:: html

  .. image:: /recipes/recipes/images/thumb/sphx_glr_changing_default_device_thumb.png
    :alt:

  :ref:`sphx_glr_recipes_recipes_changing_default_device.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Changing default device</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Installation ---------------------- PyTorch should be installed to log models and metrics into TensorBoard log  directory. The following command will install PyTorch 1.4+ via  Anaconda (recommended):">

.. only:: html

  .. image:: /recipes/recipes/images/thumb/sphx_glr_tensorboard_with_pytorch_thumb.png
    :alt:

  :ref:`sphx_glr_recipes_recipes_tensorboard_with_pytorch.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">How to use TensorBoard with PyTorch</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Introduction ------------ A state_dict is an integral entity if you are interested in saving or loading models from PyTorch. Because state_dict objects are Python dictionaries, they can be easily saved, updated, altered, and restored, adding a great deal of modularity to PyTorch models and optimizers. Note that only layers with learnable parameters (convolutional layers, linear layers, etc.) and registered buffers (batchnorm’s running_mean) have entries in the model’s state_dict. Optimizer objects (``torch.optim``) also have a state_dict, which contains information about the optimizer’s state, as well as the hyperparameters used. In this recipe, we will see how state_dict is used with a simple model.">

.. only:: html

  .. image:: /recipes/recipes/images/thumb/sphx_glr_what_is_state_dict_thumb.png
    :alt:

  :ref:`sphx_glr_recipes_recipes_what_is_state_dict.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">What is a state_dict in PyTorch</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Introduction ------------ Whether you are loading from a partial state_dict, which is missing some keys, or loading a state_dict with more keys than the model that you are loading into, you can set the strict argument to False in the load_state_dict() function to ignore non-matching keys. In this recipe, we will experiment with warmstarting a model using parameters of a different model.">

.. only:: html

  .. image:: /recipes/recipes/images/thumb/sphx_glr_warmstarting_model_using_parameters_from_a_different_model_thumb.png
    :alt:

  :ref:`sphx_glr_recipes_recipes_warmstarting_model_using_parameters_from_a_different_model.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Warmstarting model using parameters from a different model in PyTorch</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="When writing models with PyTorch, it is commonly the case that the parameters to a given layer depend on the shape of the output of the previous layer. For example, the in_features of an nn.Linear layer must match the size(-1) of the input. For some layers, the shape computation involves complex equations, for example convolution operations.">

.. only:: html

  .. image:: /recipes/recipes/images/thumb/sphx_glr_reasoning_about_shapes_thumb.png
    :alt:

  :ref:`sphx_glr_recipes_recipes_reasoning_about_shapes.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Reasoning about Shapes in PyTorch</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="If you&#x27;re loading a checkpoint and want to reduce compute and memory as much as possible, this tutorial shares some recommended practices. In particular, we will discuss">

.. only:: html

  .. image:: /recipes/recipes/images/thumb/sphx_glr_module_load_state_dict_tips_thumb.png
    :alt:

  :ref:`sphx_glr_recipes_recipes_module_load_state_dict_tips.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Tips for Loading an nn.Module from a Checkpoint</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Introduction ------------ PyTorch provides the elegantly designed modules and classes, including torch.nn, to help you create and train neural networks. An nn.Module contains layers, and a method forward(input) that returns the output.">

.. only:: html

  .. image:: /recipes/recipes/images/thumb/sphx_glr_defining_a_neural_network_thumb.png
    :alt:

  :ref:`sphx_glr_recipes_recipes_defining_a_neural_network.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Defining a Neural Network in PyTorch</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Introduction ------------ When training your neural network, models are able to increase their accuracy through gradient descent. In short, gradient descent is the process of minimizing our loss (or error) by tweaking the weights and biases in our model.">

.. only:: html

  .. image:: /recipes/recipes/images/thumb/sphx_glr_zeroing_out_gradients_thumb.png
    :alt:

  :ref:`sphx_glr_recipes_recipes_zeroing_out_gradients.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Zeroing out gradients in PyTorch</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this tutorial, we&#x27;re going to cover the primary APIs of torch.utils.benchmark.Timer. The PyTorch Timer is based on the timeit.Timer API, with several PyTorch specific modifications. Familiarity with the builtin Timer class is not required for this tutorial, however we assume that the reader is familiar with the fundamentals of performance work.">

.. only:: html

  .. image:: /recipes/recipes/images/thumb/sphx_glr_timer_quick_start_thumb.png
    :alt:

  :ref:`sphx_glr_recipes_recipes_timer_quick_start.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Timer quick start</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Model Interpretability using Captum">

.. only:: html

  .. image:: /recipes/recipes/images/thumb/sphx_glr_Captum_Recipe_thumb.png
    :alt:

  :ref:`sphx_glr_recipes_recipes_Captum_Recipe.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Model Interpretability using Captum</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This recipe introduces a new utility function torch.utils.swap_tensors as well as two new extension points where it has been integrated in nn.Module:">

.. only:: html

  .. image:: /recipes/recipes/images/thumb/sphx_glr_swap_tensors_thumb.png
    :alt:

  :ref:`sphx_glr_recipes_recipes_swap_tensors.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Extension points in nn.Module for load_state_dict and tensor subclasses</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="PyTorch Profiler">

.. only:: html

  .. image:: /recipes/recipes/images/thumb/sphx_glr_profiler_recipe_thumb.png
    :alt:

  :ref:`sphx_glr_recipes_recipes_profiler_recipe.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">PyTorch Profiler</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="torch.cuda.amp provides convenience methods for mixed precision, where some operations use the torch.float32 (``float``) datatype and other operations use torch.float16 (``half``). Some ops, like linear layers and convolutions, are much faster in float16 or bfloat16. Other ops, like reductions, often require the dynamic range of float32.  Mixed precision tries to match each op to its appropriate datatype, which can reduce your network&#x27;s runtime and memory footprint.">

.. only:: html

  .. image:: /recipes/recipes/images/thumb/sphx_glr_amp_recipe_thumb.png
    :alt:

  :ref:`sphx_glr_recipes_recipes_amp_recipe.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Automatic Mixed Precision</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Introduction ------------ Benchmarking is an important step in writing code. It helps us validate that our code meets performance expectations, compare different approaches to solving the same problem and prevent performance regressions.">

.. only:: html

  .. image:: /recipes/recipes/images/thumb/sphx_glr_benchmark_thumb.png
    :alt:

  :ref:`sphx_glr_recipes_recipes_benchmark.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">PyTorch Benchmark</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /recipes/recipes/tuning_guide
   /recipes/recipes/changing_default_device
   /recipes/recipes/tensorboard_with_pytorch
   /recipes/recipes/what_is_state_dict
   /recipes/recipes/warmstarting_model_using_parameters_from_a_different_model
   /recipes/recipes/reasoning_about_shapes
   /recipes/recipes/module_load_state_dict_tips
   /recipes/recipes/defining_a_neural_network
   /recipes/recipes/zeroing_out_gradients
   /recipes/recipes/timer_quick_start
   /recipes/recipes/Captum_Recipe
   /recipes/recipes/swap_tensors
   /recipes/recipes/profiler_recipe
   /recipes/recipes/amp_recipe
   /recipes/recipes/benchmark