# PyTorch Tutorials

## Docs

- [ONNX Live Tutorial](https://docs.pytorch.org/tutorials/advanced/ONNXLive.html)
- [TorchRL objectives: Coding a DDPG loss](https://docs.pytorch.org/tutorials/advanced/coding_ddpg.html)
- [Autograd in C++ Frontend](https://docs.pytorch.org/tutorials/advanced/cpp_autograd.html)
- [Custom C++ and CUDA Operators](https://docs.pytorch.org/tutorials/advanced/cpp_custom_ops.html)
- [Custom SYCL Operators](https://docs.pytorch.org/tutorials/advanced/cpp_custom_ops_sycl.html)
- [<no title>](https://docs.pytorch.org/tutorials/advanced/cpp_export.html)
- [Using the PyTorch C++ Frontend](https://docs.pytorch.org/tutorials/advanced/cpp_frontend.html)
- [Supporting Custom C++ Classes in torch.compile/torch.export](https://docs.pytorch.org/tutorials/advanced/custom_class_pt2.html)
- [Extending PyTorch with Custom C++ Classes](https://docs.pytorch.org/tutorials/advanced/custom_classes.html)
- [PyTorch Custom Operators](https://docs.pytorch.org/tutorials/advanced/custom_ops_landing_page.html)
- [Training Transformer models using Distributed Data Parallel and Pipeline Parallelism](https://docs.pytorch.org/tutorials/advanced/ddp_pipeline.html)
- [Registering a Dispatched Operator in C++](https://docs.pytorch.org/tutorials/advanced/dispatcher.html)
- [Extending dispatcher for a new backend in C++](https://docs.pytorch.org/tutorials/advanced/extend_dispatcher.html)
- [Distributed Training with Uneven Inputs Using the Join Context Manager](https://docs.pytorch.org/tutorials/advanced/generic_join.html)
- [Neural Transfer Using PyTorch](https://docs.pytorch.org/tutorials/advanced/neural_style_tutorial.html)
- [Creating Extensions Using NumPy and SciPy](https://docs.pytorch.org/tutorials/advanced/numpy_extensions_tutorial.html)
- [Pendulum: Writing your environment and transforms with TorchRL](https://docs.pytorch.org/tutorials/advanced/pendulum.html)
- [Facilitating New Backend Integration by PrivateUse1](https://docs.pytorch.org/tutorials/advanced/privateuseone.html)
- [Custom Python Operators](https://docs.pytorch.org/tutorials/advanced/python_custom_ops.html)
- [Combining Distributed DataParallel with Distributed RPC Framework](https://docs.pytorch.org/tutorials/advanced/rpc_ddp_tutorial.html)
- [(beta) Accelerating BERT with semi-structured (2:4) sparsity](https://docs.pytorch.org/tutorials/advanced/semi_structured_sparse.html)
- [Computation times](https://docs.pytorch.org/tutorials/advanced/sg_execution_times.html)
- [Exploring TorchRec sharding](https://docs.pytorch.org/tutorials/advanced/sharding.html)
- [<no title>](https://docs.pytorch.org/tutorials/advanced/torch-script-parallelism.html)
- [<no title>](https://docs.pytorch.org/tutorials/advanced/torch_script_custom_ops.html)
- [Semi-Supervised Learning using USB built upon PyTorch](https://docs.pytorch.org/tutorials/advanced/usb_semisup_learn.html)
- [Audio Data Augmentation](https://docs.pytorch.org/tutorials/beginner/audio_data_augmentation_tutorial.html)
- [Audio Datasets](https://docs.pytorch.org/tutorials/beginner/audio_datasets_tutorial.html)
- [Audio Feature Augmentation](https://docs.pytorch.org/tutorials/beginner/audio_feature_augmentation_tutorial.html)
- [Audio Feature Extractions](https://docs.pytorch.org/tutorials/beginner/audio_feature_extractions_tutorial.html)
- [Audio Resampling](https://docs.pytorch.org/tutorials/beginner/audio_resampling_tutorial.html)
- [Automatic Differentiation with torch.autograd](https://docs.pytorch.org/tutorials/beginner/basics/autogradqs_tutorial.html)
- [Build the Neural Network](https://docs.pytorch.org/tutorials/beginner/basics/buildmodel_tutorial.html)
- [Datasets & DataLoaders](https://docs.pytorch.org/tutorials/beginner/basics/data_tutorial.html)
- [Learn the Basics](https://docs.pytorch.org/tutorials/beginner/basics/index.html)
- [Learn the Basics](https://docs.pytorch.org/tutorials/beginner/basics/intro.html)
- [Optimizing Model Parameters](https://docs.pytorch.org/tutorials/beginner/basics/optimization_tutorial.html)
- [Quickstart](https://docs.pytorch.org/tutorials/beginner/basics/quickstart_tutorial.html)
- [Save and Load the Model](https://docs.pytorch.org/tutorials/beginner/basics/saveloadrun_tutorial.html)
- [Computation times](https://docs.pytorch.org/tutorials/beginner/basics/sg_execution_times.html)
- [Tensors](https://docs.pytorch.org/tutorials/beginner/basics/tensorqs_tutorial.html)
- [Transforms](https://docs.pytorch.org/tutorials/beginner/basics/transforms_tutorial.html)
- [Fast Transformer Inference with Better Transformer](https://docs.pytorch.org/tutorials/beginner/bettertransformer_tutorial.html)
- [A Gentle Introduction to torch.autograd](https://docs.pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html)
- [Training a Classifier](https://docs.pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html)
- [Optional: Data Parallelism](https://docs.pytorch.org/tutorials/beginner/blitz/data_parallel_tutorial.html)
- [Deep Learning with PyTorch: A 60 Minute Blitz](https://docs.pytorch.org/tutorials/beginner/blitz/index.html)
- [Neural Networks](https://docs.pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html)
- [Computation times](https://docs.pytorch.org/tutorials/beginner/blitz/sg_execution_times.html)
- [Tensors](https://docs.pytorch.org/tutorials/beginner/blitz/tensor_tutorial.html)
- [Chatbot Tutorial](https://docs.pytorch.org/tutorials/beginner/chatbot_tutorial.html)
- [Running Tutorials in Google Colab](https://docs.pytorch.org/tutorials/beginner/colab.html)
- [Writing Custom Datasets, DataLoaders and Transforms](https://docs.pytorch.org/tutorials/beginner/data_loading_tutorial.html)
- [DCGAN Tutorial](https://docs.pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html)
- [Fault-tolerant Distributed Training with torchrun](https://docs.pytorch.org/tutorials/beginner/ddp_series_fault_tolerance.html)
- [Distributed Data Parallel in PyTorch - Video Tutorials](https://docs.pytorch.org/tutorials/beginner/ddp_series_intro.html)
- [Multi GPU training with DDP](https://docs.pytorch.org/tutorials/beginner/ddp_series_multigpu.html)
- [What is Distributed Data Parallel (DDP)](https://docs.pytorch.org/tutorials/beginner/ddp_series_theory.html)
- [Deep Learning with PyTorch: A 60 Minute Blitz](https://docs.pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html)
- [Image Segmentation DeepLabV3 on Android](https://docs.pytorch.org/tutorials/beginner/deeplabv3_on_android.html)
- [Image Segmentation DeepLabV3 on iOS](https://docs.pytorch.org/tutorials/beginner/deeplabv3_on_ios.html)
- [PyTorch Distributed Overview](https://docs.pytorch.org/tutorials/beginner/dist_overview.html)
- [<no title>](https://docs.pytorch.org/tutorials/beginner/examples_autograd/index.html)
- [PyTorch: Tensors and autograd](https://docs.pytorch.org/tutorials/beginner/examples_autograd/polynomial_autograd.html)
- [PyTorch: Defining New autograd Functions](https://docs.pytorch.org/tutorials/beginner/examples_autograd/polynomial_custom_function.html)
- [Computation times](https://docs.pytorch.org/tutorials/beginner/examples_autograd/sg_execution_times.html)
- [PyTorch: Control Flow + Weight Sharing](https://docs.pytorch.org/tutorials/beginner/examples_nn/dynamic_net.html)
- [<no title>](https://docs.pytorch.org/tutorials/beginner/examples_nn/index.html)
- [PyTorch: Custom nn Modules](https://docs.pytorch.org/tutorials/beginner/examples_nn/polynomial_module.html)
- [PyTorch: nn](https://docs.pytorch.org/tutorials/beginner/examples_nn/polynomial_nn.html)
- [PyTorch: optim](https://docs.pytorch.org/tutorials/beginner/examples_nn/polynomial_optim.html)
- [Computation times](https://docs.pytorch.org/tutorials/beginner/examples_nn/sg_execution_times.html)
- [<no title>](https://docs.pytorch.org/tutorials/beginner/examples_tensor/index.html)
- [Warm-up: numpy](https://docs.pytorch.org/tutorials/beginner/examples_tensor/polynomial_numpy.html)
- [PyTorch: Tensors](https://docs.pytorch.org/tutorials/beginner/examples_tensor/polynomial_tensor.html)
- [Computation times](https://docs.pytorch.org/tutorials/beginner/examples_tensor/sg_execution_times.html)
- [Adversarial Example Generation](https://docs.pytorch.org/tutorials/beginner/fgsm_tutorial.html)
- [Finetuning Torchvision Models](https://docs.pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html)
- [Autograd](https://docs.pytorch.org/tutorials/beginner/former_torchies/autograd_tutorial_old.html)
- [nn Package](https://docs.pytorch.org/tutorials/beginner/former_torchies/nnft_tutorial.html)
- [Multi-GPU Examples](https://docs.pytorch.org/tutorials/beginner/former_torchies/parallelism_tutorial.html)
- [Tensors](https://docs.pytorch.org/tutorials/beginner/former_torchies/tensor_tutorial_old.html)
- [PyTorch for Former Torch Users](https://docs.pytorch.org/tutorials/beginner/former_torchies_tutorial.html)
- [Introduction to Holistic Trace Analysis](https://docs.pytorch.org/tutorials/beginner/hta_intro_tutorial.html)
- [Trace Diff using Holistic Trace Analysis](https://docs.pytorch.org/tutorials/beginner/hta_trace_diff_tutorial.html)
- [Hyperparameter tuning using Ray Tune](https://docs.pytorch.org/tutorials/beginner/hyperparameter_tuning_tutorial.html)
- [Introduction to PyTorch - YouTube Series](https://docs.pytorch.org/tutorials/beginner/introyt.html)
- [The Fundamentals of Autograd](https://docs.pytorch.org/tutorials/beginner/introyt/autogradyt_tutorial.html)
- [Model Understanding with Captum](https://docs.pytorch.org/tutorials/beginner/introyt/captumyt.html)
- [Introduction to PyTorch on YouTube](https://docs.pytorch.org/tutorials/beginner/introyt/index.html)
- [Introduction to PyTorch](https://docs.pytorch.org/tutorials/beginner/introyt/introyt1_tutorial.html)
- [Introduction to PyTorch - YouTube Series](https://docs.pytorch.org/tutorials/beginner/introyt/introyt_index.html)
- [Building Models with PyTorch](https://docs.pytorch.org/tutorials/beginner/introyt/modelsyt_tutorial.html)
- [Computation times](https://docs.pytorch.org/tutorials/beginner/introyt/sg_execution_times.html)
- [PyTorch TensorBoard Support](https://docs.pytorch.org/tutorials/beginner/introyt/tensorboardyt_tutorial.html)
- [Introduction to PyTorch Tensors](https://docs.pytorch.org/tutorials/beginner/introyt/tensors_deeper_tutorial.html)
- [Training with PyTorch](https://docs.pytorch.org/tutorials/beginner/introyt/trainingyt.html)
- [Knowledge Distillation Tutorial](https://docs.pytorch.org/tutorials/beginner/knowledge_distillation_tutorial.html)
- [Mosaic: Memory Profiling for PyTorch](https://docs.pytorch.org/tutorials/beginner/mosaic_memory_profiling_tutorial.html)
- [Advanced: Making Dynamic Decisions and the Bi-LSTM CRF](https://docs.pytorch.org/tutorials/beginner/nlp/advanced_tutorial.html)
- [Deep Learning with PyTorch](https://docs.pytorch.org/tutorials/beginner/nlp/deep_learning_tutorial.html)
- [Deep Learning for NLP with Pytorch](https://docs.pytorch.org/tutorials/beginner/nlp/index.html)
- [Introduction to PyTorch](https://docs.pytorch.org/tutorials/beginner/nlp/pytorch_tutorial.html)
- [Sequence Models and Long Short-Term Memory Networks](https://docs.pytorch.org/tutorials/beginner/nlp/sequence_models_tutorial.html)
- [Computation times](https://docs.pytorch.org/tutorials/beginner/nlp/sg_execution_times.html)
- [Word Embeddings: Encoding Lexical Semantics](https://docs.pytorch.org/tutorials/beginner/nlp/word_embeddings_tutorial.html)
- [What is torch.nn really?](https://docs.pytorch.org/tutorials/beginner/nn_tutorial.html)
- [Export a model with control flow to ONNX](https://docs.pytorch.org/tutorials/beginner/onnx/export_control_flow_model_to_onnx_tutorial.html)
- [Export a PyTorch model to ONNX](https://docs.pytorch.org/tutorials/beginner/onnx/export_simple_model_to_onnx_tutorial.html)
- [ONNX](https://docs.pytorch.org/tutorials/beginner/onnx/index.html)
- [Introduction to ONNX](https://docs.pytorch.org/tutorials/beginner/onnx/intro_onnx.html)
- [Extending the ONNX Exporter Operator Support](https://docs.pytorch.org/tutorials/beginner/onnx/onnx_registry_tutorial.html)
- [Computation times](https://docs.pytorch.org/tutorials/beginner/onnx/sg_execution_times.html)
- [Profiling your PyTorch Module](https://docs.pytorch.org/tutorials/beginner/profiler.html)
- [Learning PyTorch with Examples](https://docs.pytorch.org/tutorials/beginner/pytorch_with_examples.html)
- [Saving and Loading Models](https://docs.pytorch.org/tutorials/beginner/saving_loading_models.html)
- [Computation times](https://docs.pytorch.org/tutorials/beginner/sg_execution_times.html)
- [T5-Base Model for Summarization, Sentiment Classification, and Translation](https://docs.pytorch.org/tutorials/beginner/t5_tutoria.html)
- [Template Tutorial](https://docs.pytorch.org/tutorials/beginner/template_tutorial.html)
- [Text classification with the torchtext library](https://docs.pytorch.org/tutorials/beginner/text_sentiment_ngrams_tutorial.html)
- [Preprocess custom text dataset using torchtext](https://docs.pytorch.org/tutorials/beginner/torchtext_custom_dataset_tutorial.html)
- [Transfer Learning for Computer Vision Tutorial](https://docs.pytorch.org/tutorials/beginner/transfer_learning_tutorial.html)
- [Language Modeling with nn.Transformer and torchtext](https://docs.pytorch.org/tutorials/beginner/transformer_tutorial.html)
- [Language Translation with nn.Transformer and torchtext](https://docs.pytorch.org/tutorials/beginner/translation_transformer.html)
- [Understanding requires_grad, retain_grad, Leaf, and Non-leaf Tensors](https://docs.pytorch.org/tutorials/beginner/understanding_leaf_vs_nonleaf_tutorial.html)
- [Compilers](https://docs.pytorch.org/tutorials/compilers_index.html)
- [Deep Dive](https://docs.pytorch.org/tutorials/deep-dive.html)
- [Distributed](https://docs.pytorch.org/tutorials/distributed.html)
- [Domains](https://docs.pytorch.org/tutorials/domains.html)
- [Ecosystem](https://docs.pytorch.org/tutorials/ecosystem.html)
- [Extension](https://docs.pytorch.org/tutorials/extension.html)
- [Welcome to PyTorch Tutorials](https://docs.pytorch.org/tutorials/index.html)
- [Getting Started with Fully Sharded Data Parallel(FSDP)](https://docs.pytorch.org/tutorials/intermediate/FSDP1_tutorial.html)
- [Advanced Model Training with Fully Sharded Data Parallel (FSDP)](https://docs.pytorch.org/tutorials/intermediate/FSDP_advanced_tutorial.html)
- [Getting Started with Fully Sharded Data Parallel (FSDP2)](https://docs.pytorch.org/tutorials/intermediate/FSDP_tutorial.html)
- [Introduction to Libuv TCPStore Backend](https://docs.pytorch.org/tutorials/intermediate/TCPStore_libuv_backend.html)
- [Large Scale Transformer model training with Tensor Parallel (TP)](https://docs.pytorch.org/tutorials/intermediate/TP_tutorial.html)
- [Hooks for autograd saved tensors](https://docs.pytorch.org/tutorials/intermediate/autograd_saved_tensors_hooks_tutorial.html)
- [Multi-Objective NAS with Ax](https://docs.pytorch.org/tutorials/intermediate/ax_multiobjective_nas_tutorial.html)
- [NLP From Scratch: Classifying Names with a Character-Level RNN](https://docs.pytorch.org/tutorials/intermediate/char_rnn_classification_tutorial.html)
- [NLP From Scratch: Generating Names with a Character-Level RNN](https://docs.pytorch.org/tutorials/intermediate/char_rnn_generation_tutorial.html)
- [Compiled Autograd: Capturing a larger backward graph for torch.compile](https://docs.pytorch.org/tutorials/intermediate/compiled_autograd_tutorial.html)
- [Fusing Convolution and Batch Norm using Custom Function](https://docs.pytorch.org/tutorials/intermediate/custom_function_conv_bn_tutorial.html)
- [Double Backward with Custom Functions](https://docs.pytorch.org/tutorials/intermediate/custom_function_double_backward_tutorial.html)
- [Training “real-world” models with DDP](https://docs.pytorch.org/tutorials/intermediate/ddp_series_minGPT.html)
- [Multinode Training](https://docs.pytorch.org/tutorials/intermediate/ddp_series_multinode.html)
- [Getting Started with Distributed Data Parallel](https://docs.pytorch.org/tutorials/intermediate/ddp_tutorial.html)
- [Distributed Pipeline Parallelism Using RPC](https://docs.pytorch.org/tutorials/intermediate/dist_pipeline_parallel_tutorial.html)
- [Writing Distributed Applications with PyTorch](https://docs.pytorch.org/tutorials/intermediate/dist_tuto.html)
- [Recurrent DQN: Training recurrent policies](https://docs.pytorch.org/tutorials/intermediate/dqn_with_rnn_tutorial.html)
- [Model ensembling](https://docs.pytorch.org/tutorials/intermediate/ensembling.html)
- [Forced Alignment with Wav2Vec2](https://docs.pytorch.org/tutorials/intermediate/forced_alignment_with_torchaudio_tutorial.html)
- [Forward-mode Automatic Differentiation (Beta)](https://docs.pytorch.org/tutorials/intermediate/forward_ad_usage.html)
- [(beta) Building a Simple CPU Performance Profiler with FX](https://docs.pytorch.org/tutorials/intermediate/fx_profiling_tutorial.html)
- [Inductor CPU backend debugging and profiling](https://docs.pytorch.org/tutorials/intermediate/inductor_debug_cpu.html)
- [Jacobians, Hessians, hvp, vhp, and more: composing function transforms](https://docs.pytorch.org/tutorials/intermediate/jacobians_hessians.html)
- [Train a Mario-playing RL Agent](https://docs.pytorch.org/tutorials/intermediate/mario_rl_tutorial.html)
- [Channels Last Memory Format in PyTorch](https://docs.pytorch.org/tutorials/intermediate/memory_format_tutorial.html)
- [<no title>](https://docs.pytorch.org/tutorials/intermediate/mnist_train_nas.html)
- [Single-Machine Model Parallel Best Practices](https://docs.pytorch.org/tutorials/intermediate/model_parallel_tutorial.html)
- [Interactive Distributed Applications with Monarch](https://docs.pytorch.org/tutorials/intermediate/monarch_distributed_tutorial.html)
- [Neural Tangent Kernels](https://docs.pytorch.org/tutorials/intermediate/neural_tangent_kernels.html)
- [NLP from Scratch](https://docs.pytorch.org/tutorials/intermediate/nlp_from_scratch_index.html)
- [Getting Started - Accelerate Your Scripts with nvFuser](https://docs.pytorch.org/tutorials/intermediate/nvfuser_intro_tutorial.html)
- [How to save memory by fusing the optimizer step into the backward pass](https://docs.pytorch.org/tutorials/intermediate/optimizer_step_in_backward_tutorial.html)
- [Parametrizations Tutorial](https://docs.pytorch.org/tutorials/intermediate/parametrizations.html)
- [Per-sample-gradients](https://docs.pytorch.org/tutorials/intermediate/per_sample_grads.html)
- [A guide on good usage of non_blocking and pin_memory() in PyTorch](https://docs.pytorch.org/tutorials/intermediate/pinmem_nonblock.html)
- [Training Transformer models using Pipeline Parallelism](https://docs.pytorch.org/tutorials/intermediate/pipeline_tutorial.html)
- [Introduction to Distributed Pipeline Parallelism](https://docs.pytorch.org/tutorials/intermediate/pipelining_tutorial.html)
- [Customize Process Group Backends Using Cpp Extensions](https://docs.pytorch.org/tutorials/intermediate/process_group_cpp_extension_tutorial.html)
- [Pruning Tutorial](https://docs.pytorch.org/tutorials/intermediate/pruning_tutorial.html)
- [Real Time Inference on Raspberry Pi 4 and 5 (40 fps!)](https://docs.pytorch.org/tutorials/intermediate/realtime_rpi.html)
- [Reinforcement Learning (PPO) with TorchRL Tutorial](https://docs.pytorch.org/tutorials/intermediate/reinforcement_ppo.html)
- [Reinforcement Learning (DQN) Tutorial](https://docs.pytorch.org/tutorials/intermediate/reinforcement_q_learning.html)
- [Implementing Batch RPC Processing Using Asynchronous Executions](https://docs.pytorch.org/tutorials/intermediate/rpc_async_execution.html)
- [Implementing a Parameter Server Using Distributed RPC Framework](https://docs.pytorch.org/tutorials/intermediate/rpc_param_server_tutorial.html)
- [Getting Started with Distributed RPC Framework](https://docs.pytorch.org/tutorials/intermediate/rpc_tutorial.html)
- [(Beta) Implementing High-Performance Transformers with Scaled Dot Product Attention (SDPA)](https://docs.pytorch.org/tutorials/intermediate/scaled_dot_product_attention_tutorial.html)
- [NLP From Scratch: Translation with a Sequence to Sequence Network and Attention](https://docs.pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html)
- [Computation times](https://docs.pytorch.org/tutorials/intermediate/sg_execution_times.html)
- [Spatial Transformer Networks Tutorial](https://docs.pytorch.org/tutorials/intermediate/spatial_transformer_tutorial.html)
- [Speech Recognition with Wav2Vec2](https://docs.pytorch.org/tutorials/intermediate/speech_recognition_pipeline_tutorial.html)
- [PyTorch Profiler With TensorBoard](https://docs.pytorch.org/tutorials/intermediate/tensorboard_profiler_tutorial.html)
- [Visualizing Models, Data, and Training with TensorBoard](https://docs.pytorch.org/tutorials/intermediate/tensorboard_tutorial.html)
- [Text-to-speech with Tacotron2](https://docs.pytorch.org/tutorials/intermediate/text_to_speech_with_torchaudio.html)
- [Building a Convolution/Batch Norm fuser with torch.compile](https://docs.pytorch.org/tutorials/intermediate/torch_compile_conv_bn_fuser.html)
- [torch.compile End-to-End Tutorial](https://docs.pytorch.org/tutorials/intermediate/torch_compile_full_example.html)
- [Introduction to torch.compile](https://docs.pytorch.org/tutorials/intermediate/torch_compile_tutorial.html)
- [torch.export Nightly Tutorial](https://docs.pytorch.org/tutorials/intermediate/torch_export_nightly_tutorial.html)
- [torch.export Tutorial](https://docs.pytorch.org/tutorials/intermediate/torch_export_tutorial.html)
- [Introduction to TorchRec](https://docs.pytorch.org/tutorials/intermediate/torchrec_intro_tutorial.html)
- [Introduction to TorchRec](https://docs.pytorch.org/tutorials/intermediate/torchrec_tutorial.html)
- [TorchVision Object Detection Finetuning Tutorial](https://docs.pytorch.org/tutorials/intermediate/torchvision_tutorial.html)
- [Accelerating PyTorch Transformers by replacing nn.Transformer with Nested Tensors and torch.compile()](https://docs.pytorch.org/tutorials/intermediate/transformer_building_blocks.html)
- [Using Variable Length Attention in PyTorch](https://docs.pytorch.org/tutorials/intermediate/variable_length_attention_tutorial.html)
- [Visualizing Gradients](https://docs.pytorch.org/tutorials/intermediate/visualizing_gradients_tutorial.html)
- [Intro](https://docs.pytorch.org/tutorials/intro.html)
- [Making Native Android Application that uses PyTorch prebuilt libraries](https://docs.pytorch.org/tutorials/recipes/android_native_app_with_custom_op.html)
- [(beta) Compiling the optimizer with torch.compile](https://docs.pytorch.org/tutorials/recipes/compiling_optimizer.html)
- [(beta) Running the compiled optimizer with an LR Scheduler](https://docs.pytorch.org/tutorials/recipes/compiling_optimizer_lr_scheduler.html)
- [DebugMode: Recording Dispatched Operations and Numerical Debugging](https://docs.pytorch.org/tutorials/recipes/debug_mode_tutorial.html)
- [Asynchronous Saving with Distributed Checkpoint (DCP)](https://docs.pytorch.org/tutorials/recipes/distributed_async_checkpoint_recipe.html)
- [Getting Started with Distributed Checkpoint (DCP)](https://docs.pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html)
- [Getting Started with CommDebugMode](https://docs.pytorch.org/tutorials/recipes/distributed_comm_debug_mode.html)
- [Getting Started with DeviceMesh](https://docs.pytorch.org/tutorials/recipes/distributed_device_mesh.html)
- [<no title>](https://docs.pytorch.org/tutorials/recipes/distributed_optim_torchscript.html)
- [Profiling PyTorch RPC-Based Workloads](https://docs.pytorch.org/tutorials/recipes/distributed_rpc_profiling.html)
- [Explicit horizontal fusion with foreach_map and torch.compile](https://docs.pytorch.org/tutorials/recipes/foreach_map.html)
- [Ease-of-use quantization for PyTorch with Intel® Neural Compressor](https://docs.pytorch.org/tutorials/recipes/intel_neural_compressor_for_pytorch.html)
- [Loading data in PyTorch](https://docs.pytorch.org/tutorials/recipes/loading_data_recipe.html)
- [(beta) Efficient mobile interpreter in Android and iOS](https://docs.pytorch.org/tutorials/recipes/mobile_interpreter.html)
- [Pytorch Mobile Performance Recipes](https://docs.pytorch.org/tutorials/recipes/mobile_perf.html)
- [Model Preparation for Android Recipe](https://docs.pytorch.org/tutorials/recipes/model_preparation_android.html)
- [Model Preparation for iOS Recipe](https://docs.pytorch.org/tutorials/recipes/model_preparation_ios.html)
- [Profiling PyTorch workloads with The Instrumentation and Tracing Technology (ITT) API](https://docs.pytorch.org/tutorials/recipes/profile_with_itt.html)
- [Summary of PyTorch Mobile Recipes](https://docs.pytorch.org/tutorials/recipes/ptmobile_recipes_summary.html)
- [Model Interpretability using Captum](https://docs.pytorch.org/tutorials/recipes/recipes/Captum_Recipe.html)
- [Automatic Mixed Precision](https://docs.pytorch.org/tutorials/recipes/recipes/amp_recipe.html)
- [SyntaxError](https://docs.pytorch.org/tutorials/recipes/recipes/benchmark.html)
- [Changing default device](https://docs.pytorch.org/tutorials/recipes/recipes/changing_default_device.html)
- [Defining a Neural Network in PyTorch](https://docs.pytorch.org/tutorials/recipes/recipes/defining_a_neural_network.html)
- [PyTorch Recipes](https://docs.pytorch.org/tutorials/recipes/recipes/index.html)
- [Tips for Loading an nn.Module from a Checkpoint](https://docs.pytorch.org/tutorials/recipes/recipes/module_load_state_dict_tips.html)
- [PyTorch Profiler](https://docs.pytorch.org/tutorials/recipes/recipes/profiler_recipe.html)
- [Reasoning about Shapes in PyTorch](https://docs.pytorch.org/tutorials/recipes/recipes/reasoning_about_shapes.html)
- [Save Load Across Devices](https://docs.pytorch.org/tutorials/recipes/recipes/save_load_across_devices.html)
- [Saving And Loading A General Checkpoint](https://docs.pytorch.org/tutorials/recipes/recipes/saving_and_loading_a_general_checkpoint.html)
- [Saving And Loading Models For Inference](https://docs.pytorch.org/tutorials/recipes/recipes/saving_and_loading_models_for_inference.html)
- [Saving Multiple Models In One File](https://docs.pytorch.org/tutorials/recipes/recipes/saving_multiple_models_in_one_file.html)
- [Computation times](https://docs.pytorch.org/tutorials/recipes/recipes/sg_execution_times.html)
- [Extension points in nn.Module for load_state_dict and tensor subclasses](https://docs.pytorch.org/tutorials/recipes/recipes/swap_tensors.html)
- [How to use TensorBoard with PyTorch](https://docs.pytorch.org/tutorials/recipes/recipes/tensorboard_with_pytorch.html)
- [Timer quick start](https://docs.pytorch.org/tutorials/recipes/recipes/timer_quick_start.html)
- [Performance Tuning Guide](https://docs.pytorch.org/tutorials/recipes/recipes/tuning_guide.html)
- [Warmstarting model using parameters from a different model in PyTorch](https://docs.pytorch.org/tutorials/recipes/recipes/warmstarting_model_using_parameters_from_a_different_model.html)
- [What is a state_dict in PyTorch](https://docs.pytorch.org/tutorials/recipes/recipes/what_is_state_dict.html)
- [Zeroing out gradients in PyTorch](https://docs.pytorch.org/tutorials/recipes/recipes/zeroing_out_gradients.html)
- [Reducing AoT cold start compilation time with regional compilation](https://docs.pytorch.org/tutorials/recipes/regional_aot.html)
- [Reducing torch.compile cold start compilation time with regional compilation](https://docs.pytorch.org/tutorials/recipes/regional_compilation.html)
- [Script and Optimize for Mobile Recipe](https://docs.pytorch.org/tutorials/recipes/script_optimized.html)
- [Computation times](https://docs.pytorch.org/tutorials/recipes/sg_execution_times.html)
- [Compile Time Caching Configuration](https://docs.pytorch.org/tutorials/recipes/torch_compile_caching_configuration_tutorial.html)
- [Compile Time Caching in torch.compile](https://docs.pytorch.org/tutorials/recipes/torch_compile_caching_tutorial.html)
- [(beta) Utilizing Torch Function modes with torch.compile](https://docs.pytorch.org/tutorials/recipes/torch_compile_torch_function_modes.html)
- [Using User-Defined Triton Kernels with torch.compile](https://docs.pytorch.org/tutorials/recipes/torch_compile_user_defined_triton_kernel_tutorial.html)
- [Dynamic Compilation Control with torch.compiler.set_stance](https://docs.pytorch.org/tutorials/recipes/torch_compiler_set_stance_tutorial.html)
- [torch.export AOTInductor Tutorial for Python runtime (Beta)](https://docs.pytorch.org/tutorials/recipes/torch_export_aoti_python.html)
- [Demonstration of torch.export flow, common challenges and the solutions to address them](https://docs.pytorch.org/tutorials/recipes/torch_export_challenges_solutions.html)
- [(beta) Using TORCH_LOGS python API with torch.compile](https://docs.pytorch.org/tutorials/recipes/torch_logs.html)
- [<no title>](https://docs.pytorch.org/tutorials/recipes/torchscript_inference.html)
- [Optimizing CPU Performance on Intel® Xeon® with run_cpu Script](https://docs.pytorch.org/tutorials/recipes/xeon_run_cpu.html)
- [Shard Optimizer States with ZeroRedundancyOptimizer](https://docs.pytorch.org/tutorials/recipes/zero_redundancy_optimizer.html)
- [Recipes](https://docs.pytorch.org/tutorials/recipes_index.html)
- [Computation times](https://docs.pytorch.org/tutorials/sg_execution_times.html)
- [Introduction to Context Parallel](https://docs.pytorch.org/tutorials/unstable/context_parallel.html)
- [Profiling PyTorch RPC-Based Workloads](https://docs.pytorch.org/tutorials/unstable/distributed_rpc_profiling.html)
- [Flight Recorder for Debugging Stuck Jobs](https://docs.pytorch.org/tutorials/unstable/flight_recorder_tutorial.html)
- [(prototype) Accelerating torch.save and torch.load with GPUDirect Storage](https://docs.pytorch.org/tutorials/unstable/gpu_direct_storage.html)
- [(prototype) GPU Quantization with TorchAO](https://docs.pytorch.org/tutorials/unstable/gpu_quantization_torchao_tutorial.html)
- [TorchInductor C++ Wrapper Tutorial](https://docs.pytorch.org/tutorials/unstable/inductor_cpp_wrapper_tutorial.html)
- [How to use torch.compile on Windows CPU/XPU](https://docs.pytorch.org/tutorials/unstable/inductor_windows.html)
- [<no title>](https://docs.pytorch.org/tutorials/unstable/inductor_windows_cpu.html)
- [(Prototype) Convert Mobilenetv2 to Core ML](https://docs.pytorch.org/tutorials/unstable/ios_coreml_workflow.html)
- [(Prototype) Use iOS GPU in PyTorch](https://docs.pytorch.org/tutorials/unstable/ios_gpu_workflow.html)
- [(Prototype) Introduce lite interpreter workflow in Android and iOS](https://docs.pytorch.org/tutorials/unstable/lite_interpreter.html)
- [Efficiently writing “sparse” semantics for Adagrad with MaskedTensor](https://docs.pytorch.org/tutorials/unstable/maskedtensor_adagrad.html)
- [MaskedTensor Advanced Semantics](https://docs.pytorch.org/tutorials/unstable/maskedtensor_advanced_semantics.html)
- [MaskedTensor Overview](https://docs.pytorch.org/tutorials/unstable/maskedtensor_overview.html)
- [MaskedTensor Sparsity](https://docs.pytorch.org/tutorials/unstable/maskedtensor_sparsity.html)
- [Using Max-Autotune Compilation on CPU for Better Performance](https://docs.pytorch.org/tutorials/unstable/max_autotune_on_CPU_tutorial.html)
- [Getting Started with Nested Tensors](https://docs.pytorch.org/tutorials/unstable/nestedtensor.html)
- [(Beta) Convert MobileNetV2 to NNAPI](https://docs.pytorch.org/tutorials/unstable/nnapi_mobilenetv2.html)
- [PyTorch 2 Export Quantization for OpenVINO torch.compile Backend](https://docs.pytorch.org/tutorials/unstable/openvino_quantizer.html)
- [Autoloading Out-of-Tree Extension](https://docs.pytorch.org/tutorials/unstable/python_extension_autoload.html)
- [Computation times](https://docs.pytorch.org/tutorials/unstable/sg_execution_times.html)
- [(prototype) Tracing-based Selective Build Mobile Interpreter in Android and iOS](https://docs.pytorch.org/tutorials/unstable/tracing_based_selective_build.html)
- [torch.vmap](https://docs.pytorch.org/tutorials/unstable/vmap_recipe.html)
- [Unstable](https://docs.pytorch.org/tutorials/unstable_index.html)