Deep Dive# Focused on enhancing model performance, this section includes tutorials on profiling, hyperparameter tuning, quantization, and other techniques to optimize PyTorch models for better efficiency and speed. All Profiling PyTorch Learn how to profile a PyTorch application Profiling Parametrizations Tutorial Learn how to use torch.nn.utils.parametrize to put constraints on your parameters (e.g. make them orthogonal, symmetric positive definite, low-rank...) Model-Optimization,Best-Practice Pruning Tutorial Learn how to use torch.nn.utils.prune to sparsify your neural networks, and how to extend it to implement your own custom pruning technique. Model-Optimization,Best-Practice Inductor CPU Backend Debugging and Profiling Learn the usage, debugging and performance profiling for ``torch.compile`` with Inductor CPU backend. Model-Optimization,inductor (beta) Implementing High-Performance Transformers with SCALED DOT PRODUCT ATTENTION This tutorial explores the new torch.nn.functional.scaled_dot_product_attention and how it can be used to construct Transformer components. Model-Optimization,Attention,Transformer Knowledge Distillation in Convolutional Neural Networks Learn how to improve the accuracy of lightweight models using more powerful models as teachers. Model-Optimization,Image/Video (beta) Channels Last Memory Format in PyTorch Get an overview of Channels Last memory format and understand how it is used to order NCHW tensors in memory preserving dimensions. Memory-Format,Best-Practice,Frontend-APIs Forward-mode Automatic Differentiation Learn how to use forward-mode automatic differentiation. Frontend-APIs Jacobians, Hessians, hvp, vhp, and more Learn how to compute advanced autodiff quantities using torch.func Frontend-APIs Model Ensembling Learn how to ensemble models using torch.vmap Frontend-APIs Per-Sample-Gradients Learn how to compute per-sample-gradients using torch.func Frontend-APIs Neural Tangent Kernels Learn how to compute neural tangent kernels using torch.func Frontend-APIs Using the PyTorch C++ Frontend Walk through an end-to-end example of training a model with the C++ frontend by training a DCGAN – a kind of generative model – to generate images of MNIST digits. Frontend-APIs,C++ Autograd in C++ Frontend The autograd package helps build flexible and dynamic nerural netorks. In this tutorial, exploreseveral examples of doing autograd in PyTorch C++ frontend Frontend-APIs,C++