.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "tutorials/sdk-integration-tutorial.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_tutorials_sdk-integration-tutorial.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_tutorials_sdk-integration-tutorial.py:


Using the ExecuTorch SDK to Profile a Model
========================

**Author:** `Jack Khuu <https://github.com/Jack-Khuu>`__

.. GENERATED FROM PYTHON SOURCE LINES 16-26

The `ExecuTorch SDK <../sdk-overview.html>`__ is a set of tools designed to
provide users with the ability to profile, debug, and visualize ExecuTorch
models.

This tutorial will show a full end-to-end flow of how to utilize the SDK.
Specifically, it will:

1. Generate the artifacts consumed by the SDK (`ETRecord <../sdk-etrecord>`__, `ETDump <../sdk-etdump.html>`__).
2. Create an Inspector class consuming these artifacts.
3. Utilize the Inspector class to analyze the model.

.. GENERATED FROM PYTHON SOURCE LINES 28-47

Prerequisites
-------------

To run this tutorial, you’ll need to install ExecuTorch.

Set up a conda environment. To set up a conda environment in Google Colab::

      !pip install -q condacolab
      import condacolab
      condacolab.install()

      !conda create --name executorch python=3.10
      !conda install -c conda-forge flatbuffers

Install ExecuTorch from source. If cloning is failing on Google Colab, make
sure Colab -> Setting -> Github -> Access Private Repo is checked::

      !git clone --branch v0.1.0 https://{github_username}:{token}@github.com/pytorch/executorch.git
      !cd executorch && bash ./install_requirements.sh

.. GENERATED FROM PYTHON SOURCE LINES 49-61

Generate ETRecord (Optional)
----------------------------

The first step is to generate an ``ETRecord``. ``ETRecord`` contains model
graphs and metadata for linking runtime results (such as profiling) to
the eager model. This is generated via ``executorch.sdk.generate_etrecord``.

``executorch.sdk.generate_etrecord`` takes in an output file path (str), the
edge dialect model (``EdgeProgramManager``), the ExecuTorch dialect model
(``ExecutorchProgramManager``), and an optional dictionary of additional models

In this tutorial, the mobilenet v2 example model is used to demonstrate.

.. GENERATED FROM PYTHON SOURCE LINES 61-97

.. code-block:: default


    import copy

    import torch

    from executorch.examples.models.mobilenet_v2 import MV2Model
    from executorch.exir import (
        EdgeCompileConfig,
        EdgeProgramManager,
        ExecutorchProgramManager,
        to_edge,
    )
    from executorch.sdk import generate_etrecord
    from torch.export import export, ExportedProgram


    # Generate MV2 Model
    model: torch.nn.Module = MV2Model()

    aten_model: ExportedProgram = export(
        model.get_eager_model().eval(),
        model.get_example_inputs(),
    )

    edge_program_manager: EdgeProgramManager = to_edge(
        aten_model, compile_config=EdgeCompileConfig(_check_ir_validity=True)
    )
    edge_program_manager_copy = copy.deepcopy(edge_program_manager)
    et_program_manager: ExecutorchProgramManager = edge_program_manager.to_executorch()


    # Generate ETRecord
    etrecord_path = "etrecord.bin"
    generate_etrecord(etrecord_path, edge_program_manager_copy, et_program_manager)






.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Downloading: "https://download.pytorch.org/models/mobilenet_v2-7ebf99e0.pth" to /var/lib/ci-user/.cache/torch/hub/checkpoints/mobilenet_v2-7ebf99e0.pth

      0%|          | 0.00/13.6M [00:00<?, ?B/s]
    100%|██████████| 13.6M/13.6M [00:00<00:00, 198MB/s]
    /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/utils/_pytree.py:590: UserWarning: pytree_to_str is deprecated. Please use treespec_dumps
      warnings.warn("pytree_to_str is deprecated. Please use treespec_dumps")




.. GENERATED FROM PYTHON SOURCE LINES 102-108

.. warning::
   Users should do a deepcopy of the output of to_edge() and pass in the
   deepcopy to the generate_etrecord API. This is needed because the
   subsequent call, to_executorch(), does an in-place mutation and will
   lose debug data in the process.


.. GENERATED FROM PYTHON SOURCE LINES 111-134

Generate ETDump
---------------

Next step is to generate an ``ETDump``. ``ETDump`` contains runtime results
from executing the model. To generate, users have two options:

**Option 1:**

Use Buck::

      python3 -m examples.sdk.scripts.export_bundled_program -m mv2
      buck2 run -c executorch.event_tracer_enabled=true examples/sdk/sdk_example_runner:sdk_example_runner -- --bundled_program_path mv2_bundled.bp

**Option 2:**

Use CMake::

      cd executorch
      python3 -m examples.sdk.scripts.export_bundled_program -m mv2
      rm -rf cmake-out && mkdir cmake-out && cd cmake-out && cmake -DBUCK2=buck2 -DEXECUTORCH_BUILD_SDK=1 -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=1 ..
      cd ..
      cmake --build cmake-out -j8 -t sdk_example_runner
      ./cmake-out/examples/sdk/sdk_example_runner --bundled_program_path mv2_bundled.bp

.. GENERATED FROM PYTHON SOURCE LINES 136-147

Creating an Inspector
---------------------

Final step is to create the ``Inspector`` by passing in the artifact paths.
Inspector takes the runtime results from ``ETDump`` and correlates them to
the operators of the Edge Dialect Graph.

Note: An ``ETRecord`` is not required. If an ``ETRecord`` is not provided,
the Inspector will show runtime results without operator correlation.

To visualize all runtime events, call Inspector's ``print_data_tabular``.

.. GENERATED FROM PYTHON SOURCE LINES 147-155

.. code-block:: default


    from executorch.sdk import Inspector

    etdump_path = "etdump.etdp"
    inspector = Inspector(etdump_path=etdump_path, etrecord_path=etrecord_path)
    inspector.print_data_tabular()






.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    False



.. GENERATED FROM PYTHON SOURCE LINES 169-177

Analyzing with an Inspector
---------------------------

``Inspector`` provides 2 ways of accessing ingested information: `EventBlocks <../sdk-inspector#eventblock-class>`__
and ``DataFrames``. These mediums give users the ability to perform custom
analysis about their model performance.

Below are examples usages, with both ``EventBlock`` and ``DataFrame`` approaches.

.. GENERATED FROM PYTHON SOURCE LINES 177-186

.. code-block:: default


    # Set Up
    import pprint as pp

    import pandas as pd

    pd.set_option("display.max_colwidth", None)
    pd.set_option("display.max_columns", None)








.. GENERATED FROM PYTHON SOURCE LINES 187-189

If a user wants the raw profiling results, they would do something similar to
finding the raw runtime data of an ``addmm.out`` event.

.. GENERATED FROM PYTHON SOURCE LINES 189-202

.. code-block:: default


    for event_block in inspector.event_blocks:
        # Via EventBlocks
        for event in event_block.events:
            if event.name == "native_call_addmm.out":
                print(event.name, event.perf_data.raw)

        # Via Dataframe
        df = event_block.to_dataframe()
        df = df[df.event_name == "native_call_addmm.out"]
        print(df[["event_name", "raw"]])
        print()








.. GENERATED FROM PYTHON SOURCE LINES 203-206

If a user wants to trace an operator back to their model code, they would do
something similar to finding the module hierarchy and stack trace of the
slowest ``convolution.out`` call.

.. GENERATED FROM PYTHON SOURCE LINES 206-232

.. code-block:: default


    for event_block in inspector.event_blocks:
        # Via EventBlocks
        slowest = None
        for event in event_block.events:
            if event.name == "native_call_convolution.out":
                if slowest is None or event.perf_data.p50 > slowest.perf_data.p50:
                    slowest = event
        if slowest is not None:
            print(slowest.name)
            print()
            pp.pprint(slowest.stack_traces)
            print()
            pp.pprint(slowest.module_hierarchy)

        # Via Dataframe
        df = event_block.to_dataframe()
        df = df[df.event_name == "native_call_convolution.out"]
        if len(df) > 0:
            slowest = df.loc[df["p50"].idxmax()]
            print(slowest.event_name)
            print()
            pp.pprint(slowest.stack_traces)
            print()
            pp.pprint(slowest.module_hierarchy)








.. GENERATED FROM PYTHON SOURCE LINES 233-235

If a user wants the total runtime of a module, they can use
``find_total_for_module``.

.. GENERATED FROM PYTHON SOURCE LINES 235-239

.. code-block:: default


    print(inspector.find_total_for_module("L__self___features"))
    print(inspector.find_total_for_module("L__self___features_14"))





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    0.0
    0.0




.. GENERATED FROM PYTHON SOURCE LINES 240-242

Note: ``find_total_for_module`` is a special first class method of
`Inspector <../sdk-inspector.html>`__

.. GENERATED FROM PYTHON SOURCE LINES 244-258

Conclusion
----------

In this tutorial, we learned about the steps required to consume an ExecuTorch
model with the ExecuTorch SDK. It also showed how to use the Inspector APIs
to analyze the model run results.

Links Mentioned
^^^^^^^^^^^^^^^

- `ExecuTorch SDK <../sdk-overview.html>`__
- `ETRecord <../sdk-etrecord>`__
- `ETDump <../sdk-etdump.html>`__
- `Inspector <../sdk-inspector.html>`__


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 20.215 seconds)


.. _sphx_glr_download_tutorials_sdk-integration-tutorial.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example




    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: sdk-integration-tutorial.py <sdk-integration-tutorial.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: sdk-integration-tutorial.ipynb <sdk-integration-tutorial.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_