.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "tutorials/nvdec_tutorial.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_tutorials_nvdec_tutorial.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_tutorials_nvdec_tutorial.py:


Accelerated video decoding with NVDEC
=====================================

.. _nvdec_tutorial:

**Author**: `Moto Hira <moto@meta.com>`__

.. warning::
    Starting with version 2.8, we are refactoring TorchAudio to transition it
    into a maintenance phase. As a result:

    - The APIs described in this tutorial are deprecated in 2.8 and will be removed in 2.9.
    - The decoding and encoding capabilities of PyTorch for both audio and video
      are being consolidated into TorchCodec.

    Please see https://github.com/pytorch/audio/issues/3902 for more information.

This tutorial shows how to use NVIDIA’s hardware video decoder (NVDEC)
with TorchAudio, and how it improves the performance of video decoding.

.. GENERATED FROM PYTHON SOURCE LINES 24-33

.. note::

   This tutorial requires FFmpeg libraries compiled with HW
   acceleration enabled.

   Please refer to
   :ref:`Enabling GPU video decoder/encoder <enabling_hw_decoder>`
   for how to build FFmpeg with HW acceleration.


.. GENERATED FROM PYTHON SOURCE LINES 34-41

.. code-block:: default


    import torch
    import torchaudio

    print(torch.__version__)
    print(torchaudio.__version__)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    2.8.0+cu126
    2.8.0




.. GENERATED FROM PYTHON SOURCE LINES 43-49

.. code-block:: default

    import os
    import time

    import matplotlib.pyplot as plt
    from torchaudio.io import StreamReader








.. GENERATED FROM PYTHON SOURCE LINES 50-56

Check the prerequisites
-----------------------

First, we check that TorchAudio correctly detects FFmpeg libraries
that support HW decoder/encoder.


.. GENERATED FROM PYTHON SOURCE LINES 57-60

.. code-block:: default


    from torchaudio.utils import ffmpeg_utils








.. GENERATED FROM PYTHON SOURCE LINES 62-67

.. code-block:: default


    print("FFmpeg Library versions:")
    for k, ver in ffmpeg_utils.get_versions().items():
        print(f"  {k}:\t{'.'.join(str(v) for v in ver)}")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    FFmpeg Library versions:
    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:64: UserWarning: torio.utils.ffmpeg_utils.get_versions has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      for k, ver in ffmpeg_utils.get_versions().items():
      libavcodec:   60.3.100
      libavdevice:  60.1.100
      libavfilter:  9.3.100
      libavformat:  60.3.100
      libavutil:    58.2.100




.. GENERATED FROM PYTHON SOURCE LINES 69-75

.. code-block:: default


    print("Available NVDEC Decoders:")
    for k in ffmpeg_utils.get_video_decoders().keys():
        if "cuvid" in k:
            print(f" - {k}")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Available NVDEC Decoders:
    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:71: UserWarning: torio.utils.ffmpeg_utils.get_video_decoders has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      for k in ffmpeg_utils.get_video_decoders().keys():
     - av1_cuvid
     - h264_cuvid
     - hevc_cuvid
     - mjpeg_cuvid
     - mpeg1_cuvid
     - mpeg2_cuvid
     - mpeg4_cuvid
     - vc1_cuvid
     - vp8_cuvid
     - vp9_cuvid




.. GENERATED FROM PYTHON SOURCE LINES 77-81

.. code-block:: default


    print("Avaialbe GPU:")
    print(torch.cuda.get_device_properties(0))





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Avaialbe GPU:
    _CudaDeviceProperties(name='NVIDIA A10G', major=8, minor=6, total_memory=22598MB, multi_processor_count=80, uuid=566db26d-0405-a011-4198-2699df443f87, pci_bus_id=0, pci_device_id=30, pci_domain_id=0, L2_cache_size=6MB)




.. GENERATED FROM PYTHON SOURCE LINES 82-94

We will use the following video which has the following properties;

- Codec: H.264
- Resolution: 960x540
- FPS: 29.97
- Pixel format: YUV420P

.. raw:: html

   <video style="max-width: 100%" controls>
     <source src="https://download.pytorch.org/torchaudio/tutorial-assets/stream-api/NASAs_Most_Scientifically_Complex_Space_Observatory_Requires_Precision-MP4_small.mp4" type="video/mp4">
   </video>

.. GENERATED FROM PYTHON SOURCE LINES 98-103

.. code-block:: default


    src = torchaudio.utils.download_asset(
        "tutorial-assets/stream-api/NASAs_Most_Scientifically_Complex_Space_Observatory_Requires_Precision-MP4_small.mp4"
    )





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:99: UserWarning: torchaudio.utils.download.download_asset has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      src = torchaudio.utils.download_asset(

      0%|          | 0.00/31.8M [00:00<?, ?B/s]
     88%|########7 | 27.9M/31.8M [00:00<00:00, 291MB/s]
    100%|##########| 31.8M/31.8M [00:00<00:00, 291MB/s]




.. GENERATED FROM PYTHON SOURCE LINES 104-111

Decoding videos with NVDEC
--------------------------

To use HW video decoder, you need to specify the HW decoder when
defining the output video stream by passing ``decoder`` option to
:py:meth:`~torchaudio.io.StreamReader.add_video_stream` method.


.. GENERATED FROM PYTHON SOURCE LINES 111-117

.. code-block:: default


    s = StreamReader(src)
    s.add_video_stream(5, decoder="h264_cuvid")
    s.fill_buffer()
    (video,) = s.pop_chunks()





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:112: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(src)




.. GENERATED FROM PYTHON SOURCE LINES 118-119

The video frames are decoded and returned as tensor of NCHW format.

.. GENERATED FROM PYTHON SOURCE LINES 120-123

.. code-block:: default


    print(video.shape, video.dtype)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    torch.Size([5, 3, 540, 960]) torch.uint8




.. GENERATED FROM PYTHON SOURCE LINES 124-126

By default, the decoded frames are sent back to CPU memory, and
CPU tensors are created.

.. GENERATED FROM PYTHON SOURCE LINES 127-130

.. code-block:: default


    print(video.device)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    cpu




.. GENERATED FROM PYTHON SOURCE LINES 131-144

By specifying ``hw_accel`` option, you can convert the decoded frames
to CUDA tensor.
``hw_accel`` option takes string values and pass it
to :py:class:`torch.device`.

.. note::

   Currently, ``hw_accel`` option and
   :py:meth:`~torchaudio.io.StreamReader.add_basic_video_stream`
   are not compatible. ``add_basic_video_stream`` adds post-decoding
   process, which is designed for frames in CPU memory.
   Please use :py:meth:`~torchaudio.io.StreamReader.add_video_stream`.


.. GENERATED FROM PYTHON SOURCE LINES 145-154

.. code-block:: default


    s = StreamReader(src)
    s.add_video_stream(5, decoder="h264_cuvid", hw_accel="cuda:0")
    s.fill_buffer()
    (video,) = s.pop_chunks()

    print(video.shape, video.dtype, video.device)






.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:146: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(src)
    torch.Size([5, 3, 540, 960]) torch.uint8 cuda:0




.. GENERATED FROM PYTHON SOURCE LINES 155-190

.. note::

   When there are multiple of GPUs available, ``StreamReader`` by
   default uses the first GPU. You can change this by providing
   ``"gpu"`` option.

.. code::

   # Video data is sent to CUDA device 0, decoded and
   # converted on the same device.
   s.add_video_stream(
       ...,
       decoder="h264_cuvid",
       decoder_option={"gpu": "0"},
       hw_accel="cuda:0",
   )

.. note::

   ``"gpu"`` option and ``hw_accel`` option can be specified
   independently. If they do not match, decoded frames are
   transfered to the device specified by ``hw_accell``
   automatically.

.. code::

   # Video data is sent to CUDA device 0, and decoded there.
   # Then it is transfered to CUDA device 1, and converted to
   # CUDA tensor.
   s.add_video_stream(
       ...,
       decoder="h264_cuvid",
       decoder_option={"gpu": "0"},
       hw_accel="cuda:1",
   )

.. GENERATED FROM PYTHON SOURCE LINES 192-200

Visualization
-------------

Let's look at the frames decoded by HW decoder and compare them
against equivalent results from software decoders.

The following function seeks into the given timestamp and decode one
frame with the specificed decoder.

.. GENERATED FROM PYTHON SOURCE LINES 200-211

.. code-block:: default



    def test_decode(decoder: str, seek: float):
        s = StreamReader(src)
        s.seek(seek)
        s.add_video_stream(1, decoder=decoder)
        s.fill_buffer()
        (video,) = s.pop_chunks()
        return video[0]









.. GENERATED FROM PYTHON SOURCE LINES 213-219

.. code-block:: default


    timestamps = [12, 19, 45, 131, 180]

    cpu_frames = [test_decode(decoder="h264", seek=ts) for ts in timestamps]
    cuda_frames = [test_decode(decoder="h264_cuvid", seek=ts) for ts in timestamps]





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:203: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(src)




.. GENERATED FROM PYTHON SOURCE LINES 220-226

.. note::

   Currently, HW decoder does not support colorspace conversion.
   Decoded frames are YUV format.
   The following function performs YUV to RGB covnersion
   (and axis shuffling for plotting).

.. GENERATED FROM PYTHON SOURCE LINES 227-248

.. code-block:: default



    def yuv_to_rgb(frames):
        frames = frames.cpu().to(torch.float)
        y = frames[..., 0, :, :]
        u = frames[..., 1, :, :]
        v = frames[..., 2, :, :]

        y /= 255
        u = u / 255 - 0.5
        v = v / 255 - 0.5

        r = y + 1.14 * v
        g = y + -0.396 * u - 0.581 * v
        b = y + 2.029 * u

        rgb = torch.stack([r, g, b], -1)
        rgb = (rgb * 255).clamp(0, 255).to(torch.uint8)
        return rgb.numpy()









.. GENERATED FROM PYTHON SOURCE LINES 249-251

Now we visualize the resutls.


.. GENERATED FROM PYTHON SOURCE LINES 252-269

.. code-block:: default



    def plot():
        n_rows = len(timestamps)
        fig, axes = plt.subplots(n_rows, 2, figsize=[12.8, 16.0])
        for i in range(n_rows):
            axes[i][0].imshow(yuv_to_rgb(cpu_frames[i]))
            axes[i][1].imshow(yuv_to_rgb(cuda_frames[i]))

        axes[0][0].set_title("Software decoder")
        axes[0][1].set_title("HW decoder")
        plt.setp(axes, xticks=[], yticks=[])
        plt.tight_layout()


    plot()




.. image-sg:: /tutorials/images/sphx_glr_nvdec_tutorial_001.png
   :alt: Software decoder, HW decoder
   :srcset: /tutorials/images/sphx_glr_nvdec_tutorial_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 270-273

They are indistinguishable to the eyes of the author.
Feel free to let us know if you spot something. :)


.. GENERATED FROM PYTHON SOURCE LINES 277-295

HW resizing and cropping
------------------------

You can use ``decoder_option`` argument to provide decoder-specific
options.

The following options are often relevant in preprocessing.

- ``resize``: Resize the frame into ``(width)x(height)``.
- ``crop``: Crop the frame ``(top)x(bottom)x(left)x(right)``.
  Note that the specified values are the amount of rows/columns removed.
  The final image size is ``(width - left - right)x(height - top -bottom)``.
  If ``crop`` and ``resize`` options are used together,
  ``crop`` is performed first.

For other available options, please run
``ffmpeg -h decoder=h264_cuvid``.


.. GENERATED FROM PYTHON SOURCE LINES 295-307

.. code-block:: default



    def test_options(option):
        s = StreamReader(src)
        s.seek(87)
        s.add_video_stream(1, decoder="h264_cuvid", hw_accel="cuda:0", decoder_option=option)
        s.fill_buffer()
        (video,) = s.pop_chunks()
        print(f"Option: {option}:\t{video.shape}")
        return video[0]









.. GENERATED FROM PYTHON SOURCE LINES 309-316

.. code-block:: default


    original = test_options(option=None)
    resized = test_options(option={"resize": "480x270"})
    cropped = test_options(option={"crop": "135x135x240x240"})
    cropped_and_resized = test_options(option={"crop": "135x135x240x240", "resize": "640x360"})






.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:298: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(src)
    Option: None:   torch.Size([1, 3, 540, 960])
    Option: {'resize': '480x270'}:  torch.Size([1, 3, 270, 480])
    Option: {'crop': '135x135x240x240'}:    torch.Size([1, 3, 270, 480])
    Option: {'crop': '135x135x240x240', 'resize': '640x360'}:       torch.Size([1, 3, 360, 640])




.. GENERATED FROM PYTHON SOURCE LINES 318-337

.. code-block:: default



    def plot():
        fig, axes = plt.subplots(2, 2, figsize=[12.8, 9.6])
        axes[0][0].imshow(yuv_to_rgb(original))
        axes[0][1].imshow(yuv_to_rgb(resized))
        axes[1][0].imshow(yuv_to_rgb(cropped))
        axes[1][1].imshow(yuv_to_rgb(cropped_and_resized))

        axes[0][0].set_title("Original")
        axes[0][1].set_title("Resized")
        axes[1][0].set_title("Cropped")
        axes[1][1].set_title("Cropped and resized")
        plt.tight_layout()
        return fig


    plot()




.. image-sg:: /tutorials/images/sphx_glr_nvdec_tutorial_002.png
   :alt: Original, Resized, Cropped, Cropped and resized
   :srcset: /tutorials/images/sphx_glr_nvdec_tutorial_002.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    <Figure size 1280x960 with 4 Axes>



.. GENERATED FROM PYTHON SOURCE LINES 338-360

Comparing resizing methods
--------------------------

Unlike software scaling, NVDEC does not provide an option to choose
the scaling algorithm.
In ML applicatoins, it is often necessary to construct a
preprocessing pipeline with a similar numerical property.
So here we compare the result of hardware resizing with software
resizing of different algorithms.

We will use the following video, which contains the test pattern
generated using the following command.

.. code::

   ffmpeg -y -f lavfi -t 12.05 -i mptestsrc -movflags +faststart mptestsrc.mp4

.. raw:: html

   <video style="max-width: 100%" controls>
     <source src="https://download.pytorch.org/torchaudio/tutorial-assets/mptestsrc.mp4" type="video/mp4">
   </video>

.. GENERATED FROM PYTHON SOURCE LINES 364-368

.. code-block:: default


    test_src = torchaudio.utils.download_asset("tutorial-assets/mptestsrc.mp4")






.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:365: UserWarning: torchaudio.utils.download.download_asset has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      test_src = torchaudio.utils.download_asset("tutorial-assets/mptestsrc.mp4")

      0%|          | 0.00/232k [00:00<?, ?B/s]
    100%|##########| 232k/232k [00:00<00:00, 128MB/s]




.. GENERATED FROM PYTHON SOURCE LINES 369-372

The following function decodes video and
apply the specified scaling algorithm.


.. GENERATED FROM PYTHON SOURCE LINES 372-384

.. code-block:: default



    def decode_resize_ffmpeg(mode, height, width, seek):
        filter_desc = None if mode is None else f"scale={width}:{height}:sws_flags={mode}"
        s = StreamReader(test_src)
        s.add_video_stream(1, filter_desc=filter_desc)
        s.seek(seek)
        s.fill_buffer()
        (chunk,) = s.pop_chunks()
        return chunk









.. GENERATED FROM PYTHON SOURCE LINES 385-387

The following function uses HW decoder to decode video and resize.


.. GENERATED FROM PYTHON SOURCE LINES 387-398

.. code-block:: default



    def decode_resize_cuvid(height, width, seek):
        s = StreamReader(test_src)
        s.add_video_stream(1, decoder="h264_cuvid", decoder_option={"resize": f"{width}x{height}"}, hw_accel="cuda:0")
        s.seek(seek)
        s.fill_buffer()
        (chunk,) = s.pop_chunks()
        return chunk.cpu()









.. GENERATED FROM PYTHON SOURCE LINES 399-400

Now we execute them and visualize the resulting frames.

.. GENERATED FROM PYTHON SOURCE LINES 400-416

.. code-block:: default


    params = {"height": 224, "width": 224, "seek": 3}

    frames = [
        decode_resize_ffmpeg(None, **params),
        decode_resize_ffmpeg("neighbor", **params),
        decode_resize_ffmpeg("bilinear", **params),
        decode_resize_ffmpeg("bicubic", **params),
        decode_resize_cuvid(**params),
        decode_resize_ffmpeg("spline", **params),
        decode_resize_ffmpeg("lanczos:param0=1", **params),
        decode_resize_ffmpeg("lanczos:param0=3", **params),
        decode_resize_ffmpeg("lanczos:param0=5", **params),
    ]






.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:376: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(test_src)
    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:390: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(test_src)




.. GENERATED FROM PYTHON SOURCE LINES 418-442

.. code-block:: default



    def plot():
        fig, axes = plt.subplots(3, 3, figsize=[12.8, 15.2])
        for i, f in enumerate(frames):
            h, w = f.shape[2:4]
            f = f[..., : h // 4, : w // 4]
            axes[i // 3][i % 3].imshow(yuv_to_rgb(f[0]))
        axes[0][0].set_title("Original")
        axes[0][1].set_title("nearest neighbor")
        axes[0][2].set_title("bilinear")
        axes[1][0].set_title("bicubic")
        axes[1][1].set_title("NVDEC")
        axes[1][2].set_title("spline")
        axes[2][0].set_title("lanczos(1)")
        axes[2][1].set_title("lanczos(3)")
        axes[2][2].set_title("lanczos(5)")

        plt.setp(axes, xticks=[], yticks=[])
        plt.tight_layout()


    plot()




.. image-sg:: /tutorials/images/sphx_glr_nvdec_tutorial_003.png
   :alt: Original, nearest neighbor, bilinear, bicubic, NVDEC, spline, lanczos(1), lanczos(3), lanczos(5)
   :srcset: /tutorials/images/sphx_glr_nvdec_tutorial_003.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 443-446

None of them is exactly the same. To the eyes of authors, lanczos(1)
appears to be most similar to NVDEC.
The bicubic looks close as well.

.. GENERATED FROM PYTHON SOURCE LINES 448-454

Benchmark NVDEC with StreamReader
---------------------------------

In this section, we compare the performace of software video
decoding and HW video decoding.


.. GENERATED FROM PYTHON SOURCE LINES 457-476

Decode as CUDA frames
---------------------

First, we compare the time it takes for software decoder and
hardware encoder to decode the same video.
To make the result comparable, when using software decoder, we move
the resulting tensor to CUDA.

The procedures to test look like the following

- Use hardware decoder and place data on CUDA directly
- Use software decoder, generate CPU Tensors and move them to CUDA.

.. note:

   Because HW decoder currently only supports reading videos as
   YUV444P format, we decode frames into YUV444P format for the case of
   software decoder as well.


.. GENERATED FROM PYTHON SOURCE LINES 479-480

The following function implements the hardware decoder test case.

.. GENERATED FROM PYTHON SOURCE LINES 480-498

.. code-block:: default



    def test_decode_cuda(src, decoder, hw_accel="cuda", frames_per_chunk=5):
        s = StreamReader(src)
        s.add_video_stream(frames_per_chunk, decoder=decoder, hw_accel=hw_accel)

        num_frames = 0
        chunk = None
        t0 = time.monotonic()
        for (chunk,) in s.stream():
            num_frames += chunk.shape[0]
        elapsed = time.monotonic() - t0
        print(f" - Shape: {chunk.shape}")
        fps = num_frames / elapsed
        print(f" - Processed {num_frames} frames in {elapsed:.2f} seconds. ({fps:.2f} fps)")
        return fps









.. GENERATED FROM PYTHON SOURCE LINES 499-500

The following function implements the software decoder test case.

.. GENERATED FROM PYTHON SOURCE LINES 500-520

.. code-block:: default



    def test_decode_cpu(src, threads, decoder=None, frames_per_chunk=5):
        s = StreamReader(src)
        s.add_video_stream(frames_per_chunk, decoder=decoder, decoder_option={"threads": f"{threads}"})

        num_frames = 0
        device = torch.device("cuda")
        t0 = time.monotonic()
        for i, (chunk,) in enumerate(s.stream()):
            if i == 0:
                print(f" - Shape: {chunk.shape}")
            num_frames += chunk.shape[0]
            chunk = chunk.to(device)
        elapsed = time.monotonic() - t0
        fps = num_frames / elapsed
        print(f" - Processed {num_frames} frames in {elapsed:.2f} seconds. ({fps:.2f} fps)")
        return fps









.. GENERATED FROM PYTHON SOURCE LINES 521-523

For each resolution of video, we run multiple software decoder test
cases with different number of threads.

.. GENERATED FROM PYTHON SOURCE LINES 523-536

.. code-block:: default



    def run_decode_tests(src, frames_per_chunk=5):
        fps = []
        print(f"Testing: {os.path.basename(src)}")
        for threads in [1, 4, 8, 16]:
            print(f"* Software decoding (num_threads={threads})")
            fps.append(test_decode_cpu(src, threads))
        print("* Hardware decoding")
        fps.append(test_decode_cuda(src, decoder="h264_cuvid"))
        return fps









.. GENERATED FROM PYTHON SOURCE LINES 537-541

Now we run the tests with videos of different resolutions.

QVGA
----

.. GENERATED FROM PYTHON SOURCE LINES 541-545

.. code-block:: default


    src_qvga = torchaudio.utils.download_asset("tutorial-assets/testsrc2_qvga.h264.mp4")
    fps_qvga = run_decode_tests(src_qvga)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:542: UserWarning: torchaudio.utils.download.download_asset has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      src_qvga = torchaudio.utils.download_asset("tutorial-assets/testsrc2_qvga.h264.mp4")

      0%|          | 0.00/1.06M [00:00<?, ?B/s]
    100%|##########| 1.06M/1.06M [00:00<00:00, 267MB/s]
    Testing: testsrc2_qvga.h264.mp4
    * Software decoding (num_threads=1)
    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:503: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(src)
     - Shape: torch.Size([5, 3, 240, 320])
     - Processed 900 frames in 0.50 seconds. (1792.93 fps)
    * Software decoding (num_threads=4)
     - Shape: torch.Size([5, 3, 240, 320])
     - Processed 900 frames in 0.35 seconds. (2551.90 fps)
    * Software decoding (num_threads=8)
     - Shape: torch.Size([5, 3, 240, 320])
     - Processed 900 frames in 0.34 seconds. (2662.90 fps)
    * Software decoding (num_threads=16)
     - Shape: torch.Size([5, 3, 240, 320])
     - Processed 895 frames in 0.35 seconds. (2551.38 fps)
    * Hardware decoding
    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:483: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(src)
     - Shape: torch.Size([5, 3, 240, 320])
     - Processed 900 frames in 2.01 seconds. (447.51 fps)




.. GENERATED FROM PYTHON SOURCE LINES 546-548

VGA
---

.. GENERATED FROM PYTHON SOURCE LINES 548-552

.. code-block:: default


    src_vga = torchaudio.utils.download_asset("tutorial-assets/testsrc2_vga.h264.mp4")
    fps_vga = run_decode_tests(src_vga)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:549: UserWarning: torchaudio.utils.download.download_asset has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      src_vga = torchaudio.utils.download_asset("tutorial-assets/testsrc2_vga.h264.mp4")

      0%|          | 0.00/3.59M [00:00<?, ?B/s]
    100%|##########| 3.59M/3.59M [00:00<00:00, 407MB/s]
    Testing: testsrc2_vga.h264.mp4
    * Software decoding (num_threads=1)
    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:503: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(src)
     - Shape: torch.Size([5, 3, 480, 640])
     - Processed 900 frames in 1.26 seconds. (711.65 fps)
    * Software decoding (num_threads=4)
     - Shape: torch.Size([5, 3, 480, 640])
     - Processed 900 frames in 0.69 seconds. (1302.73 fps)
    * Software decoding (num_threads=8)
     - Shape: torch.Size([5, 3, 480, 640])
     - Processed 900 frames in 0.73 seconds. (1239.85 fps)
    * Software decoding (num_threads=16)
     - Shape: torch.Size([5, 3, 480, 640])
     - Processed 895 frames in 0.66 seconds. (1353.78 fps)
    * Hardware decoding
    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:483: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(src)
     - Shape: torch.Size([5, 3, 480, 640])
     - Processed 900 frames in 0.34 seconds. (2639.67 fps)




.. GENERATED FROM PYTHON SOURCE LINES 553-555

XGA
---

.. GENERATED FROM PYTHON SOURCE LINES 555-560

.. code-block:: default


    src_xga = torchaudio.utils.download_asset("tutorial-assets/testsrc2_xga.h264.mp4")
    fps_xga = run_decode_tests(src_xga)






.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:556: UserWarning: torchaudio.utils.download.download_asset has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      src_xga = torchaudio.utils.download_asset("tutorial-assets/testsrc2_xga.h264.mp4")

      0%|          | 0.00/9.22M [00:00<?, ?B/s]
    100%|##########| 9.22M/9.22M [00:00<00:00, 319MB/s]
    Testing: testsrc2_xga.h264.mp4
    * Software decoding (num_threads=1)
    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:503: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(src)
     - Shape: torch.Size([5, 3, 768, 1024])
     - Processed 900 frames in 2.68 seconds. (335.81 fps)
    * Software decoding (num_threads=4)
     - Shape: torch.Size([5, 3, 768, 1024])
     - Processed 900 frames in 1.17 seconds. (771.75 fps)
    * Software decoding (num_threads=8)
     - Shape: torch.Size([5, 3, 768, 1024])
     - Processed 900 frames in 1.07 seconds. (838.61 fps)
    * Software decoding (num_threads=16)
     - Shape: torch.Size([5, 3, 768, 1024])
     - Processed 895 frames in 1.01 seconds. (881.89 fps)
    * Hardware decoding
    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:483: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(src)
     - Shape: torch.Size([5, 3, 768, 1024])
     - Processed 900 frames in 0.61 seconds. (1473.41 fps)




.. GENERATED FROM PYTHON SOURCE LINES 561-565

Result
------

Now we plot the result.

.. GENERATED FROM PYTHON SOURCE LINES 565-590

.. code-block:: default



    def plot():
        fig, ax = plt.subplots(figsize=[9.6, 6.4])

        for items in zip(fps_qvga, fps_vga, fps_xga, "ov^sx"):
            ax.plot(items[:-1], marker=items[-1])
        ax.grid(axis="both")
        ax.set_xticks([0, 1, 2], ["QVGA (320x240)", "VGA (640x480)", "XGA (1024x768)"])
        ax.legend(
            [
                "Software Decoding (threads=1)",
                "Software Decoding (threads=4)",
                "Software Decoding (threads=8)",
                "Software Decoding (threads=16)",
                "Hardware Decoding (CUDA Tensor)",
            ]
        )
        ax.set_title("Speed of processing video frames")
        ax.set_ylabel("Frames per second")
        plt.tight_layout()


    plot()




.. image-sg:: /tutorials/images/sphx_glr_nvdec_tutorial_004.png
   :alt: Speed of processing video frames
   :srcset: /tutorials/images/sphx_glr_nvdec_tutorial_004.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 591-609

We observe couple of things

- Increasing the number of threads in software decoding makes the
  pipeline faster, but the performance saturates around 8 threads.
- The performance gain from using hardware decoder depends on the
  resolution of video.
- At lower resolutions like QVGA, hardware decoding is slower than
  software decoding
- At higher resolutions like XGA, hardware decoding is faster
  than software decoding.


It is worth noting that the performance gain also depends on the
type of GPU.
We observed that when decoding VGA videos using V100 or A100 GPUs,
hardware decoders are slower than software decoders. But using A10
GPU hardware deocder is faster than software decodr.


.. GENERATED FROM PYTHON SOURCE LINES 612-636

Decode and resize
-----------------

Next, we add resize operation to the pipeline.
We will compare the following pipelines.

1. Decode video using software decoder and read the frames as
   PyTorch Tensor. Resize the tensor using
   :py:func:`torch.nn.functional.interpolate`, then send
   the resulting tensor to CUDA device.
2. Decode video using software decoder, resize the frame with
   FFmpeg's filter graph, read the resized frames as PyTorch tensor,
   then send it to CUDA device.
3. Decode and resize video simulaneously with HW decoder, read the
   resulting frames as CUDA tensor.

The pipeline 1 represents common video loading implementations.

The pipeline 2 uses FFmpeg's filter graph, which allows to manipulate
raw frames before converting them to Tensors.

The pipeline 3 has the minimum amount of data transfer from CPU to
CUDA, which significantly contribute to performant data loading.


.. GENERATED FROM PYTHON SOURCE LINES 639-644

The following function implements the pipeline 1. It uses PyTorch's
:py:func:`torch.nn.functional.interpolate`.
We use ``bincubic`` mode, as we saw that the resulting frames are
closest to NVDEC resizing.


.. GENERATED FROM PYTHON SOURCE LINES 644-665

.. code-block:: default



    def test_decode_then_resize(src, height, width, mode="bicubic", frames_per_chunk=5):
        s = StreamReader(src)
        s.add_video_stream(frames_per_chunk, decoder_option={"threads": "8"})

        num_frames = 0
        device = torch.device("cuda")
        chunk = None
        t0 = time.monotonic()
        for (chunk,) in s.stream():
            num_frames += chunk.shape[0]
            chunk = torch.nn.functional.interpolate(chunk, [height, width], mode=mode, antialias=True)
            chunk = chunk.to(device)
        elapsed = time.monotonic() - t0
        fps = num_frames / elapsed
        print(f" - Shape: {chunk.shape}")
        print(f" - Processed {num_frames} frames in {elapsed:.2f} seconds. ({fps:.2f} fps)")
        return fps









.. GENERATED FROM PYTHON SOURCE LINES 666-672

The following function implements the pipeline 2. Frames are resized
as part of decoding process, then sent to CUDA device.

We use ``bincubic`` mode, to make the result comparable with
PyTorch-based implementation above.


.. GENERATED FROM PYTHON SOURCE LINES 672-694

.. code-block:: default



    def test_decode_and_resize(src, height, width, mode="bicubic", frames_per_chunk=5):
        s = StreamReader(src)
        s.add_video_stream(
            frames_per_chunk, filter_desc=f"scale={width}:{height}:sws_flags={mode}", decoder_option={"threads": "8"}
        )

        num_frames = 0
        device = torch.device("cuda")
        chunk = None
        t0 = time.monotonic()
        for (chunk,) in s.stream():
            num_frames += chunk.shape[0]
            chunk = chunk.to(device)
        elapsed = time.monotonic() - t0
        fps = num_frames / elapsed
        print(f" - Shape: {chunk.shape}")
        print(f" - Processed {num_frames} frames in {elapsed:.2f} seconds. ({fps:.2f} fps)")
        return fps









.. GENERATED FROM PYTHON SOURCE LINES 695-697

The following function implements the pipeline 3. Resizing is
performed by NVDEC and the resulting tensor is placed on CUDA memory.

.. GENERATED FROM PYTHON SOURCE LINES 697-715

.. code-block:: default



    def test_hw_decode_and_resize(src, decoder, decoder_option, hw_accel="cuda", frames_per_chunk=5):
        s = StreamReader(src)
        s.add_video_stream(5, decoder=decoder, decoder_option=decoder_option, hw_accel=hw_accel)

        num_frames = 0
        chunk = None
        t0 = time.monotonic()
        for (chunk,) in s.stream():
            num_frames += chunk.shape[0]
        elapsed = time.monotonic() - t0
        fps = num_frames / elapsed
        print(f" - Shape: {chunk.shape}")
        print(f" - Processed {num_frames} frames in {elapsed:.2f} seconds. ({fps:.2f} fps)")
        return fps









.. GENERATED FROM PYTHON SOURCE LINES 716-718

The following function run the benchmark functions on given sources.


.. GENERATED FROM PYTHON SOURCE LINES 719-733

.. code-block:: default



    def run_resize_tests(src):
        print(f"Testing: {os.path.basename(src)}")
        height, width = 224, 224
        print("* Software decoding with PyTorch interpolate")
        cpu_resize1 = test_decode_then_resize(src, height=height, width=width)
        print("* Software decoding with FFmpeg scale")
        cpu_resize2 = test_decode_and_resize(src, height=height, width=width)
        print("* Hardware decoding with resize")
        cuda_resize = test_hw_decode_and_resize(src, decoder="h264_cuvid", decoder_option={"resize": f"{width}x{height}"})
        return [cpu_resize1, cpu_resize2, cuda_resize]









.. GENERATED FROM PYTHON SOURCE LINES 734-735

Now we run the tests.

.. GENERATED FROM PYTHON SOURCE LINES 738-740

QVGA
----

.. GENERATED FROM PYTHON SOURCE LINES 740-743

.. code-block:: default


    fps_qvga = run_resize_tests(src_qvga)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Testing: testsrc2_qvga.h264.mp4
    * Software decoding with PyTorch interpolate
    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:647: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(src)
     - Shape: torch.Size([5, 3, 224, 224])
     - Processed 900 frames in 0.59 seconds. (1517.03 fps)
    * Software decoding with FFmpeg scale
    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:675: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(src)
     - Shape: torch.Size([5, 3, 224, 224])
     - Processed 900 frames in 0.38 seconds. (2391.42 fps)
    * Hardware decoding with resize
    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:700: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(src)
     - Shape: torch.Size([5, 3, 224, 224])
     - Processed 900 frames in 2.01 seconds. (447.62 fps)




.. GENERATED FROM PYTHON SOURCE LINES 744-746

VGA
---

.. GENERATED FROM PYTHON SOURCE LINES 746-749

.. code-block:: default


    fps_vga = run_resize_tests(src_vga)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Testing: testsrc2_vga.h264.mp4
    * Software decoding with PyTorch interpolate
    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:647: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(src)
     - Shape: torch.Size([5, 3, 224, 224])
     - Processed 900 frames in 1.36 seconds. (659.64 fps)
    * Software decoding with FFmpeg scale
    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:675: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(src)
     - Shape: torch.Size([5, 3, 224, 224])
     - Processed 900 frames in 0.68 seconds. (1333.05 fps)
    * Hardware decoding with resize
    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:700: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(src)
     - Shape: torch.Size([5, 3, 224, 224])
     - Processed 900 frames in 1.47 seconds. (611.88 fps)




.. GENERATED FROM PYTHON SOURCE LINES 750-752

XGA
---

.. GENERATED FROM PYTHON SOURCE LINES 752-755

.. code-block:: default


    fps_xga = run_resize_tests(src_xga)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Testing: testsrc2_xga.h264.mp4
    * Software decoding with PyTorch interpolate
    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:647: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(src)
     - Shape: torch.Size([5, 3, 224, 224])
     - Processed 900 frames in 2.55 seconds. (352.97 fps)
    * Software decoding with FFmpeg scale
    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:675: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(src)
     - Shape: torch.Size([5, 3, 224, 224])
     - Processed 900 frames in 1.06 seconds. (847.18 fps)
    * Hardware decoding with resize
    /pytorch/audio/examples/tutorials/nvdec_tutorial.py:700: UserWarning: torio.io._streaming_media_decoder.StreamingMediaDecoder has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
      s = StreamReader(src)
     - Shape: torch.Size([5, 3, 224, 224])
     - Processed 900 frames in 0.61 seconds. (1480.00 fps)




.. GENERATED FROM PYTHON SOURCE LINES 756-760

Result
------
Now we plot the result.


.. GENERATED FROM PYTHON SOURCE LINES 760-784

.. code-block:: default



    def plot():
        fig, ax = plt.subplots(figsize=[9.6, 6.4])

        for items in zip(fps_qvga, fps_vga, fps_xga, "ov^sx"):
            ax.plot(items[:-1], marker=items[-1])
        ax.grid(axis="both")
        ax.set_xticks([0, 1, 2], ["QVGA (320x240)", "VGA (640x480)", "XGA (1024x768)"])
        ax.legend(
            [
                "Software decoding\nwith resize\n(PyTorch interpolate)",
                "Software decoding\nwith resize\n(FFmpeg scale)",
                "NVDEC\nwith resizing",
            ]
        )
        ax.set_title("Speed of processing video frames")
        ax.set_xlabel("Input video resolution")
        ax.set_ylabel("Frames per second")
        plt.tight_layout()


    plot()




.. image-sg:: /tutorials/images/sphx_glr_nvdec_tutorial_005.png
   :alt: Speed of processing video frames
   :srcset: /tutorials/images/sphx_glr_nvdec_tutorial_005.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 785-797

Hardware deocder shows a similar trend as previous experiment.
In fact, the performance is almost the same. Hardware resizing has
almost zero overhead for scaling down the frames.

Software decoding also shows a similar trend. Performing resizing as
part of decoding is faster. One possible explanation is that, video
frames are internally stored as YUV420P, which has half the number
of pixels compared to RGB24, or YUV444P. This means that if resizing
before copying frame data to PyTorch tensor, the number of pixels
manipulated and copied are smaller than the case where applying
resizing after frames are converted to Tensor.


.. GENERATED FROM PYTHON SOURCE LINES 800-801

Tag: :obj:`torchaudio.io`


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  30.874 seconds)


.. _sphx_glr_download_tutorials_nvdec_tutorial.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example


    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: nvdec_tutorial.py <nvdec_tutorial.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: nvdec_tutorial.ipynb <nvdec_tutorial.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_