Arm VGF Backend#
The Arm® VGF backend is the ExecuTorch solution for lowering PyTorch models to VGF compatible hardware. It leverages the TOSA operator set and the ML SDK for Vulkan® to produce a .PTE file. The VGF backend also supports execution from a .PTE file and provides functionality to extract the corresponding VGF file for integration into various applications.
Features#
Wide operator support for delegating large parts of models to the VGF target.
A quantizer that optimizes quantization for the VGF target.
Target Requirements#
The target system must include ML SDK for Vulkan and a Vulkan driver with Vulkan API >= 1.3.
Development Requirements#
Tip
All requirements can be downloaded using examples/arm/setup.sh --enable-mlsdk-deps --disable-ethos-u-deps and added to the path using
source examples/arm/ethos-u-scratch/setup_path.sh
For the AOT flow, compilation of a model to .pte format using the VGF backend, the requirements are:
TOSA Serialization Library for serializing the Exir IR graph into TOSA IR.
ML SDK Model Converter for converting TOSA flatbuffers to VGF files.
And for building and running your application using the generic executor_runner:
Vulkan API should be set up locally for GPU execution support.
ML Emulation Layer for Vulkan for testing on Vulkan API.
Using the Arm VGF Backend#
The VGF Minimal Example demonstrates how to lower a module using the VGF backend.
The main configuration point for the lowering is the VgfCompileSpec consumed by the partitioner and quantizer.
The full user-facing API is documented below.
class VgfCompileSpec(tosa_spec: executorch.backends.arm.tosa.specification.TosaSpecification | str | None = None, compiler_flags: list[str] | None = None)
Compile spec for VGF compatible targets.
Args:
tosa_spec: TOSA specification that should be targeted.
compiler_flags: Extra compiler flags for converter_backend.
def VgfCompileSpec.dump_debug_info(self, debug_mode: executorch.backends.arm.common.arm_compile_spec.ArmCompileSpec.DebugMode | None):
Dump debugging information into the intermediates path.
Args:
debug_mode: The debug mode to use for dumping debug information.
def VgfCompileSpec.dump_intermediate_artifacts_to(self, output_path: str | None):
Sets a path for dumping intermediate results during such as tosa and pte.
Args:
output_path: Path to dump intermediate results to.
def VgfCompileSpec.get_intermediate_path(self) -> str | None:
Gets the path used for dumping intermediate results such as tosa and pte.
Returns: Path where intermediate results are saved.
def VgfCompileSpec.get_output_format() -> str:
Returns a constant string that is the output format of the class.
Partitioner API#
See Partitioner API for more information of the Partitioner API.
Quantization#
The VGF quantizer supports Post Training Quantization (PT2E) and Quantization-Aware Training (QAT).
For more information on quantization, see Quantization.
Runtime Integration#
The VGF backend can use the default ExecuTorch runner. The steps required for building and running it are explained in the VGF Backend Tutorial. The example application is recommended to use for testing basic functionality of your lowered models, as well as a starting point for developing runtime integrations for your own targets.
Reference#
→Partitioner API — Partitioner options.
→Quantization — Supported quantization schemes.
→Arm VGF Troubleshooting — Debug common issues.
→Arm VGF Backend Tutorials — Tutorials.
