NXP eIQ Neutron Kernel Selective Kernel Registration

The NXP ExecuTorch backend supports selective Neutron kernel registration for Neutron-C targets, which reduces the size of the Neutron Firmware. During the backend’s conversion to the Neutron representation by the Neutron Converter, microcode for the Neutron accelerator is generated. The microcode consists of kernel calls executed by the Neutron Driver. The code for kernel call functions is distributed in the Neutron Firmware.

The eiq_neutron_sdk.neutron_converter optionally generates a *_kernel_selection.c file, registering only kernels that are required for a particular model or, in the case of ExecuTorch, a delegated subgraph. This *_kernel_selection.c, when used during application linking, takes precedence over the default list of registered kernels in the Neutron Firmware, and allows the linker to include only the necessary Neutron kernels. The Neutron Firmware is required for deployment on an edge device (e.g. i.MX RT700) and is distributed via the MCUXpresso SDK. The MCUXpresso SDK enables the building of a final application that is then flashed on the edge device. For more details about this process, see eIQ ExecuTorch Library User Guide.

By default, for Neutron-C targets like i.MX RT700, all kernel implementations are present in the Neutron Firmware, which is linked to the final application. This enables an easy build process for any model, but increases the size of the final application with unused code. In memory-constrained environments, you can link only the kernels required by the deployed models. This way you can reduce the size of the final application by linking only selected kernels, used in one or more models.

The feature works as follows: The Neutron Converter with the appropriate flag exports a kernel selection file for each converted subgraph, the kernel selection files are then merged and ready to be included in the MCUXpresso SDK to use for a selection-only build.

Note: This feature applies only to Neutron-C targets. Neutron-S has a different implementation and links only kernel calls included in the model’s microcode by default.

Export kernel selection file

To enable this feature in the NXP ExecuTorch backend, use the parameter --dump_kernel_selection_code in aot_neutron_compile.py. An example with the CifarNet model:

python -m examples.nxp.aot_neutron_compile --quantize \
    --delegate -m cifar10 \
    --dump_kernel_selection_code

This command will create a *_kernel_selection.c file alongside the converted PTE file in the working directory.

Kernel Registration for Multiple Models

If you want to use or experiment with multiple models in one application while having a reduced kernel set, you can create one kernel selection file with the script merge_kernel_selection_code.py:

python -m eiq_neutron_sdk.neutron_library_utils.merge_kernel_selection_code \
    -input-files model1_kernel_selection.c model2_kernel_selection.c [modelX_kernel_selection.c ...] \ 
    -output-file merged_kernel_selection.c

Each particular model must be converted by the same Neutron converter version, so the *_kernel_selection.c files share the same version.

MCUXpresso SDK build with kernel selection

Copy your model PTE file and kernel selection file into the MCUXpresso SDK and follow its documentation.