Qualcomm QNN deployment and ABI troubleshooting

Using QNN HTP through ONNX Runtime on a Qualcomm Linux device requires ORT, a QNN EP, QAIRT userspace libraries and the device's FastRPC environment. Having all the library files is the first step. Also verify the versions actually loaded, plugin registration and communication between ARM and DSP.

This article follows the deployment chain, focusing on Linux ARM64 with glibc and a standalone QNN Plugin EP. /opt/qnn-runtime is an example directory; replace it with the actual extracted location. Android uses different system libraries and toolchains, so these Linux binaries cannot simply be copied there.

1. Confirm the system and runtime combination

Start by recording system information on the target:

uname -m
cat /etc/os-release
getconf GNU_LIBC_VERSION

aarch64 describes the CPU instruction set only. Linux/glibc, Android/Bionic and Linux/musl have different userspaces. Check system type, toolchain directory and BSP when choosing an SDK. An x86_64 build tool on the development host is not an ARM64 runtime library.

List application-side files separately from device-system dependencies:

ORT has two distinct integration paths here. The built-in legacy QNN EP comes with an ORT build that includes QNN support. A standalone Plugin EP is registered at runtime. Establish which distribution is in use before choosing initialization APIs; a dynamic-library filename alone does not establish the integration type.

One combination verified with an independent model in the archive was ORT Core 1.24.4, QNN Plugin EP 2.2.0 and QAIRT 2.46.0. This records a tested combination. It does not mean that independently upgrading all three to their newest releases will work. Preserve full versions, package sources and target-device details in the deployment manifest.

2. Check ELF architecture and symbol requirements

Inspect files on Linux or in a build environment with suitable ELF tools:

runtime_dir=/opt/qnn-runtime
file "$runtime_dir/lib/libonnxruntime.so"
file "$runtime_dir/lib/libonnxruntime_providers_qnn.so"
file "$runtime_dir/lib/libQnnHtp.so"

readelf -h "$runtime_dir/lib/libonnxruntime_providers_qnn.so"
readelf -d "$runtime_dir/lib/libonnxruntime_providers_qnn.so"
readelf -V "$runtime_dir/lib/libonnxruntime_providers_qnn.so"

-h checks ELF class and machine architecture. -d shows dynamic-linking information including NEEDED, RPATH and RUNPATH. -V shows symbol versions. For the application executable, readelf -l ./qnn_probe can also check the interpreter path. Shared libraries need not contain an interpreter segment; its absence does not establish corruption.

The following errors go beyond a missing file:

version `GLIBC_2.xx' not found
version `GLIBCXX_3.4.xx' not found
undefined symbol: ...

GLIBC_* identifies versioned glibc symbols, while GLIBCXX_* usually belongs to libstdc++. Read the file's version needs and compare them with the provider libraries actually loaded on the target. Comparing GCC versions or taking the largest version number from strings cannot establish compatibility of the entire dependency chain. Indirect application, plugin and backend dependencies may impose higher requirements.

When compiling your own application, prefer the device's matching sysroot or a build environment with the target userspace. The earlier llama.cpp deployment on Qualcomm used a toolchain fixed to glibc 2.35 and checked output symbol requirements during the build. The same approach helps avoid discovering application-side QNN ABI problems only on the device. Precompiled QAIRT libraries still need separate checks; rebuilding the application cannot lower their system requirements.

For unsatisfied symbol versions, choose a matching distribution or rebuild components you control. Replacing the device's system libc affects the whole device and does not resolve Plugin API, DSP-architecture or context-format mismatches.

3. Configure ARM and DSP loading paths

HTP execution passes through both runtime environments:

flowchart TB
  A["应用与 ORT Core"] --> B["QNN Plugin EP"]
  B --> C["HTP backend 与 ARM Stub"]
  C --> D["FastRPC 用户态与驱动"]
  D --> E["DSP 侧 HTP Skel"]
  E --> F["HTP 执行"]

LD_LIBRARY_PATH affects ARM-side dynamic-library search; ADSP_LIBRARY_PATH is for DSP-side search. This example shows directory organization. Replace hexagon-vXX and system DSP paths according to the actual SDK and device:

runtime_dir=/opt/qnn-runtime
export LD_LIBRARY_PATH="$runtime_dir/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
export ADSP_LIBRARY_PATH="$runtime_dir/dsp/hexagon-vXX/unsigned;/usr/lib/rfsa/adsp;/dsp"

The variables use different separators. The DSP example uses semicolons and quotes around the entire value. Check the current SDK/BSP's path requirements. Do not mix DSP Skel directories into ARM library search paths merely to make everything discoverable.

Check host-side dependencies in that same target environment:

ldd "$runtime_dir/lib/libonnxruntime_providers_qnn.so"
ldd "$runtime_dir/lib/libQnnHtp.so"

not found means a dependency cannot currently be resolved. If resolution finds another same-named library in a system directory, verify its version too. ldd covers dynamic dependency resolution only. Backends and Stubs loaded through dlopen, and DSP-side Skels, still require actual initialization checks.

For a running Linux process, inspect mapped files:

process_id=1234  # 替换为目标进程 PID
rg 'libonnxruntime|libQnn|libcdsprpc|libstdc\+\+' "/proc/$process_id/maps"

This can expose mixing of Python-environment, system and application libraries. If initialization exits too early, enable glibc loader diagnostics on a minimal probe:

LD_DEBUG=libs,versions ./qnn_probe 2>loader.log

qnn_probe means a small ORT test program you write; it is not a fixed SDK command. Search paths and loaded filenames in the log are more informative than an installation-directory listing. RPATH/RUNPATH and environment variables affect resolution. Setting LD_LIBRARY_PATH alone does not establish that the intended version was loaded.

Pass the environment explicitly to systemd services and containers. Variables set in an interactive terminal do not automatically enter service configuration. A container with complete userspace libraries still depends on host device nodes, permissions, drivers and firmware.

4. Register the Plugin EP and select a device

Initialize a standalone Plugin EP in this order:

创建 Ort::Env
    → RegisterExecutionProviderLibrary(name, plugin_path)
    → GetEpDevices()
    → 选择实际枚举到的 QNN EP 设备
    → SessionOptionsAppendExecutionProvider_V2(devices, options)
    → 创建 Session 并执行模型
释放 Session
    → 注销插件库

This describes call order. Use the installed ORT headers for exact arguments and device-name accessors. The API used at compilation must be compatible with the deployed Core. A C++ example from another documentation version cannot be assumed to fit an older adapter.

The application chooses the library registration name; enumeration supplies the device's EP name. They are separate concepts. Official QNN examples consistently use QNNExecutionProvider, which aids diagnosis, but that does not mean every plugin mandates a fixed registration name. Device-selection logs should show actual EP names, vendors and device information, rather than adapter-specific abbreviations such as QNN or qcom.

Keep at least the following records:

In the archived investigation, the independent probe could create a session and run a model with the 1.24.4 / 2.2.0 / 2.46.0 combination, but the application returned:

QNN execution provider is not supported in this build

After failing to select a QNN device, the application fell back to legacy AppendExecutionProvider("QNN", ...). That Core did not contain the legacy QNN EP, so the final exception hid the preceding device-discovery problem. The fix removed that fallback, preserved registration and enumeration results at the selection failure, and initialized through the Plugin EP V2 path.

This error did not establish unsupported model operators, so re-exporting or quantizing the model was not the first action. If an application supports both ORT distribution types, configure the integration explicitly and initialize each separately. A failed Plugin EP initialization is not a basis for switching to a legacy API.

The lifetimes of Ort::Env and the registered plugin must cover the relevant sessions. Unregister the library only after releasing all sessions that use it.

5. Verify device communication and a minimal model

If the plugin loads but HTP initialization fails, check device communication first. Where the device supplies the corresponding QAIRT tool, run it in the same terminal with library paths configured:

qnn-platform-validator --backend dsp

Consult the packaged help to confirm that option is available. Check FastRPC device nodes, CDSP services or listeners, DSP firmware and permissions according to the board's BSP. Node and service names differ across Linux BSPs. Installation commands from another board are not automatically applicable.

A passing transport probe establishes that communication test only. Next, exercise the full path with a minimal independent ORT model: a known-supported static graph and fixed inputs, minimizing application and model complexity. Disable CPU EP fallback when testing the intended HTP subgraph:

session_options.AddConfigEntry("session.disable_cpu_ep_fallback", "1");

Inspect node assignments in logs or profiles, then validate output shapes, numerical values and repeated execution. QNN appearing in a device list, or successful session creation, does not independently prove that the intended computation completed correctly on HTP.

For an explicitly split HTP encoder and CPU decoder, validate both sessions separately. Disabling fallback checks the part intended for HTP; it should not be used to reject a CPU segment retained by design. See DEIM numerical errors and graph partitioning on Qualcomm HTP.

These are investigation priorities rather than one-to-one error causes. Some lower-level dependencies are loaded only at a later stage, so retain the original error chain.

6. Distinguish model and context formats

Plain ONNX, an explicitly quantized QDQ graph and an EPContext model carry different deployment requirements:

The archived newer runtime combination ran a floating-point graph on HTP. Whether QDQ is needed depends on the actual SoC, QAIRT and EP support. Do not carry forward the older blanket assumption that HTP can run only quantized graphs.

Generate a context using the method supported by the selected QNN EP version, then reload it in a new process and compare results with the same inputs. This checks for missing external files, path dependencies and runtime mismatches. Creating the first session and writing a context completes generation only.

External mode requires delivering the wrapper ONNX together with its referenced binary, preserving that reference. Embed mode puts the context inside ONNX and reduces external files, but still needs the QNN EP, backend and device environment. A model containing a QNN EPContext cannot be handed to a CPU-only EP as though it were the original ONNX graph.

Record the generating SoC, HTP architecture, QAIRT version and compile options. Determine reuse from official compatibility guidance and reload tests on the target. A message such as Using newer context binary on old SDK calls for checking generator/runtime compatibility, then choosing a matching runtime or regenerating the artifact. Renaming a file does not change its context format.

7. Package deployment files and verification records

A deployment directory can use the following layout. Actual required libraries depend on the distribution:

qnn-runtime/
├── bin/                 # 应用与独立探针
├── lib/                 # ARM64 ORT、Plugin、QAIRT 用户态库
├── dsp/                 # 配套 HTP Skel，保留架构信息
├── models/              # ONNX、context 及必要的外部文件
└── manifest.json        # 版本、目标环境、文件哈希与验证结果

The manifest should record ORT, Plugin and QAIRT versions; system and glibc information; SoC/HTP architecture; BSP/firmware versions; and SHA-256 hashes for models, contexts and key runtime libraries. Include only components whose licenses permit redistribution. Document BSP drivers, firmware and system services separately as device prerequisites.

Finally, rerun the independent probe and production model in the actual service-startup environment. After upgrading a layer, retain reference inputs and check loading, registration, communication, graph execution and context reload in sequence. Replacing the model, SDK and BSP together and retaining only the final error makes diagnosis harder. Saving each stage's logs and output baseline lets the verified environment be reused for the next model.

8. References

• QNN Plugin EP configuration, model support and usage
• ONNX Runtime Plugin EP registration and device selection
• EPContext design and file organization
• GNU readelf: ELF and symbol-version inspection
• Linux dynamic linker: search paths and diagnostics