Skip to main content

​
Install / pip

Common on networks with TLS inspection — including Qualcomm Developer Cloud (QDC). Pre-download the SDK and point pip at the local file. See the full PowerShell snippet in Python install.
The .exe is not yet code-signed. Click More info → Run anyway in the SmartScreen dialog.

​
CLI

The installer doesn’t add itself to PATH. Run:
Qualcomm AI Engine Direct is NPU-only. Use --compute npu (or omit the flag — npu is the default for qairt). To run on CPU/GPU, switch to a GGUF model on the llama.cpp runtime.
The conversation outgrew the context window (--nctx, default 4096).
  • llama.cpp (GGUF): raise it at runtime, e.g. geniex infer <model> --nctx 8192, up to the model’s trained maximum. A larger window uses more memory.
  • Qualcomm AI Engine Direct (NPU): the window is fixed in the compiled bundle and --nctx has no effect. Add --sliding-window to keep chatting (evicts the oldest context), or pull a bundle built for a longer context.
See Increasing the context length.

​
Server

The server doesn’t auto-download. Pull the model first:
Then restart geniex serve.
The --privileged flag is required for NPU access. Make sure your docker run includes it, plus the volume mounts for /usr/lib. See CLI install (Docker).

​
Linux

Two separate things have to be right:
  1. Registry login. Docker Hub (docker.io/qualcomm/geniex) is public and needs no login. For the Qualcomm Container Registry, log in first:
    bash
    You should see Login Succeeded.
  2. Docker group membership. If docker pull returns permission denied while trying to connect to the docker API at unix:///var/run/docker.sock, your user isn’t in the docker group:
    bash
    Then retry the pull.
The container can’t reach the NPU. Make sure your docker run includes --privileged and the /usr/lib mount, and that the host’s Qualcomm driver packages (qcom-adreno1, qcom-fastrpc1) are installed — see Linux install → Install host dependencies.
The llama.cpp Hexagon backend dlopens the unversioned libcdsprpc.so, but qcom-fastrpc1 only ships libcdsprpc.so.1. On bare metal install.sh creates the symlink; if you deployed the release tarball directly (no install.sh), create it yourself:
bash
If the Hexagon driver can’t load (see the entry above), hybrid still produces correct output — it just runs entirely on CPU, so the failure is silent. Confirm the NPU is actually engaged by running with GGML_HEX_VERBOSE=1; you should see Hexagon Arch version vNN and a libggml-htp-vNN.so session. No such lines means it fell back to CPU.
The aarch64 Linux build is compiled for armv8.2-a with the fp16, dotprod, lse (atomics), and rdm extensions. Baseline armv8.0 boards — some Dragonwing IoT SoCs with no NPU — don’t implement these, so geniex stops at startup with a clear error rather than crashing with a raw SIGILL: illegal instruction partway through a run.This check is global: every backend needs those instructions, so switching --compute to cpu, gpu, or npu won’t help on such a device. Check what your CPU reports:
bash
If those features are absent, install the CPU-only build instead. It is compiled for plain armv8.0-a with no ISA extensions, ships CPU inference only (no QAIRT/NPU, no OpenCL/GPU, no Hexagon), and is correspondingly slower — ggml falls back to its fp32 quant kernels without dotprod/fp16. Every channel asks for it explicitly; nothing is auto-detected:
bash
geniex-qairt has no CPU-only build at all: QAIRT needs an NPU, and these boards have none. Use geniex-llama-cpp (or the geniex meta package, which drops the QAIRT backend automatically on such a board).If the CPU-only build still fails, report it in GitHub Issues or Slack with the output above.
As a VLM the model needs ~4.94 GB — 3.04 GB of Q4_0 weights plus a 1.9 GB F32 mmproj projector — against the ~4.7 GB this board has available, so the kernel kills the process while the projector loads:
Record the model as an llm to run its text path only — that skips the projector and fits. Image and audio input are unavailable in this mode.
bash
See geniex model set-type.

​
Android

The demo (or your code) is passing the raw user text into generateStreamFlow instead of the chat-templated prompt. Qualcomm AI Engine Direct pipelines treat their input as already-templated — pass applyChatTemplate().formattedText, not the raw user message.
Android needs an explicit chipset for Qualcomm AI Hub pulls — auto-detect only runs on Windows on Snapdragon. Set ModelPullInput.chipset to "SM8750" (Snapdragon 8 Elite) or "SM8850" (Snapdragon 8 Elite Gen 5). See Android API reference → ModelPullInput.
The model id returned by Qualcomm AI Hub must match an entry in the Qualcomm AI Engine Direct runtime’s registry (qwen3_4b_instruct_2507, qwen2_5_vl_7b_instruct, etc.). To add a new Qualcomm AI Hub Model, register it on the C++ side first — see third-party/geniex-qairt/models/{llm,vlm}_model_registry.h.
Qualcomm AI Hub Models are compiled with fixed KV cache and context length. Leave both nGpuLayers and nCtx at their defaults; tune max_tokens and enable_thinking instead.
The same startup guard as on Linux. The default AAR is compiled for armv8.7-a with the fp16, dotprod, and i8mm extensions, but minSdk 27 still admits phones that predate them, so geniex_init returns NOT_SUPPORTED instead of crashing with a raw SIGILL partway through a run.Ship the CPU-only AAR on those devices — geniex-android-aar-cpu-<tag>.aar from the release assets. It is compiled for plain armv8.0-a with no ISA extensions, ships CPU inference only (no Qualcomm AI Engine Direct, no OpenCL, no Hexagon), and is correspondingly slower. Otherwise it is a drop-in replacement: same com.geniex.sdk namespace and API. Only the default AAR is published to Maven Central.

​
Still stuck?

GitHub Issues

File a bug, request a feature, or browse open issues.

Slack

Developer collaboration.