CLI reference

Model inference

`geniex pull`

Download a model and store it locally.

geniex pull <model-name>[:<precision>]

Flag	Description
`--model-hub`	Model source: `aihub` \| `hf` \| `modelscope` \| `docker` \| `localfs`. Auto-detected when omitted.
`--local-path`	Path to a local directory or AI Hub `.zip` file. Implies `--model-hub localfs`.
`--model-type`	Model type: `llm` \| `vlm`. Auto-detected when omitted.

Pulling from a local path:

geniex pull local/my-model --local-path /path/to/model-dir

pull copies files into the GenieX cache. After a successful pull you can safely delete the source to avoid keeping two copies.

Precision (Quantization) (llama.cpp only) For GGUF models the CLI prompts you to pick a precision:

Choose a precision version to download
> Q4_0       [1.2 GiB] (default)
  Q8_0       [2.0 GiB]
  F16        [3.8 GiB]

Q4_0 has the best Hexagon NPU support. See Precisions (Quantizations) Supported.

Qualcomm AI Hub Models are pre-quantized — no choice needed.

`geniex infer` — LLM

Launch an interactive chat session with a language model.

geniex infer ai-hub-models/Qwen3-4B

Thinking mode — control whether the model shows reasoning before responding:

geniex infer ai-hub-models/Qwen3-4B --think         # show reasoning steps
geniex infer ai-hub-models/Qwen3-4B --think=false   # respond directly

Compute unit selection (via --compute) — pick which compute unit runs the model (default: npu):

# llama.cpp models support all compute units
geniex infer unsloth/Qwen3.5-0.8B-GGUF --compute npu
geniex infer unsloth/Qwen3.5-0.8B-GGUF --compute gpu
geniex infer unsloth/Qwen3.5-0.8B-GGUF --compute cpu

# Qualcomm AI Hub Models only support NPU
geniex infer ai-hub-models/Qwen3-4B --compute npu

# Explicit device list — spread across multiple HTP cores (llama.cpp + multi-DSP devices only)
GGML_HEXAGON_NDEV=4 geniex infer google/gemma-4-26B-A4B-it-qat-q4_0-gguf:Q4_0 --compute HTP0,HTP1,HTP2,HTP3

Set GGML_HEXAGON_NDEV to match the number of device IDs in --compute. qairt models coerce a device list to NPU with a warning. See Spreading across multiple HTP cores for MTP performance numbers.

Qualcomm AI Hub Models run on NPU only. Using --compute cpu or --compute gpu returns an error.

`geniex infer` — VLM

Run vision-language inference with text-only, image, or audio input:

geniex infer google/gemma-4-E2B-it-qat-q4_0-gguf

For text-only, just launch and chat. For image or audio input, provide the absolute path or drag the file into your terminal — image (.jpg, .jpeg, .png, .webp) and audio (.wav, .mp3) paths are auto-detected, and a single prompt can carry both. google/gemma-4-E2B-it-qat-q4_0-gguf ships an audio-capable mmproj (~4.0 GiB total: Q4_0 weights + conformer mmproj, pulled together):

geniex pull google/gemma-4-E2B-it-qat-q4_0-gguf

curl -L -o jfk.wav https://github.com/ggml-org/whisper.cpp/raw/master/samples/jfk.wav
curl -L -o landmark.jpg "https://images.pexels.com/photos/402028/pexels-photo-402028.jpeg?w=1024"

geniex infer google/gemma-4-E2B-it-qat-q4_0-gguf \
  -p "Describe the image and transcribe the audio. Image: /full/path/to/landmark.jpg Audio: /full/path/to/jfk.wav"

Output (verified on --compute npu, Snapdragon X Elite):

**Image Description:**
This is a scenic, panoramic photograph that features a traditional Japanese temple ... The overall mood of the image is serene and beautiful.

**Audio Transcription:**
"And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country."

Audio input runs on the llama.cpp backend only and needs an audio-capable model (mmproj with a conformer encoder). Audio also runs on --compute cpu / gpu, but QAIRT models report audio: false and a QAIRT model given audio fails with GenieXError(-201201): Multimodal generation failed.

In an interactive session with an audio-capable model, /mic records a clip instead of loading one from disk — Ctrl-C stops the recording and transcribes it. This needs SoX on your PATH (sudo apt install sox, or winget install --id=ChrisBagwell.SoX -e on Windows); without it GenieX warns SoX is not installed at startup. See the audio input tutorial for the full walkthrough.

`geniex model`

Inspect and reconfigure cached models.

`geniex model set-type`

Override the model type recorded in a cached model’s manifest. Omit the type to pick it interactively.

geniex model set-type <model-name> [llm|vlm]

The type decides which path geniex infer takes: vlm loads the mmproj projector alongside the weights and accepts image / audio input, while llm loads the language weights only. Forcing a multimodal GGUF to llm is how you run its text path on a board where weights + projector don’t fit in memory.

geniex model set-type google/gemma-4-E2B-it-qat-q4_0-gguf llm

geniex pull --model-type llm <model> writes the same field at download time, so a fresh pull needs no set-type afterwards.

`geniex model list`

List Qualcomm AI Hub Models that have a qairt (NPU) build. Only models compatible with the detected chipset are listed unless you pass --all.

geniex model list
geniex model list --all

`geniex serve`

Start the OpenAI-compatible local server. See Local server for the API.

geniex serve

To move a thinking model’s chain-of-thought out of message.content into message.reasoning_content, set the per-request reasoning_format field — see Separating reasoning.

Logging

--log is a global flag controlling the CLI’s log output. It is equivalent to the GENIEX_LOG environment variable and takes precedence over it when both are set.

geniex --log debug list

Value	Emits
`none`	nothing (CLI default)
`error`	errors only
`warn`	warnings + errors
`info`	info + warnings + errors
`debug`	debug + info + warnings + errors
`trace`	everything

Configuration flags

These flags can be passed to geniex infer to control model loading and generation.

Sampler flags

Control how the model selects tokens during generation.

Flag	Type	Default	Description
`--temperature`	float	—	Sampling temperature. Higher values increase randomness.
`--top-p`	float	—	Top-p (nucleus) sampling threshold.
`--top-k`	int	—	Top-k sampling. Only consider the top-k most likely tokens.
`--min-p`	float	—	Min-p sampling threshold.
`--repetition-penalty`	float	`1`	Penalize repeated tokens. Values > 1 reduce repetition.
`--presence-penalty`	float	—	Penalize tokens that have appeared at all.
`--frequency-penalty`	float	—	Penalize tokens proportional to their frequency.
`--seed`	int	—	Random seed for reproducible outputs.
`--grammar-path`	string	—	Path to a GBNF grammar file for constrained generation.
`--grammar-string`	string	—	Inline grammar in GBNF string format.
`--enable-json`	—	—	Force JSON-only output.

Model flags

Control model loading, context, and generation limits.

Flag	Type	Default	Description
`-n`, `--ngl`	int	`-1`	Number of layers to offload to GPU/NPU, `-1` = all (llama_cpp only).
`--nctx`	int	`4096`	Context window size (max input + output tokens).
`--max-tokens`	int	`2048`	Maximum tokens to generate per response.
`--stop`	string[]	—	Stop sequences (can be specified multiple times).
`--stop-file`	string	—	File containing stop sequences (one per line).
`--think` / `--think=false`	bool	`true`	Enable or disable thinking mode for reasoning models.
`-s`, `--system-prompt`	string	—	System prompt to set model behavior.
`--sliding-window`	—	`false`	(`qairt` only) Evict the oldest context above a small anchored prefix instead of erroring when the context length is exceeded, letting the conversation continue.
`--power-mode`	string	`burst`	HTP power/clock-management mode, shared by `qairt` and `llama_cpp`: `low_power_saver`, `power_saver`, `high_power_saver`, `low_balanced`, `balanced`, `high_performance`, `sustained_high_performance`, `burst`.

Increasing the context length

The context window (--nctx) is how much the model can hold at once. When a conversation grows past it you get a context length exceeded error. How to raise it depends on the runtime:

llama.cpp (GGUF): --nctx <N> raises the window at runtime, up to the model’s trained maximum. A larger window uses more KV-cache memory.
geniex infer unsloth/Qwen3-8B-GGUF --nctx 8192
Qualcomm AI Engine Direct (NPU): the context length is baked into the compiled bundle and cannot be raised at runtime — --nctx has no effect. Instead:
- Add --sliding-window to keep chatting past the limit: the oldest context above a small anchored prefix is evicted instead of erroring.
- For a genuinely larger window, pull a bundle compiled for a longer context from Qualcomm AI Hub. See Qualcomm AI Engine Direct runtime constraints.

`geniex-bench`

geniex-bench is a standalone benchmark binary — separate from the geniex CLI — that measures TTFT, prefill speed, and decode speed directly against the GenieX C API. It ships as a pre-built archive for Windows ARM64 and Linux ARM64. See the Benchmarking tutorial for download instructions and a full walkthrough. Quick reference:

Flag	Default	Description
`--plugin`	—	`llama_cpp` \| `qairt`. Required.
`--device`	`auto`	`cpu` \| `gpu` \| `npu` \| `hybrid`.
`-m` / `--model`	—	Local path (`.gguf` or bundle dir) or model-manager id (`org/repo[:quant]`). Required.
`-p`	`512`	Random-token prefill length.
`-n` / `--n-gen`	`128`	Tokens to generate per run.
`-r`	`5`	Measured repetitions.
`--warmup`	`1`	Warmup runs (not counted in stats).
`--temperature`	`0.0`	Sampling temperature. `0.0` = deterministic (greedy).
`--seed`	`42`	Random seed for prefill token generation.
`--power-mode`	`burst`	HTP power/clock-management mode, shared by `qairt` and `llama_cpp`.
`--output-json`	—	Write per-cell JSON with aggregated stats to this path.
`--matrix-file`	—	TSV file (`cell_id`, `plugin`, `device`, `model`) to run multiple cells in one session.
`--output-json-dir`	—	Output directory for matrix-mode JSON files.
`--accuracy`	—	Print generated text instead of timing (1 run, no warmup). Pair with `--prompt-file`.

Utility commands

Command	Description	Example
`geniex list`	Display all downloaded models with their names and sizes.	`geniex list`
`geniex remove <model>`	Remove a specific local model by name.	`geniex remove unsloth/Qwen3-0.6B-GGUF`
`geniex clean`	Delete all locally cached models.	`geniex clean`
`geniex infer -h`	Show help for `geniex infer`.	`geniex infer -h`
`geniex serve -h`	Show help for `geniex serve`.	`geniex serve -h`

Was this page helpful?

Yes

Get Started

Run

Tutorials

Resources

Model inference

`geniex pull`

`geniex infer` — LLM

`geniex infer` — VLM

`geniex model`

`geniex model set-type`

`geniex model list`

`geniex serve`

Logging

Configuration flags

Sampler flags

Model flags

Increasing the context length

`geniex-bench`

Utility commands

​Model inference

​geniex pull

​geniex infer — LLM

​geniex infer — VLM

​geniex model

​geniex model set-type

​geniex model list

​geniex serve

​Logging

​Configuration flags

​Sampler flags

​Model flags

​Increasing the context length

​geniex-bench

​Utility commands

Model inference

`geniex pull`

`geniex infer` — LLM

`geniex infer` — VLM

`geniex model`

`geniex model set-type`

`geniex model list`

`geniex serve`

Logging

Configuration flags

Sampler flags

Model flags

Increasing the context length

`geniex-bench`

Utility commands