CLI reference

Model inference

`geniex pull`

Download a model and store it locally.

geniex pull <model-name>[:<precision>]

Flag	Description
`--model-hub`	Model source: `aihub` \| `hf` \| `modelscope` \| `docker` \| `localfs`. Auto-detected when omitted.
`--local-path`	Path to a local directory or AI Hub `.zip` file. Implies `--model-hub localfs`.
`--model-type`	Model type: `llm` \| `vlm`. Auto-detected when omitted.

Pulling from a local path:

geniex pull local/my-model --local-path /path/to/model-dir

pull copies files into the GenieX cache. After a successful pull you can safely delete the source to avoid keeping two copies.

Precision (Quantization) (llama.cpp only) For GGUF models the CLI prompts you to pick a precision:

Choose a precision version to download
> Q4_0       [1.2 GiB]
  Q8_0       [2.0 GiB]
  F16        [3.8 GiB]

Q4_0 has the best Hexagon NPU support. See Precisions (Quantizations) Supported.

Qualcomm AI Hub Models are pre-quantized — no choice needed.

`geniex infer` — LLM

Launch an interactive chat session with a language model.

geniex infer ai-hub-models/Qwen3-4B

Thinking mode — control whether the model shows reasoning before responding:

geniex infer ai-hub-models/Qwen3-4B --think         # show reasoning steps
geniex infer ai-hub-models/Qwen3-4B --think=false   # respond directly

Compute unit selection (via --compute) — pick which compute unit runs the model (default: npu):

# llama.cpp models support all compute units
geniex infer unsloth/Qwen3.5-0.8B-GGUF --compute npu
geniex infer unsloth/Qwen3.5-0.8B-GGUF --compute gpu
geniex infer unsloth/Qwen3.5-0.8B-GGUF --compute cpu

# Qualcomm AI Hub Models only support NPU
geniex infer ai-hub-models/Qwen3-4B --compute npu

# Explicit device list — spread across multiple HTP cores (llama.cpp + multi-DSP devices only)
GGML_HEXAGON_DEVICES=4 geniex infer google/gemma-4-26B-A4B-it-qat-q4_0-gguf:Q4_0 --compute HTP0,HTP1,HTP2,HTP3

Set GGML_HEXAGON_DEVICES to match the number of device IDs in --compute. qairt models coerce a device list to NPU with a warning. See Spreading across multiple HTP cores for MTP performance numbers.

For VLMs on --compute npu, the vision encoder shares the language model’s HTP session by default, which can abort on some devices. Run it on a second session with GGML_HEXAGON_DEVICES=2 geniex infer <vlm> --compute HTP0 --vit-compute HTP1. See Troubleshooting.

Qualcomm AI Hub Models run on NPU only. --compute cpu or --compute gpu is ignored with a warning and the model runs on NPU.

`geniex infer` — VLM

Run vision-language inference with text-only, image, or audio input:

geniex infer google/gemma-4-E2B-it-qat-q4_0-gguf

For text-only, just launch and chat. For image or audio input, provide the absolute path or drag the file into your terminal — image (.jpg, .jpeg, .png, .webp) and audio (.wav, .mp3) paths are auto-detected, and a single prompt can carry both. google/gemma-4-E2B-it-qat-q4_0-gguf ships an audio-capable mmproj (~4.0 GiB total: Q4_0 weights + conformer mmproj, pulled together):

geniex pull google/gemma-4-E2B-it-qat-q4_0-gguf

curl -L -o jfk.wav https://github.com/ggml-org/whisper.cpp/raw/master/samples/jfk.wav
curl -L -o landmark.jpg "https://images.pexels.com/photos/402028/pexels-photo-402028.jpeg?w=1024"

geniex infer google/gemma-4-E2B-it-qat-q4_0-gguf \
  -p "Describe the image and transcribe the audio. Image: /full/path/to/landmark.jpg Audio: /full/path/to/jfk.wav"

Output (verified on --compute npu, Snapdragon X Elite):

**Image Description:**
This is a scenic, panoramic photograph that features a traditional Japanese temple ... The overall mood of the image is serene and beautiful.

**Audio Transcription:**
"And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country."

Audio input runs on the llama.cpp backend only and needs an audio-capable model (mmproj with a conformer encoder). Audio also runs on --compute cpu / gpu, but QAIRT models report audio: false and a QAIRT model given audio fails with GenieXError(-201201): Multimodal generation failed.

In an interactive session with an audio-capable model, /mic records a clip instead of loading one from disk — Ctrl-C stops the recording and transcribes it. This needs SoX on your PATH (sudo apt install sox, or winget install --id=ChrisBagwell.SoX -e on Windows); without it GenieX warns SoX is not installed at startup. See the audio input tutorial for the full walkthrough.

`geniex decide`

Make a local structured decision with a llama_cpp LLM or VLM. Select predicate, choice, or score with --kind. Models with a systemone template use the matching OpenJEV, LEV, or Nimble decision path and model-specific temperature; if decision.type is missing, they are treated as OpenJEV. Models without systemone fall back to a generic choice prompt using their regular chat template and temperature 1.0. LEV choices average both option orders and LEV predicates use a nine-rating readout; all paths use logits and return probabilities and confidence. Add --image or --audio (repeatable) to send local media to a compatible VLM.

geniex decide <model-name>[:<precision>] --kind predicate \
  --input "The battery will not charge" --question "Is this a hardware issue?"
geniex decide <model-name>[:<precision>] --kind choice \
  --input "The battery will not charge" --question "Choose the next route" \
  --choice 'retry=Try again' --choice 'replace=Replace the battery'
geniex decide <model-name>[:<precision>] --kind score \
  --input "The battery temperature is 85°C" --question "Rate the severity" \
  --level low --level medium --level "high=Immediate shutdown required"
# Visual choice (the model must be classified as a VLM)
geniex decide <vlm-name> --kind choice --image photo.png \
  --input "A support photo" --question "Choose the issue" \
  --choice battery --choice screen

Repeat --choice VALUE for each candidate. In --choice 'retry=Try again', retry is the value returned in JSON and Try again is optional context for the model; it is not a key-value map. Quote the argument if the description has spaces. --level similarly accepts LABEL=DESCRIPTION. --image and --audio accept existing local regular files and may each be repeated up to 8 times. --compute, --ngl, --nctx, and --power-mode control local model loading, as with geniex infer. Both LLM and VLM decisions use logits, with model-specific temperature and candidate encoding. --vit-compute selects the VLM vision encoder’s compute unit. geniex decide --help groups these flags under Model Loading, LLM/VLM Decision, and VLM Specific. By default, geniex decide uses the CLI theme to display both predicate probabilities (true and false), the selected choice and every candidate probability, or the zero-based expected score and every level probability, plus confidence when available. Pass --json to print the original full single-line JSON result for scripts. The /v1/decisions response remains JSON.

`geniex model`

Inspect and reconfigure cached models.

`geniex model set-type`

Override the model type recorded in a cached model’s manifest. Omit the type to pick it interactively.

geniex model set-type <model-name> [llm|vlm]

The type decides which path geniex infer takes: vlm loads the mmproj projector alongside the weights and accepts image / audio input, while llm loads the language weights only. Forcing a multimodal GGUF to llm is how you run its text path on a board where weights + projector don’t fit in memory.

geniex model set-type google/gemma-4-E2B-it-qat-q4_0-gguf llm

geniex pull --model-type llm <model> writes the same field at download time, so a fresh pull needs no set-type afterwards.

`geniex model list`

List Qualcomm AI Hub Models that have a qairt (NPU) build. Only models compatible with the detected chipset are listed unless you pass --all.

geniex model list
geniex model list --all

`geniex serve`

Start the OpenAI-compatible local server. See Local server for the API.

geniex serve

Thinking models put chain-of-thought in message.reasoning_content by default. Set the per-request reasoning_format field to none to keep it inline in message.content — see Separating reasoning. Chat requests think by default. Start the server with --think=false (or GENIEX_THINK=false) to turn thinking off for requests that omit enable_think; the request field always takes precedence.

Logging

--log is a global flag controlling the CLI’s log output. It is equivalent to the GENIEX_LOG environment variable and takes precedence over it when both are set.

geniex --log debug list

Value	Emits
`none`	nothing (CLI default)
`error`	errors only
`warn`	warnings + errors
`info`	info + warnings + errors
`debug`	debug + info + warnings + errors
`trace`	everything

Configuration flags

These flags can be passed to geniex infer to control model loading and generation.

Sampler flags

Control how the model selects tokens during generation.

Flag	Type	Default	Description
`--temperature`	float	—	Sampling temperature. Higher values increase randomness.
`--top-p`	float	—	Top-p (nucleus) sampling threshold.
`--top-k`	int	—	Top-k sampling. Only consider the top-k most likely tokens.
`--min-p`	float	—	Min-p sampling threshold.
`--repetition-penalty`	float	`1`	Penalize repeated tokens. Values > 1 reduce repetition.
`--presence-penalty`	float	—	Penalize tokens that have appeared at all.
`--frequency-penalty`	float	—	Penalize tokens proportional to their frequency.
`--seed`	int	—	Random seed for reproducible outputs.
`--grammar-path`	string	—	Path to a GBNF grammar file for constrained generation.
`--grammar-string`	string	—	Inline grammar in GBNF string format.

Model flags

Control model loading, context, and generation limits.

Flag	Type	Default	Description
`-n`, `--ngl`	int	`-1`	Number of layers to offload to GPU/NPU, `-1` = all (llama_cpp only).
`--nctx`	int	`4096`	Context window size (max input + output tokens).
`--max-tokens`	int	`2048`	Maximum tokens to generate per response.
`--stop`	string[]	—	Stop sequences (can be specified multiple times).
`--stop-file`	string	—	File containing stop sequences (one per line).
`--think` / `--think=false`	bool	`true`	Enable or disable thinking mode for reasoning models.
`-s`, `--system-prompt`	string	—	System prompt to set model behavior.
`--sliding-window`	—	`false`	(`qairt` only) Evict the oldest context above a small anchored prefix instead of erroring when the context length is exceeded, letting the conversation continue.
`--power-mode`	string	`burst`	HTP power/clock-management mode, shared by `qairt` and `llama_cpp`: `low_power_saver`, `power_saver`, `high_power_saver`, `low_balanced`, `balanced`, `high_performance`, `sustained_high_performance`, `burst`.

Increasing the context length

The context window (--nctx) is how much the model can hold at once. When a conversation grows past it you get a context length exceeded error. How to raise it depends on the runtime:

llama.cpp (GGUF): --nctx <N> raises the window at runtime, up to the model’s trained maximum. A larger window uses more KV-cache memory.
geniex infer unsloth/Qwen3-8B-GGUF --nctx 8192
Qualcomm AI Engine Direct (NPU): the context length is baked into the compiled bundle and cannot be raised at runtime — --nctx has no effect. Instead:
- Add --sliding-window to keep chatting past the limit: the oldest context above a small anchored prefix is evicted instead of erroring.
- For a genuinely larger window, pull a bundle compiled for a longer context from Qualcomm AI Hub. See Qualcomm AI Engine Direct runtime constraints.

`geniex-bench`

geniex-bench is a standalone benchmark binary — separate from the geniex CLI — that measures TTFT, prefill speed, and decode speed directly against the GenieX C API. It ships as a pre-built archive for Windows ARM64 and Linux ARM64. See the Benchmarking tutorial for download instructions and a full walkthrough. Quick reference:

Flag	Default	Description
`--plugin`	—	`llama_cpp` \| `qairt`. Required for a filesystem path; optional for a model-manager id.
`--device`	`auto`	`cpu` \| `gpu` \| `npu` \| `hybrid`.
`-m` / `--model`	—	Local path (`.gguf` or bundle dir) or model-manager id (`org/repo[:quant]`). Required.
`-p`	`512`	Random-token prefill length.
`-n` / `--n-gen`	`128`	Tokens to generate per run.
`-r`	`5`	Measured repetitions.
`--warmup`	`1`	Warmup runs (not counted in stats).
`--temperature`	`0.0`	Sampling temperature. `0.0` = deterministic (greedy).
`--seed`	`42`	Random seed for prefill token generation.
`--power-mode`	`burst`	HTP power/clock-management mode, shared by `qairt` and `llama_cpp`.
`--output-json`	—	Write per-cell JSON with aggregated stats to this path.
`--matrix-file`	—	TSV file (`cell_id`, `plugin`, `device`, `model`) to run multiple cells in one session.
`--output-json-dir`	—	Output directory for matrix-mode JSON files.
`--accuracy`	—	Print generated text instead of timing (1 run, no warmup). Pair with `--prompt-file`.

Utility commands

Command	Description	Example
`geniex list`	Display all downloaded models with their names and sizes.	`geniex list`
`geniex remove <model>`	Remove a specific local model by name.	`geniex remove unsloth/Qwen3-0.6B-GGUF`
`geniex clean`	Delete all locally cached models.	`geniex clean`
`geniex infer -h`	Show help for `geniex infer`.	`geniex infer -h`
`geniex serve -h`	Show help for `geniex serve`.	`geniex serve -h`

Was this page helpful?

Yes

Get Started

Run

Tutorials

Resources

Model inference

`geniex pull`

`geniex infer` — LLM

`geniex infer` — VLM

`geniex decide`

`geniex model`

`geniex model set-type`

`geniex model list`

`geniex serve`

Logging

Configuration flags

Sampler flags

Model flags

Increasing the context length

`geniex-bench`

Utility commands

​Model inference

​geniex pull

​geniex infer — LLM

​geniex infer — VLM

​geniex decide

​geniex model

​geniex model set-type

​geniex model list

​geniex serve

​Logging

​Configuration flags

​Sampler flags

​Model flags

​Increasing the context length

​geniex-bench

​Utility commands

Model inference

`geniex pull`

`geniex infer` — LLM

`geniex infer` — VLM

`geniex decide`

`geniex model`

`geniex model set-type`

`geniex model list`

`geniex serve`

Logging

Configuration flags

Sampler flags

Model flags

Increasing the context length

`geniex-bench`

Utility commands