Instructions to use UraionLabs/LFM3-1.7B-MLX-oQ4e with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use UraionLabs/LFM3-1.7B-MLX-oQ4e with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("UraionLabs/LFM3-1.7B-MLX-oQ4e") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use UraionLabs/LFM3-1.7B-MLX-oQ4e with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "UraionLabs/LFM3-1.7B-MLX-oQ4e"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "UraionLabs/LFM3-1.7B-MLX-oQ4e" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use UraionLabs/LFM3-1.7B-MLX-oQ4e with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "UraionLabs/LFM3-1.7B-MLX-oQ4e"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "UraionLabs/LFM3-1.7B-MLX-oQ4e" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8080/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "UraionLabs/LFM3-1.7B-MLX-oQ4e", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use UraionLabs/LFM3-1.7B-MLX-oQ4e with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "UraionLabs/LFM3-1.7B-MLX-oQ4e"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default UraionLabs/LFM3-1.7B-MLX-oQ4e
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use UraionLabs/LFM3-1.7B-MLX-oQ4e with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "UraionLabs/LFM3-1.7B-MLX-oQ4e"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "UraionLabs/LFM3-1.7B-MLX-oQ4e" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Uraion Labs
Foundational systems research.
LFM3-1.7B — oQ4e (MLX)
Liquid AI's LFM3-1.7B, quantized for Apple Silicon with oMLX oQe mixed precision. Smallest and fastest. About 1 GB on disk.
LFM3-1.7B-MLX-oQ4e is an MLX quantization of LiquidAI/LFM3-1.7B, Liquid AI's tri-hybrid text model for local agents, tool calling and structured outputs. It was quantized and published by Uraion Labs with the oMLX oQ4e preset and runs natively on Apple Silicon Macs (M1 through M5) with oMLX and mlx-vlm >= 0.7.11.
- Size: 1.04 GB of weights (~4.8 effective bits per weight, including group scales)
- Speed: ~203 tokens/s decode on an M5 Pro (measured by Uraion Labs, see below)
- Context: 256K tokens, native reasoning (
<think>), XML tool calls, deferred tool search - Languages: 16, including English, Chinese, Arabic, Japanese, Korean, Hindi and the major European languages
Runtime note: LFM3 support in
mlx-lmis still in progress upstream. Use oMLX or mlx-vlm 0.7.11+ (which loads LFM3 as a text-only model) today.
Pick your quant
| Quant | Base bits | Mixed-precision profile | Output head | Size | Decode speed* | Best for | Repository |
|---|---|---|---|---|---|---|---|
| oQ8e | 8-bit | Uniform 8-bit | 8-bit | 1.84 GB | ~130 tok/s | Maximum quality | UraionLabs/LFM3-1.7B-MLX-oQ8e |
| oQ6e | 6-bit | Mixed 6/8-bit (33 layers @ 8-bit) | 6-bit | 1.45 GB | ~152 tok/s | Recommended: near-8-bit quality | UraionLabs/LFM3-1.7B-MLX-oQ6e |
| oQ5e | 5-bit | Mixed 5/6/8-bit (29 @ 6b, 8 @ 8b) | 6-bit | 1.23 GB | ~174 tok/s | Faster, still strong quality | UraionLabs/LFM3-1.7B-MLX-oQ5e |
| oQ4e (this repo) | 4-bit | Mixed 4/5/6/8-bit (37 @ 5b, 2 @ 6b, 4 @ 8b) | 5-bit | 1.04 GB | ~203 tok/s | Smallest and fastest (~1 GB) | UraionLabs/LFM3-1.7B-MLX-oQ4e |
*Single-stream decode speed measured by Uraion Labs on an Apple M5 Pro (64 GB), macOS 27, oMLX 0.7.3, 800 generated tokens, average of 2 runs. Your numbers will vary with chip, context length and settings.
All four variants answered a short math check correctly and produced a valid OpenAI-style get_weather tool call through the oMLX server (oQ4e and oQ8e tool-call tested). This is a smoke test, not a full benchmark run.
About this quantization
| Property | Value |
|---|---|
| Original model | LiquidAI/LFM3-1.7B (Liquid AI) |
| Quantized by | Uraion Labs |
| Quantization preset | oQ4e (oMLX oQ, enhanced mixed precision) |
| Base weight precision | 4-bit |
| Mixed-precision profile | Mixed 4/5/6/8-bit (37 @ 5b, 2 @ 6b, 4 @ 8b) |
| Layer overrides | 43 overrides: 37 boosted to 5-bit (attention q/k/v/o, FFN down-proj, embeddings), 2 to 6-bit, 4 attention gates to 8-bit |
| Embeddings / output head (tied) | 5-bit |
| Group size / mode | 64 / affine (per-group scale and bias) |
| Bit allocation | Chosen per layer by oMLX oQ sensitivity analysis |
| Weight file | 1.04 GB (1,037,627,479 bytes), single model.safetensors |
| Effective bits per weight | ~4.80 |
| Format | MLX safetensors |
| Tested runtimes | oMLX 0.7.3, mlx-vlm 0.7.11 |
Model information
| Property | Value |
|---|---|
| Architecture | Lfm3ForCausalLM (lfm3), tri-hybrid |
| Parameters | 1.73B |
| Layers | 24: 13 gated short convolutions, 5 sliding-window attention (window 256), 6 global attention |
| Attention | GQA, 16 query heads / 4 KV heads, head dim 128 |
| Context length | 262,144 tokens (input + output) |
| Vocabulary | 128,000 tokens |
| Languages | English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian, Polish |
| License | LFM Open License v1.0 |
Quickstart
oMLX (recommended)
oMLX is an OpenAI-compatible server for Apple Silicon with continuous batching and SSD-backed KV cache. Download the model into your oMLX models folder (or use the model downloader in the oMLX app), then serve it:
hf download UraionLabs/LFM3-1.7B-MLX-oQ4e --local-dir ~/.omlx/models/LFM3-1.7B-MLX-oQ4e
omlx serve --model-dir ~/.omlx/models --port 8000
Call it from any OpenAI client, tools included:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="your-omlx-key")
tools = [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
},
}]
response = client.chat.completions.create(
model="LFM3-1.7B-MLX-oQ4e",
messages=[{"role": "user", "content": "What's the weather in Paris right now?"}],
tools=tools,
temperature=0.1,
extra_body={"top_k": 50, "repetition_penalty": 1.1},
)
print(response.choices[0].message.tool_calls)
mlx-vlm (CLI)
pip install -U "mlx-vlm>=0.7.11"
python -m mlx_vlm.generate \
--model UraionLabs/LFM3-1.7B-MLX-oQ4e \
--prompt "Explain how a heat pump works." \
--max-tokens 1024 \
--temperature 0.1 \
--top-k 50 \
--repetition-penalty 1.1
mlx-vlm (Python)
from mlx_vlm import load, generate
model, processor = load("UraionLabs/LFM3-1.7B-MLX-oQ4e")
messages = [{"role": "user", "content": "Explain how a heat pump works."}]
prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
result = generate(model, processor, prompt, max_tokens=1024, temperature=0.1, verbose=True)
print(result.text)
Agent harnesses
LFM3 was post-trained inside agent harnesses such as Hermes Agent, Pi and OpenClaw. Point any OpenAI-compatible harness at the oMLX endpoint above. See Liquid AI's Agent Harnesses guide.
Recommended sampling
From Liquid AI's official model card:
| Use case | Settings |
|---|---|
| General (chat, tool use, agents, coding) | temperature=0.1, top_k=50, repetition_penalty=1.1 |
| Math and STEM | temperature=1.0, top_p=0.95, top_k=20 |
The bundled generation_config.json already uses the general settings.
Chat format, reasoning and tools
- ChatML template (
<|im_start|>/<|im_end|>). Useapply_chat_template(..., add_generation_prompt=True). - Reasoning is on by default. Pass
enable_thinking=Falseto the template to turn it off (upstream notes this can be inconsistent on reasoning-heavy tasks). Earlier-turn reasoning is dropped unlesspreserve_thinking=True. - Tool calls are emitted as XML (
<tool_call><function=name><parameter=...>). oMLX converts them to OpenAItool_callsautomatically. - Deferred tool search: mark tools with
defer_loading=Trueand returntool_referenceparts from your search tool. Details in Liquid AI's tool-use docs.
Upstream benchmarks
Reported by Liquid AI for the original BF16 LFM3-1.7B (vLLM, full context). Uraion Labs has not re-run these on the quantized weights. Lower-bit quants may score somewhat lower.
| Benchmark | LFM3-1.7B | LFM2.5-2.6B | MiniCPM5-2B | Gemma 4 E4B | Qwen3.5-4B | Qwen3.5-9B |
|---|---|---|---|---|---|---|
| LiveClaw | 45.60 | 14.70 | 30.37 | 20.36 | 19.94 | 35.77 |
| Claw-Eval General (EN), average | 68.64 | 62.63 | 60.74 | 56.14 | 61.96 | 66.53 |
| BFCL V4 | 71.39 | 55.89 | 62.65 | 48.25 | 55.12 | 62.83 |
| ToolSandbox | 80.36 | 78.04 | 68.16 | 69.27 | 75.40 | 76.41 |
| τ³-bench (4 domains), average | 68.01 | 56.77 | 69.44 | 33.53 | 65.98 | 67.11 |
| MCP-Atlas | 32.57 | 16.45 | 26.38 | 6.19 | 41.37 | 42.67 |
| SWE-bench Verified (OpenCode) | 43.40 | 18.00 | 34.80 | 0.50 | 41.20 | 51.20 |
| LiveCodeBench v6 | 73.46 | 60.57 | 70.52 | 71.00 | 61.99 | 69.86 |
| IFEval | 96.66 | 95.39 | 87.83 | 90.02 | 92.52 | 92.08 |
| IFBench | 73.03 | 60.48 | 62.08 | 44.56 | 38.16 | 48.84 |
| IFStruct | 93.72 | 82.18 | 72.65 | 78.45 | 87.05 | 89.20 |
| GPQA Diamond | 63.37 | 57.78 | 65.15 | 57.88 | 75.76 | 80.40 |
| AA-Omniscience non-hallucination (public) | 92.93 | 53.69 | 84.44 | 72.09 | 13.64 | 52.59 |
| AA-LCR | 52.83 | 6.00 | 49.34 | 39.33 | 52.67 | 62.00 |
| RULER 128K | 65.56 | 1.15 | 37.96 | 18.82 | 84.63 | 87.11 |
| Average | 49.37 | 31.55 | 39.19 | 31.74 | 39.27 | 47.24 |
Full table, settings and per-benchmark details: LiquidAI/LFM3-1.7B.
Limitations
- Liquid AI positions LFM3-1.7B for local agents, tool calling, structured outputs and fine-tuning. It is not recommended for agentic coding or complex multi-step reasoning because of its size. Pair it with retrieval or search for knowledge-heavy tasks.
- Quantization reduces precision. The 4-bit and 5-bit variants trade some accuracy for size and speed; use oQ6e or oQ8e when quality matters most.
- Like any language model, it can produce inaccurate or biased output. Do not treat answers on legal, medical or financial topics as professional advice. Provided "AS IS", without warranty.
Attribution and modifications
LFM3-1.7B was created by Liquid AI. The architecture, training, benchmarks and capabilities described here come from the upstream release. All credit for the model belongs to Liquid AI.
Changes made by Uraion Labs: the weights in model.safetensors were quantized to MLX format, and config.json was updated with quantization metadata. The tokenizer, chat template, generation config and custom code files are unchanged from upstream. No retraining or fine-tuning was performed.
Upstream resources: model card · LFM3 blog post · docs · LFM3 collection · Discord
License
Distributed under the LFM Open License v1.0 (copy included in LICENSE), the same license as the original model. In short: free for research and non-commercial use, and for commercial use by organizations under US$10M in annual revenue. Commercial use above that threshold is not covered; contact Liquid AI. Read the full license before use.
Citation
@article{liquidAI2026LFM3,
author = {Liquid AI},
title = {Introducing LFM3: Every Modality for Specialized Intelligence},
journal = {Liquid AI Blog},
year = {2026},
note = {www.liquid.ai/blog/lfm3},
}
Quantized and published by Uraion Labs. Foundational systems research.
- Downloads last month
- 335
4-bit