okayuji/KeyVoice-Refiner-2B-v0.1

KeyVoice Refiner 2B (v0.1)

A text-refinement model for cleaning up speech-to-text output: punctuation, polite-form conversion, translation, summarization, and filler-word removal. This repository contains a complete Q4_K_M GGUF text model, ready for a compatible runtime.

For: developers adding local text cleanup, or Mac users who want to try it in KeyVoice. Not for: general chat, code generation, mathematical reasoning, or long-form creative writing. Platform: GGUF runtimes with Qwen3.5 support; the original performance report used an Apple Silicon M4 Mac.

Quick start

Use KeyVoice for Mac for the app experience. For your own integration, download the file with the Hugging Face CLI:

hf download okayuji/KeyVoice-Refiner-2B-v0.1 refiner-2b-v0.1-Q4_K_M.gguf --local-dir ./models

Run it with llama.cpp using its CLI instructions. Select the downloaded GGUF and let the runtime apply its embedded chat template. The runtime and app provide text generation; this repository distributes the model file.

Intended use

• Punctuation and line-break formatting on transcribed speech.
• Polite-form / honorific conversion, including Japanese desu/masu tone.
• Multilingual translation; the original internal evaluation reported coverage of 46 languages (see verification below).
• Summarization and compression of verbose text.
• Filler-word removal, including English “um” / “uh” and Japanese “eeto” / “ano”.
• Rewriting with a custom instruction.

Model details and files

Item	Value
Base model	Qwen/Qwen3.5-2B
Teacher	Qwen/Qwen3.5-9B, sequence-level distillation
Training	LoRA, rank 32, trained on the teacher's refined outputs
Architecture	Qwen3.5 dense model; qwen3_5 in the original card, qwen35 in the distributed GGUF
Parameter count	2.27B, as reported in the original card
Model file	refiner-2b-v0.1-Q4_K_M.gguf, Q4_K_M
Exact file size	1,384,619,488 bytes; the original card rounded this to 1.3 GB
License	Apache 2.0

The exact file size comes from the Hugging Face file metadata at revision bd3a9f9, checked on 2026-10-23.

Reported performance and verification

The following values are retained from the original model card. They describe an internal evaluation, not a newly reproduced benchmark.

Reported result	Conditions recorded in the original card	Missing information
About 104 tokens/s	Apple Silicon M4, llama.cpp, Q4_K_M	OS, llama.cpp revision, prompt/output lengths, sampling, concurrency, and measurement date
100% pass rate across 46 languages	Internal Phase C data-augmentation evaluation set	Public test set, sample count, pass criteria, and per-language results
About 2 GB memory	Q4_K_M	Measurement method, context length, and peak versus steady-state memory

These figures should not be used as cross-machine comparisons or as a guarantee for unseen text. No new inference, performance measurement, or language evaluation was run for the 2026-10-23 documentation revision. No reproducible prompt/output/latency sample is published with this release.

The GGUF header was inspected directly on 2026-10-23, without loading the full model or running inference. It contains the qwen35 architecture and a tokenizer.chat_template; the template, rather than the abbreviated example below, is the source of truth for message formatting.

Prompt formatting

Pass a refinement instruction and the text to a runtime that supports the embedded GGUF chat template. An instruction such as “You are an expert at refining transcribed text.” was shown in the original card; it is an application instruction, not a special token.

The original card illustrated the roles with this abbreviated layout:

<|im_start|>system
You are an expert at refining transcribed text.
<|im_end|>
<|im_start|>user
{user instruction}
<|im_end|>
<|im_start|>assistant

That illustration is not the full serialized prompt. The distributed template also handles whitespace and the thinking prefix; with thinking disabled it appends an empty <think>...</think> block after the assistant prefix. Do not substitute the abbreviated layout for the actual template. A runtime's Qwen3.5 support is required; support for another Qwen model alone does not establish compatibility.

The original card also showed the legacy entry point ./llama-cli -m ./models/refiner-2b-v0.1-Q4_K_M.gguf -p "..." for llama.cpp / llama-cpp-python users. This was a placeholder, not a recorded generation example. Use the current runtime documentation linked above for its executable name, chat-template handling, and Python binding support.

Training data

The teacher generated refined outputs for punctuation and line breaks, polite-form conversion, translation across 46 languages, summarization, and filler-word removal. This was small-scale LoRA distillation by a single developer. The exact dataset is not published; no training examples or private source data are included here.

Operational notes

• The GGUF is the model asset; install a compatible runtime separately.
• Confirm Qwen3.5 architecture and chat-template support before diagnosing output quality.
• The reported 2 GB memory figure is not a maximum. Runtime configuration and context length affect memory use.
• Runtime versions and platforms beyond the original M4 report have not been revalidated for this card.

Limitations and bias

• General conversation is weak: the model is specialized for refinement.
• It is not designed for code generation, mathematical reasoning, or safety-critical medical, legal, or financial advice.
• Creative and long-form writing were outside the training target; the original card suggested using the Qwen3.5-9B teacher for those tasks.
• Base-model knowledge limits and biases carry over. General-knowledge and reasoning ability may be lower than the base model.
• As a 2B-class model, it can misinterpret complex instructions.
• Japanese honorific handling targets desu/masu. Respectful and humble forms (sonkeigo / kenjogo) are weaker than the 9B teacher.
• The internal 46-language result does not establish uniform quality across languages or independent benchmark accuracy.

Related samples

For Core AI and Foundation Models examples on Apple platforms, see coreai-samples. Those examples use their own model assets; this GGUF model follows the llama.cpp-compatible runtime instructions above.

License and attribution

Released under Apache License 2.0. The base model license and teacher model license are Apache 2.0. Original Qwen model copyright: Copyright 2026 Alibaba Cloud. The LoRA fine-tune, distillation, and GGUF distribution are by okayuji.

Qwen and other product names belong to their respective owners. This distribution is not affiliated with or endorsed by the Qwen team. Runtime software has its own license.

Citation

@misc{keyvoice-refiner-2b-v0.1,
  author = {okayuji},
  title = {KeyVoice Refiner 2B (v0.1)},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/okayuji/KeyVoice-Refiner-2B-v0.1}}
}
@misc{qwen3.5,
  title = {Qwen3.5},
  author = {Qwen Team},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Qwen/Qwen3.5-2B}}
}