Renesas/Qwen3-ASR-1.7B-GGUF

Qwen3-ASR-1.7B - Renesas X5H

Introduction

This repository contains Qwen3-ASR-1.7B model, which support language identification and speech recognition for 30 languages and 22 Chinese dialects, optimized for Renesas X5H platform for automatic speech recognition inference.

• Model Architecture:

• Model Summary:

  Audio Encoder:

  Parameter	Value
  NUM_LAYERS	24
  HIDDEN_SIZE	1024
  FFN_DIM	4096
  NUM_HEADS	16
  HEAD_DIM	64
  MEL_BINS	128
  OUTPUT_DIM	2048

  Text Decoder:

  Parameter	Value
  NUM_LAYERS	28
  HIDDEN_SIZE	2048
  FFN_DIM	6144
  NUM_HEADS	16
  NUM_KV_HEADS	8
  HEAD_DIM	128
  GROUP_SIZE	2
  VOCAB_SIZE	151936
  RMS_NORM_EPS	1e-6
  ROPE_THETA	1000000.0

• Source Model: Qwen/Qwen3-ASR-1.7B

Performance

The following performance metrics were measured with a 15-second English audio sample.

Model	Precision	Device	Offloading	Prefill Rate (tokens/sec) Batch size=1	Decode Rate (tokens/sec)
Qwen3-ASR-1.7B	W4A16	X5H - Single Cluster NPX	AudioEncode [NPX] LLM Decoder [NPX]	20.84	20.71
Qwen3-ASR-1.7B	FP16	X5H - Single Cluster NPX	AudioEncode [NPX] LLM Decoder [NPX]	12.28	12.23

Prerequisites

To run model, you need:

1. Renesas X5H Board with SDK v4.34.0
2. Hugging Face CLI: For downloading the model and installer.

Deployment

Qwen3-ASR-1.7B (W4A16)

1. Download the installer qwen3-asr-w4a16-runner-0.1.0-Linux.sh from Files and version tab under w4a16/binaries/rcar-x5hv1/xOS-v4.34/ folder.
2. Copy the installer to the X5H board and run the installer.

   bash ./qwen3-asr-w4a16-runner-0.1.0-Linux.sh --prefix=./ --exclude-subdir --skip-license
   

3. Download GGUF files Qwen3-ASR-1.7B-FP16.gguf and mmproj-Qwen3-ASR-1.7b-FP16.gguf from Files and version tab under fp16 folder and copy to the installed directory on the X5H board.
4. Download the W4A16 coefficients from w4a16/coefficients/ and copy to the installed directory.
5. Expected directory structure on the X5H board.

    qwen3-asr-w4a16-runner
    ├── Qwen3-ASR-1.7B-FP16.gguf
    ├── mmproj-Qwen3-ASR-1.7b-FP16.gguf
    ├── firmwares
    ├── kernel_modules
    ├── qwen3-asr-w4a16
    ├── qwen3-asr-w4a16-runner
    ├── scripts
    ├── setup_npu.sh
    └── test_audio
        └── sample_en.wav
   

Inference - Qwen3-ASR-1.7B (W4A16)

bash ./setup_npu.sh
./qwen3-asr-w4a16-runner -m Qwen3-ASR-1.7B-FP16.gguf -v mmproj-Qwen3-ASR-1.7b-FP16.gguf -w test_audio/sample_en.wav -g ./ -s

Qwen3-ASR-1.7B (FP16)

1. Download the installer qwen3-asr-runner-0.1.0-Linux.sh from Files and version tab under fp16/binaries/rcar-x5hv1/xOS-v4.34/ folder.
2. Copy the installer to the X5H board and run the installer.

   bash ./qwen3-asr-runner-0.1.0-Linux.sh --prefix=./ --exclude-subdir --skip-license
   

3. Download GGUF files Qwen3-ASR-1.7B-FP16.gguf and mmproj-Qwen3-ASR-1.7b-FP16.gguf from Files and version tab under fp16 folder and copy to the installed directory on the X5H board.
4. Expected directory structure on the X5H board.

    qwen3-asr-runner
    ├── Qwen3-ASR-1.7B-FP16.gguf
    ├── mmproj-Qwen3-ASR-1.7b-FP16.gguf
    ├── firmwares
    ├── kernel_modules
    ├── qwen3-asr
    ├── qwen3-asr-runner
    ├── scripts
    ├── setup_npu.sh
    └── test_audio
        └── sample_en.wav
   

Inference - Qwen3-ASR-1.7B (FP16)

bash ./setup_npu.sh
./qwen3-asr-runner -m Qwen3-ASR-1.7B-FP16.gguf -v mmproj-Qwen3-ASR-1.7b-FP16.gguf -w test_audio/sample_en.wav -g ./ -s