SashaSobchuk/opus-mt-onnx-int8

OPUS-MT ONNX int8 — models for local (in-browser) translation

A collection of Helsinki-NLP OPUS-MT translation models, quantized to int8, so they can run locally (in the browser or on-device) without a server. Each model lives in its own folder.

What is this for

Running machine translation fully on the user's device. The app downloads only the model folder for the language pair the user needs, and translation happens locally.

How it was made

1. Original models were downloaded from Hugging Face (Helsinki-NLP/opus-mt-*).
2. Weights were quantized to int8 (*_quantized.onnx) to reduce size and speed up inference.
3. Every language pair was tested on sample words to check whether a working translation is produced.

These are modified versions of the original models (format conversion and quantization). Quantization can slightly reduce translation quality compared to the original fp32 weights.

Repository layout

pairs_map.json              # language pair -> model folder, language tag, test score
opus-mt-en-uk/
  config.json
  generation_config.json
  tokenizer.json
  encoder_model_quantized.onnx
  decoder_model_merged_quantized.onnx
opus-mt-ine-ine/
  ...

Download a single file with:

https://huggingface.co/slova/opus-mt-onnx-int8/resolve/main/<model-folder>/<file>

pairs_map.json

A flat object. The key is source_target (language codes).

{
  "hi_fy": ["opus-mt-ine-ine", ">>fry<<", 0.5041985923391694],
  "hi_fj": null
}

• null — no working translation was obtained for this pair.
• otherwise [model folder, language tag, score]:
  • model folder — folder in this repository that translates this pair;
  • language tag — target-language token such as >>fry<< that multilingual models expect at the start of the input (null for models that translate a single direction);
  • score — automatic test score of the pair (higher is better).

One multilingual model folder can serve many language pairs.

Visualization of all tested pairs: https://sashasobchuk.github.io/local_translation_visualize/

License and attribution

Original models: © Helsinki-NLP / the OPUS-MT project, released under CC-BY 4.0. Please check the license on each original model card. If you use these models, credit the original authors:

Tiedemann, J. and Thottingal, S. (2020). OPUS-MT — Building open translation services for the World. Proceedings of EAMT 2020.

This repository only redistributes converted/quantized versions of those models and is released under the same CC-BY 4.0 license, with the modifications described above.