Instructions to use TalasCloud/UMT5-XXL-Encoder-GSQ-RCO-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use TalasCloud/UMT5-XXL-Encoder-GSQ-RCO-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf TalasCloud/UMT5-XXL-Encoder-GSQ-RCO-GGUF:Q6_K # Run inference directly in the terminal: llama cli -hf TalasCloud/UMT5-XXL-Encoder-GSQ-RCO-GGUF:Q6_K
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf TalasCloud/UMT5-XXL-Encoder-GSQ-RCO-GGUF:Q6_K # Run inference directly in the terminal: llama cli -hf TalasCloud/UMT5-XXL-Encoder-GSQ-RCO-GGUF:Q6_K
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf TalasCloud/UMT5-XXL-Encoder-GSQ-RCO-GGUF:Q6_K # Run inference directly in the terminal: ./build/bin/llama cli -hf TalasCloud/UMT5-XXL-Encoder-GSQ-RCO-GGUF:Q6_K
Use Docker
docker model run hf.co/TalasCloud/UMT5-XXL-Encoder-GSQ-RCO-GGUF:Q6_K
- Llama new
- LM Studio
- Jan
- Ollama
How to use TalasCloud/UMT5-XXL-Encoder-GSQ-RCO-GGUF with Ollama:
ollama run hf.co/TalasCloud/UMT5-XXL-Encoder-GSQ-RCO-GGUF:Q6_K
- Unsloth Desktop
- Docker Model Runner
How to use TalasCloud/UMT5-XXL-Encoder-GSQ-RCO-GGUF with Docker Model Runner:
docker model run hf.co/TalasCloud/UMT5-XXL-Encoder-GSQ-RCO-GGUF:Q6_K
- Lemonade
How to use TalasCloud/UMT5-XXL-Encoder-GSQ-RCO-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull TalasCloud/UMT5-XXL-Encoder-GSQ-RCO-GGUF:Q6_K
Run and chat with the model
lemonade run user.UMT5-XXL-Encoder-GSQ-RCO-GGUF-Q6_K
List all available models
lemonade list
- Atomic Chat
UMT5-XXL Encoder · GSQ/RCO GGUF
Calibrated text encoder for the Wan video pipeline.
Русский. Текстовый энкодер UMT5-XXL для видео Wan. Смешанные кванты Q3_K, Q4_K и Q6_K под заданный бюджет весов.
Evidence / Проверка
Recorded projection error 60,640 → 37,930. The encoder GGUF is 3.62 GB.
Ошибка проекций 60 640 → 37 930 на записанной выборке. GGUF энкодера занимает 3,62 ГБ.
Projection error measures calibration fit on the recorded workload. Ошибка проекций характеризует подгонку на записанной выборке.
Calibration runs / Замеры калибровки
Files / Файлы
- umt5-xxl-encoder-Q3_K-Q4_K-Q6_K-GSQ-RCO-Vulkan-2026-11-17.gguf · 3,618,248,512 bytes
Companions / Компоненты
Wan2.2-TI2V-5B-Turbo-GSQ-RCO-GGUF
Sources
Weights and hardware
Format: Q3_K, Q4_K, Q6_K. Tensor layout: Q6_K × 31, F32 × 73, Q4_K × 136, Q3_K × 2.
Vulkan GSQ calibrated 168 matrix projections. RCO selected Q3_K, Q4_K and Q6_K tensor layouts within the checkpoint byte budget. Remaining tensors use FP32.
Target system: Windows, AMD Radeon RX 580 2048SP (Polaris, 8 GB) and Intel Xeon E5-2696 v3 (Haswell, AVX2). Use as the Wan text encoder with a compatible Wan diffusion checkpoint and VAE. Media runtime
SHA-256 cec23a999e52f7edc7db5e7816469d8034d9a4ce5ece144ee013ea78f7f2b602.
- Downloads last month
- 185
6-bit
Model tree for TalasCloud/UMT5-XXL-Encoder-GSQ-RCO-GGUF
Base model
google/umt5-xxl