Instructions to use meituan-longcat/LongCat-Flash-Lite-Sparse with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use meituan-longcat/LongCat-Flash-Lite-Sparse with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="meituan-longcat/LongCat-Flash-Lite-Sparse")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import LongcatCausalLM
model = LongcatCausalLM.from_pretrained("meituan-longcat/LongCat-Flash-Lite-Sparse", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use meituan-longcat/LongCat-Flash-Lite-Sparse with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "meituan-longcat/LongCat-Flash-Lite-Sparse"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "meituan-longcat/LongCat-Flash-Lite-Sparse",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/meituan-longcat/LongCat-Flash-Lite-Sparse

SGLang

How to use meituan-longcat/LongCat-Flash-Lite-Sparse with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "meituan-longcat/LongCat-Flash-Lite-Sparse" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "meituan-longcat/LongCat-Flash-Lite-Sparse",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "meituan-longcat/LongCat-Flash-Lite-Sparse" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "meituan-longcat/LongCat-Flash-Lite-Sparse",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use meituan-longcat/LongCat-Flash-Lite-Sparse with Docker Model Runner:
```
docker model run hf.co/meituan-longcat/LongCat-Flash-Lite-Sparse
```

GGUF version please!!

by cimilarkes - opened 16 days ago

Discussion

cimilarkes

16 days ago

GGUF version please!!

elbobinas

16 days ago

and llama.cpp support (if you allow me to wish)

jc2375

16 days ago

Why are there no weights on the repo?

llmfan46

16 days ago

•

edited 16 days ago

GGUF how? Mainline llama.cpp doesn't even support the original model that this model is built on.

Wen-Z

LongCat org 15 days ago

Why are there no weights on the repo?

Thanks for your patience! The weights are now available on the repo.

nonetrix

15 days ago

•

edited 15 days ago

Just commenting so I'll get notified if anything happens hopefully, nice to see medium smallish model that people can run locally soon hopefully if it doesn't fall into void of models that never get llama.cpp support

leouon

15 days ago

Thanks Meituan team. I've been a longcat fan since the first release earlier last year. This model looks to be the perfect size. Even if the benchmarks are not as good for coding/agentic tasks compared to other chinese models I always thought it was good for RP and had solid general knowledge.

maternion

10 days ago

•

edited 10 days ago

https://github.com/ggml-org/llama.cpp/pull/19167

The llama.cpp pr adding support for ~~this~~ longcat-flash-lite arch just got new commits merged.

llmfan46

10 days ago

•

edited 10 days ago

https://github.com/ggml-org/llama.cpp/pull/19167

The llama.cpp pr adding support for this arch just got new commits merged.

This is for LongCat-Flash-Lite, not for LongCat-Flash-Lite-Sparse, I already made a fork that adds support on llama.cpp for LongCat-Flash-Lite, here:

https://github.com/erm14254/llama.cpp-minimax-m3-combined/tree/longcat-mtp

And I made GGUFs too:

https://huggingface.co/llmfan46/LongCat-Flash-Lite-Uncensored-Heretic-Native-MTP-Preserved-GGUF

Support for LongCat-Flash-Lite-Sparse would require a lot of extra work to add into llama.cpp

TomLucidor

3 days ago

•

edited 3 days ago

@Wen-Z will there be QAT version of LongCat-Flash-Lite-Sparse (or maybe LongCat-Flash-Lite-2.0?)
Also who here knows how to make quantized MLX versions of this? @llmfan46 any recommendations?
P.S. quantized DFlash/DSpark support would be cool too

llmfan46

3 days ago

@Wen-Z will there be QAT version of LongCat-Flash-Lite-Sparse (or maybe LongCat-Flash-Lite-2.0?)
Also who here knows how to make quantized MLX versions of this? @llmfan46 any recommendations?
P.S. quantized DFlash/DSpark support would be cool too

Sorry I never made any MLX, I made GGUFs, NVFP4, NVFP4 GGUFs, GPTQs, AWQs and FP8.

Anyway the other issue with LongCat models is the complete lack of architecture support, even if you made a quant, most likely nothing would support it unless someone added support themselves.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment