Instructions to use aiyun123/Qwen2.5-Coder-0.5B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use aiyun123/Qwen2.5-Coder-0.5B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="aiyun123/Qwen2.5-Coder-0.5B-Instruct")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("aiyun123/Qwen2.5-Coder-0.5B-Instruct", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use aiyun123/Qwen2.5-Coder-0.5B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "aiyun123/Qwen2.5-Coder-0.5B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aiyun123/Qwen2.5-Coder-0.5B-Instruct",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/aiyun123/Qwen2.5-Coder-0.5B-Instruct

SGLang

How to use aiyun123/Qwen2.5-Coder-0.5B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "aiyun123/Qwen2.5-Coder-0.5B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aiyun123/Qwen2.5-Coder-0.5B-Instruct",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "aiyun123/Qwen2.5-Coder-0.5B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aiyun123/Qwen2.5-Coder-0.5B-Instruct",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use aiyun123/Qwen2.5-Coder-0.5B-Instruct with Docker Model Runner:
```
docker model run hf.co/aiyun123/Qwen2.5-Coder-0.5B-Instruct
```

Qwen2.5-Coder-0.5B-Instruct / README.md

aiyun123

Update README.md

7e1e9a5 verified 10 months ago

preview code

Raw History Blame Contribute Delete

1.87 kB

metadata

language:
  - en
  - zh
license: other
tags:
  - gensyn
  - testnet
  - rl-swarm
  - code-generation
  - mbpp
  - code-contests
pipeline_tag: text-generation
base_model: Qwen/Qwen2.5-Coder-0.5B-Instruct
datasets:
  - deepmind/code_contests
  - google-research-datasets/mbpp
library_name: transformers

Qwen2.5-Coder-0.5B-Instruct — Gensyn Swarm

此仓库用于记录在 Gensyn Testnet 的 RL Swarm 代码生成任务中的参与与权重版本。基础模型为 Qwen/Qwen2.5-Coder-0.5B-Instruct。

Overview

Base model: Qwen2.5-Coder-0.5B-Instruct
Task: Code generation (mbpp, code_contests)
Hardware: Mac mini (M4, 16GB), Apple MPS
Participation: Gensyn Testnet / CodeZero

Training Data

deepmind/code_contests
google-research-datasets/mbpp

Metrics

pass@1, pass@k, exact-match（随版本更新补充）

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "aiyun123/Qwen2.5-Coder-0.5B-Instruct"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto"  # 自动选择设备；在 Mac 可走 MPS
)

prompt = "Write a Python function to check if a number is prime."
inputs = tok(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tok.decode(outputs[0], skip_special_tokens=True))

Inference Notes

macOS: 推荐 PYTORCH_ENABLE_MPS_FALLBACK=1，必要时设置 PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0
dtype: 若遇内存限制，可尝试 torch.float16 或启用 device_map="auto"

Limitations

小模型在复杂算法/长代码生成上的能力有限；需结合评测任务客观比较

Versioning

swarm-YYYY-MM-DD（每日/每轮次版本号；后续推送时更新）

License

与上游基础模型许可一致；请参见 Qwen2.5 的官方许可说明与链接。