Instructions to use Sweaterdog/Andy-4-tiny with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Sweaterdog/Andy-4-tiny with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Sweaterdog/Andy-4-tiny")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Sweaterdog/Andy-4-tiny")
model = AutoModelForCausalLM.from_pretrained("Sweaterdog/Andy-4-tiny", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Sweaterdog/Andy-4-tiny with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Sweaterdog/Andy-4-tiny"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sweaterdog/Andy-4-tiny",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Sweaterdog/Andy-4-tiny

SGLang

How to use Sweaterdog/Andy-4-tiny with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Sweaterdog/Andy-4-tiny" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sweaterdog/Andy-4-tiny",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Sweaterdog/Andy-4-tiny" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sweaterdog/Andy-4-tiny",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Sweaterdog/Andy-4-tiny with Docker Model Runner:
```
docker model run hf.co/Sweaterdog/Andy-4-tiny
```

Andy-4-tiny / README.md

Sweaterdog

Update README.md (#1)

2880937 verified 11 months ago

preview code

Raw

History Blame Contribute Delete

3.55 kB

	---
	datasets:
	- Sweaterdog/Andy-4-base
	- Sweaterdog/Andy-4-ft
	- Sweaterdog/Andy-base-2
	language:
	- en
	base_model:
	- HuggingFaceTB/SmolLM2-360M-Instruct
	tags:
	- gaming
	- minecraft
	- mindcraft
	library_name: transformers
	---

	# 🧠 Andy‑4-tiny 🐜

	![file_0000000057e4622f835ec6ade102adfc.png](https://cdn-uploads.huggingface.co/production/uploads/66960602f0ffd8e3a381106a/hXe0j2BbfohvOmtfdZyJu.png)


	Andy‑4-tiny is an 360 Million‑parameter specialist model tuned for Minecraft gameplay via the Mindcraft framework.
	The Current version of Andy-4-tiny is `Andy-4-tiny-0522`.

	> ⚠️ Certification:
	> Andy‑4 is not yet certified by the Mindcraft developers. Use in production at your own discretion.


	## 🔍 Model Specifications

	- Parameters: 360M
	- Training Hardware: 1 × NVIDIA RTX 3070
	- Duration: ~ 36 hours total
	- Data Volumes:
	- Messages: 179,384
	- Tokens: 425,535,198
	- Conversations: 62,149

	- Base Architecture: SmolLM2
	- License: [Andy 1.0 License](LICENSE)
	- Repository: https://huggingface.co/Sweaterdog/Andy‑4

	---

	## 📊 Training Regimen

	1. Andy‑4‑base‑1 dataset
	- Epochs: 2
	- Learning Rate: 5e-5
	- Dataset Size: 47.4k

	2. Andy‑4‑base-2 dataset
	- Epochs: 2
	- Learning Rate: 7e-5
	- Dataset Size: 49.2k

	3. Fine‑tune (FT) dataset
	- Epochs: 2.5
	- Learning Rate: 2e-5
	- Dataset Size: 4.12k

	- Optimizer: AdamW_8bit with cosine decay
	- Quantization: 4‑bit (`bnb-4bit`) for inference
	- Warm Up Steps: 0.1% of each dataset

	---

	## 🚀 Installation

	Andy-4-tiny is an Edge-case model, built to run on the CPU and use minimal ram. These are the requirements to Run Them, not to use them while Minecraft is also running.

	\| Quantization \| RAM Required \|
	\|--------------\|---------------\|
	\| F16 \| CPU 2GB \|
	\| Q8_0 \| CPU 1GB \|
	\| Q4_K_M \| CPU 0.8GB \|

	### 1. Installation directly on Ollama

	1. Visit [Andy-4 on Ollama](https://ollama.com/Sweaterdog/Andy-4)
	2. Copy the command after choosing model type / quantization
	3. Run the command in the terminal
	4. Set the profile's model to be what you installed, such as `ollama/sweaterdog/andy-4:tiny-q8_0`

	### 2. Manual Download & Modelfile

	1. Download
	- From the HF Files tab, grab your chosen `.GGUF` quant weights (e.g. `Andy-4-tiny.Q4_K_M.gguf`).
	- Download the provided `Modelfile`.


	2. Edit

	Change
	```text
	FROM YOUR/PATH/HERE
	```
	to
	```text
	FROM /path/to/Andy-4-tiny.Q4_K_M.gguf
	```
	Optional:
	Increase the parameter `num_ctx` to a higher value for longer conversations if you:

	A. Have extra VRAM

	B. Quantized the context window

	C. Can use a smaller model

	3. Create
	```bash
	ollama create andy-4-tiny -f Modelfile
	```

	This registers the Andy‑4-tiny model locally.

	---

	## 📌 Acknowledgments

	<details>
	<summary>Click to expand</summary>

	- Data & Models by: @Sweaterdog
	- Framework: Mindcraft (https://github.com/kolbytn/mindcraft)
	- LoRA Weights: https://huggingface.co/Sweaterdog/Andy-4-LoRA
	- *Explicit credit is not granted to Meta since this model was trained off of a slightly different architecture, from [DeepSeek-R1](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B)
	</details>

	---

	## ⚖️ License

	See [Andy 1.0 License](LICENSE).

	This work uses data and models created by @Sweaterdog.