Text Generation
Transformers
Safetensors
PyTorch
llama
facebook
meta
llama-3
text-generation-inference
Instructions to use meta-llama/Llama-3.2-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use meta-llama/Llama-3.2-3B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="meta-llama/Llama-3.2-3B")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-3B") model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-3B") - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use meta-llama/Llama-3.2-3B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "meta-llama/Llama-3.2-3B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "meta-llama/Llama-3.2-3B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/meta-llama/Llama-3.2-3B
- SGLang
How to use meta-llama/Llama-3.2-3B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "meta-llama/Llama-3.2-3B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "meta-llama/Llama-3.2-3B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "meta-llama/Llama-3.2-3B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "meta-llama/Llama-3.2-3B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use meta-llama/Llama-3.2-3B with Docker Model Runner:
docker model run hf.co/meta-llama/Llama-3.2-3B
Access request
#82 opened 7 days ago
by
ehexplorer123
Access request for TRIBE v2 research and educational use
#81 opened 10 days ago
by
Silvianl87
Access request for educational purposes
#80 opened 16 days ago
by
gzverev
Access Request
#79 opened 18 days ago
by
jason952722
Access Request
#78 opened 19 days ago
by
dariust
Requesting Access
#77 opened 20 days ago
by
abhi7991
Access denied
#76 opened about 1 month ago
by
dredozubov
Access Request
#75 opened about 1 month ago
by
FoxYaDigg
Request for Access to meta-llama/Llama-3.2-3B for Academic Research
#74 opened about 1 month ago
by
xxx123lwh
Access Request
👍 1
#73 opened about 1 month ago
by
LazyPierogi
Access please
#72 opened about 1 month ago
by
ALEXE-AI
Access Request
#71 opened about 2 months ago
by
dominiconorton
Access request
#70 opened about 2 months ago
by
jsc323
Request Access
🔥 1
#69 opened about 2 months ago
by
joshuatreepaik
Request to re-evaluate access for [dr777] - Research on Neuroscience Visualization
#68 opened about 2 months ago
by
dr777
Access Request
2
#65 opened about 2 months ago
by
ManavDavis83
fix: set `clean_up_tokenization_spaces` to `false`
#64 opened 2 months ago
by
maxsloef
Access needed
#63 opened 2 months ago
by
zhengsihua
Install & run meta-llama/Llama-3.2-3B easily using llmpm
#62 opened 2 months ago
by
sarthak-saxena
Need access to the repo
#61 opened 4 months ago
by
Basthy28
Access Please
#60 opened 8 months ago
by
Ved2005
model meta-llama/Llama-3.2-1B-Instruc
#59 opened 9 months ago
by
Kalyansai2001
Request: DOI
#58 opened 9 months ago
by
darkshadow03
Access needed
#57 opened 9 months ago
by
prafull
Access for model
#56 opened 9 months ago
by
paulzy
Request for Access
1
#55 opened 9 months ago
by
sjanoe
Request for model access
#54 opened 10 months ago
by
Lalson
PLEASE I Need Access to the Llama 3 Collections
#53 opened 11 months ago
by
AllenTAN
I Need Access to Llama 3.2-3B Model
#52 opened 11 months ago
by
Sam-Dev26
Your request to access this repo has been rejected by the repo's authors.
#51 opened 12 months ago
by
PaolaGhione
Request rejected
#50 opened 12 months ago
by
nikos99n
Request: DOI
#49 opened 12 months ago
by
arnav751
Request: DOI
#48 opened about 1 year ago
by
aradhya2001
Request: DOI
#47 opened about 1 year ago
by
MohaOthman
Request: DOI
#45 opened about 1 year ago
by
aarush67
Please grant me the permission for Llama-3.2-3B for project purpose.
6
#44 opened about 1 year ago
by
ashishmukka2112
please grant access to meta-llama/Llama-3.2-3B for personal research demand
#43 opened about 1 year ago
by
jowal1
Mir fehlt die tokenizer.model
#42 opened about 1 year ago
by
jkuphal
Upload 344473.gif
#41 opened about 1 year ago
by
Usagi-Chen
Please provide access to llama3.2-3B model, I want to perform a poc
#40 opened about 1 year ago
by
LokeshKh
Request: DOI
#39 opened about 1 year ago
by
S01M
Access for personal research
#38 opened about 1 year ago
by
Gabrieliam
RuntimeError: The size of tensor a (24) must match the size of tensor b (128) at non-singleton dimension 3
#37 opened about 1 year ago
by
Ink
Access Rejected
#36 opened about 1 year ago
by
sajjadhadi
Request: DOI
#35 opened over 1 year ago
by
baraalsedih
Why rejected me repo authors. :-(
🤝 4
3
#31 opened over 1 year ago
by
dawnbreaker887
request rejected so re-requesting for access
➕ 1
1
#30 opened over 1 year ago
by
stepyoun
why rejected?
1
#29 opened over 1 year ago
by deleted
Request: DOI
#27 opened over 1 year ago
by
gracine