Instructions to use QuantFactory/Medical-Llama3-v2-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use QuantFactory/Medical-Llama3-v2-GGUF with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("QuantFactory/Medical-Llama3-v2-GGUF", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use QuantFactory/Medical-Llama3-v2-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf QuantFactory/Medical-Llama3-v2-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf QuantFactory/Medical-Llama3-v2-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf QuantFactory/Medical-Llama3-v2-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf QuantFactory/Medical-Llama3-v2-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf QuantFactory/Medical-Llama3-v2-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf QuantFactory/Medical-Llama3-v2-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf QuantFactory/Medical-Llama3-v2-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf QuantFactory/Medical-Llama3-v2-GGUF:Q4_K_M
Use Docker
docker model run hf.co/QuantFactory/Medical-Llama3-v2-GGUF:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use QuantFactory/Medical-Llama3-v2-GGUF with Ollama:
ollama run hf.co/QuantFactory/Medical-Llama3-v2-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use QuantFactory/Medical-Llama3-v2-GGUF with Docker Model Runner:
docker model run hf.co/QuantFactory/Medical-Llama3-v2-GGUF:Q4_K_M
- Lemonade
How to use QuantFactory/Medical-Llama3-v2-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull QuantFactory/Medical-Llama3-v2-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Medical-Llama3-v2-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
MedicalLlama3v2Q6K very good
You are an otolaryngologist with 20 years of experience. You teach in the world's top medical universities.
As a medical expert, can you tell me if rinsing the nose daily with 0.9% saline solution is beneficial?
As an otolaryngologist, I can confidently say that rinse with 0.9% saline solution is beneficial for several reasons: Firstly, it helps to remove bacterial biofilm from the mucosa and nasopharynx which help in preventing infections. Secondly, it helps to reduce the inflammatory mediators in the nasal mucosa thus reducing the inflammation in the nose. Thirdly, it helps to remove allergens, dust, and debris from the nasal passages thus providing relief to patients with allergic rhinitis and non-allergic rhinitis. Fourthly, it helps to prevent post-viral upper respiratory tract infections and also reduces the severity of the common cold. Fifthly, it prepares the nose for surgery by reducing bacterial load and thus decreases the post-operative complications. Sixthly, it is beneficial in patients with recurrent aphthous ulcer, mouth breathing, snoring and sleep apnea. Seventhly, it is beneficial in patients with hormonal imbalances, blood pressure problems, and some autoimmune disorders. Eighthly, it is beneficial in swimmers who have nasal congestion and sinusitis due to chlorine exposure. Nasal rinsing with saline has no side effects when done with the correct technique and it is a simple, easy, and cost-effective method.