Instructions to use OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelWithLMHead

tokenizer = AutoTokenizer.from_pretrained("OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B")
model = AutoModelWithLMHead.from_pretrained("OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B

SGLang

How to use OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B with Docker Model Runner:
```
docker model run hf.co/OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B
```

Arabic_Text-To-SQL_using_Qwen2.5-1.5B / LLaMA-Factory /examples /README.md

OsamaMo

Training in progress, step 500

93adfea verified over 1 year ago

preview code

Raw

History Blame Contribute Delete

6.16 kB

We provide diverse examples about fine-tuning LLMs.

Make sure to execute these commands in the LLaMA-Factory directory.

LoRA Fine-Tuning
QLoRA Fine-Tuning
Full-Parameter Fine-Tuning
Merging LoRA Adapters and Quantization
Inferring LoRA Fine-Tuned Models
Extras

Use CUDA_VISIBLE_DEVICES (GPU) or ASCEND_RT_VISIBLE_DEVICES (NPU) to choose computing devices.

By default, LLaMA-Factory uses all visible computing devices.

Examples

LoRA Fine-Tuning

(Continuous) Pre-Training

llamafactory-cli train examples/train_lora/llama3_lora_pretrain.yaml

Supervised Fine-Tuning

llamafactory-cli train examples/train_lora/llama3_lora_sft.yaml

Multimodal Supervised Fine-Tuning

llamafactory-cli train examples/train_lora/llava1_5_lora_sft.yaml
llamafactory-cli train examples/train_lora/qwen2vl_lora_sft.yaml

DPO/ORPO/SimPO Training

llamafactory-cli train examples/train_lora/llama3_lora_dpo.yaml

Multimodal DPO/ORPO/SimPO Training

llamafactory-cli train examples/train_lora/qwen2vl_lora_dpo.yaml

Reward Modeling

llamafactory-cli train examples/train_lora/llama3_lora_reward.yaml

PPO Training

llamafactory-cli train examples/train_lora/llama3_lora_ppo.yaml

KTO Training

llamafactory-cli train examples/train_lora/llama3_lora_kto.yaml

Preprocess Dataset

It is useful for large dataset, use tokenized_path in config to load the preprocessed dataset.

llamafactory-cli train examples/train_lora/llama3_preprocess.yaml

Evaluating on MMLU/CMMLU/C-Eval Benchmarks

llamafactory-cli eval examples/train_lora/llama3_lora_eval.yaml

Supervised Fine-Tuning on Multiple Nodes

FORCE_TORCHRUN=1 NNODES=2 NODE_RANK=0 MASTER_ADDR=192.168.0.1 MASTER_PORT=29500 llamafactory-cli train examples/train_lora/llama3_lora_sft.yaml
FORCE_TORCHRUN=1 NNODES=2 NODE_RANK=1 MASTER_ADDR=192.168.0.1 MASTER_PORT=29500 llamafactory-cli train examples/train_lora/llama3_lora_sft.yaml

Supervised Fine-Tuning with DeepSpeed ZeRO-3 (Weight Sharding)

FORCE_TORCHRUN=1 llamafactory-cli train examples/train_lora/llama3_lora_sft_ds3.yaml

Supervised Fine-Tuning with Ray on 4 GPUs

USE_RAY=1 llamafactory-cli train examples/train_lora/llama3_lora_sft_ray.yaml

QLoRA Fine-Tuning

Supervised Fine-Tuning with 4/8-bit Bitsandbytes/HQQ/EETQ Quantization (Recommended)

llamafactory-cli train examples/train_qlora/llama3_lora_sft_otfq.yaml

Supervised Fine-Tuning with 4-bit Bitsandbytes Quantization on Ascend NPU

llamafactory-cli train examples/train_qlora/llama3_lora_sft_bnb_npu.yaml

Supervised Fine-Tuning with 4/8-bit GPTQ Quantization

llamafactory-cli train examples/train_qlora/llama3_lora_sft_gptq.yaml

Supervised Fine-Tuning with 4-bit AWQ Quantization

llamafactory-cli train examples/train_qlora/llama3_lora_sft_awq.yaml

Supervised Fine-Tuning with 2-bit AQLM Quantization

llamafactory-cli train examples/train_qlora/llama3_lora_sft_aqlm.yaml

Full-Parameter Fine-Tuning

Supervised Fine-Tuning on Single Node

FORCE_TORCHRUN=1 llamafactory-cli train examples/train_full/llama3_full_sft.yaml

Supervised Fine-Tuning on Multiple Nodes

FORCE_TORCHRUN=1 NNODES=2 NODE_RANK=0 MASTER_ADDR=192.168.0.1 MASTER_PORT=29500 llamafactory-cli train examples/train_full/llama3_full_sft.yaml
FORCE_TORCHRUN=1 NNODES=2 NODE_RANK=1 MASTER_ADDR=192.168.0.1 MASTER_PORT=29500 llamafactory-cli train examples/train_full/llama3_full_sft.yaml

Multimodal Supervised Fine-Tuning

FORCE_TORCHRUN=1 llamafactory-cli train examples/train_full/qwen2vl_full_sft.yaml

Merging LoRA Adapters and Quantization

Merge LoRA Adapters

Note: DO NOT use quantized model or quantization_bit when merging LoRA adapters.

llamafactory-cli export examples/merge_lora/llama3_lora_sft.yaml

Quantizing Model using AutoGPTQ

llamafactory-cli export examples/merge_lora/llama3_gptq.yaml

Save Ollama modelfile

llamafactory-cli export examples/merge_lora/llama3_full_sft.yaml

Inferring LoRA Fine-Tuned Models

Batch Generation using vLLM Tensor Parallel

python scripts/vllm_infer.py --model_name_or_path path_to_merged_model --dataset alpaca_en_demo

Use CLI ChatBox

llamafactory-cli chat examples/inference/llama3_lora_sft.yaml

Use Web UI ChatBox

llamafactory-cli webchat examples/inference/llama3_lora_sft.yaml

Launch OpenAI-style API

llamafactory-cli api examples/inference/llama3_lora_sft.yaml

Extras

Full-Parameter Fine-Tuning using GaLore

llamafactory-cli train examples/extras/galore/llama3_full_sft.yaml

Full-Parameter Fine-Tuning using APOLLO

llamafactory-cli train examples/extras/apollo/llama3_full_sft.yaml

Full-Parameter Fine-Tuning using BAdam

llamafactory-cli train examples/extras/badam/llama3_full_sft.yaml

Full-Parameter Fine-Tuning using Adam-mini

llamafactory-cli train examples/extras/adam_mini/qwen2_full_sft.yaml

LoRA+ Fine-Tuning

llamafactory-cli train examples/extras/loraplus/llama3_lora_sft.yaml

PiSSA Fine-Tuning

llamafactory-cli train examples/extras/pissa/llama3_lora_sft.yaml

Mixture-of-Depths Fine-Tuning

llamafactory-cli train examples/extras/mod/llama3_full_sft.yaml

LLaMA-Pro Fine-Tuning

bash examples/extras/llama_pro/expand.sh
llamafactory-cli train examples/extras/llama_pro/llama3_freeze_sft.yaml

FSDP+QLoRA Fine-Tuning

bash examples/extras/fsdp_qlora/train.sh

Computing BLEU and ROUGE Scores

llamafactory-cli train examples/extras/nlg_eval/llama3_lora_predict.yaml

Table of Contents

Examples

LoRA Fine-Tuning

(Continuous) Pre-Training

Supervised Fine-Tuning

Multimodal Supervised Fine-Tuning

DPO/ORPO/SimPO Training

Multimodal DPO/ORPO/SimPO Training

Reward Modeling

PPO Training

KTO Training

Preprocess Dataset

Evaluating on MMLU/CMMLU/C-Eval Benchmarks

Supervised Fine-Tuning on Multiple Nodes

Supervised Fine-Tuning with DeepSpeed ZeRO-3 (Weight Sharding)

Supervised Fine-Tuning with Ray on 4 GPUs

QLoRA Fine-Tuning

Supervised Fine-Tuning with 4/8-bit Bitsandbytes/HQQ/EETQ Quantization (Recommended)

Supervised Fine-Tuning with 4-bit Bitsandbytes Quantization on Ascend NPU

Supervised Fine-Tuning with 4/8-bit GPTQ Quantization

Supervised Fine-Tuning with 4-bit AWQ Quantization

Supervised Fine-Tuning with 2-bit AQLM Quantization

Full-Parameter Fine-Tuning

Supervised Fine-Tuning on Single Node

Supervised Fine-Tuning on Multiple Nodes

Multimodal Supervised Fine-Tuning

Merging LoRA Adapters and Quantization

Merge LoRA Adapters

Quantizing Model using AutoGPTQ

Save Ollama modelfile

Inferring LoRA Fine-Tuned Models

Batch Generation using vLLM Tensor Parallel

Use CLI ChatBox

Use Web UI ChatBox

Launch OpenAI-style API

Extras

Full-Parameter Fine-Tuning using GaLore

Full-Parameter Fine-Tuning using APOLLO

Full-Parameter Fine-Tuning using BAdam

Full-Parameter Fine-Tuning using Adam-mini

LoRA+ Fine-Tuning

PiSSA Fine-Tuning

Mixture-of-Depths Fine-Tuning

LLaMA-Pro Fine-Tuning

FSDP+QLoRA Fine-Tuning

Computing BLEU and ROUGE Scores