Instructions to use OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelWithLMHead

tokenizer = AutoTokenizer.from_pretrained("OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B")
model = AutoModelWithLMHead.from_pretrained("OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B

SGLang

How to use OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B with Docker Model Runner:
```
docker model run hf.co/OsamaMo/Arabic_Text-To-SQL_using_Qwen2.5-1.5B
```

Arabic_Text-To-SQL_using_Qwen2.5-1.5B / LLaMA-Factory /examples /README_zh.md

OsamaMo

Training in progress, step 500

93adfea verified about 1 year ago

preview code

raw

history blame contribute delete

5.9 kB

	我们提供了多样化的大模型微调示例脚本。

	请确保在 `LLaMA-Factory` 目录下执行下述命令。

	## 目录

	- [LoRA 微调](#lora-微调)
	- [QLoRA 微调](#qlora-微调)
	- [全参数微调](#全参数微调)
	- [合并 LoRA 适配器与模型量化](#合并-lora-适配器与模型量化)
	- [推理 LoRA 模型](#推理-lora-模型)
	- [杂项](#杂项)

	使用 `CUDA_VISIBLE_DEVICES`（GPU）或 `ASCEND_RT_VISIBLE_DEVICES`（NPU）选择计算设备。

	LLaMA-Factory 默认使用所有可见的计算设备。

	## 示例

	### LoRA 微调

	#### （增量）预训练

	```bash
	llamafactory-cli train examples/train_lora/llama3_lora_pretrain.yaml
	```

	#### 指令监督微调

	```bash
	llamafactory-cli train examples/train_lora/llama3_lora_sft.yaml
	```

	#### 多模态指令监督微调

	```bash
	llamafactory-cli train examples/train_lora/llava1_5_lora_sft.yaml
	llamafactory-cli train examples/train_lora/qwen2vl_lora_sft.yaml
	```

	#### DPO/ORPO/SimPO 训练

	```bash
	llamafactory-cli train examples/train_lora/llama3_lora_dpo.yaml
	```

	#### 多模态 DPO/ORPO/SimPO 训练

	```bash
	llamafactory-cli train examples/train_lora/qwen2vl_lora_dpo.yaml
	```

	#### 奖励模型训练

	```bash
	llamafactory-cli train examples/train_lora/llama3_lora_reward.yaml
	```

	#### PPO 训练

	```bash
	llamafactory-cli train examples/train_lora/llama3_lora_ppo.yaml
	```

	#### KTO 训练

	```bash
	llamafactory-cli train examples/train_lora/llama3_lora_kto.yaml
	```

	#### 预处理数据集

	对于大数据集有帮助，在配置中使用 `tokenized_path` 以加载预处理后的数据集。

	```bash
	llamafactory-cli train examples/train_lora/llama3_preprocess.yaml
	```

	#### 在 MMLU/CMMLU/C-Eval 上评估

	```bash
	llamafactory-cli eval examples/train_lora/llama3_lora_eval.yaml
	```

	#### 多机指令监督微调

	```bash
	FORCE_TORCHRUN=1 NNODES=2 NODE_RANK=0 MASTER_ADDR=192.168.0.1 MASTER_PORT=29500 llamafactory-cli train examples/train_lora/llama3_lora_sft.yaml
	FORCE_TORCHRUN=1 NNODES=2 NODE_RANK=1 MASTER_ADDR=192.168.0.1 MASTER_PORT=29500 llamafactory-cli train examples/train_lora/llama3_lora_sft.yaml
	```

	#### 使用 DeepSpeed ZeRO-3 平均分配显存

	```bash
	FORCE_TORCHRUN=1 llamafactory-cli train examples/train_lora/llama3_lora_sft_ds3.yaml
	```

	#### 使用 Ray 在 4 张 GPU 上微调

	```bash
	USE_RAY=1 llamafactory-cli train examples/train_lora/llama3_lora_sft_ray.yaml
	```

	### QLoRA 微调

	#### 基于 4/8 比特 Bitsandbytes/HQQ/EETQ 量化进行指令监督微调（推荐）

	```bash
	llamafactory-cli train examples/train_qlora/llama3_lora_sft_otfq.yaml
	```

	#### 在 NPU 上基于 4 比特 Bitsandbytes 量化进行指令监督微调

	```bash
	llamafactory-cli train examples/train_qlora/llama3_lora_sft_bnb_npu.yaml
	```

	#### 基于 4/8 比特 GPTQ 量化进行指令监督微调

	```bash
	llamafactory-cli train examples/train_qlora/llama3_lora_sft_gptq.yaml
	```

	#### 基于 4 比特 AWQ 量化进行指令监督微调

	```bash
	llamafactory-cli train examples/train_qlora/llama3_lora_sft_awq.yaml
	```

	#### 基于 2 比特 AQLM 量化进行指令监督微调

	```bash
	llamafactory-cli train examples/train_qlora/llama3_lora_sft_aqlm.yaml
	```

	### 全参数微调

	#### 在单机上进行指令监督微调

	```bash
	FORCE_TORCHRUN=1 llamafactory-cli train examples/train_full/llama3_full_sft.yaml
	```

	#### 在多机上进行指令监督微调

	```bash
	FORCE_TORCHRUN=1 NNODES=2 NODE_RANK=0 MASTER_ADDR=192.168.0.1 MASTER_PORT=29500 llamafactory-cli train examples/train_full/llama3_full_sft.yaml
	FORCE_TORCHRUN=1 NNODES=2 NODE_RANK=1 MASTER_ADDR=192.168.0.1 MASTER_PORT=29500 llamafactory-cli train examples/train_full/llama3_full_sft.yaml
	```

	#### 多模态指令监督微调

	```bash
	FORCE_TORCHRUN=1 llamafactory-cli train examples/train_full/qwen2vl_full_sft.yaml
	```

	### 合并 LoRA 适配器与模型量化

	#### 合并 LoRA 适配器

	注：请勿使用量化后的模型或 `quantization_bit` 参数来合并 LoRA 适配器。

	```bash
	llamafactory-cli export examples/merge_lora/llama3_lora_sft.yaml
	```

	#### 使用 AutoGPTQ 量化模型

	```bash
	llamafactory-cli export examples/merge_lora/llama3_gptq.yaml
	```

	### 保存 Ollama 配置文件

	```bash
	llamafactory-cli export examples/merge_lora/llama3_full_sft.yaml
	```

	### 推理 LoRA 模型

	#### 使用 vLLM+TP 批量推理

	```
	python scripts/vllm_infer.py --model_name_or_path path_to_merged_model --dataset alpaca_en_demo
	```

	#### 使用命令行对话框

	```bash
	llamafactory-cli chat examples/inference/llama3_lora_sft.yaml
	```

	#### 使用浏览器对话框

	```bash
	llamafactory-cli webchat examples/inference/llama3_lora_sft.yaml
	```

	#### 启动 OpenAI 风格 API

	```bash
	llamafactory-cli api examples/inference/llama3_lora_sft.yaml
	```

	### 杂项

	#### 使用 GaLore 进行全参数训练

	```bash
	llamafactory-cli train examples/extras/galore/llama3_full_sft.yaml
	```

	#### 使用 APOLLO 进行全参数训练

	```bash
	llamafactory-cli train examples/extras/apollo/llama3_full_sft.yaml
	```

	#### 使用 BAdam 进行全参数训练

	```bash
	llamafactory-cli train examples/extras/badam/llama3_full_sft.yaml
	```

	#### 使用 Adam-mini 进行全参数训练

	```bash
	llamafactory-cli train examples/extras/adam_mini/qwen2_full_sft.yaml
	```

	#### LoRA+ 微调

	```bash
	llamafactory-cli train examples/extras/loraplus/llama3_lora_sft.yaml
	```

	#### PiSSA 微调

	```bash
	llamafactory-cli train examples/extras/pissa/llama3_lora_sft.yaml
	```

	#### 深度混合微调

	```bash
	llamafactory-cli train examples/extras/mod/llama3_full_sft.yaml
	```

	#### LLaMA-Pro 微调

	```bash
	bash examples/extras/llama_pro/expand.sh
	llamafactory-cli train examples/extras/llama_pro/llama3_freeze_sft.yaml
	```

	#### FSDP+QLoRA 微调

	```bash
	bash examples/extras/fsdp_qlora/train.sh
	```

	#### 计算 BLEU 和 ROUGE 分数

	```bash
	llamafactory-cli train examples/extras/nlg_eval/llama3_lora_predict.yaml
	```