Instructions to use KSU-HW-SEC/Hardware_Phi_30k_version with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use KSU-HW-SEC/Hardware_Phi_30k_version with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="KSU-HW-SEC/Hardware_Phi_30k_version", trust_remote_code=True)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("KSU-HW-SEC/Hardware_Phi_30k_version", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use KSU-HW-SEC/Hardware_Phi_30k_version with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "KSU-HW-SEC/Hardware_Phi_30k_version"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KSU-HW-SEC/Hardware_Phi_30k_version",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/KSU-HW-SEC/Hardware_Phi_30k_version

SGLang

How to use KSU-HW-SEC/Hardware_Phi_30k_version with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "KSU-HW-SEC/Hardware_Phi_30k_version" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KSU-HW-SEC/Hardware_Phi_30k_version",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "KSU-HW-SEC/Hardware_Phi_30k_version" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KSU-HW-SEC/Hardware_Phi_30k_version",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use KSU-HW-SEC/Hardware_Phi_30k_version with Docker Model Runner:
```
docker model run hf.co/KSU-HW-SEC/Hardware_Phi_30k_version
```

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

HardwarePhi 30k Iteration Result

(have a upload issue, we would repair soon)

Overview

This document presents the results of the HardwarePhi model after 30,000 iterations. The model is based on the Phi1.5B architecture (old version) and has been trained using a specific dataset.

Author:

Weimin Fu from Kansas State University's Hardware Security Group

Corresponding Author:

Xiaolong Guo
E-mail: guoxiaolong@k-state.edu

Acknowledgments:

Common Crawl, CAD4Assurance, Trust-Hub, and open-source designers on GitHub and OpenCore support the data collection efforts. Their commitment to knowledge sharing is greatly appreciated.
Special thanks to Shijie Li and Yifang Zhao from the University of Science and Technology of China for their assistance in data collection.
Gratitude is extended to Kaichen Yang from the Electrical and Computer Engineering Department at Michigan Technological University for providing the training platform support.

Dataset

The training was performed using the following dataset:

Hardware Code and Security Median Dataset

Model Architecture

The base model architecture used for this iteration is Phi1.5B (old version). This architecture provides the foundational structure and parameters for the HardwarePhi model.

Iteration Details

Iteration Count: 30,000

Citation Information

Please cite the following paper

@article{fuhardware,
  title={Hardware Phi-1.5 B: A Large Language Model Encodes Hardware Domain Specific Knowledge},
  author={Fu, Weimin and Li, Shijie and Zhao, Yifang and Ma, Haocheng and Dutta, Raj and Zhang, Xuan and Yang, Kaichen and Jin, Yier and Guo, Xiaolong},
  journal={29th IEEE/ACM Asia and South Pacific Design Automation Conference (ASP-DAC)},
  year={2024}
}

You can view the paper from arxiv: arxiv.org/abs/2402.01728

Update from our group for Hardware domain-specific LLM:

Blog: Large Language Model for Hardware Security

HomePage: Hardware Security Lab

Acknowledgment

Portions of this work were supported by the National Science Foundation (CCF-2019310, First Award Program of ARISE in EPSCoR 2148878).

Downloads last month: -

Collection including KSU-HW-SEC/Hardware_Phi_30k_version

Our Picking

Collection

Portions of this work were supported by the National Science Foundation (2340949). • 8 items • Updated Mar 2

Paper for KSU-HW-SEC/Hardware_Phi_30k_version

Hardware Phi-1.5B: A Large Language Model Encodes Hardware Domain Specific Knowledge

Paper • 2402.01728 • Published Jan 27, 2024