Instructions to use CCB/abstracts_to_tweet_model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use CCB/abstracts_to_tweet_model with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="CCB/abstracts_to_tweet_model")

# Load model directly
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("CCB/abstracts_to_tweet_model")
model = AutoModelForSeq2SeqLM.from_pretrained("CCB/abstracts_to_tweet_model")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use CCB/abstracts_to_tweet_model with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "CCB/abstracts_to_tweet_model"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CCB/abstracts_to_tweet_model",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/CCB/abstracts_to_tweet_model

SGLang

How to use CCB/abstracts_to_tweet_model with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "CCB/abstracts_to_tweet_model" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CCB/abstracts_to_tweet_model",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "CCB/abstracts_to_tweet_model" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CCB/abstracts_to_tweet_model",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use CCB/abstracts_to_tweet_model with Docker Model Runner:
```
docker model run hf.co/CCB/abstracts_to_tweet_model
```

abstracts_to_tweet_model / datadreamer.json

CCB

Pushed by DataDreamer

63f68f9 verified almost 2 years ago

raw

history blame contribute delete

3.92 kB

	{
	"model_card": {
	"Date & Time": "2024-06-22T12:12:32.445548",
	"Model Card": [
	"https://huggingface.co/google/t5-v1_1-base"
	],
	"License Information": [
	"apache-2.0"
	],
	"Citation Information": [
	"\n@inproceedings{Wolf_Transformers_State-of-the-Art_Natural_2020,\n author = {Wolf, Thomas and Debut, Lysandre and Sanh, Victor and Chaumond, Julien",
	"\n@Misc{peft,\n title = {PEFT: State-of-the-art Parameter-Efficient Fine-Tuning methods},\n author = {Sourab Mangrulkar and Sylvain Gugger and Lysandre Debut and Younes"
	]
	},
	"data_card": {
	"Generate Research Paper Abstracts": {
	"Date & Time": "2024-06-22T10:58:16.666642",
	"Model Name": [
	"gpt-4"
	],
	"Model Card": [
	"https://cdn.openai.com/papers/gpt-4-system-card.pdf"
	],
	"License Information": [
	"https://openai.com/policies"
	],
	"Citation Information": [
	"@article{OpenAI2023GPT4TR,\n title={GPT-4 Technical Report},\n author={OpenAI},\n journal={ArXiv},\n year={2023},\n volume={abs/2303.08774},\n url={https://api.semanticscholar.org/CorpusID:257532815}\n}",
	"@article{ouyang2022training,\n title={Training language models to follow instructions with human feedback},\n author={Ouyang, Long and Wu, Jeffrey and Jiang, Xu and Almeida, Diogo and Wainwright, Carroll and Mishkin, Pamela and Zhang, Chong and Agarwal, Sandhini and Slama, Katarina and Ray, Alex and others},\n journal={Advances in Neural Information Processing Systems},\n volume={35},\n pages={27730--27744},\n year={2022}\n}"
	]
	},
	"Generate Tweets from Abstracts": {
	"Date & Time": "2024-06-22T10:58:55.064072",
	"Model Name": [
	"gpt-4"
	],
	"Model Card": [
	"https://cdn.openai.com/papers/gpt-4-system-card.pdf"
	],
	"License Information": [
	"https://openai.com/policies"
	],
	"Citation Information": [
	"@article{OpenAI2023GPT4TR,\n title={GPT-4 Technical Report},\n author={OpenAI},\n journal={ArXiv},\n year={2023},\n volume={abs/2303.08774},\n url={https://api.semanticscholar.org/CorpusID:257532815}\n}",
	"@article{ouyang2022training,\n title={Training language models to follow instructions with human feedback},\n author={Ouyang, Long and Wu, Jeffrey and Jiang, Xu and Almeida, Diogo and Wainwright, Carroll and Mishkin, Pamela and Zhang, Chong and Agarwal, Sandhini and Slama, Katarina and Ray, Alex and others},\n journal={Advances in Neural Information Processing Systems},\n volume={35},\n pages={27730--27744},\n year={2022}\n}"
	]
	},
	"Generate Tweets from Abstracts (train split)": {
	"Date & Time": "2024-06-22T10:58:55.091788"
	}
	},
	"__version__": "0.28.0",
	"datetime": "2024-06-22T10:58:56.752118",
	"type": "TrainHFFineTune",
	"name": "Train an Abstract => Tweet Model",
	"version": 1.0,
	"fingerprint": "7ac09ad132f2a9e6",
	"req_versions": {
	"dill": "0.3.8",
	"sqlitedict": "2.1.0",
	"torch": "2.3.1",
	"numpy": "1.26.4",
	"transformers": "4.41.2",
	"datasets": "2.20.0",
	"huggingface_hub": "0.23.4",
	"accelerate": "0.31.0",
	"peft": "0.11.1",
	"tiktoken": "0.7.0",
	"tokenizers": "0.19.1",
	"openai": "1.35.3",
	"ctransformers": "0.2.27",
	"optimum": "1.20.0",
	"bitsandbytes": "0.42.0",
	"litellm": "1.31.14",
	"trl": "0.8.1",
	"setfit": "1.0.3"
	},
	"interpreter": "3.10.14 (main, Mar 19 2024, 21:46:16) [Clang 15.0.0 (clang-1500.3.9.4)]"
	}