teknium/trismegistus-project
Viewer • Updated • 13.5k • 133 • 77
How to use aloobun/tinyllama_1.8B_Trismegistus with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("text-generation", model="aloobun/tinyllama_1.8B_Trismegistus") # Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("aloobun/tinyllama_1.8B_Trismegistus")
model = AutoModelForCausalLM.from_pretrained("aloobun/tinyllama_1.8B_Trismegistus")How to use aloobun/tinyllama_1.8B_Trismegistus with vLLM:
# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "aloobun/tinyllama_1.8B_Trismegistus"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "aloobun/tinyllama_1.8B_Trismegistus",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'docker model run hf.co/aloobun/tinyllama_1.8B_Trismegistus
How to use aloobun/tinyllama_1.8B_Trismegistus with SGLang:
# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
--model-path "aloobun/tinyllama_1.8B_Trismegistus" \
--host 0.0.0.0 \
--port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "aloobun/tinyllama_1.8B_Trismegistus",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "aloobun/tinyllama_1.8B_Trismegistus" \
--host 0.0.0.0 \
--port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "aloobun/tinyllama_1.8B_Trismegistus",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'How to use aloobun/tinyllama_1.8B_Trismegistus with Docker Model Runner:
docker model run hf.co/aloobun/tinyllama_1.8B_Trismegistus
Total Parameters: 1804769280
from transformers import AutoTokenizer
import transformers
import torch
model = "aloobun/tinyllama_1.8B_Trismegistus"
tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="auto",
)
CHAT_EOS_TOKEN_ID = 2
prompt = "I want to perform an occult practice. Draft a procedure detailing the stages of conjuring a desired outcome using sigil sorcery, specifically within the context of Temple Smaragdum Thallasses. "
sequences = pipeline(
prompt,
do_sample=True,
top_k=50,
top_p = 0.9,
num_return_sequences=1,
repetition_penalty=1.1,
max_new_tokens=512,
eos_token_id=CHAT_EOS_TOKEN_ID,
)
for seq in sequences:
print(f"Result: {seq['generated_text']}")
Five years from now I will have the ability to manipulate and create mystical energy in any form which is capable of changing the current state of consciousness within our universe."
"""
def set_state(self):
# set the state of the temple to the "Glorantha" state. (the state that would be represented by the "Scar" artifact).
return "Glorantha"
def perform_sorcery(self):
# Initiate two-dimensional text that contains the Gematria symbol (Ashura) on the first page of a magnum opus.
self.Gematria = "CWLXAYIJI" <|
return "Lorem ipsum dolor sit amet consectetur adipisic, Lorem Ipsum dolor si mim ponit urcae, sed lend, lend, dormient, et immortel, in nefas talisa"
def write_a_poem(self):
# perform a magic act to bring forth a poetic composition with a character that will be "Avenged" as the chosen target for the Temple Smaragdum Thallasses'
# Sorcery.
# Explanation: As a master sorcerer, I will provide you with my most personal thoughts on a specific person or object.
# You shall conjure, unveil and reveal their unique energy. The "Avenged", for all eternal glory!
#
# End: Aventure's Voice: "Nostral Goggian glyph" - A mystical artwork inspired by the work of GoggledGod1970 and depicts the creation of a
# Gnostic demon, a hateful, malevolent entity which wields powerful power over the world. The ancient name - "Ouroboros"
# refers to the ancient saga that revolves around the "Anthropony" - A massive 100-meter balefully-headed creature -
# a symbol of destruction which eats into existence and vanquishes, destroying all that approach it.
slices:
- sources:
- model: mychen76/tinyllama-colorist-v2
layer_range: [0, 22]
- sources:
- model: aloobun/tinyllama_0_16_trismegistus_v1
layer_range: [0, 16]
merge_method: passthrough
dtype: float16