Instructions to use OccultAI/Ouroboros-24B-v1.4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use OccultAI/Ouroboros-24B-v1.4 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="OccultAI/Ouroboros-24B-v1.4") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("OccultAI/Ouroboros-24B-v1.4") model = AutoModelForCausalLM.from_pretrained("OccultAI/Ouroboros-24B-v1.4", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use OccultAI/Ouroboros-24B-v1.4 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "OccultAI/Ouroboros-24B-v1.4" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OccultAI/Ouroboros-24B-v1.4", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/OccultAI/Ouroboros-24B-v1.4
- SGLang
How to use OccultAI/Ouroboros-24B-v1.4 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "OccultAI/Ouroboros-24B-v1.4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OccultAI/Ouroboros-24B-v1.4", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "OccultAI/Ouroboros-24B-v1.4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OccultAI/Ouroboros-24B-v1.4", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use OccultAI/Ouroboros-24B-v1.4 with Docker Model Runner:
docker model run hf.co/OccultAI/Ouroboros-24B-v1.4
Better
This model is creative and generates minimal slop. It mostly avoids the annoying name-calling of Goetia 1.4 and Doppleganger Twist although its original alignment still influences responses during summarisation tasks and when writing conclusions ("horrific", "fucked up", "twisted"). It occasionally subverts expectations by reimagining prompts in interesting ways while still adhering to the user's requests.
The name-calling is probably due to blackxordolphtronGOAT, which heavily featured blacksheep, of which the 12B version especially is well known for its "insulting demeanor" and, being fully uncensored, adds a bit of "zest" to merges.
I wonder if DPO would be a better way to alter its alignment than SFT.
reimagining prompts in interesting ways That could be the "spectral phantoms" at work. I asked the LLM that helped code this method to "simulate projections of alternate versions of the model using combinations of existing merge methods" like the salience from arcee_fusion, the magnitude from della, variance from SCE, etc. It referred to these as phantom versions of itself. I'm glad to see it worked at least somewhat as intended.
I wonder if DPO would be a better way to alter its alignment than SFT.
SFT is generally regarded as being ineffective for alignment tasks.
I'll probably switch from SFT to SFT->DPO to first implant the style, then refine the alignment. I have not tested DPO yet, but it should be simple to modify a SFT setup.