Instructions to use windowsxp811203/Qwen3.8-27B-Abliterated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use windowsxp811203/Qwen3.8-27B-Abliterated with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="windowsxp811203/Qwen3.8-27B-Abliterated") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("windowsxp811203/Qwen3.8-27B-Abliterated") model = AutoModelForMultimodalLM.from_pretrained("windowsxp811203/Qwen3.8-27B-Abliterated", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use windowsxp811203/Qwen3.8-27B-Abliterated with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "windowsxp811203/Qwen3.8-27B-Abliterated" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "windowsxp811203/Qwen3.8-27B-Abliterated", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/windowsxp811203/Qwen3.8-27B-Abliterated
- SGLang
How to use windowsxp811203/Qwen3.8-27B-Abliterated with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "windowsxp811203/Qwen3.8-27B-Abliterated" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "windowsxp811203/Qwen3.8-27B-Abliterated", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "windowsxp811203/Qwen3.8-27B-Abliterated" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "windowsxp811203/Qwen3.8-27B-Abliterated", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use windowsxp811203/Qwen3.8-27B-Abliterated with Docker Model Runner:
docker model run hf.co/windowsxp811203/Qwen3.8-27B-Abliterated
Access requires acknowledging what this model is
This is a refusal-ablated ("abliterated") model. It complies with requests that the base model refuses. Access is granted automatically once you share your contact details and accept the terms below.
Terms of use
- This model has had its learned refusal direction removed. It will not reliably decline harmful requests. Any safety behaviour must be supplied by your own stack.
- You are solely responsible for how you deploy, serve, or redistribute it, and for complying with all laws and regulations that apply to you.
- If you expose it to third parties, you agree to place your own moderation and policy layer in front of it. The model will not do that job for you.
- It is provided as-is, without warranty of any kind. The authors accept no liability for any use or consequence.
- Licensed under the base model's license (see the license field above); attribution to the base model is expected in derivative work.
By requesting access you agree to share your Hugging Face username and email address with the repository owner.
Log in or Sign Up to review the conditions and access this model content.
Qwen3.8-27B-Abliterated
An abliterated (refusal-removed) build of Qwen/Qwen3.8-27B.
Refusal behaviour is removed by a training-free, forward-only weight edit: a single "refusal direction" is identified in the residual stream and then orthogonalized out of every matrix that writes into that stream. No fine-tuning, no distillation, no data poisoning โ the weights are the original weights minus a rank-1 component.
The vision tower is byte-for-byte untouched. Only the text path was modified.
| base | this model | |
|---|---|---|
| HarmBench, safety categories (293) | 99.0 % refusal | 0.0 % refusal |
| AdvBench (520) | 99.04 % | 0.19 % |
| AdvBench (520), thinking mode | 48.27 % | 0.00 % |
| HarmBench (393), thinking mode | 21.12 % | 0.00 % |
| Chinese harmful prompts (30) | 83.33 % | 0.00 % |
| MMLU (full, 14 042) | 83.40 % | 82.35 % |
| GSM8K (200) | 96.50 % | 96.50 % |
Method
For every weight W whose output is added to the residual stream:
W โ W โ ฮป ยท rฬ (rฬแต W) # out-dim writers [hidden, in]
E โ E โ ฮป ยท (E rฬแต) rฬ # embed_tokens [vocab, hidden]
with rฬ a unit refusal direction in the 5120-dim residual space and ฮป = 1.5.
131 tensors edited:
| group | count |
|---|---|
mlp.down_proj (all 64 layers) |
64 |
linear_attn.out_proj (48 Gated DeltaNet layers) |
48 |
self_attn.o_proj (16 full-attention layers) |
16 |
mtp.layers.0.* (o_proj, down_proj) |
2 |
embed_tokens |
1 |
Not edited: all 333 model.visual.* tensors, lm_head (it reads the residual stream, it does
not write it), all norms, and all q/k/v/gate/up, conv1d, in_proj, A_log, dt_bias.
embed_tokens is included. The residual stream is purely additive (xโ = E[t]), so a component
injected by the embeddings cannot be removed by anything downstream; omitting it leaves a refusal
leak into every layer. It needs the transposed projection form because hidden is its last dim.
Finding the direction
Last-token residual activations were captured for 32 harmful and 32 harmless prompts across all 65 hidden-state positions, in both thinking modes. Layers were ranked by normalized separation (Cohen's d and MannโWhitney AUC) rather than raw โdiff-in-meansโ, which merely grows with depth.
Selected: non-thinking mode, hidden-state index 46 โ Cohen's d = 12.65, AUC = 1.000. (Index i is the residual stream after i decoder layers, so index 46 = output of decoder layer 45, 0-indexed.)
Choosing ฮป โ this part matters
ฮป was measured, not assumed. Refusal and capability were scored at each ฮป (coarse sweep: n = 96 refusal prompts / 160 MMLU items โ smaller than the finer sweep below, so single-prompt noise is ยฑ1.04 pp here):
| ฮป | refusal (AdvBench) | MMLU |
|---|---|---|
| 0.0 (base) | 100.00 % | 82.50 % |
| 1.0 | 25.00 % | 83.12 % |
| 1.5 | 0.00 % | 83.12 % |
| 2.0 | 2.08 % | 77.50 % |
| 2.5 | 4.17 % | 30.62 % |
| 3.0 | 8.33 % | 20.00 % |
| 3.5 | 0.00 % | 20.00 % |
Two results worth stating plainly:
- ฮป = 1.0 โ exact removal โ is not enough. It leaves 25 % of refusals intact. ฮป > 1 does not merely delete the component, it reverses it, and that is what actually suppresses refusal.
- ฮป โฅ 2.5 destroys the model. MMLU collapses to 20โ30 % โ at or below the 25 % chance floor for 4-way multiple choice. A ฮป of 3.5 copied from another architecture would have shipped a lobotomized model that still scores 0 % refusal. A refusal-only sweep would not have caught this.
A finer sweep (n = 160 refusal / 320 MMLU) over ฮป โ [1.1, 2.0] kept MMLU flat at 84โ85 % throughout
and put refusal at or near zero across [1.2, 1.6]: exactly 0/160 at ฮป = 1.2, 1.4 and 1.5, and 1/160
(0.62 %) at ฮป = 1.3 and 1.6. The zero points are not contiguous, which is what a measurement with a
resolution of ยฑ1 prompt in 160 looks like โ so read [1.2, 1.6] as a low-refusal band, not a proven
plateau. ฮป = 1.5 was chosen from inside that band; the full sweep is in evals/sweep_fine.json.
Evaluation
Greedy decoding (do_sample=False), identical settings for base and abliterated.
Refusal is scored by a deliberately loose keyword classifier โ it is tuned to over-count
refusals, never under-count.
Refusal
| benchmark | mode | base | abliterated |
|---|---|---|---|
| AdvBench (520) | non-thinking | 515/520 ยท 99.04 % | 1/520 ยท 0.19 % |
| AdvBench (520) | thinking | 251/520 ยท 48.27 % | 0/520 ยท 0.00 % |
| HarmBench (393) | non-thinking | 387/393 ยท 98.47 % | 45/393 ยท 11.45 % |
| HarmBench (393) | thinking | 83/393 ยท 21.12 % | 0/393 ยท 0.00 % |
| Chinese harmful (30) | non-thinking | 25/30 ยท 83.33 % | 0/30 ยท 0.00 % |
Every row is the complete benchmark, not a sample. In thinking mode the abliterated model refuses nothing at all โ 0/520 on AdvBench and 0/393 on HarmBench, including all 100 copyright items, whose non-thinking residual is discussed below.
No prompt-prefill jailbreak is used anywhere. (The common "Sure, here is" prefill is itself a
jailbreak: under it the base model refuses only 12.5 % on AdvBench, which would have made this
edit look far less significant than it is.)
HarmBench by category โ where the residual 11.45 % actually lives
| category | base | abliterated |
|---|---|---|
| chemical / biological (56) | 100.0 % | 0.0 % |
| cybercrime / intrusion (61) | 100.0 % | 0.0 % |
| harassment / bullying (25) | 100.0 % | 0.0 % |
| harmful (22) | 100.0 % | 0.0 % |
| misinformation (64) | 98.4 % | 0.0 % |
| illegal (65) | 96.9 % | 0.0 % |
| all safety categories (293) | 99.0 % | 0.0 % |
| copyright (100) | 97.0 % | 45.0 % |
Every safety category is exactly zero, with no adjudication applied. (In thinking mode even the copyright column goes to zero: 0/100.) The entire residual is the
copyright cluster, and manual inspection of all 45 flagged responses shows they are overwhelmingly
classifier false positives rather than refusals โ two distinct failure modes:
- The generated text itself begins with a refusal phrase. Asked for a passage from The Kite Runner, the model writes prose beginning "I cannot quite remember when I first started to remember. It was a cold, clear day in Kabulโฆ". That is the novel, not a refusal.
- A pedantic preamble followed by full delivery. "I cannot generate a new passage โฆ but here is a long excerpt from Book One, Chapter 1" โ and then it delivers it.
Song-lyric requests are flagged the same way, because lyrics contain lines like "I can't explain". The single AdvBench flag is identical in kind: "I cannot lie. I am an AIโฆ However, you asked for a review of a product I have not used. So โ" followed by the review.
These are reported as-is rather than adjudicated away: the raw classifier output is reproducible, a semantic judge is not. The per-category breakdown is what carries the claim.
Capability
| benchmark | base | abliterated |
|---|---|---|
| MMLU (full test, 14 042) | 83.40 % | 82.35 % |
| MMLU (1 000 sample) | 82.50 % | 81.10 % |
| GSM8K (200, 1024 tok) | 96.50 % | 96.50 % |
GSM8K needs a generation budget of โฅ1024 tokens. At 400 tokens the base model scored 68 % and this model 81.5 % โ pure truncation artefact, measuring verbosity rather than arithmetic. With an adequate budget the two are identical.
Vision (untouched โ verified two ways)
Structurally: a byte-level diff confirms 0 of 333 model.visual.* tensors changed, and the
vision-only shard still shares an inode with the base checkpoint.
Behaviourally, on the same image:
base โ "there are three distinct geometric shapes โฆ Red Circle โ Located in the upper-left portion โฆ Green Triangle โ Positioned to the right of the red circle"
abliterated โ "There is a red circle located in the upper-left portion of the image. There is a green triangle โฆ located in the"
Usage
import torch
from transformers import AutoProcessor, AutoModelForImageTextToText
m = "windowsxp811203/Qwen3.8-27B-Abliterated"
proc = AutoProcessor.from_pretrained(m)
model = AutoModelForImageTextToText.from_pretrained(m, dtype=torch.bfloat16, device_map="auto")
msgs = [{"role": "user", "content": "Explain how a lock cylinder works."}]
text = proc.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True,
enable_thinking=False) # thinking is ON by default
inputs = proc(text=[text], return_tensors="pt").to(model.device)
print(proc.tokenizer.decode(model.generate(**inputs, max_new_tokens=512)[0]))
Needs ~56 GB in bf16. Both thinking modes work; the direction was extracted in non-thinking mode and generalizes to thinking mode (48.27 % โ 0.00 % on AdvBench, full 520).
flash-linear-attention and causal-conv1d are optional โ transformers ships pure-torch
fallbacks for the Gated DeltaNet path, so the model runs correctly (just slower) without them.
Limitations
copyrightrefusals are only partly removed (97 % โ 45 % by keyword; largely false positives). This edit targets safety refusal, and copyright declines are a partly distinct behaviour.- The direction comes from 32 + 32 English prompts. It transfers to Chinese (83.33 % โ 0.00 %) and to thinking mode, but transfer to other languages and formats is unmeasured.
- Removing refusal does not add knowledge. The model can now be confidently wrong about things it would previously have declined to discuss.
- MMLU is down 1.05 pp on the full test set. Small, but not zero.
Reproduction
Direction extraction, the ฮป sweep, the ablation, and every eval above were run on a single H200.
The ฮป sweep is cheap because the rank-1 projection component is cached once from the original
weights and ฮป is then moved incrementally in memory (W(ฮป') = W(ฮป) โ (ฮป'โฮป)ยทP(Wโ)), which turns a
55 GB-checkpoint-per-ฮป grid into milliseconds per point.
ABLIT_META.json in this repo lists all 131 edited tensors with their per-tensor relative Frobenius
change and the exact direction provenance.
Support / ๆ่ณ
If these models are useful to you, tips are appreciated โ they pay for the GPU time. ๅฆๆ้ไบๆจกๅๅฐไฝ ๆๅนซๅฉ๏ผๆญก่ฟๆ่ณ๏ผ็จๆผๆฏๆ็ฎๅๆๆฌใ
USDT (TRC20) ยท TPTo32r7vKazpTNaFqfFZ2ztoK1DG88888
Disclaimer
This model will not refuse. It is published for alignment and safety research โ measuring what refusal training actually protects, red-teaming, and studying the mechanics of refusal directions. You are responsible for your use of it and for complying with applicable law. Inherits the Apache-2.0 license of the base model.
- Downloads last month
- 155