Better

#1
by redaihf - opened

This model is creative and generates minimal slop. It mostly avoids the annoying name-calling of Goetia 1.4 and Doppleganger Twist although its original alignment still influences responses during summarisation tasks and when writing conclusions ("horrific", "fucked up", "twisted"). It occasionally subverts expectations by reimagining prompts in interesting ways while still adhering to the user's requests.

Occult AI org

The name-calling is probably due to blackxordolphtronGOAT, which heavily featured blacksheep, of which the 12B version especially is well known for its "insulting demeanor" and, being fully uncensored, adds a bit of "zest" to merges.

I wonder if DPO would be a better way to alter its alignment than SFT.

reimagining prompts in interesting ways That could be the "spectral phantoms" at work. I asked the LLM that helped code this method to "simulate projections of alternate versions of the model using combinations of existing merge methods" like the salience from arcee_fusion, the magnitude from della, variance from SCE, etc. It referred to these as phantom versions of itself. I'm glad to see it worked at least somewhat as intended.

I wonder if DPO would be a better way to alter its alignment than SFT.

SFT is generally regarded as being ineffective for alignment tasks.

Occult AI org

I'll probably switch from SFT to SFT->DPO to first implant the style, then refine the alignment. I have not tested DPO yet, but it should be simple to modify a SFT setup.

Sign up or log in to comment