mradermacher

/

model_requests

like 159

Link for convenience: https://huggingface.co/DavidAU/Qwen3.5-13B-Deckard-Heretic-Uncensored-Thinking

yes, DavidAU notifies us himself =)

https://huggingface.co/mradermacher/model_requests/discussions/2097

@RichardErkhov
Can you help me with something? I want to apply hybrid layer quantization to a good model later on, but I don't know how to make my quantized layers look like the ones in the screenshot:

I think you attached wrong screenshot lol. and what is hybrid layer quantization? you mean gguf? just ask me, I will quant it

I think you attached wrong screenshot lol. and what is hybrid layer quantization? you mean gguf? just ask me, I will quant it

1. I mean that when I create a new repository for a quantized model, it doesn't automatically appear in the "Quantizations" section on the original model page. I'd like to understand how to ensure my quants are listed there.
2. By "hybrid quantization," I'm referring to a technique where, for a target quantization level (e.g., Q5_K_M), I use higher precision for specific components—for instance, keeping embeddings in a higher precision like Q8 and applying higher-bit quantization to the Self-Attention blocks. I also know that keeping the first and last layers at higher precision improves the quality of responses, as these layers are more critical than the intermediate ones. I am already capable of creating such quantized models.
   P.S. English isn't my native language, so I'm asking Gemini to translate this text into English.

1. ah, it's something to do with the top of readme. explore the source code of any model from mradermacher and you will find what you are searching for, I dont remember it myself as I didnt bother doing it as when I started it wasnt even a feature and I was too lazy to edit the code lol

2. uhm not sure about that, as I didnt do that for a while, I think googling and checking llama cpp repository would be of a better help, I will also ask nico if he knows how to do that

which is your native language?

1. ah, it's something to do with the top of readme. explore the source code of any model from mradermacher and you will find what you are searching for, I dont remember it myself as I didnt bother doing it as when I started it wasnt even a feature and I was too lazy to edit the code lol

2. uhm not sure about that, as I didnt do that for a while, I think googling and checking llama cpp repository would be of a better help, I will also ask nico if he knows how to do that

which is your native language?

I'm from Ukraine; I speak both Ukrainian and Russian, but I'm chatting with the model in Russian.
For some reason, the model doesn't seem to be loading.

@RichardErkhov , I managed to upload the model and write a description. Now I need to figure out how to display my quantization on the model's home page, but I'll get to that later.

I think I've figured it out, but for some reason the files aren't showing up here:

You can for sure speak to me in russian if it helps you understand better =)

Not sure why huggingface doesnt know how to figure the size out, perhaps you want to rename it to something like modelname.IQ4_XS.gguf
Notice the "."after the filename instead of "-"
I dont think it matters, but you never know

You can for sure speak to me in russian if it helps you understand better =)

Not sure why huggingface doesnt know how to figure the size out, perhaps you want to rename it to something like modelname.IQ4_XS.gguf
Notice the "."after the filename instead of "-"
I dont think it matters, but you never know

I made it. My quantization has already been downloaded 471 times. I didn’t think anyone would really need it.

@RichardErkhov
I just created a quantized version of Google/Gemma-4-E4B and only later realized that it’s a base model. Does that mean I didn’t need to quantize it?

You can do any quantization you want =)
but if you wanted to quantize your model, no, you dont need to quantize a base model, you need to quantize only your model