Could you maybe just open source this quantization method?

#3
by BlueNipples - opened

Manually doing a limited number of quants yourself doesn't seem terribly efficient, and you don't appear to want to somehow make money out of it.

Would be beneficial if people could just use this themselves πŸ€·β€β™‚οΈ

Unsloth AI org

I've been telling people many times we already did: https://github.com/unslothai/llama.cpp

shimmyshimmer changed discussion status to closed

Sign up or log in to comment