Engram weights for domain specialization?

#63
by AlexAlex321 - opened

Question: any plans for custom / swappable n-gram / Engram weights for domain specialization?

Since this lookup-based memory is addressable by surface tokens/n-grams, it seems like a natural fit for domain-specialized, swappable memory packs (e.g. coding / legal / construction) that could be hot-swapped on top of a frozen core model.

Are there plans to expose it as a pluggable, domain-specializable component (custom weights per domain), or will it stay a single table trained end-to-end?

Thanks!

Sign up or log in to comment