AI & ML interests
Pretraining, QAT, SLM, Overtraining, AI Interpretability.
Recent Activity
BananaMind AI
We build small, efficient AI models that run on real hardware — phones, laptops, a single consumer GPU — instead of only in datacenters. Our releases cover compact language models, speech, vision, safety tooling and developer utilities.
We believe AI shouldn't need a datacenter to be smart.
What we work on
- Small language models, pretrained from scratch
- Quantization and QAT for local / on-device inference
- Text-to-speech and audio generation
- Computer vision and detection models
- AI safety and interpretability
- Benchmarks, datasets and evaluation tooling
- Mobile AI apps and developer tools
Models
All BananaMind 2 models are decoder-only Transformers trained from scratch on a single RTX 5070 Ti.
Shared design across the series:
- Digit-aware byte-level BPE tokenizer — digits are isolated, so numbers never merge into a single token
- Grouped-query attention with QK norm
- SwiGLU, RoPE, RMSNorm, tied embeddings
They ship custom architecture code, so load them with trust_remote_code=True.
BananaMind 2 — Base
| Model | Params | Context | Vocab | Tokens | Notes |
|---|---|---|---|---|---|
| BananaMind-2-Pro-Preview | 139M | 3,072 | 32,768 | 51.9B / 100B | First public checkpoint of the ongoing Pro run (96K steps). INT Index 23.04, Base Bench 1.1 Elo 1106 |
| BananaMind-2-Medium | 49.6M | 3,072 | 12,288 | 50B | 39.27 benchmark average — HellaSwag 32.43, ARC-E 43.81, PIQA 61.86 |
| BananaMind-2-Mini | 25.2M | 4,096 | 8,192 | 30B | First model of the BananaMind 2 series |
| BananaMind-2-Nano | 10.0M | 4,096 | 8,192 | 30B | Smallest base model, 35.77 average |
Try them in the browser: Pro Preview demo · Medium demo
BananaMind 2 — Chat
Instruction-tuned on smol-smoltalk with full-parameter SFT and assistant-only loss masking. System prompts and multi-turn conversations are supported.
| Model | Params | Base model | Instruct Bench 1.1 Elo |
|---|---|---|---|
| BananaMind-2-Pro-Preview-Chat | 139M | Pro Preview | 888 |
| BananaMind-2-Medium-Chat | 49.6M | Medium | 787 |
| BananaMind-2-Mini-Chat | 25.2M | Mini | 654 |
| BananaMind-2-Nano-Chat | 10.0M | Nano | 643 |
Earlier generations
| Model | Params | Context | Tokens | Notes |
|---|---|---|---|---|
| BananaMind-1.5-Base | 75.1M | 4,096 | ~27B | Llama-style architecture, our first fully pretrained medium model |
Licensing
- BananaMind Community License 1.0 — BananaMind 2 Pro Preview and Pro Preview Chat
- Apache 2.0 — all other models listed above
Benchmarks
The Elo numbers above come from our own evaluation suites:
- Base Bench 1.1 — base model comparison
- Instruct Bench 1.1 — chat / instruct comparison
- Safety Bench 1.1 — safety evaluation
Live rankings: BananaMindBench Leaderboard · SLM Leaderboard
Also in the org
- Safety and detection — Content-Safety-Mini-1.5, Qwen3.5-4B-Activation-Safety-Classifier, 2-AI-Detect
- Task-specific — MathBananaMind-1.1, Title-1.0
- Speech — TTS-Neon
- Mixture of experts — 2-MoE, 2-SLMoE
- Experiments — MiniBananaMind and MicroBananaMind series, StoryBananaMind, the KV1-8M quantization tests (1-bit / 2-bit / 16-bit), Overfitter-1.0
Versioning
0.x → 1.0 → 1.5 → 2.0 → 2.1 → ...
Intermediate versions (1.1, 1.7) only ship when there is something worth releasing between two major steps.
Roadmap
- BananaMind 2 Ultra — planned
- BananaMind 2.1 — Q4 2026 (preview checkpoints already public: 2.1-Unified, 2.1-Pico-Preview)
- BananaMind 3 — Q1–Q2 2027
Join us
We're looking for people already active in the community working on pretraining, QAT, small language models, AI safety, interpretability, benchmarks or datasets. If that's you, press the Join button on the organization page.
Who we are
Partners
If our work is useful to you, follow the org to get new releases in your feed.
buckets 1
-
BananaMind/BananaMind-2-Pro
Text Generation • 0.2B • Updated • 1.43k • 33 -
BananaMind/BananaMind-2-Pro-Preview
Text Generation • 0.2B • Updated • 635 • 24 -
BananaMind/BananaMind-2-Medium
Text Generation • 55.9M • Updated • 2.38k • 18 -
BananaMind/BananaMind-2-SLMoE
Text Generation • 27.5M • Updated • 569 • 13
-
BananaMind/BananaMind-2-Pro
Text Generation • 0.2B • Updated • 1.43k • 33 -
BananaMind/BananaMind-2-Pro-Preview
Text Generation • 0.2B • Updated • 635 • 24 -
BananaMind/BananaMind-2-Medium
Text Generation • 55.9M • Updated • 2.38k • 18 -
BananaMind/BananaMind-2-SLMoE
Text Generation • 27.5M • Updated • 569 • 13
