AI & ML interests

Pretraining, QAT, SLM, Overtraining, AI Interpretability.

Recent Activity

Organization Card

BananaMind AI banner

BananaMind AI

We build small, efficient AI models that run on real hardware — phones, laptops, a single consumer GPU — instead of only in datacenters. Our releases cover compact language models, speech, vision, safety tooling and developer utilities.

We believe AI shouldn't need a datacenter to be smart.


What we work on

  • Small language models, pretrained from scratch
  • Quantization and QAT for local / on-device inference
  • Text-to-speech and audio generation
  • Computer vision and detection models
  • AI safety and interpretability
  • Benchmarks, datasets and evaluation tooling
  • Mobile AI apps and developer tools

Models

All BananaMind 2 models are decoder-only Transformers trained from scratch on a single RTX 5070 Ti.

Shared design across the series:

  • Digit-aware byte-level BPE tokenizer — digits are isolated, so numbers never merge into a single token
  • Grouped-query attention with QK norm
  • SwiGLU, RoPE, RMSNorm, tied embeddings

They ship custom architecture code, so load them with trust_remote_code=True.

BananaMind 2 — Base

Model Params Context Vocab Tokens Notes
BananaMind-2-Pro-Preview 139M 3,072 32,768 51.9B / 100B First public checkpoint of the ongoing Pro run (96K steps). INT Index 23.04, Base Bench 1.1 Elo 1106
BananaMind-2-Medium 49.6M 3,072 12,288 50B 39.27 benchmark average — HellaSwag 32.43, ARC-E 43.81, PIQA 61.86
BananaMind-2-Mini 25.2M 4,096 8,192 30B First model of the BananaMind 2 series
BananaMind-2-Nano 10.0M 4,096 8,192 30B Smallest base model, 35.77 average

Try them in the browser: Pro Preview demo · Medium demo

BananaMind 2 — Chat

Instruction-tuned on smol-smoltalk with full-parameter SFT and assistant-only loss masking. System prompts and multi-turn conversations are supported.

Model Params Base model Instruct Bench 1.1 Elo
BananaMind-2-Pro-Preview-Chat 139M Pro Preview 888
BananaMind-2-Medium-Chat 49.6M Medium 787
BananaMind-2-Mini-Chat 25.2M Mini 654
BananaMind-2-Nano-Chat 10.0M Nano 643

Earlier generations

Model Params Context Tokens Notes
BananaMind-1.5-Base 75.1M 4,096 ~27B Llama-style architecture, our first fully pretrained medium model

Licensing

  • BananaMind Community License 1.0 — BananaMind 2 Pro Preview and Pro Preview Chat
  • Apache 2.0 — all other models listed above

Benchmarks

The Elo numbers above come from our own evaluation suites:

Live rankings: BananaMindBench Leaderboard · SLM Leaderboard


Also in the org


Versioning

0.x → 1.0 → 1.5 → 2.0 → 2.1 → ...

Intermediate versions (1.1, 1.7) only ship when there is something worth releasing between two major steps.

Roadmap

  • BananaMind 2 Ultra — planned
  • BananaMind 2.1 — Q4 2026 (preview checkpoints already public: 2.1-Unified, 2.1-Pico-Preview)
  • BananaMind 3 — Q1–Q2 2027

Join us

We're looking for people already active in the community working on pretraining, QAT, small language models, AI safety, interpretability, benchmarks or datasets. If that's you, press the Join button on the organization page.

Who we are

Partners


If our work is useful to you, follow the org to get new releases in your feed.