karay/demo-models

Demo models

Vision models converted to ONNX for the in-browser demos on karay.me: the LaFAM demo and the region similarity demo. The demos run them on the visitor's device with ONNX Runtime Web (WebGPU, or WebAssembly on the CPU); no image leaves the browser.

Each file is the largest model of its family that stays under 100 MB, except where a specific model is the point: CLIP RN50x4 (the model of the typographic attacks of Goh et al., 2021) and the pathology models Hibou-B and QuiltNet-B-32.

Interface

Every model takes one input and returns the last feature map before pooling:

	Shape	Meaning
input image	(1, 3, S, S)	RGB in [0, 1]; S = 224 (288 for CLIP RN50x4); the model's own normalization is part of the graph
output features	(1, C, H, W)	the last feature map; for ViTs, the patch tokens after the final norm, as a grid
output cell_logits	(1, K, H, W)	classifiers only: the classification head applied to every cell (K = 1000 for ImageNet, 5 tissue classes, 2 for the tumour detector)
output logits	(1, K)	only CLIP RN50x4 and QuiltNet-B-32: the model's own zero-shot logits (attention pooling or class token), which are not the mean of the cells

For the classifiers that pool their features (ResNet, ConvNeXt, Swin, MaxViT), the mean of cell_logits over the cells equals the model's logits, so the per-cell readout is exact (Karjauv, 2026). For the two CLIP models with a zero-shot head, cell_logits classifies each cell on its own: the cell's value embedding through the attention pooling's output projection (RN50x4, as in MaskCLIP, Zhou et al., 2022), or the patch token through the final projection (QuiltNet). Weights are stored as float16 and cast to float32 in the graph; the files match the PyTorch models to a correlation of at least 0.9999 on the demo images.

Files

File	Model	Source checkpoint (timm)	Grid	License
resnet50.onnx	ResNet-50, ImageNet classifier	resnet50.tv2_in1k	7×7	BSD-3-Clause
convnext_small.onnx	ConvNeXt-S, ImageNet classifier	convnext_small.fb_in1k	7×7	Apache-2.0
swin_small.onnx	Swin-S, ImageNet classifier	swin_small_patch4_window7_224.ms_in1k	7×7	MIT
maxvit_tiny.onnx	MaxViT-T, ImageNet classifier	maxvit_tiny_tf_224.in1k	7×7	Apache-2.0
dino_resnet50.onnx	DINO ResNet-50, self-supervised	DINO dino_resnet50_pretrain.pth (torchvision)	7×7	Apache-2.0
dino_vit_small.onnx	DINO ViT-S/16, self-supervised	vit_small_patch16_224.dino	14×14	Apache-2.0
dinov2_vit_small.onnx	DINOv2 ViT-S/14, self-supervised	vit_small_patch14_dinov2.lvd142m	16×16	Apache-2.0
dinov2_reg_vit_small.onnx	DINOv2 ViT-S/14 with registers, self-supervised	vit_small_patch14_reg4_dinov2.lvd142m	16×16	Apache-2.0
dinov3_vit_small_plus.onnx	DINOv3 ViT-S+/16, self-supervised	vit_small_plus_patch16_dinov3.lvd1689m	14×14	DINOv3 License
dinov3_convnext_small.onnx	DINOv3 ConvNeXt-S, self-supervised	convnext_small.dinov3_lvd1689m	7×7	DINOv3 License
fcmae_convnextv2_tiny.onnx	ConvNeXt V2-T, masked autoencoder (FCMAE)	convnextv2_tiny.fcmae	7×7	CC-BY-NC-4.0 (non-commercial use only)
clip_resnet101.onnx	CLIP ResNet-101 (OpenAI), image encoder trunk	resnet101_clip_gap.openai	7×7	MIT
tinyclip_vit_medium.onnx	TinyCLIP ViT-M/16, image encoder	vit_medium_patch16_clip_224.tinyclip_yfcc15m	14×14	MIT
clip_rn50x4.onnx	CLIP RN50x4 (OpenAI) with its zero-shot ImageNet classifier (80 prompts)	open_clip RN50x4, openai	9×9	MIT
hibou_b.onnx	Hibou-B (pathology, DINOv2 ViT-B/14 with registers) + a linear tissue head (tumour, stroma, lymphocytes, necrosis, fat)	histai/hibou-b via hibou_port.py; head hibou_b_tissue_head.pt	16×16	Apache-2.0
quiltnet_b32.onnx	QuiltNet-B-32 (pathology CLIP) with a zero-shot head over the same tissue classes	open_clip wisdomik/QuiltNet-B-32	7×7	MIT
tumour_detector.onnx	breast tumour detector, ResNet-34 (other, tumour)	kaczmarj/breast-tumor-resnet34.tcga-brca (torchvision)	7×7	CC-BY-4.0
kaiko_vit_small.onnx	Kaiko ViT-S/16 (pathology, self-supervised)	torch.hub kaiko-ai/towards_large_pathology_fms vits16	14×14	Kaiko Non-Commercial
ctranspath.onnx	CTransPath (pathology, Swin-T with a convolutional stem)	1aurent/swin_tiny_patch4_window7_224.CTransPath	7×7	GPL-3.0

The pathology models expect H&E tiles at about 0.5 µm per pixel (224 pixels = 112 µm); the tumour detector was trained on 0.25 µm per pixel tiles of 350 pixels resized to 224 (about 0.39 µm per pixel). The tissue head of Hibou-B is logistic regression on the mean patch token, trained with one label per 112 µm tile (the majority tissue of the pathologist's annotation) on 6,692 tiles of 132 BCSS cases, none lobular (hibou_head.py in the author's project); as it is linear on the mean, the mean of its cell_logits is its prediction.

Each file is a derivative of its source checkpoint and is distributed under that checkpoint's license (see LICENSE.md). The DINOv3 models are distributed under the DINOv3 License, a copy of which is included; ConvNeXt V2 (FCMAE) and Kaiko ViT-S/16 are for non-commercial use only; CTransPath is GPL-3.0, with its conversion source in this repository.

Reproducing

export_models.py exports every file from its source checkpoint and checks it against PyTorch on a folder of .jpg images (any will do):

SAMPLES=path/to/images uv run --no-project --with torch --with torchvision --with timm --with onnx --with onnxruntime \
    --with pillow --with open_clip_torch --with safetensors --with huggingface_hub python export_models.py NAME