HIPIE: Hierarchical Open-vocabulary Universal Image Segmentation

PyTorch implementation of HIPIE from "Hierarchical Open-vocabulary Universal Image Segmentation" (Wang et al., NeurIPS 2023).

Pretrained Weights

We provide ViT-H and ResNet-50 weights for hierarchical and part-aware image segmentation across multiple datasets:

Format	Filename	Description
ViT-H (O365, COCO, RefCOCO, PACO)	vit_h_cloud.pth	Pretrained with O365,COCO,RefCOCO,PACO
ViT-H (COCO, RefCOCO, Pascal-Parts)	vit_h_cloud_parts.pth	Finetuned on COCO,RefCOCO,Pascal-Parts
ResNet-50 (Pascal-Parts)	r50_parts.pth	Pretrained with O365,COCO,RefCOCO,Pascal Panoptic Parts

Usage

For demo notebooks, model configs, and inference scripts, see the GitHub repository.

Citation

@inproceedings{wang2023hierarchical,
title={Hierarchical Open-vocabulary Universal Image Segmentation},
author={Wang, Xudong and Li, Shufan and Kallidromitis, Konstantinos and Kato, Yusuke and Kozuka, Kazuki and Darrell, Trevor},
booktitle={Thirty-seventh Conference on Neural Information Processing Systems},
year={2023}
}