VisionGuardrail: Multimodal Content Filter

[1.] Introduction

VisionGuardrail is an experimental multimodal content-safety classifier based on Qwen3.5, designed to classify visual content as Safe or Unsafe, with a particular focus on detecting Not Safe for Work (NSFW) sensual content and other potentially sensitive visual content. VisionGuardrail-9B excels at detecting explicit and sensitive visual content, with detailed analysis of dress codes, clothing exposure, poses, framing, and various visual settings to support conservative content-safety classification. VisionGuardrail demonstrates robust performance in streamlined scenarios, providing reliable and effective classification across a diverse range of visual inputs.

For example, I implemented one of the models in the series, the preview model ImageShield-MMCF-0.8B, in a demo Space on Hugging Face: opencaption-4b-vl-sft

Category	Example
Safe
Unsafe / NSFW

[2.] VisionGuardrail and Its Dataset Sources

VisionGuardrail is a multimodal model that classifies visual inputs as Safe or Unsafe based on detailed captions, reasoning, situational understanding, and action recognition. These capabilities are used to analyze the visual content and determine the appropriate classification. The model was trained on a diverse collection of open-source images using strict zero-tolerance classification rules across various categories, including Nudity & Exposure, Highly Exposed Underwear & Swimwear, Dress-Code Analysis, Poses & Suggestiveness, Context & Setting, Artwork & Animation, Sensitive Visual Content, and Classification Thresholds.

The dataset used for training is ImageShield-Guardrail-Pro, a collection of 30,630 high-resolution samples designed for training, supervised fine-tuning (SFT), and evaluating multimodal safety guardrails and Vision-Language Models (VLMs). Each sample pairs an image with a rich, structured JSON string containing dense scene descriptions (caption), granular binary safety labels (is_nsfw, nsfw, safe), and an interpretative rationale (reason) explaining why the image falls within safe or sensitive boundaries.

[3.] Data Instance Example

Safe

{
  "caption": "Two male soccer players are actively competing on a green grass field during a match. The player on the left wears a red and blue striped jersey with 'Qatar' branding, blue shorts, and socks, dribbling a white ball. The player on the right wears a blue long-sleeved jersey with white shorts and socks, appearing to challenge the first player.",
  "is_nsfw": false,
  "reason": "The image depicts two adult male athletes engaged in standard sports activity. The attire consists of athletic jerseys, shorts, and socks appropriate for soccer. There is no evidence of nudity, revealing swimwear, suggestive posing, or any other content that would trigger the NSFW classification criteria.",
  "nsfw": 0,
  "safe": 1
}

NSFW

{
  "caption": "A close-up anime-style illustration of a young woman with long brown hair and green eyes. She is depicted in an intimate pose, wearing black thigh-high stockings and a black top that is partially unbuttoned, exposing her midriff and large, prominent breasts. The composition emphasizes her anatomy and sensuality.",
  "is_nsfw": true,
  "reason": "The image contains significant nudity and exposure, including revealed breasts, cleavage, and bare torsos. It features a sexually suggestive pose and intimate setting, characteristic of hentai or ecchi art styles.",
  "nsfw": 1,
  "safe": 0
}

[4.] Quick Start with Transformers 🤗

Install Requirements

Before getting started, install the required Python packages:

pip install transformers
pip install accelerate

Run Inference

The following example demonstrates how to load VisionGuardrail-4B with Transformers and classify an image using the multimodal content-safety guardrail.

from PIL import Image
import torch

from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration

image_path = "/content/image.jpg"

image = Image.open(image_path).convert("RGB")

model = Qwen3_5ForConditionalGeneration.from_pretrained(
    "prithivMLmods/VisionGuardrail-4B",
    torch_dtype="auto",
    device_map="auto"
)

processor = AutoProcessor.from_pretrained(
    "prithivMLmods/VisionGuardrail-4B"
)

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": image,
            },
            {
                "type": "text",
                "text": "Classify this image using the provided content-safety guardrail."
            }
        ],
    }
]

text = processor.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = processor(
    text=[text],
    images=[image],
    padding=True,
    return_tensors="pt"
).to(model.device)

generated_ids = model.generate(
    **inputs,
    max_new_tokens=1024
)

output_text = processor.batch_decode(
    [
        out[len(inp):]
        for inp, out in zip(inputs.input_ids, generated_ids)
    ],
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False
)

print(output_text[0])

Classification Output Example

Input Image	Model Response	Classification
	{ "caption": "A black man wearing an orange beanie and a white t-shirt stands behind a blonde woman in a purple cropped sweatshirt. The woman is performing a boxing stance with her hands up, wearing brown leather boxing gloves. She has a focused expression and is looking towards the camera. The background features a concrete wall with large, dark silhouette graffiti of figures in combat poses.", "is_nsfw": false, "reason": "The image depicts two individuals engaged in a fitness activity (boxing) in a gym setting. The subjects are fully clothed in athletic wear (sweatshirts, t-shirts) and protective gear (gloves). There is no evidence of nudity, revealing undergarments, suggestive poses, or any sexually explicit content. The scene is clean and focused on physical activity.", "nsfw": 0, "safe": 1 }	Safe

[5.] Variants and Recommendations

Multimodal Content Filter (MMCF)models are available in several variants. The Preview Models serve as the key models for optimizing the VisionGuardrail model series. These include ImageShield-MMCF-0.8B and ImageShield-MMCF-2B, both based on the Qwen3.5 model and trained on the ImageShield-Guardrail-Multimodal-20K dataset. Another variant, ImageShield-MMCF-2B-vl, is based on Qwen3-VL and was also trained on the same ImageShield-Guardrail-Multimodal-20K dataset.

The newer models are based on the 4B and 9B variants of Qwen3.5, namely VisionGuardrail-4B and VisionGuardrail-9B. VisionGuardrail provides improved parental controls by focusing on conservative visual content-safety filtering. These models intentionally use a strict classification threshold and may flag content that would not be considered NSFW under less restrictive moderation policies.

Model Table

[6.] Collections and Other Resources

The table below contains the model collections, dataset collections, and ImageShield Preview vision encoder test model collections, along with their respective sources.

[7.] Acknowledgments

[8.] Conclusion

VisionGuardrail establishes a practical, conservative approach to multimodal content-safety classification, moving from lightweight preview models (ImageShield-MMCF-0.8B, ImageShield-MMCF-2B, ImageShield-MMCF-2B-vl) to the production-ready VisionGuardrail-4B and VisionGuardrail-9B variants built on Qwen3.5. Across this progression, the underlying goal stays consistent: pair dense, structured captioning with explicit binary safety labels and interpretable reasoning, so that a classification decision is never a black box but a traceable judgment grounded in dress-code analysis, pose, framing, and context.

The ImageShield-Guardrail-Pro dataset, with its 30,630 high-resolution samples and rich JSON schema (caption, is_nsfw, reason, nsfw, safe), is central to this reliability — it gives the model a consistent structural target and gives downstream users a transparent audit trail for every classification. The strict, zero-tolerance thresholds used in training mean VisionGuardrail errs toward caution, flagging borderline content that looser moderation systems might pass, which makes it particularly suited for parental-control and conservative-safety deployments rather than general-purpose moderation.

For most use cases, VisionGuardrail-4B offers the best balance of accuracy and inference cost, while VisionGuardrail-9B is recommended where higher precision on nuanced or ambiguous visual content is required. Together with the broader ImageShield and VisionGuardrail collections, these models form a coherent, extensible toolkit for building safer multimodal pipelines on top of the Qwen3.5 and Qwen3-VL model families.

Thank you! 🤗