Image-to-Text
PEFT
Safetensors
English
qwen3.5
image-captioning
natural-language-captioning
lora
sankaku
multimodal
Not-For-All-Audiences
colab
T4
notebook
jupyter
adapter
natural-language
qwen

Qwen 3.5 Natural Language Image Captioning Adapter

This is an adapter model for the Qwen 3.5 4B Abliterated model DreamFast/Qwen3.5-4B-Uncensored-HauhauCS-Aggressive-Safetensor-Benchmark .

The adapter is trained on image captioning using natural language output.

When using this adapter on the Qwen 4B model , we can do batches of 30 captions at a time on a single T4 , roughly 7 seconds per caption.

Its trained on my own prompts , natural lanugage captions of images from sankaku complex using a mix of gemini flash and joycaption beta , and prompts from (NSFW) posts on civitai The main goal of this project is to create a fast, compact natural language captioning model that can be used to generate training captions for image-to-text and image-generation models.

This is the current dataset distribution

ffff

Testing the adapter

You can test the adapter using the Google Colab notebook provided in this repository:

qwen35_sankaku_captioning.ipynb

The notebook provides an interface for testing image caption generation using the adapter.

Training

The adapter was trained on 2,000 images using:

Parameter Details
Base model DreamFast/Qwen3.5-4B-Uncensored-HauhauCS-Aggressive-Safetensor-Benchmark
Training images 10,000
Dataset source Sankaku Complex Database
Training hardware 2 × NVIDIA T4 GPUs (Kaggle)
Training duration 24 hours
Training steps 13000

Training methodology

The dataset preparation involved an initial captioning and summarization stage.

During the first training run, the base Qwen 3.5 model was used to caption the images. Its generated descriptions were then summarized by the model itself.

The adapter was subsequently trained on these summaries rather than the original, longer model responses.

This approach was intended to remove unnecessary preambles and verbose introductory text from the captioning output, producing more direct and compact natural language descriptions.

An important distinction: this process does not add new visual understanding or knowledge to the underlying model. It primarily teaches the model to produce a more concise version of its existing captioning capabilities.

Performance

Inference speed is one of the most promising aspects of this adapter.

On an NVIDIA T4 (16 GB), the model can process approximately 40 image items in a batch, with an average caption processing time of approximately 6–7 seconds per image.

These are observed performance figures and may vary depending on image resolution, generation settings, software configuration and available GPU memory.

The combination of relatively fast inference and access to the Sankaku database opens up interesting possibilities for on-the-fly dataset preparation.

Potential applications

One particularly interesting possibility is integrating natural language caption generation directly into an image model training session.

For example:

  1. Fetch candidate images from the Sankaku database.
  2. Generate natural language captions using this adapter.
  3. Use the resulting image-caption pairs to train a LoRA.
  4. Repeat the process to continuously expand the training dataset.

This could reduce the need to prepare and manually caption large datasets in advance.

For LoRA training on GPUs with 16 GB of VRAM, MOSS Image is one of the first model families I would like to explore for this approach.

Limitations and future development

The current adapter is primarily optimized for concise natural language captioning.

Because the training dataset was generated through a caption-and-summarization process, the resulting descriptions can lack some of the detail and depth that would be desirable for more demanding image understanding and image-generation training tasks.

My next step is to manually edit and improve the dataset, focusing on more detailed and informative descriptions.

The intention is to move beyond simply removing unnecessary verbosity and towards genuinely richer natural language image descriptions.

Development roadmap

  • Collect 2,000 images from the Sankaku Complex Database.
  • Generate initial natural language captions.
  • Summarize the original model responses.
  • Train the first captioning adapter.
  • Manually edit the dataset to improve caption depth and detail.
  • Retrain the adapter using the improved descriptions.
  • Explore on-the-fly caption generation for LoRA training.

Next update expected: November 20, 2026.

Project goals

The long-term goal is to create a compact and efficient natural language captioning model that makes it practical to generate image-text training datasets dynamically.

Combining a fast captioning model with large image databases such as Sankaku could make it possible to generate training material on demand, rather than relying exclusively on manually prepared datasets.

This is an initial experiment, and I plan to continue improving the quality and depth of the generated captions.

Downloads last month
157
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for codeShare/qwen35_image_captioning_adapter

Datasets used to train codeShare/qwen35_image_captioning_adapter