Instructions to use codeShare/qwen35_image_captioning_adapter with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use codeShare/qwen35_image_captioning_adapter with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("DreamFast/Qwen3.5-4B-Uncensored-HauhauCS-Aggressive-Safetensor-Benchmark") model = PeftModel.from_pretrained(base_model, "codeShare/qwen35_image_captioning_adapter") - Notebooks
- Google Colab
- Kaggle
Qwen 3.5 Natural Language Image Captioning Adapter
This is an adapter model for the Qwen 3.5 4B Abliterated model DreamFast/Qwen3.5-4B-Uncensored-HauhauCS-Aggressive-Safetensor-Benchmark .
The adapter is trained on image captioning using natural language output.
When using this adapter on the Qwen 4B model , we can do batches of 30 captions at a time on a single T4 , roughly 7 seconds per caption.
Its trained on my own prompts , natural lanugage captions of images from sankaku complex using a mix of gemini flash and joycaption beta , and prompts from (NSFW) posts on civitai The main goal of this project is to create a fast, compact natural language captioning model that can be used to generate training captions for image-to-text and image-generation models.
This is the current dataset distribution
Testing the adapter
You can test the adapter using the Google Colab notebook provided in this repository:
qwen35_sankaku_captioning.ipynb
The notebook provides an interface for testing image caption generation using the adapter.
Training
The adapter was trained on 2,000 images using:
| Parameter | Details |
|---|---|
| Base model | DreamFast/Qwen3.5-4B-Uncensored-HauhauCS-Aggressive-Safetensor-Benchmark |
| Training images | 10,000 |
| Dataset source | Sankaku Complex Database |
| Training hardware | 2 × NVIDIA T4 GPUs (Kaggle) |
| Training duration | 24 hours |
| Training steps | 13000 |
Training methodology
The dataset preparation involved an initial captioning and summarization stage.
During the first training run, the base Qwen 3.5 model was used to caption the images. Its generated descriptions were then summarized by the model itself.
The adapter was subsequently trained on these summaries rather than the original, longer model responses.
This approach was intended to remove unnecessary preambles and verbose introductory text from the captioning output, producing more direct and compact natural language descriptions.
An important distinction: this process does not add new visual understanding or knowledge to the underlying model. It primarily teaches the model to produce a more concise version of its existing captioning capabilities.
Performance
Inference speed is one of the most promising aspects of this adapter.
On an NVIDIA T4 (16 GB), the model can process approximately 40 image items in a batch, with an average caption processing time of approximately 6–7 seconds per image.
These are observed performance figures and may vary depending on image resolution, generation settings, software configuration and available GPU memory.
The combination of relatively fast inference and access to the Sankaku database opens up interesting possibilities for on-the-fly dataset preparation.
Potential applications
One particularly interesting possibility is integrating natural language caption generation directly into an image model training session.
For example:
- Fetch candidate images from the Sankaku database.
- Generate natural language captions using this adapter.
- Use the resulting image-caption pairs to train a LoRA.
- Repeat the process to continuously expand the training dataset.
This could reduce the need to prepare and manually caption large datasets in advance.
For LoRA training on GPUs with 16 GB of VRAM, MOSS Image is one of the first model families I would like to explore for this approach.
Limitations and future development
The current adapter is primarily optimized for concise natural language captioning.
Because the training dataset was generated through a caption-and-summarization process, the resulting descriptions can lack some of the detail and depth that would be desirable for more demanding image understanding and image-generation training tasks.
My next step is to manually edit and improve the dataset, focusing on more detailed and informative descriptions.
The intention is to move beyond simply removing unnecessary verbosity and towards genuinely richer natural language image descriptions.
Development roadmap
- Collect 2,000 images from the Sankaku Complex Database.
- Generate initial natural language captions.
- Summarize the original model responses.
- Train the first captioning adapter.
- Manually edit the dataset to improve caption depth and detail.
- Retrain the adapter using the improved descriptions.
- Explore on-the-fly caption generation for LoRA training.
Next update expected: November 20, 2026.
Project goals
The long-term goal is to create a compact and efficient natural language captioning model that makes it practical to generate image-text training datasets dynamically.
Combining a fast captioning model with large image databases such as Sankaku could make it possible to generate training material on demand, rather than relying exclusively on manually prepared datasets.
This is an initial experiment, and I plan to continue improving the quality and depth of the generated captions.
- Downloads last month
- 157
Model tree for codeShare/qwen35_image_captioning_adapter
Base model
Qwen/Qwen3.5-4B-Base