Hugging Face's logo

ibm-granite
/
granitelib-rag-r1.0

Granite Library
Safetensors
GGUF
English
Model card Files
xet
 Community
61

Models and configs for Ollama backend

#4
by kndtran - opened
base: refs/heads/main
←
from: refs/pr/4
Discussion Files changed
+630
-79
kndtran
•
edited Dec 31, 2025

For granite-common issue https://github.com/ibm-granite/granite-common/issues/94.

PR notes

  1. Conversion scripts for Ollama backend. Designed to run once to create the files in this PR.
    1. convert_io_yaml_files.py - rewrites the vLLM io.yaml for an Ollama backend
      1. The response_format value is remapped to a different location in the io.yaml for Ollama. No modification of the value is currently needed.
    2. convert_to_gguf.py - converts the .safetensors to .gguf format and writes the Modelfiles for each LoRA adapter
  2. Ollama model naming scheme on filesystem renamed from granite4:micro to granite4_micro to avoid issues on Windows systems.
  3. Simplify run_ollama.sh script to assume that repo already contains converted files.
feat: Add conversion scripts for Ollama backend. Simplify run Ollama script, move functionalities to conversion scripts.3459def4
feat: Add granite4:micro LoRA files.1fc300c9
fix: Remove absolute path in Modelfiles.2fff1a73
kndtran changed pull request status to open
fix: Change max_completion_tokens to max_tokens in io.yaml conversion script.79323231
kndtran
•
edited Dec 29, 2025

Please do not merge this branch yet. There are issues with some of the quantized LoRA adapters as reported in the granite-common issue above. I can't seem to "unpublish" this PR.
fix: change to new directory ordering scheme46b66915
chore: change yaml output without sorted keys, preserving original ordering7ba01477
fix: add more specific instructions for output fields2fbc7056
fix: add more specific output field to promptcc861fd3
feat: new response_body format, consistently adheres to given json structuref29ba53f
fix: remove .cache folder from find search919e3b4a
fix: remove .cache folder from find search1c0798a5
kndtran

This branch is ready to merge. The corresponding granite-common issue above will need to be updated to use the main branch once this PR is merged.
frreiss changed pull request status to merged

Sign up or log in to comment