Naija-Petro 8B

Naija-Petro 8B is a Qwen3-8B model fine-tuned (QLoRA, Unsloth) on ~20,000 synthetic petroleum-engineering instruction–response pairs. It is the lightweight, fast-inference variant of the Naija-Petro family and the model served behind the project's retrieval-augmented assistant.

⚠️ The base training data is general/global petroleum knowledge. For Nigeria-specific facts (regulation, the PIA 2021, NUPRC/NMDPRA/NNPC), pair this model with the Naija-Petro RAG system, which grounds answers in verifiable Nigerian sources.

Model details

  • Developed by: Naija-Petro project (Hugging Face: Shinzmann)
  • Model type: Decoder-only causal LM, instruction-tuned
  • Language: English
  • License: Apache-2.0 (inherited from Qwen3-8B)
  • Finetuned from: Qwen/Qwen3-8B
  • Domain: Petroleum engineering — drilling, reservoir, production, completions, EOR, well testing, petroleum geoscience

Model sources

Uses

Direct use

Technical question answering and explanation across petroleum-engineering subdomains: concepts, equations and derivations, workflow guidance, and terminology — as a study aid and engineering decision-support tool.

Downstream use

Backbone for retrieval-augmented assistants (see the project repo), further domain fine-tuning, or distillation.

Out-of-scope use

Not for autonomous operational, safety-critical, or financial decisions; not a substitute for licensed engineering judgment, official regulations, or field data. General-purpose chat is not its focus.

Bias, risks, and limitations

  • Trained largely on synthetic data generated from a scraped corpus; it can be confidently wrong ("hallucinate"), especially on numerical specifics and Nigeria-specific regulation/economics.
  • English only. May reflect biases of its base model and source literature.
  • Knowledge is static as of training; use the RAG layer for current/local facts.

Recommendation: Always validate outputs with qualified engineers and primary sources before any operational use.

How to get started

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Shinzmann/naija-petro-8b"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

SYSTEM = (
    "You are Naija-Petro, an expert petroleum-engineering AI assistant. Provide "
    "precise, technically accurate answers; include equations, units, and "
    "practical considerations."
)
messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": "Explain the material balance equation for an undersaturated reservoir."},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True,
                                 enable_thinking=False, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=512, temperature=0.4, top_p=0.9)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

Ollama (GGUF):

ollama run hf.co/Shinzmann/naija-petro-8b-GGUF:Q4_K_M

Training details

Data

~20,000 instruction–response pairs generated with NVIDIA NeMo Data Designer from a scraped, de-duplicated petroleum corpus (arXiv, Semantic Scholar, OpenAlex, Crossref, DOE/OSTI, PetroWiki, the SLB glossary, EIA, and more), with an LLM-as-judge quality-scoring pass. Pipeline and EDA are in the project notebooks.

Procedure

QLoRA (4-bit NF4) with Unsloth on a single A100 80GB.

Hyperparameter Value
Base model Qwen3-8B
Method QLoRA, 4-bit NF4
LoRA rank / alpha / dropout 32 / 64 / 0.0
Epochs 3
Effective batch size 64 (32 × 2 grad-accum)
Learning rate / scheduler 2e-4 / cosine, 5% warmup
Weight decay 0.01
Max sequence length 2048
Seed 42

Training results (Weights & Biases)

Trained for 3 epochs (669 optimiser steps) on a single A100 80GB.

Metric Value
Final training loss ≈ 0.83
Final validation loss ≈ 0.86
Epochs / steps 3 / 669

Evaluation

Training converged to a low validation loss (≈ 0.86; see Training results). A downstream 30-question internal benchmark across six subdomains (drilling, reservoir, production, completions, EOR, well testing), scored by an LLM-as-judge on technical accuracy, completeness, and terminology, was also run; the automated judge did not differentiate reliably in that pass, so a corrected evaluation is in progress and task-level scores are not yet reported. Qualitatively, the fine-tuned model produces well-structured, equation- and unit-aware answers in the target domain. Treat all outputs as expert-validated decision support.

Technical specifications

  • Architecture: Qwen3 (decoder-only transformer), 8B parameters
  • Adapter: LoRA merged into 16-bit weights; also distributed as GGUF quantizations
  • Software: Unsloth, 🤗 Transformers, PEFT, TRL, bitsandbytes
  • Hardware: 1× NVIDIA A100 80GB

Citation

@misc{naijapetro8b2025,
  title  = {Naija-Petro 8B: A Petroleum-Engineering Language Model},
  author = {Naija-Petro project},
  year   = {2025},
  url    = {https://huggingface.co/Shinzmann/naija-petro-8b}
}

Model card contact

Open an issue at https://github.com/Mystique1337/naija-petro.

Downloads last month
437
Safetensors
Model size
8B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Shinzmann/naija-petro-8b

Finetuned
Qwen/Qwen3-8B
Adapter
(1439)
this model
Quantizations
2 models

Space using Shinzmann/naija-petro-8b 1