AutoTraderX Financial Advisor (QLoRA)

Model Description

AutoTraderX Financial Advisor is a QLoRA fine-tuned adapter built on top of unsloth/Llama-3.2-3B-Instruct.

The model is optimized for:

  • Financial advisory conversations
  • Market analysis generation
  • Investment reasoning
  • Structured financial insights

This repository contains LoRA adapter weights, not the full base model.

  • Developed by: Swayam (Hugging Face: Swayam132)
  • Model type: Causal Language Model with LoRA adapters
  • Language: English
  • Base model: unsloth/Llama-3.2-3B-Instruct
  • Training method: QLoRA (4-bit quantized base + PEFT)

Intended Use

Direct Use

  • Financial advisory chatbots
  • AI-powered investment dashboards
  • Market commentary generation
  • Financial education tools

Out-of-Scope Use

  • Licensed financial advisory replacement
  • Legal, tax, or regulatory compliance advice
  • Automated trading without human validation
  • High-stakes financial decision-making

โš ๏ธ Important Disclaimer

This model is an AI language model fine-tuned on financial text data. It:

  • Does NOT have real-time market access.
  • Does NOT provide certified financial advice.
  • May generate inaccurate, outdated, or speculative information.
  • Should NOT be used as a sole basis for investment decisions.

Users must independently verify financial information and consult licensed financial professionals before making investment decisions.

The developer assumes no responsibility for financial losses resulting from use of this model.


Training Data

  • Dataset size: 500 instruction-response samples
  • Format: Instruction + optional input + response
  • Domain: Financial advisory and market reasoning
  • Data type: Custom curated dataset

Training Procedure

Fine-tuned using QLoRA with PEFT.

Base Model Configuration

  • Base model: unsloth/Llama-3.2-3B-Instruct
  • Quantization: 4-bit (QLoRA)
  • Precision: bf16 mixed precision

LoRA Parameters

Parameter Value
LoRA Rank (r) 16
LoRA Alpha 32
LoRA Dropout 0.05
Target Modules q_proj, k_proj, v_proj, o_proj (attention layers)
Bias none
Task Type CAUSAL_LM

Training Hyperparameters

Parameter Value
Epochs 3
Batch Size 4
Learning Rate 2e-4
Gradient Accumulation 4
Max Sequence Length 1024
Optimizer AdamW
Training Steps ~93

Evaluation

Evaluation Method

Evaluation was conducted qualitatively and quantitatively on a held-out subset of financial prompts.

Metrics

Since the task is generative and advisory in nature, traditional classification metrics are not fully applicable. The following evaluation approaches were used:

  • Perplexity (approximate): Reduced compared to base model on financial prompts
  • Human Evaluation:
    • Coherence: High
    • Relevance: High
    • Financial reasoning structure: Improved vs base model
  • Instruction-following accuracy: Improved relative to base Llama-3.2-3B-Instruct baseline

Note: No formal benchmark dataset (e.g., MMLU, GSM8K) was used for scoring.


Limitations

  • Small training dataset (500 samples)
  • Domain-specific fine-tuning may reduce general knowledge performance
  • No real-time data integration
  • Financial predictions may be speculative

How to Use

from transformers import pipeline

fin_advisor = pipeline(
    "text-generation",
    model="Swayam132/autotraderx-financial-advisor",
    torch_dtype="auto",
    device_map="auto"
)

instruction = "What is your view on long-term investment in Bitcoin?"
prompt = f"### Instruction:\n{instruction}\n\n### Response:\n"

outputs = fin_advisor(
    prompt,
    max_new_tokens=300,
    temperature=0.7,
    top_k=50,
    top_p=0.95,
    repetition_penalty=1.15,
)

response = outputs[0]["generated_text"].split("### Response:\n")[-1].strip()
print(response)
Downloads last month
205
Safetensors
Model size
3B params
Tensor type
F16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Swayam132/autotraderx-qlora-adapter

Finetuned
(613)
this model