Jeremmmyyyyy/Open-R1-QA

open_r1

This model is a fine-tuned version of open-r1/OpenR1-Qwen-7B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0084

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• distributed_type: multi-GPU
• num_devices: 16
• total_train_batch_size: 16
• total_eval_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 224
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
0.0811	0.196	49	0.0713
0.0502	0.392	98	0.0410
0.0364	0.588	147	0.0331
0.0289	0.784	196	0.0294
0.0276	0.98	245	0.0263
0.0255	1.176	294	0.0233
0.0217	1.3720	343	0.0208
0.0197	1.568	392	0.0178
0.0167	1.764	441	0.0155
0.0111	1.96	490	0.0136
0.0109	2.156	539	0.0124
0.0117	2.352	588	0.0109
0.0091	2.548	637	0.0097
0.0102	2.7440	686	0.0089
0.0086	2.94	735	0.0084

Framework versions

• Transformers 4.49.0
• Pytorch 2.6.0+cu124
• Datasets 3.3.2
• Tokenizers 0.21.0