TRIC-Trilingual Recognition of Irony with Confidence
Collection
This collections contains data and models used for the TRIC (Trilingual Recognition of Irony with Confidence) paper (under review) โข 17 items โข Updated
This model is a fine-tuned version of vinai/bertweet-base on an unknown dataset. It achieves the following results on the evaluation set:
This is the best-performing REGRESSION model for English irony detection. The model was fine-tuned both a mix of English and Dutch tweets. The model predicts one numerical value indicating irony likelihood, where 0 is not ironic and 6 is ironic.
More information needed
More information needed
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss | Mse | Rmse | Mae | R2 | F1 | Precision | Recall | Accuracy |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1.7333 | 0.4630 | 100 | 1.8475 | 9.7883 | 3.1286 | 2.2877 | -0.4094 | 0.4570 | 0.3669 | 0.6057 | 0.6057 |
| 1.6952 | 0.9259 | 200 | 1.7889 | 8.8708 | 2.9784 | 2.2442 | -0.2773 | 0.4570 | 0.3669 | 0.6057 | 0.6057 |
| 1.6175 | 1.3889 | 300 | 1.6295 | 7.6123 | 2.7590 | 2.0223 | -0.0961 | 0.4570 | 0.3669 | 0.6057 | 0.6057 |
| 1.4401 | 1.8519 | 400 | 1.4962 | 6.6368 | 2.5762 | 1.8601 | 0.0444 | 0.4570 | 0.3669 | 0.6057 | 0.6057 |
| 1.2553 | 2.3148 | 500 | 1.3949 | 5.9003 | 2.4291 | 1.7518 | 0.1504 | 0.4570 | 0.3669 | 0.6057 | 0.6057 |
| 1.2296 | 2.7778 | 600 | 1.3520 | 5.9339 | 2.4360 | 1.6730 | 0.1456 | 0.4570 | 0.3669 | 0.6057 | 0.6057 |
| 1.0909 | 3.2407 | 700 | 1.2565 | 5.3251 | 2.3076 | 1.5831 | 0.2332 | 0.4570 | 0.3669 | 0.6057 | 0.6057 |
| 1.0031 | 3.7037 | 800 | 1.2159 | 4.7598 | 2.1817 | 1.5709 | 0.3146 | 0.4570 | 0.3669 | 0.6057 | 0.6057 |
| 0.9833 | 4.1667 | 900 | 1.1544 | 4.6141 | 2.1480 | 1.5031 | 0.3356 | 0.7296 | 0.8018 | 0.7572 | 0.7572 |
| 0.825 | 4.6296 | 1000 | 1.1512 | 5.0019 | 2.2365 | 1.4608 | 0.2798 | 0.7757 | 0.7943 | 0.7859 | 0.7859 |
| 0.8187 | 5.0926 | 1100 | 1.1150 | 4.9111 | 2.2161 | 1.4352 | 0.2928 | 0.7815 | 0.7849 | 0.7859 | 0.7859 |
| 0.7138 | 5.5556 | 1200 | 1.0724 | 4.8492 | 2.2021 | 1.3871 | 0.3018 | 0.7766 | 0.7791 | 0.7807 | 0.7807 |
| 0.6706 | 6.0185 | 1300 | 1.0560 | 4.9024 | 2.2141 | 1.3650 | 0.2941 | 0.7786 | 0.7823 | 0.7833 | 0.7833 |
| 0.6112 | 6.4815 | 1400 | 1.0594 | 5.0772 | 2.2533 | 1.3694 | 0.2689 | 0.7750 | 0.7759 | 0.7781 | 0.7781 |
| 0.5906 | 6.9444 | 1500 | 1.0611 | 5.1421 | 2.2676 | 1.3794 | 0.2596 | 0.7736 | 0.7734 | 0.7755 | 0.7755 |
| 0.5597 | 7.4074 | 1600 | 1.0286 | 5.0419 | 2.2454 | 1.3290 | 0.2740 | 0.7839 | 0.7879 | 0.7885 | 0.7885 |
| 0.5422 | 7.8704 | 1700 | 1.0531 | 5.2061 | 2.2817 | 1.3596 | 0.2504 | 0.7672 | 0.7678 | 0.7702 | 0.7702 |
| 0.5255 | 8.3333 | 1800 | 1.0478 | 5.2565 | 2.2927 | 1.3372 | 0.2431 | 0.7811 | 0.7853 | 0.7859 | 0.7859 |
| 0.5116 | 8.7963 | 1900 | 1.0544 | 5.2090 | 2.2823 | 1.3546 | 0.2500 | 0.7721 | 0.7733 | 0.7755 | 0.7755 |
| 0.5213 | 9.2593 | 2000 | 1.0423 | 5.1715 | 2.2741 | 1.3341 | 0.2554 | 0.7819 | 0.7846 | 0.7859 | 0.7859 |
| 0.4999 | 9.7222 | 2100 | 1.0566 | 5.2819 | 2.2982 | 1.3481 | 0.2395 | 0.7721 | 0.7733 | 0.7755 | 0.7755 |
Base model
vinai/bertweet-base