# Full VoxLingua107 dev evaluation after model integration

`espnet/geolid_vl107only_shared_trainable` classified **1526 / 1,609 utterances
correctly (94.84%)** across 33 languages.
This is a complete new inference run using `ESPnetLIDModel` at
`301372e4da311d2a3d9574a5a43b3037ae6e67c1`, completed at `2026-10-23T01:05:59.999050+00:00`.

All 1,609 predicted labels match the earlier full-dev evaluation at `ca777d59eb49519e3827e40cc34997f2123d6682`.

The measured checkpoint SHA-256 is `d77542f79e39f876336b650a70dbb980240b30494a67d75ddf1d10096a09619e`.
The model inputs came from this immutable
[HF revision](https://huggingface.co/espnet/geolid_vl107only_shared_trainable/tree/82c40013919a914cb3a77c237df1b0e67ece2f15).
The checkpoint and published model configuration have not been modified by
this evaluation. Only the local path to `languages.txt` was resolved for the
run; its published `./languages.txt` value works from the model directory.

## Reproduce inference

Install the ESPnet revision documented in the model card, and download this
model's `config.yaml`, `model.pth` and `languages.txt`. Run the following command
from that model directory. Prepare `data/voxlingua107-dev/wav.scp` as sorted
`<utterance-id> <absolute-waveform-path>` lines for all 1,609 dev utterances;
use the original full-length, mono, 16 kHz waveforms. Choose a fresh output
directory because the CLI can skip inference when a completion marker exists.

```bash
OMP_NUM_THREADS=4 MKL_NUM_THREADS=4 python -m espnet2.bin.lid_inference \
  --output_dir evaluation_output \
  --ngpu 1 --dtype float32 \
  --data_path_and_name_and_type data/voxlingua107-dev/wav.scp,speech,sound \
  --valid_batch_size 1 --num_workers 8 \
  --lid_train_config config.yaml --lid_model_file model.pth \
  --use_preprocessor false --fix_duration false \
  --extract_embd false --save_embd_per_utt false \
  --save_embd_avg_lang false --save_tsne_plot false \
  --checkpoint_interval 200
```

The output `evaluation_output/voxlingua107-dev_lids` contains the predicted
labels. Compare each utterance ID with the reference language in
[evaluation_predictions.jsonl](evaluation_predictions.jsonl). The reported
accuracy is `100 * correct / 1609`; no duration weighting or macro averaging
is used. The JSONL also records the predictions from this run and a correctness
flag for every utterance, so the score can be independently recomputed.

## Evidence and scope

- [Evaluation metadata](evaluation.json): counts, settings, per-language counts,
  environment, source revision and hashes.
- [All predictions and reference labels](evaluation_predictions.jsonl).
- [Complete captured inference log](evaluation.log): paths refer to the actual
  evaluation workspace; its output is retained unchanged.
- [Prediction differences from the previous run](evaluation_prediction_differences.json).
- [Previous evaluation metadata](evaluation_previous.json), preserved unchanged.
- [Validation history](validation.json): earlier strict-loading checks plus a
  separate record of this full evaluation through the unmodified CLI.

This run evaluates VoxLingua107 dev only. Historical paper results for other
test sets have not been remeasured here. The new full run uses the standard
loader without forcing strict loading; the earlier explicit strict-loading
check is a separate historical validation.