Full VoxLingua107 dev evaluation after model integration

espnet/geolid_vl107only_shared_trainable classified 1526 / 1,609 utterances correctly (94.84%) across 33 languages. This is a complete new inference run using ESPnetLIDModel at 301372e4da311d2a3d9574a5a43b3037ae6e67c1, completed at 2026-10-23T01:05:59.999050+00:00.

All 1,609 predicted labels match the earlier full-dev evaluation at ca777d59eb49519e3827e40cc34997f2123d6682.

The measured checkpoint SHA-256 is d77542f79e39f876336b650a70dbb980240b30494a67d75ddf1d10096a09619e. The model inputs came from this immutable HF revision. The checkpoint and published model configuration have not been modified by this evaluation. Only the local path to languages.txt was resolved for the run; its published ./languages.txt value works from the model directory.

Reproduce inference

Install the ESPnet revision documented in the model card, and download this model's config.yaml, model.pth and languages.txt. Run the following command from that model directory. Prepare data/voxlingua107-dev/wav.scp as sorted <utterance-id> <absolute-waveform-path> lines for all 1,609 dev utterances; use the original full-length, mono, 16 kHz waveforms. Choose a fresh output directory because the CLI can skip inference when a completion marker exists.

OMP_NUM_THREADS=4 MKL_NUM_THREADS=4 python -m espnet2.bin.lid_inference \
  --output_dir evaluation_output \
  --ngpu 1 --dtype float32 \
  --data_path_and_name_and_type data/voxlingua107-dev/wav.scp,speech,sound \
  --valid_batch_size 1 --num_workers 8 \
  --lid_train_config config.yaml --lid_model_file model.pth \
  --use_preprocessor false --fix_duration false \
  --extract_embd false --save_embd_per_utt false \
  --save_embd_avg_lang false --save_tsne_plot false \
  --checkpoint_interval 200

The output evaluation_output/voxlingua107-dev_lids contains the predicted labels. Compare each utterance ID with the reference language in evaluation_predictions.jsonl. The reported accuracy is 100 * correct / 1609; no duration weighting or macro averaging is used. The JSONL also records the predictions from this run and a correctness flag for every utterance, so the score can be independently recomputed.

Evidence and scope

Evaluation metadata: counts, settings, per-language counts, environment, source revision and hashes.
All predictions and reference labels.
Complete captured inference log: paths refer to the actual evaluation workspace; its output is retained unchanged.
Prediction differences from the previous run.
Previous evaluation metadata, preserved unchanged.
Validation history: earlier strict-loading checks plus a separate record of this full evaluation through the unmodified CLI.

This run evaluates VoxLingua107 dev only. Historical paper results for other test sets have not been remeasured here. The new full run uses the standard loader without forcing strict loading; the earlier explicit strict-loading check is a separate historical validation.