Automatic Speech Recognition
Transformers
PyTorch
JAX
Safetensors
whisper
audio
hf-asr-leaderboard
Eval Results
Instructions to use openai/whisper-large-v3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use openai/whisper-large-v3 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="openai/whisper-large-v3")# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq processor = AutoProcessor.from_pretrained("openai/whisper-large-v3") model = AutoModelForSpeechSeq2Seq.from_pretrained("openai/whisper-large-v3", device_map="auto") - Inference
- Notebooks
- Google Colab
- Kaggle
Multi task finetuning (transcribe and translate)
#174
by Phil-AB - opened
I want to finetune this model for multi-tasking (Transcribe and translate)
Essentially, I want to transcribe (Akan speech to Akan text) and translate (Akan speech to English text).
Individually, it works
but I want to use one model and finetune for both transcription and translation to English language.
My datasets contain Akan speech with its transcription and its translated English text.
Any help as to how I can go about this will be greatly appreciated.
Also is there a notebook with a multitask done that I can use for reference or any resource I could draw insights from?
Thank you.