using model on CPU
#10
by mooncy - opened
Hi, may I ask if this model can be run on a CPU instead of a GPU? Any guidance would be greatly appreciated. Thank you!
Hi.
Sortformer is developed by NVIDIA NeMo team and we are not currently optimizing on any CPU models.
Having said that, this does not mean that Sortformer cannot be running on CPU - you can optimize and run it on CPU.
Thank you for showing your interest on this model!
FYI: I exported the model as ONNX, and it works really nice on the CPU!
see parakeet + sortformer example (it's Rust):
https://github.com/altunenes/parakeet-rs/blob/master/examples/diarization.rs
Note, I also wanted to export v2 models (streaming models), but failed currently :-(
https://github.com/NVIDIA-NeMo/NeMo/issues/15077