Inferecing an audio file using onnxruntime with TensorRT
Audio duration: 02:37:47
Inference duration: ~4 minutes
Repo: https://github.com/egorsmkv/pyannote-onnx-rust
#ai #speech #rust
Audio duration: 02:37:47
Inference duration: ~4 minutes
Repo: https://github.com/egorsmkv/pyannote-onnx-rust
#ai #speech #rust
doing something
Large model (FP32) of RF-DETR has these speed and memory usage #cv #ai #rust #onnx
Also, tested it on A100 with TensorRT:
https://colab.research.google.com/drive/1-agoo5ll-hWEecWQAtO1FM39sqavJxph?usp=sharing
Results are not so obvious, but it works base_rfdetr_fp16.onnx model and gives ~10ms/img
https://colab.research.google.com/drive/1-agoo5ll-hWEecWQAtO1FM39sqavJxph?usp=sharing
Results are not so obvious, but it works base_rfdetr_fp16.onnx model and gives ~10ms/img
Google
RF-DETR + USLS on TensorRT (A100).ipynb
Colab notebook
Forwarded from Hacker News
One main problem in adapatation of ML models from Python to Rust is feature extraction.
I can run ONNX models, but some specific feature extractors can not be implemented in the Rust ecosystem.
Ways to implement them:
1) In the AI era we can fastly prototype them by feeding Python code into LLMs.
2) Load the entire Python interpreter into Rust like this https://peterprototypes.com/blog/huggingface-from-rust/
#rust #ml
I can run ONNX models, but some specific feature extractors can not be implemented in the Rust ecosystem.
Ways to implement them:
1) In the AI era we can fastly prototype them by feeding Python code into LLMs.
2) Load the entire Python interpreter into Rust like this https://peterprototypes.com/blog/huggingface-from-rust/
#rust #ml
Peterprototypes
🤗 Calling Hugging Face models from Rust | Peter Todorov 💻 Blog
This post describes a simple approach on calling Hugging Face ML models from a Rust codebase via Python interop
👍1
doing something
Idk why, FP16 execution is slower than FP32 shown on this picture
Speed on TensorRT even faster