Tentando fazer o elixir liberar VRAM quando não tiver usando modelo, to querendo migrar o escrivão pra elixir
Forwarded from Lunks
eu fiz uns testes e só matando o processo. Eu cheguei nisso aqui https://gist.github.com/lunks/fcceaeb13e57dc4e1711003c1907b809
Gist
Switching between models with NX/Bumblebee
Switching between models with NX/Bumblebee. GitHub Gist: instantly share code, notes, and snippets.
Achei que perdia ben mais precisão, seloco https://emschwartz.me/binary-vector-embeddings-are-so-cool/
Evan Schwartz
Binary vector embeddings are so cool
Vector embeddings by themselves are pretty neat. Binary quantized vector embeddings are extra impressive. In short, they can retain 95+% retrieval accuracy with 32x compression 🤯.
Tenho que testar isso com o ollama
https://github.com/Nutlope/llama-ocr/blob/13bc0fb2bd06171a950896ffa133dfdeef1f3e8f/src/index.ts#L36
https://github.com/Nutlope/llama-ocr/blob/13bc0fb2bd06171a950896ffa133dfdeef1f3e8f/src/index.ts#L36
GitHub
llama-ocr/src/index.ts at 13bc0fb2bd06171a950896ffa133dfdeef1f3e8f · Nutlope/llama-ocr
Document to Markdown OCR library with Llama 3.2 vision - Nutlope/llama-ocr
Um projetinho pra brincar FDS, se tiver como integrar com ollama sepa eu uso no dia a dia https://github.com/gregpr07/browser-use