Phi 3.5 vision + HQQ (4-bits), weights are in bfloat16
vRAM usage decreased: 8.9 GB -> 3.8 GB
https://colab.research.google.com/drive/176YsmMtdy-o0g19HVUPx7gKZ4OQ2tdxl?usp=sharing
#ai #cv #quantization
vRAM usage decreased: 8.9 GB -> 3.8 GB
https://colab.research.google.com/drive/176YsmMtdy-o0g19HVUPx7gKZ4OQ2tdxl?usp=sharing
#ai #cv #quantization
Google
OCR images using Phi-3.5 vision + HQQ (default backend).ipynb
Colab notebook
doing something
Phi 3.5 vision + HQQ (4-bits), weights are in bfloat16 vRAM usage decreased: 8.9 GB -> 3.8 GB https://colab.research.google.com/drive/176YsmMtdy-o0g19HVUPx7gKZ4OQ2tdxl?usp=sharing #ai #cv #quantization
It is possible to quantize this model as the following:
from transformers import AutoModelForCausalLM, HqqConfig
quant_config = HqqConfig(nbits=4, group_size=64)
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="cuda",
torch_dtype=torch.bfloat16,
trust_remote_code=True,
attn_implementation=attn_implementation,
quantization_config=quant_config
)
https://huggingface.co/apple/DepthPro
In this demo I've used Apple's model to extract an object (a man) with transparent background
Also, attached the code you can use to do the same
#ai #cv
In this demo I've used Apple's model to extract an object (a man) with transparent background
Also, attached the code you can use to do the same
#ai #cv
huggingface.co
apple/DepthPro · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
doing something
https://numfocus.org
Read their report, they have interesting projects in the pool. For instance, this one - https://docs.xarray.dev
xarray
Xarray documentation
Xarray makes working with labelled multi-dimensional arrays in Python simple, efficient, and fun! Version: 2026.4.0- What’s New Useful links: Home| Code Repository| Issues| Discussions| Releases| T...
doing something
https://github.com/psqlpy-python/psqlpy #rust
Used this project in https://github.com/egorsmkv/taskiq-pgsql-rabbitmq where Taskiq works as a good replacement of Celery
#python #backend
#python #backend
GitHub
GitHub - egorsmkv/taskiq-pgsql-rabbitmq: Taskiq: PostgreSQL (store results) and RabbitMQ (broker)
Taskiq: PostgreSQL (store results) and RabbitMQ (broker) - egorsmkv/taskiq-pgsql-rabbitmq
doing something
Used this project in https://github.com/egorsmkv/taskiq-pgsql-rabbitmq where Taskiq works as a good replacement of Celery #python #backend
Some improvements can be applied to this integration
For example, here, we can store not only a pickle object but full information about a task like the unpickled object (as shown on the picture)
For example, here, we can store not only a pickle object but full information about a task like the unpickled object (as shown on the picture)