https://www.dlology.com/blog/how-to-do-unsupervised-clustering-with-keras/ #tips #clustering

https://www.tensorflow.org/hub/tutorials/tf2_object_detection
Object Detection with TF2 explained #tips

https://towardsdatascience.com/10-tensorflow-tricks-every-ml-practitioner-must-know-96b860e53c1 #tips

https://github.com/qubvel/keras_telegram_callback
Telegram-bot callback for your Keras model
#keras #callback #tips #libs #code

https://landing.ai/tips-for-a-data-centric-ai-approach/ #tips Andrew Ng

#Tips Efficient Training Large Models on Multiple GPUs, Main Concepts (from https://huggingface.co/docs/transformers/perf_train_gpu_many):

DataParallel (DP) - the same setup is replicated multiple times, and each being fed a slice of the data. The processing is done in parallel and all setups are synchronized at the end of each training step.
TensorParallel (TP) - each tensor is split up into multiple chunks, so instead of having the whole tensor reside on a single gpu, each shard of the tensor resides on its designated gpu. During processing each shard gets processed separately and in parallel on different GPUs and the results are synced at the end of the step. This is what one may call horizontal parallelism, as the splitting happens on horizontal level.
PipelineParallel (PP) - the model is split up vertically (layer-level) across multiple GPUs, so that only one or several layers of the model are places on a single gpu. Each gpu processes in parallel different stages of the pipeline and working on a small chunk of the batch.
Zero Redundancy Optimizer (ZeRO) - Also performs sharding of the tensors somewhat similar to TP, except the whole tensor gets reconstructed in time for a forward or backward computation, therefore the model doesn’t need to be modified. It also supports various offloading techniques to compensate for limited GPU memory.
Sharded DDP - is another name for the foundational ZeRO concept as used by various other implementations of ZeRO.

#Frameworks :
https://www.deepspeed.ai/
https://fairscale.readthedocs.io/en/latest/
https://github.com/tunib-ai/oslo
https://github.com/microsoft/varuna

https://encord.com/blog/dimentionality-reduction-techniques-machine-learning/
Dimensionality reduction techniques in one place #FYI #Tips

Dino/Dino v2 explained: Self-distillation with no labels & etc. #FYI #Tips #Explained #Tutorial
1. https://medium.com/@anuj.dutt9/emerging-properties-in-self-supervised-vision-transformers-dino-paper-summary-4c7a6ed68161 Original Dino
2. https://encord.com/blog/dinov2-self-supervised-learning-explained/
3. https://www.picsellia.com/post/dinov2-steps-by-steps-explanations-picsellia
4. https://www.ai-bites.net/dino-v2-learning-robust-visual-features-without-supervision-model-explained/
5. https://blog.marvik.ai/2023/05/16/dinov2-exploring-self-supervised-vision-transformers/

Original papers:
1. https://arxiv.org/abs/2104.14294 Emerging Properties in Self-Supervised Vision Transformers (Dino)
2. https://arxiv.org/abs/2304.07193 DINOv2: Learning Robust Visual Features without Supervision
3. https://arxiv.org/abs/2309.16588 Vision Transformers Need Registers

https://arxiv.org/pdf/2411.17525
https://huggingface.co/docs/transformers/main/en/quantization/higgs
https://github.com/HanGuo97/flute

Large Language Model Quantization
#Frameworks #Paper #Tips