https://elastiknn.com/ Elasticsearch Plugin for Nearest Neighbor Search on dense vectors
#Tools #Library

https://github.com/EthicalML/awesome-production-machine-learning #Library #Frameworks #Tools

https://arxiv.org/pdf/2103.14030.pdf #paper
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

https://shap.readthedocs.io/en/latest/index.html
SHAP (SHapley Additive exPlanations) is a game theoretic approach to explain the output of any machine learning model. It connects optimal credit allocation with local explanations using the classic Shapley values from game theory and their related extensions.
#Framework

https://www.datagym.ai/ #annotation #tools

https://docs.determined.ai/latest/index.html#
#Framework distributed training, hyperparameter tuning

https://www.determined.ai/blog/data-version-control-determined

https://keras.io/keras_tuner/
#Framework KerasTuner is an easy-to-use, scalable hyperparameter optimization framework that solves the pain points of hyperparameter search.

https://www.microsoft.com/en-us/research/project/document-ai/
Microsoft Document AI (Intelligent Document Processing) #Framework

https://www.linkedin.com/posts/smasis_machinelearning-math-datascience-activity-6951137542079467520-4Bb8/
grid search vs Bayesian Optimization for hyperparameter tuning

https://github.com/yzhao062/pyod outlier/anomaly detection - many methods in one library #framework #library

https://faker.readthedocs.io/en/stable/index.html
https://sdv.dev/
https://gretel.ai/synthetics

Synthetic Data Generators! #Frameworks #Library

https://medmnist.com/
MedMNIST: A Large-Scale Lightweight Benchmark for 2D and 3D Biomedical Image Classification
#Dataset

https://arxiv.org/pdf/2208.07339.pdf
https://huggingface.co/blog/hf-bitsandbytes-integration
#Performance

https://landing.ai/tips-for-a-data-centric-ai-approach/ #tips Andrew Ng

https://github.com/clovaai/donut #Frameworks #Model Document Understanding Transformer

High-Performance Large-Scale Image Recognition Without Normalization
https://arxiv.org/pdf/2102.06171.pdf #Paper

https://github.com/jsbroks/coco-annotator #Frameworks #Tool Image segmentation tool

https://tf-explain.readthedocs.io/en/latest/index.html
tf-explain offers interpretability methods for Tensorflow 2.0 to ease neural network’s understanding.
#Frameworks

#Tips Efficient Training Large Models on Multiple GPUs, Main Concepts (from https://huggingface.co/docs/transformers/perf_train_gpu_many):

DataParallel (DP) - the same setup is replicated multiple times, and each being fed a slice of the data. The processing is done in parallel and all setups are synchronized at the end of each training step.
TensorParallel (TP) - each tensor is split up into multiple chunks, so instead of having the whole tensor reside on a single gpu, each shard of the tensor resides on its designated gpu. During processing each shard gets processed separately and in parallel on different GPUs and the results are synced at the end of the step. This is what one may call horizontal parallelism, as the splitting happens on horizontal level.
PipelineParallel (PP) - the model is split up vertically (layer-level) across multiple GPUs, so that only one or several layers of the model are places on a single gpu. Each gpu processes in parallel different stages of the pipeline and working on a small chunk of the batch.
Zero Redundancy Optimizer (ZeRO) - Also performs sharding of the tensors somewhat similar to TP, except the whole tensor gets reconstructed in time for a forward or backward computation, therefore the model doesn’t need to be modified. It also supports various offloading techniques to compensate for limited GPU memory.
Sharded DDP - is another name for the foundational ZeRO concept as used by various other implementations of ZeRO.

#Frameworks :
https://www.deepspeed.ai/
https://fairscale.readthedocs.io/en/latest/
https://github.com/tunib-ai/oslo
https://github.com/microsoft/varuna

[2302.14045] Language Is Not All You Need: Aligning Perception with Language Models
https://arxiv.org/abs/2302.14045
#paper New generation