When looking at WorldBank, WEF and some consulting company reports and white-papers I always wondered if anybody reads them.

Here is a possible answer - No
https://img.washingtonpost.com/blogs/wonkblog/files/2014/05/pdfs.jpg

They do not understand that making content more reachable and SEO-friendly helps long-term. But SEO-friendly websites are usually full of bullshit.

#internet

New dimensionality reduction technique - UMAP
- https://github.com/lmcinnes/umap

I will write more as I test it / learn more.

Works well with HDBSCAN and CNNs I guess
- https://goo.gl/9hYAXL

Usage examples
- https://goo.gl/QuYWJF

#data_science

Best link about convolution arithmetic
-https://github.com/vdumoulin/conv_arithmetic/blob/master/README.md

#deep_learning

What is amazing about tf and CUDA / CUDNN drivers - that documentation is not updated when newer versions are released - and they are always changing library file names which is annoying af.

Arguably Google and Nvidia are the richest companies from the whole DS stack - but their documentations is the worst of all the richest companies.

So if you are updating your docker container and libraries suddenly start producing weird errors - look for compatibility guidelines like this one - https://goo.gl/cF3Swy

Of course docs and release note will have no mention of this. Because Google.

Also docker hub contains all the versions of CUDA+CUDDNN packaged, which helps
- https://hub.docker.com/r/nvidia/cuda/

PS
Pytorch has all this embedded into their official repo list
- http://prntscr.com/i6nfsl

Google, why do you make us suffer?

#deep_learning

Dockerfile update for CUDA9 - CUDNN7:
- https://goo.gl/JwUXN5

Hello world in PyTorch and tensorflow seem to be working.

#data_science
#deep_learning

Classic / basic CNN papers

Aggregated Residual Transformations for Deep Neural Networks (ResNeXt)

- Authors Xie Saining / Girshick Ross / Dollár Piotr / Tu Zhuowen / He Kaiming
- Link http://arxiv.org/abs/1611.05431
- Resnet and VGG go deeper
- Inception nets go wider. Despite efficiency - they are hard to re-purpose and design
- key idea - add group convolutions to the residual block
- illustrations
-- basic building block https://goo.gl/L8PjUF
-- same block in terms of group convolutions https://goo.gl/fZKmgf
-- overall architecture https://goo.gl/WWSxRv
-- performance - https://goo.gl/vgLN8G - +1% vs resnet

#data_science
#deep_learning

Nice example of group convolutions via pytorch
- https://goo.gl/4d3JQC

#deep_learning
#pytorch

Some nice boilerplate on neural style transfer
- https://medium.com/artists-and-machine-intelligence/neural-artistic-style-transfer-a-comprehensive-look-f54d8649c199

#deep_learning

Simple Keras + web service deploy guidelines from FChollet + PyImageSearch
- https://www.pyimagesearch.com/wp-content/uploads/2018/01/keras_api_header.png
- https://blog.keras.io/building-a-simple-keras-deep-learning-rest-api.html?__s=jzpzanwy9jmh18omiik2
- https://www.pyimagesearch.com/2018/01/29/scalable-keras-deep-learning-rest-api/

Also an engineer guy from our team told me that this architecture sucks on high loads because redis will require object serialization, which takes a lot of time for images. Native python process management works better.


#data_science
#deep_learning

This is just EPIC
- https://habrahabr.ru/post/347760/

#internet

Some advice on using UMAP algorithm properly from the author
- https://github.com/lmcinnes/umap/issues/37#issuecomment-360807602

#data_science

Internet digest
- Ben Evans - https://goo.gl/XYKbvr
- RNNs + band names - https://goo.gl/LBBEiP
- Soldiers + fitness trackers = military bases - https://goo.gl/B4yzxX
- Google's new unit - security and ML - https://goo.gl/q1Xnjd
- Apple produces TV content - https://goo.gl/P2X9Gb
- Some bs rumours about Telegram ICO size - https://goo.gl/D4XgPD
- Twitter is plagued by bot-farms - https://goo.gl/ZLHVz1
-- Easy to detect via similar registration dates - https://goo.gl/ZLHVz1
- Podcast about financial innovations in the US - https://goo.gl/kxHUQY

#digest
#internet

Interesting intutions to understand the mean-shift algorithm
-https://spin.atomicobject.com/2015/05/26/mean-shift-clustering/
- https://goo.gl/QANV5e

Downside - sklearn implementation is slow, you will have to write your own GPU implementation.

#data_science

ML in action

Cyclical Learning rates are not merged in Pytorch yet, but they are in the PR stage
- https://github.com/pytorch/pytorch/pull/2016/files

#data_science
#pytorch

2017 DS/ML digest 2

Libraries
- One more RL library (last year saw 1 or 2) http://ray.readthedocs.io/en/latest/rllib.html
- Speech recognition from facebook - https://github.com/facebookresearch/wav2letter
- Even better speech generation than WaveNet - https://goo.gl/mTwyoV - I cannot tell computer apart

Industry (overdue news)
- Nvidia does not like it's consumer GPUs deployed in data centers https://goo.gl/n8mkxk
- Clarifai kills forevery https://goo.gl/PxcjvT
- Google search and gorillas vs. black people - https://goo.gl/t6LwLN

Blog posts
- Baidu - dataset size vs. accuracy https://goo.gl/j6M5ZP (log-scale)
-- https://goo.gl/AYan3f
-- https://goo.gl/JyVNHG

Datasets
- New Youtube actions dataset - https://arxiv.org/abs/1801.03150
-- http://arxiv.org/abs/1801.03150

Papers - current topic - meta learning / CNN optimization and tricks
- Systematic evaluation of CNN advances on the ImageNet https://arxiv.org/abs/1606.02228
-- http://prntscr.com/i8il35
- TRAINING DEEP NEURAL NETWORKS ON NOISY LABELS WITH BOOTSTRAPPING http://arxiv.org/abs/1412.6596
-- http://prntscr.com/i8iq1p
- Cyclical Learning Rates for Training Neural Networks http://arxiv.org/abs/1506.01186
-- http://prntscr.com/i8iqjx
- SEARCHING FOR ACTIVATION FUNCTIONS - http://arxiv.org/abs/1710.05941
-- http://prntscr.com/i8l0sd
-- http://prntscr.com/i8l5dp
- Large batch => train Imagenet in 15 mins
-- http://arxiv.org/abs/1711.04325
- Practical analysis of CNNs
-- http://arxiv.org/abs/1605.07678

#digest
#data_science
#deep_learning