AI, Python, Cognitive Neuroscience
3.88K subscribers
1.09K photos
47 videos
78 files
893 links
Download Telegram
💡 What's the difference between bagging and boosting?

Bagging and boosting are both ensemble methods, meaning they combine many weak predictors to create a strong predictor.

One key difference is that bagging builds independent models in parallel and "averages" their results in the end, whereas boosting builds models sequentially, at each step emphasizing reducing error that remains in the model by better fitting to the observations that were missed in previous steps.

❇️ @AI_Python_EN
Pre-Debate Material

“Yoshua Bengio, Revered Architect of AI, Has Some Ideas About What to Build Next”

The Turing Award winner wants AI systems that can reason, plan, and imagine

https://spectrum.ieee.org/tech-talk/robotics/artificial-intelligence/yoshua-bengio-revered-architect-of-ai-has-some-ideas-about-what-to-build-next

❇️ @AI_Python_EN
Machine Learning in a company is 10% Data Science & 90% other challenges It's VERY hard. Everything in this guide is ON POINT, and it's stuff you won't learn in an ML book "Best Practices of ML Engineering" This is a lifesaver.
project:
http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
Very interesting use of #AI to tackle bias in the written text by substituting words automatically to more neutral wording. However, one must also consider the challenges and ramifications such technology could mean to the written language as it can not only accidentally change the meaning of what was written, it can also change the tone and expression of the author and neutralize the point-of-view and remove emotion from language.
#NLP
https://arxiv.org/pdf/1911.09709.pdf

❇️ @AI_Python_EN
"If the future can be different from the past and you don't have deep understanding, you should not rely on AI." - a rule from Ray Dalio for when to leverage machine learning for decision-making.

Full conversation:

❇️ @AI_Python_EN
Evolutionary Powell's method is a discrete optimization algorithm I've found useful for hyperparameter tuning.

It makes weaker assumptions than Bayesian methods (and so is more robust), but stronger than random exploration (and so has better performance). It fills in the gap between then a bit.

Here's the full post on how Evolutionary Powell's method works:

We develop it as part of End-to-End Machine Learning Course 314:

The open source Ponderosa optimization package where it lives:

The line-by-line code walkthrough:

❇️ @AI_Python_EN
The Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), and the Deviance Information Criterion (DIC) are perhaps the most widely-used information criteria (IC) in model building and selection. A fourth, Minimum Description Length (MDL), is closely related to the BIC. In a nutshell, they provide guidance as which alternative model provides the most "bang for buck," i.e., the best fit after penalizing for model complexity. Penalizing for complexity is important since, given candidate models of similar predictive or explanatory power, the simplest model is most likely to be the best choice. In line with Occam's razor, complex models sometimes perform poorly on data not used in the model building. There are several others, including AIC3, SABIC, and CAIC, and no clear consensus among authorities as far as I am aware as to which is "best" overall. IC will not necessarily agree on which model should be chosen. Cross-validation, Predicted Residual Error Sum of Squares (PRESS) statistic, a kind of cross-validation, and Mallows’ Cp are also used instead of IC. Information criteria are covered in varying levels in detail in most statistics textbooks and are the subject of numerous academic papers. I know of no single go-to source on this topic.

❇️ @AI_Python_EN
best to summarize the key trends I saw at NeurIPS 2019. The themes covered are:

1. Deconstructing the deep learning black box: many papers aim to understand the theories behind deep learning including convergence and generalization. It also analyzes exciting and contentious approach is neural tangent kernel.

2. New approaches: Bayesian deep learning & uncertainty estimation, graph neural networks, and convex optimization.
3. Neuroscience in machine learning: consciousness and attention And more.

https://huyenchip.com/2019/12/18/key-trends-neurips-2019.html

❇️ @AI_Python_EN