How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks?
https://arxiv.org/abs/1911.12360
This is interesting in context of
https://arxiv.org/abs/1801.04540 who trained underparametrized networks (ShuffleNet with fixed FC layer)
🔗 How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks?
A recent line of research on deep learning focuses on the extremely over-parameterized setting, and shows that when the network width is larger than a high degree polynomial of the training sample size $n$ and the inverse of the target accuracy $ε^{-1}$, deep neural networks learned by (stochastic) gradient descent enjoy nice optimization and generalization guarantees. Very recently, it is shown that under certain margin assumption on the training data, a polylogarithmic width condition suffices for two-layer ReLU networks to converge and generalize (Ji and Telgarsky, 2019). However, how much over-parameterization is sufficient to guarantee optimization and generalization for deep neural networks still remains an open question. In this work, we establish sharp optimization and generalization guarantees for deep ReLU networks. Under various assumptions made in previous work, our optimization and generalization guarantees hold with network width polylogarithmic in $n$ and $ε^{-1}$. Our results push the study of over-parameterized deep neural networks towards more practical settings.
https://arxiv.org/abs/1911.12360
This is interesting in context of
https://arxiv.org/abs/1801.04540 who trained underparametrized networks (ShuffleNet with fixed FC layer)
🔗 How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks?
A recent line of research on deep learning focuses on the extremely over-parameterized setting, and shows that when the network width is larger than a high degree polynomial of the training sample size $n$ and the inverse of the target accuracy $ε^{-1}$, deep neural networks learned by (stochastic) gradient descent enjoy nice optimization and generalization guarantees. Very recently, it is shown that under certain margin assumption on the training data, a polylogarithmic width condition suffices for two-layer ReLU networks to converge and generalize (Ji and Telgarsky, 2019). However, how much over-parameterization is sufficient to guarantee optimization and generalization for deep neural networks still remains an open question. In this work, we establish sharp optimization and generalization guarantees for deep ReLU networks. Under various assumptions made in previous work, our optimization and generalization guarantees hold with network width polylogarithmic in $n$ and $ε^{-1}$. Our results push the study of over-parameterized deep neural networks towards more practical settings.
Kaggle Open Images 2019 — Артур Кузин
🔗 Kaggle Open Images 2019 — Артур Кузин
Artur Kuzin tells about his participation in Kaggle Open Images 2019 in English. He got a gold medal in each of the three competitions. In this video you will find out: - Description of the dataset and its markup procedures, as well as a description of the metric and its features - Architecture overview of the best models - Overview of tricks and hacks from the top3 of each competition - Approach for quick model training Presentation - https://gh.mltrainings.ru/presentations/Kuzin_KaggleOpenImages201
🔗 Kaggle Open Images 2019 — Артур Кузин
Artur Kuzin tells about his participation in Kaggle Open Images 2019 in English. He got a gold medal in each of the three competitions. In this video you will find out: - Description of the dataset and its markup procedures, as well as a description of the metric and its features - Architecture overview of the best models - Overview of tricks and hacks from the top3 of each competition - Approach for quick model training Presentation - https://gh.mltrainings.ru/presentations/Kuzin_KaggleOpenImages201
YouTube
Kaggle Open Images 2019 — Artur Kuzin
Artur Kuzin tells about his participation in Kaggle Open Images 2019 in English. He got a gold medal in each of the three competitions.
In this video you will find out:
- Description of the dataset and its markup procedures, as well as a description of…
In this video you will find out:
- Description of the dataset and its markup procedures, as well as a description of…
Practical Model Evaluation: What matters for your model? | Kaggle
🔗 Practical Model Evaluation: What matters for your model? | Kaggle
For the first day of the practical model evaluation workshop, we'll be talking about what's important to consider when deciding what model to put into production. Link to notebook: https://www.kaggle.com/rtatman/practical-model-evaluation-day-1 About Kaggle: Kaggle is the world's largest community of data scientists. Join us to compete, collaborate, learn, and do your data science work. Kaggle's platform is the fastest way to get started on a new data science project. Spin up a Jupyter notebook with a sin
🔗 Practical Model Evaluation: What matters for your model? | Kaggle
For the first day of the practical model evaluation workshop, we'll be talking about what's important to consider when deciding what model to put into production. Link to notebook: https://www.kaggle.com/rtatman/practical-model-evaluation-day-1 About Kaggle: Kaggle is the world's largest community of data scientists. Join us to compete, collaborate, learn, and do your data science work. Kaggle's platform is the fastest way to get started on a new data science project. Spin up a Jupyter notebook with a sin
YouTube
Practical Model Evaluation: What matters for your model? | Kaggle
For the first day of the practical model evaluation workshop, we'll be talking about what's important to consider when deciding what model to put into produc...
🎥 «Как визуализировать данные за две минуты, или Python для анализа данных». Павел Кнорр, DataArt.
👁 3 раз ⏳ 5357 сек.
👁 3 раз ⏳ 5357 сек.
Во время доклада Павел ответит на главный вопрос — почему Python стал языком № 1 в области Data Science! Чем вам могут помочь NumPy и Pandas, а главное — как визуализировать данные за две минуты.
ДОКЛАДЧИК: Павел Кнорр, Team Lead, DataArt.
Язык доклада: русский.
Язык презентации: английский.
00:02:20 — Почему Python.
00:07:20 — Что такое NumPy и как с этим работать.
00:22:49 — Pandas, или Excel для программистов. И как с этим работать поверх NumPy.
00:38:34 — Как это все использовать в реальных проектOn-demand last mile transportation
🔗 On-demand last mile transportation
Real-time route optimization with Location Intelligence
🔗 On-demand last mile transportation
Real-time route optimization with Location Intelligence
Medium
On-demand last mile transportation
Real-time route optimization with Location Intelligence
Why Kernelized Support Vector Machine (SVM) is MLs most beautiful Algorithm?
🔗 Why Kernelized Support Vector Machine (SVM) is MLs most beautiful Algorithm?
Machine learning has more than a few beautiful algorithms that are helping data scientists and researchers transform business models and…
🔗 Why Kernelized Support Vector Machine (SVM) is MLs most beautiful Algorithm?
Machine learning has more than a few beautiful algorithms that are helping data scientists and researchers transform business models and…
Medium
Why Kernelized Support Vector Machine (SVM) is MLs most beautiful Algorithm?
Machine learning has more than a few beautiful algorithms that are helping data scientists and researchers transform business models and…
Recreating Van Gogh’s lost painting
🔗 Recreating Van Gogh’s lost painting
Using generative models in the hunt for a masterpiece
🔗 Recreating Van Gogh’s lost painting
Using generative models in the hunt for a masterpiece
Medium
Recreating Van Gogh’s lost painting
Using generative models in the hunt for a masterpiece
🎥 Running Effective Machine Learning Teams: Common Issues, Challenges, and Solutions
👁 1 раз ⏳ 1152 сек.
👁 1 раз ⏳ 1152 сек.
Gideon Mendels, CEO, Co-Founder, Comet.ml
'Systems for AI Track'
AI Week
Yuval Ne'eman Workshop for Science, Technology and Security
Tel Aviv University
19.11.19Vk
Running Effective Machine Learning Teams: Common Issues, Challenges, and Solutions
Gideon Mendels, CEO, Co-Founder, Comet.ml
'Systems for AI Track'
AI Week
Yuval Ne'eman Workshop for Science, Technology and Security
Tel Aviv University
19.11.19
'Systems for AI Track'
AI Week
Yuval Ne'eman Workshop for Science, Technology and Security
Tel Aviv University
19.11.19
Подводные камни в управлении Machine Learning проектом
Уже полтора года я занимаю у себя в компании позицию основного ML-разработчика. Полгода управляю небольшой командой. Я накопил опыт, которым хочу поделиться. Делать это буду в формате топа заблуждений и потенциальных трудностей.
🔗 Подводные камни в управлении Machine Learning проектом
Уже полтора года я занимаю у себя в компании позицию основного ML-разработчика. Полгода управляю небольшой командой. Я накопил опыт, которым хочу поделиться. Д...
Уже полтора года я занимаю у себя в компании позицию основного ML-разработчика. Полгода управляю небольшой командой. Я накопил опыт, которым хочу поделиться. Делать это буду в формате топа заблуждений и потенциальных трудностей.
🔗 Подводные камни в управлении Machine Learning проектом
Уже полтора года я занимаю у себя в компании позицию основного ML-разработчика. Полгода управляю небольшой командой. Я накопил опыт, которым хочу поделиться. Д...
Хабр
Подводные камни в управлении Machine Learning проектом
Уже полтора года я занимаю у себя в компании позицию основного ML-разработчика. Полгода управляю небольшой командой. Я накопил опыт, которым хочу поделиться. Делать это буду в формате топа...
Practical Machine Learning with Python
Наш телеграм канал - tglink.me/ai_machinelearning_big_data
📝 Practical Machine Learning with Python (en).pdf - 💾20 333 817
Наш телеграм канал - tglink.me/ai_machinelearning_big_data
📝 Practical Machine Learning with Python (en).pdf - 💾20 333 817
Ridge Regression, Memory vs Understanding & Ice Cream!
🔗 Ridge Regression, Memory vs Understanding & Ice Cream!
Understanding where Ridge sits in the ecosystem of Linear Methods for Regression
🔗 Ridge Regression, Memory vs Understanding & Ice Cream!
Understanding where Ridge sits in the ecosystem of Linear Methods for Regression
Medium
Ridge Regression, Memory vs Understanding & Ice Cream!
Understanding where Ridge sits in the ecosystem of Linear Methods for Regression
Latest from Google brain researchers
https://www.profillic.com/paper/arxiv:1912.01603
🔗 Dream to Control: Learning Behaviors by Latent Imagination - Profillic
Explore state-of-the-art in machine learning, AI, and robotics. Browse models, source code, papers by topics and authors. Connect with researchers and engineers working on related problems in machine learning, deep learning, natural language processing, robotics, computer vision, data mining, neural networks, artificial intelligence/AI, data science... and explore working together on projects, github code
https://www.profillic.com/paper/arxiv:1912.01603
🔗 Dream to Control: Learning Behaviors by Latent Imagination - Profillic
Explore state-of-the-art in machine learning, AI, and robotics. Browse models, source code, papers by topics and authors. Connect with researchers and engineers working on related problems in machine learning, deep learning, natural language processing, robotics, computer vision, data mining, neural networks, artificial intelligence/AI, data science... and explore working together on projects, github code
Profillic
Dream to Control: Learning Behaviors by Latent Imagination - Profillic
Explore state-of-the-art in machine learning, AI, and robotics. Browse models, source code, papers by topics and authors. Connect with researchers and engineers working on related problems in machine learning, deep learning, natural language processing, robotics…
А/Б тестирование, пайплайн и ритейл: брендированная четверть по Big Data от GeekBrains и X5 Retail Group
Технологии Big Data применяются сейчас повсеместно — в промышленности, медицине, бизнесе, развлечениях. Так, без анализа больших данных не смогут нормально работать крупные ритейлеры, упадут продажи в Amazon, будут не в состоянии предсказывать погоду на многие дни, недели и месяцы наперед метеорологи. Логично, что специалисты по большим данным сейчас нарасхват, причем спрос постоянно растет.
🔗 А/Б тестирование, пайплайн и ритейл: брендированная четверть по Big Data от GeekBrains и X5 Retail Group
Технологии Big Data применяются сейчас повсеместно — в промышленности, медицине, бизнесе, развлечениях. Так, без анализа больших данных не смогут нормально раб...
Технологии Big Data применяются сейчас повсеместно — в промышленности, медицине, бизнесе, развлечениях. Так, без анализа больших данных не смогут нормально работать крупные ритейлеры, упадут продажи в Amazon, будут не в состоянии предсказывать погоду на многие дни, недели и месяцы наперед метеорологи. Логично, что специалисты по большим данным сейчас нарасхват, причем спрос постоянно растет.
🔗 А/Б тестирование, пайплайн и ритейл: брендированная четверть по Big Data от GeekBrains и X5 Retail Group
Технологии Big Data применяются сейчас повсеместно — в промышленности, медицине, бизнесе, развлечениях. Так, без анализа больших данных не смогут нормально раб...
Хабр
А/Б тестирование, пайплайн и ритейл: брендированная четверть по Big Data от GeekBrains и X5 Retail Group
Технологии Big Data применяются сейчас повсеместно — в промышленности, медицине, бизнесе, развлечениях. Так, без анализа больших данных не смогут нормально раб...
Исследование МФТИ по реконструкции изображения, которое видит человек, по ЭЭГ при помощи нейросети
https://techxplore.com/news/2019-10-neural-network-reconstructs-human-thoughts.html
https://www.youtube.com/watch?v=nf-P3b2AnZ
🔗 Neural network reconstructs human thoughts from brain waves in real time
Researchers from Russian corporation Neurobotics and the Moscow Institute of Physics and Technology have found a way to visualize a person's brain activity as actual images mimicking what they observe ...
https://techxplore.com/news/2019-10-neural-network-reconstructs-human-thoughts.html
https://www.youtube.com/watch?v=nf-P3b2AnZ
🔗 Neural network reconstructs human thoughts from brain waves in real time
Researchers from Russian corporation Neurobotics and the Moscow Institute of Physics and Technology have found a way to visualize a person's brain activity as actual images mimicking what they observe ...
Tech Xplore
Neural network reconstructs human thoughts from brain waves in real time
Researchers from Russian corporation Neurobotics and the Moscow Institute of Physics and Technology have found a way to visualize a person's brain activity as actual images mimicking what they observe ...
Дэвидсон-Пайлон К. - Вероятностное программирование на Python
Наш телеграм канал - tglink.me/ai_machinelearning_big_data
📝 Дэвидсон-Пайлон К. - Вероятностное программирование на Python (Библиотека программиста) - 2019.pdf - 💾9 841 979
Наш телеграм канал - tglink.me/ai_machinelearning_big_data
📝 Дэвидсон-Пайлон К. - Вероятностное программирование на Python (Библиотека программиста) - 2019.pdf - 💾9 841 979
Добрый вечер. Пытаюсь реализовать свою нейронную сеть с нуля без фреймворков. типа торч, тенсорфлов и керас, вроде бы так. Подскажите пожалуйста. Как правильно подбирать значения нейрона смещения? Есть ли какой способ?