Forwarded from Data Stitches
11月25号是消除对妇女的暴力行为国际日,来自metaLab的研究人员在随机选择一百万条#MeToo推文后,仔细阅读了转发次数超过 100 次的示例,在894 条推文中只有 8 条是关于性侵犯或围绕#MeToo主题的经历的实际推文,其余绝大多数是新闻媒体和政治讨论,其中大多数都忽略了#MeToo运动核心的具体问题和幸存者的声音,设计师Kim Albrecht想通过这个可视化项目来展示被忽视的针对女性暴力问题
Twitter
Kim Albrecht
Today is the International Day for the Elimination of Violence against Women. To shine some light on the enormity of the problem we visualized some unheard tweets from the #WhyIDidntReport hashtag. metoo.kimalbrecht.com #GenerationEquality #OrangeTheWorld…
#DS
Just in case you are also struggling with Python packages on Apple M1 Macs
I am using the third option: anaconda + miniforge.
https://www.anaconda.com/blog/apple-silicon-transition
Just in case you are also struggling with Python packages on Apple M1 Macs
I am using the third option: anaconda + miniforge.
https://www.anaconda.com/blog/apple-silicon-transition
Anaconda
Anaconda | A Python Data Scientist’s Guide to the Apple Silicon…
Even if you are not a Mac user, you have likely heard Apple is switching from Intel CPUs to their own custom CPUs, which they refer to collectively as "Apple Silicon." The last time Apple changed its computer architecture this dramatically was 15 years ago…
#visualization
Hmmm my plate is way off the planetary heath diet recommendation.
Source:
https://www.nature.com/articles/d41586-021-03612-1
Hmmm my plate is way off the planetary heath diet recommendation.
Source:
https://www.nature.com/articles/d41586-021-03612-1
#ml #rl
How to Train your Decision-Making AIs
https://thegradient.pub/how-to-train-your-decision-making-ais/
The author reviewed "five types of human guidance to train AIs: evaluation, preference, goals, attention, and demonstrations without action labels".
The last one reminds me of the movie Finch. In the movie, Finch was teaching the robot to walk by demonstrating walking but without "labels".
How to Train your Decision-Making AIs
https://thegradient.pub/how-to-train-your-decision-making-ais/
The author reviewed "five types of human guidance to train AIs: evaluation, preference, goals, attention, and demonstrations without action labels".
The last one reminds me of the movie Finch. In the movie, Finch was teaching the robot to walk by demonstrating walking but without "labels".
The Gradient
How to Train your Decision-Making AIs
How do humans transfer their knowledge and skills to artificial decision-making agents more efficiently? What kind of knowledge and skills should humans provide and in what format?
#DS #visualization
https://percival.ink/
A new lightweight language for data analysis and visualization. It looks promising.
I hate jupyter notebooks and I don't use them on most of my projects. One of the reasons is low reproducibility due to its non-reative nature. You changed some old cells and forgot to run a cell below, you may read wrong results.
This new language is reactive. If old cells are changed, related results are also updated.
https://percival.ink/
A new lightweight language for data analysis and visualization. It looks promising.
I hate jupyter notebooks and I don't use them on most of my projects. One of the reasons is low reproducibility due to its non-reative nature. You changed some old cells and forgot to run a cell below, you may read wrong results.
This new language is reactive. If old cells are changed, related results are also updated.
percival.ink
Percival • Web-based, reactive Datalog notebooks
Percival is a declarative data query and visualization language for exploring complex datasets, producing interactive graphics, and sharing results.
#ML #Transformers
Alammar J. The Illustrated Transformer. [cited 14 Dec 2021]. Available: http://jalammar.github.io/illustrated-transformer/
So good.
Alammar J. The Illustrated Transformer. [cited 14 Dec 2021]. Available: http://jalammar.github.io/illustrated-transformer/
So good.
jalammar.github.io
The Illustrated Transformer
Discussions:
Hacker News (65 points, 4 comments), Reddit r/MachineLearning (29 points, 3 comments)
Translations: Arabic, Chinese (Simplified) 1, Chinese (Simplified) 2, French 1, French 2, Italian, Japanese, Korean, Persian, Russian, Spanish 1, Spanish…
Hacker News (65 points, 4 comments), Reddit r/MachineLearning (29 points, 3 comments)
Translations: Arabic, Chinese (Simplified) 1, Chinese (Simplified) 2, French 1, French 2, Italian, Japanese, Korean, Persian, Russian, Spanish 1, Spanish…
#ml #science
I remember several years ago when I was still doing my PhD, there's this contest about predicting protein structure and none of them was working well. At that time, I would never have thought we could have anything like AlphaFold in a few years.
.
https://www.science.org/content/article/breakthrough-2021
I remember several years ago when I was still doing my PhD, there's this contest about predicting protein structure and none of them was working well. At that time, I would never have thought we could have anything like AlphaFold in a few years.
.
https://www.science.org/content/article/breakthrough-2021
Science
Science’s 2021 Breakthrough of the Year: AI brings protein structures to all
Bounty of new structures will forever change biology and medicine
#visualization
Pu X, Kay M. A probabilistic grammar of graphics. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. New York, NY, USA: ACM; 2020. doi:10.1145/3313831.3376466
Available at: https://dl.acm.org/doi/10.1145/3313831.3376466
A very good read if you are visualizing probability densities a lot.
The paper began with a common mistake people make when visualizing densities. Then they proposed a systematic grammar of graphics for probabilities. They also provide a package (quite preliminary, see here https://github.com/MUCollective/pgog ).
Pu X, Kay M. A probabilistic grammar of graphics. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. New York, NY, USA: ACM; 2020. doi:10.1145/3313831.3376466
Available at: https://dl.acm.org/doi/10.1145/3313831.3376466
A very good read if you are visualizing probability densities a lot.
The paper began with a common mistake people make when visualizing densities. Then they proposed a systematic grammar of graphics for probabilities. They also provide a package (quite preliminary, see here https://github.com/MUCollective/pgog ).
#data #ds
Disclaimer: I'm no expert in state diagram nor statecharts.
It might be something trivial but I find this useful: Combined with some techniques in statecharts (something frontend people like a lot), state diagram is a great way to document what our data is going through in data (pre)processing.
For complicated data transformations, we can make the corresponding state diagram and follow your code to make sure it is working as expected. The only thing is that we are focusing on the state of data not any other system.
We can use some techniques from statecharts, such as hierarchies and parallels.
State diagram is better than flowchart in this scenario because we are more interested in the different states of the data. State diagrams automatically highlights the states and we can easily spot the relevant part in the diagram and we don’t have to start from the beginning.
I documented some data transformations using state diagrams already. I haven't tired but it might also help us document our ML models.
References:
1. https://statecharts.dev
2. https://en.wikipedia.org/wiki/State_diagram
Disclaimer: I'm no expert in state diagram nor statecharts.
It might be something trivial but I find this useful: Combined with some techniques in statecharts (something frontend people like a lot), state diagram is a great way to document what our data is going through in data (pre)processing.
For complicated data transformations, we can make the corresponding state diagram and follow your code to make sure it is working as expected. The only thing is that we are focusing on the state of data not any other system.
We can use some techniques from statecharts, such as hierarchies and parallels.
State diagram is better than flowchart in this scenario because we are more interested in the different states of the data. State diagrams automatically highlights the states and we can easily spot the relevant part in the diagram and we don’t have to start from the beginning.
I documented some data transformations using state diagrams already. I haven't tired but it might also help us document our ML models.
References:
1. https://statecharts.dev
2. https://en.wikipedia.org/wiki/State_diagram
statecharts.dev
Welcome to the world of Statecharts
The world of statecharts describes what statecharts are, their benefits and drawbacks, how they differ from state machines, and practical examples on how to use them.
#python
I thought it was a trivial talk in the beginning.
But I quickly realized that I may know every each piece of the code mentioned in the video but the philosophy is what makes it exciting.
He talked about some fundamental ideas of Python, e.g., protocols.
After watching this video, an idea came to me. Pytorch lightning has implanted a lot of hooks in a very pythonic way. This is what makes pytorch lightning easy to use. (So if you do a lot of machine learning experiments, pytorch lightning is worth a try.)
https://youtu.be/cKPlPJyQrt4
I thought it was a trivial talk in the beginning.
But I quickly realized that I may know every each piece of the code mentioned in the video but the philosophy is what makes it exciting.
He talked about some fundamental ideas of Python, e.g., protocols.
After watching this video, an idea came to me. Pytorch lightning has implanted a lot of hooks in a very pythonic way. This is what makes pytorch lightning easy to use. (So if you do a lot of machine learning experiments, pytorch lightning is worth a try.)
https://youtu.be/cKPlPJyQrt4
YouTube
James Powell: So you want to be a Python expert? | PyData Seattle 2017
www.pydata.org
PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each…
PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each…
#visualization
Beautiful, elegant, and informative. It reminds me of the Netflix movie chromatic storytelling visualization.
Full image:
https://zenodo.org/record/5828349
Other discussions:
https://www.reddit.com/r/dataisbeautiful/comments/s6vh8k/dutch_astronomer_cees_bassa_took_a_photo_of_the/
Beautiful, elegant, and informative. It reminds me of the Netflix movie chromatic storytelling visualization.
Full image:
https://zenodo.org/record/5828349
Other discussions:
https://www.reddit.com/r/dataisbeautiful/comments/s6vh8k/dutch_astronomer_cees_bassa_took_a_photo_of_the/
#ds
Deepnote supports Great Expectations (GE) now.
I ran their template notebook:
https://deepnote.com/project/Reduce-Pipeline-Debt-With-Great-Expectations-mLT9DFCQSpW4kUBAzzdhBw/%2Fnotebook.ipynb/#00000-e170fae0-7e06-4a7a-85f3-343584ec4b94
Deepnote supports Great Expectations (GE) now.
I ran their template notebook:
https://deepnote.com/project/Reduce-Pipeline-Debt-With-Great-Expectations-mLT9DFCQSpW4kUBAzzdhBw/%2Fnotebook.ipynb/#00000-e170fae0-7e06-4a7a-85f3-343584ec4b94
#visualization
Seaborn is getting a new interface.
Would be great if the author defines a dunder method _ _ add _ _ () instead of using .add() method. Using dunder add, we can simply use + on layers.
Nevertheless, we can all move away from plotnine when the migration is done.
https://seaborn.pydata.org/nextgen/
Seaborn is getting a new interface.
Would be great if the author defines a dunder method _ _ add _ _ () instead of using .add() method. Using dunder add, we can simply use + on layers.
Nevertheless, we can all move away from plotnine when the migration is done.
https://seaborn.pydata.org/nextgen/
Forwarded from DPS Main
graph-basics.pdf
3.3 MB
#ML
I made some slides to bootstrap a community in my company to share papers on graph related methods (spectral, graph neural networks, etc).
These slides are mostly based on the first two chapters of the book by William Hamilton. I added some intuitive interpretations on some key ideas. Some of these are frequently used in graph neural networks even transformers. Building intuitions helps us unboxing these neural networks. But the slides are only skeleton notes so I probably have to expand them at some point.
I am thinking about drawing more about the book and on this topic. Maybe even making some short videos using these slides. Let's see how far I can go.I am way too busy now. (<-no excuse)
I made some slides to bootstrap a community in my company to share papers on graph related methods (spectral, graph neural networks, etc).
These slides are mostly based on the first two chapters of the book by William Hamilton. I added some intuitive interpretations on some key ideas. Some of these are frequently used in graph neural networks even transformers. Building intuitions helps us unboxing these neural networks. But the slides are only skeleton notes so I probably have to expand them at some point.
I am thinking about drawing more about the book and on this topic. Maybe even making some short videos using these slides. Let's see how far I can go.
#ML #RL #DeepMind
Magnetic control of tokamak plasmas through deep reinforcement learning | Nature
https://www.nature.com/articles/s41586-021-04301-9
Magnetic control of tokamak plasmas through deep reinforcement learning | Nature
https://www.nature.com/articles/s41586-021-04301-9
Nature
Magnetic control of tokamak plasmas through deep reinforcement learning
Nature - A newly designed control architecture uses deep reinforcement learning to learn to command the coils of a tokamak, and successfully stabilizes a wide variety of fusion plasma configurations.