#ml

In his MinT paper, Hyndman said he confused these two quantities in his previous paper. 😂

MinT is a simple method to make forecasts with hierarchical structure coherent. Here coherent means the sum of the lower level forecasts equals the higher level forecasts.

For example, our time series has a strucutre like sales of coca cola + sales of spirit = sales of beverages. If this relations holds for our forecasts, we have coherent forecasts.

This may sound trivial, the problem is in fact hard. There are many trivial methods such as only forecasting lower levels (coca cola, spirit) then use the sum as the higher level (sales of beverages). These are usually too naive to be effective.

MinT is a reconciliation method that combines high level forecasts and the lower level forecasts to find an optimal combination/reconciliation.

https://robjhyndman.com/papers/MinT.pdf

#ml

GPT writing papers... Both fancy and scary.

https://huggingface.co/stanford-crfm/pubmedgpt?text=Neuroplasticity

#ml

Top-10 Things in 2022 | Anima on AI
https://anima-ai.org/2022/12/31/top-10-things-in-2022/

#ml

Haha icecube

IceCube - Neutrinos in Deep Ice | Kaggle
https://www.kaggle.com/competitions/icecube-neutrinos-in-deep-ice?utm_medium=email&utm_source=gamma&utm_campaign=comp-icecube-2023

#ml

google-research/tuning_playbook: A playbook for systematically maximizing the performance of deep learning models.
https://github.com/google-research/tuning_playbook

#ml

Releasing the Skynet

https://github.com/internet-explorer-ssl/internet-explorer

#ml

https://mlcontests.com/state-of-competitive-machine-learning-2022/

Quote from the report:

Successful competitors have mostly converged on a common set of tools — Python, PyData, PyTorch, and gradient-boosted decision trees.

Deep learning still has not replaced gradient-boosted decision trees when it comes to tabular data, though it does often seem to add value when ensembled with boosting methods.
Transformers continue to dominate in NLP, and start to compete with convolutional neural nets in computer vision.

Competitions cover a broad range of research areas including computer vision, NLP, tabular data, robotics, time-series analysis, and many others.
Large ensembles remain common among winners, though single-model solutions do win too.

There are several active machine learning competition platforms, as well as dozens of purpose-built websites for individual competitions.
Competitive machine learning continues to grow in popularity, including in academia.

Around 50% of winners are solo winners; 50% of winners are first-time winners; 30% have won more than once before.

Some competitors are able to invest significantly into hardware used to train their solutions, though others who use free hardware like Google Colab are also still able to win competitions.

#ml

Pérez J, Barceló P, Marinkovic J. Attention is Turing-Complete. J Mach Learn Res. 2021;22: 1–35. Available: https://jmlr.org/papers/v22/20-302.html

#ml

Yeh, Catherine, Yida Chen, Aoyu Wu, Cynthia Chen, Fernanda Viégas, and Martin Wattenberg. 2023. “AttentionViz: A Global View of Transformer Attention.” ArXiv [Cs.HC]. arXiv. http://arxiv.org/abs/2305.03210.

#ml

https://www.kdnuggets.com/2023/06/ten-years-ai-review.html

#ml

Yes, Transformers are Effective for Time Series Forecasting (+ Autoformer)
https://huggingface.co/blog/autoformer

#ml

A family tree shows how transformers are evolving.

(HTML is probably the worst name for a model.)

https://arxiv.org/abs/2302.07730

#ml

Hand-Crafted Transformers


HandCrafted.ipynb - Colaboratory
https://colab.research.google.com/github/newhouseb/handcrafted/blob/main/HandCrafted.ipynb

#ml


Interesting idea to use Hydra in ML experiments.

https://github.com/ashleve/lightning-hydra-template

#ml

Jelassi S, Brandfonbrener D, Kakade SM, Malach E. Repeat after me: Transformers are better than state space models at copying. arXiv [cs.LG]. 2024. Available: http://arxiv.org/abs/2402.01032

Not surprising at all when you have direct access to a long context. But hey, look at this title.

#ml

I got interested in satellite data last year and played with it a bit. It's fantastic. The spatiotemporal nature of it brings up a lot of interesting questions.

Then I saw this paper today:


Rolf, Esther, Konstantin Klemmer, Caleb Robinson, and Hannah Kerner. 2024. “Mission Critical -- Satellite Data Is a Distinct Modality in Machine Learning.” arXiv [Cs.LG], February. http://arxiv.org/abs/2402.01444.

#ml

Like a dictionary


Kunc, Vladim’ir, and Jivr’i Kl’ema. 2024. “Three Decades of Activations: A Comprehensive Survey of 400 Activation Functions for Neural Networks.” arXiv [Cs.LG], February. http://arxiv.org/abs/2402.09092.

#ml

https://github.com/google-research/timesfm