Re: I don’t want to be an internet person

> I don’t want to be anything like these people. I don’t want to be an internet person.

I love this. I got a similar vibe from people who were "too good at IRC," way back in the day. They had a constant, sarcastic, tired energy about them. They had difficulty being genuine about anything. They knew so much and yet they were so stuck in their life somehow. And that sucked the life out of them.

It's like they were too tied to this vague idea of being online that they weren't willing to sacrifice it to have a better life.

The Internet is a tool, not an endpoint.

mattgreenrocks, 1 day ago

What Does the “Mean” Really Mean?

Abstract:

The arithmetic average of a collection of observed values of a homogeneous collection of quantities is often taken to be the most representative observation. There are several arguments supporting this choice the moment of inertia being the most familiar. But what does this mean?
In this note, we bring forth the Kolmogorov-Nagumo point of view that the arithmetic average is a special case of a sequence of functions of a special kind, the quadratic and the geometric means being some of the other cases. The median fails to belong to this class of functions. The Kolmogorov-Nagumo interpretation is the most defensible and the most definitive one for the arithmetic average, but its essence boils down to the fact that this average is merely an abstraction which has meaning only within its mathematical set-up.

https://arxiv.org/pdf/2003.01973.pdf

f -> average -> f^(-1)
just like change of bases

#edu The Forward-Forward Learning Algorithm

The Forward-Forward algorithm replaces the forward and
backward passes of backpropagation by two forward passes, one with positive
(i.e. real) data and the other with negative data which could be generated by the
network itself. Each layer has its own objective function which is simply to have
high goodness for positive data and low goodness for negative data.

the paper itself: https://www.cs.toronto.edu/~hinton/FFA13.pdf
a nice explanatory video: https://youtu.be/rVzDRfO2sgs

fun page

"The webpages collected here list information about classes of mathematical structures. The aim is to have a central place to check what properties are known about these structures."

https://math.chapman.edu/~jipsen/structures/doku.php

"Set records. Break records. Shatter records". 😎

Любі друзі, ця цитата сьогодні про нас усіх, про наш невпинний Студматсемінар!

Рекордне одинадцяте засідання СМС протягом одного сезону відбудеться в Сб, 20.05 об 11:00.

Тема: "Topological Machine Learning from WL algorithm to Simplicial Neural Networks".

Доповідач: Олександр Яворський (@svefn), аспірант 1-го р.н. кафедри Математичного Моделювання та Аналізу Даних у КПІ, Machine Learning Researcher у Knowledgator.

Зум: https://zoom.us/j/5197673308?pwd=eGRtaVIzbHlNT3RoRjc5U2FsVENGUT09

Запрошуємо всіх на захопливу доповідь про топологічні методи Машинного навчання та узагальнення графових нейронних мереж. Буде гаряче!

Sounds emitted by plants under stress are airborne and informative

Abstract:

Stressed plants show altered phenotypes, including changes in color, smell, and shape. Yet, airborne sounds emitted by stressed plants have not been investigated before. Here we show that stressed plants emit airborne sounds that can be recorded from a distance and classified. We recorded ultrasonic sounds emitted by tomato and tobacco plants inside an acoustic chamber, and in a greenhouse, while monitoring the plant’s physiological parameters. We developed machine learning models that succeeded in identifying the condition of the plants, including dehydration level and injury, based solely on the emitted sounds. These informative sounds may also be detectable by other organisms. This work opens avenues for understanding plants and their interactions with the environment and may have significant impact on agriculture.

https://www.cell.com/cell/fulltext/S0092-8674(23)00262-3

The False Promise of Imitating Proprietary LLMs

Abstract:

An emerging method to cheaply improve a weaker language model is to finetune it on outputs from a stronger model, such as a proprietary system like ChatGPT (e.g., Alpaca, Self-Instruct, and others). This approach looks to cheaply imitate the proprietary model's capabilities using a weaker open-source model. In this work, we critically analyze this approach. We first finetune a series of LMs that imitate ChatGPT using varying base model sizes (1.5B--13B), data sources, and imitation data amounts (0.3M--150M tokens). We then evaluate the models using crowd raters and canonical NLP benchmarks. Initially, we were surprised by the output quality of our imitation models -- they appear far better at following instructions, and crowd workers rate their outputs as competitive with ChatGPT. However, when conducting more targeted automatic evaluations, we find that imitation models close little to none of the gap from the base LM to ChatGPT on tasks that are not heavily supported in the imitation data. We show that these performance discrepancies may slip past human raters because imitation models are adept at mimicking ChatGPT's style but not its factuality. Overall, we conclude that model imitation is a false promise: there exists a substantial capabilities gap between open and closed LMs that, with current methods, can only be bridged using an unwieldy amount of imitation data or by using more capable base LMs. In turn, we argue that the highest leverage action for improving open-source models is to tackle the difficult challenge of developing better base LMs, rather than taking the shortcut of imitating proprietary systems.

> We show that these performance discrepancies may slip past human raters because imitation models are adept at mimicking ChatGPT's style but not its factuality

https://arxiv.org/pdf/2305.15717