Attention in Transformers

Quick version:

each word gets turned into a vector, a long list of numbers, where different directions in the space can carry meaning. But at the start, the representation is mostly about the token itself. “Mole” is the same token whether you’re talking about the animal, the chemistry unit, or the thing on your skin.

Attention is what lets that representation become contextual. Every token sends out a query what am I looking for? and a key what information do I offer? Match them up, softmax the scores into weights, and each token pulls in a weighted mix of everyone else’s values. So the representation of a token can change depending on what’s around it.

Sooo cool part attention has no idea what order the words are in. It’s basically doing weighted combinations, and addition doesn’t care about sequence. “The dog bit the man” and “the man bit the dog” would look the same to attention if you gave it the same token representations without any positional information. Position has to be added separately so the model knows where things are. I’d read about transformers plenty of times and somehow never registered that.

Also, the title is kinda wrong. Attention isn’t all you need. In GPT-3, roughly a third of the parameters are in the attention mechanism; a lot of the rest sits in the layers between the attention blocks. And apparently those layers are doing a lot more than I initially thought including storing some of the model’s learned knowledge. Attention moves information around. Something else is doing a lot of the processing.

I haven’t implemented it yet, which means I only half know this.

Gödel’s Incompleteness Theorem

Gödel proved that sufficiently strong, consistent mathematical systems contain true statements they can’t prove within that system.

The trick is basically encoding. He turned symbols and statements into numbers, which let arithmetic talk about statements themselves. He then constructed a sentence that essentially says:

“This statement is not provable in this system.”
Sooo...

If the system proves it, there’s a problem. If it doesn’t, the statement is true but unprovable

in that system

.

Ngl I initially confused myself. I wrote “there are truths that can’t be proved” and also “it’s wrong to say Gödel proved there are truths that can’t be proved.”
They’re not actually opposites.

Unprovable always means unprovable in a particular system.

A stronger system can prove something the original one couldn’t.

So Gödel isn’t really saying “truth is unknowable.” He’s showing that no sufficiently powerful formal system can capture all mathematical truth from within itself.

And apparently this is closely
related to the halting problem. Different proofs, but the same weird family of ideas around self-reference and limits.

how weird it is that

a system becoming capable of talking about itself is also what exposes its limits.

Let me tell u how this site has been helpful https://plato.stanford.edu