In this new world of machine intelligence, you’re gonna need men you can trust.
In a world increasingly full of useful robots, the humans turned to ancient magic to help guide them.
There are probably many pockets of neural weights in a LLM that would produce coherent text output, and training only finds one of these local optima.
The search space with billions of parameters is so large to be functionally near infinite. Each retraining finds a new pocket.
And fine tuning means starting with a known neural network trained on millions of datapoints, then freezing some percent (90%? Depends on the task) of the layers and retraining the last few layers on your data. Takes advantage of giant training sets but still is trained on your data.
The search space with billions of parameters is so large to be functionally near infinite. Each retraining finds a new pocket.
And fine tuning means starting with a known neural network trained on millions of datapoints, then freezing some percent (90%? Depends on the task) of the layers and retraining the last few layers on your data. Takes advantage of giant training sets but still is trained on your data.
It’s the height of hubris to assume we understand vast complex chaotic climate trends.
We need more precise language to communicate how the insides of our body subjectively feel - it’s too low resolution right now and there are many subtleties describing the ebbs and flows of the interoception sensations we experience on the daily.
Of course people are scared of the negative impacts of AI - people throughout history have always feared change and technological progress.
I don’t see how this time is any different from all the past times when it turns out tech has made a more egalitarian world.
When both poor people and rich people alike are using the same tech (iPhones, laptops, etc), you realize a rising tide raises all ships.
Many cultural revolutions in which power was distributed from the few to the many were due to technological advancements.
All political and social change is NOT necessarily good progress (gtfo Marxists), but nearly all technological change has been positive progress in the long run.
And there were always factions opposed to it. They’ve always been on the wrong side of history. Yes, including nuclear.
I don’t see how this time is any different from all the past times when it turns out tech has made a more egalitarian world.
When both poor people and rich people alike are using the same tech (iPhones, laptops, etc), you realize a rising tide raises all ships.
Many cultural revolutions in which power was distributed from the few to the many were due to technological advancements.
All political and social change is NOT necessarily good progress (gtfo Marxists), but nearly all technological change has been positive progress in the long run.
And there were always factions opposed to it. They’ve always been on the wrong side of history. Yes, including nuclear.
Okay hear me out - AR goggles which compress the nearly full range of EM wavelengths into the visible range and also overlay other sensors like magnetic fields and spectrograms of the air particles… imagine a less filtered view of reality, real-time, all in the visible range.
It’s so incredibly interesting to me how the closed source GPT is still significantly outcompeting all open sourced LLM’s.
Neural architecture, training data size, affording a $40M gpu training budget, genius employees versus the open source community…
The issue of our time.
Neural architecture, training data size, affording a $40M gpu training budget, genius employees versus the open source community…
The issue of our time.
“If we don't believe in freedom of expression for people we despise, we don't believe in it at all.”
― Noam Chomsky
― Noam Chomsky
If transformers are so good and nobody is working on novel neural architectures because we believed that’s close to being a “solved” problem after seeing GPT’s impressive language work, will we stall?
New neural architectures in which biological neurons merge with digital neurons will unlock new vistas of intelligence not yet seen and bring up new questions about consciousness not yet answered.
Sure, a LLM may not be conscious, but throw a few actual neurons in there grown in a Petri dish and tell me what ratio of biological to digital neurons we need to consider the monstrosity conscious. Can a single neuron have feelings? Ten? A million?
New neural architectures in which biological neurons merge with digital neurons will unlock new vistas of intelligence not yet seen and bring up new questions about consciousness not yet answered.
Sure, a LLM may not be conscious, but throw a few actual neurons in there grown in a Petri dish and tell me what ratio of biological to digital neurons we need to consider the monstrosity conscious. Can a single neuron have feelings? Ten? A million?
AI is benefiting capitalists and non-capitalists alike.
A rising tide raises all ships.
Most tech advancements since we tamed fire and turned sticks and stones into weapons and tools, have simply been a net positive despite the increased sophistication of destructive weaponry.
A rising tide raises all ships.
Most tech advancements since we tamed fire and turned sticks and stones into weapons and tools, have simply been a net positive despite the increased sophistication of destructive weaponry.
Everyone talking about alignment, I just want AI to help us cure cancer and terraform planets into interstellar ecosystems within my lifetime.
AI designer drugs for curing all sorts of diseases is gonna be dope.
I bet if you read the entire internet a few times over, you’d have some clever medical ideas too.
I bet if you read the entire internet a few times over, you’d have some clever medical ideas too.
Humans easily and naturally extrapolate patterns from minimal data - show a toddler a few pictures of cats and they'll know what "cat" is.
But it makes us prone to apophenia.
The way I understand it, AI sorta blanket-learns a global understanding of the latent space (transforming high-D data to the latent embedding representations) over all the data available to it. And then, over multiple passes of the data ("epochs"), it refines its overall understanding of the latent space.
Give an AI a few million images from ImageNet and it'll understand how to identify all sorts of things including a cat. But AI fails to do well with small N, even with some pretraining (akin to our genetically driven "instincts" in my mind).
Maybe our brains do something similar, but I don't think so. We're wired a bit differently in how we learn.
But it makes us prone to apophenia.
The way I understand it, AI sorta blanket-learns a global understanding of the latent space (transforming high-D data to the latent embedding representations) over all the data available to it. And then, over multiple passes of the data ("epochs"), it refines its overall understanding of the latent space.
Give an AI a few million images from ImageNet and it'll understand how to identify all sorts of things including a cat. But AI fails to do well with small N, even with some pretraining (akin to our genetically driven "instincts" in my mind).
Maybe our brains do something similar, but I don't think so. We're wired a bit differently in how we learn.
The problem with scientific publications in academia is that the incentives (publish or perish but only if the publication gets you grant money) is that it disincentivizes researchers from publishing results that go against what they're trying to prove and waiting to publish until the results show what they want to show.
Good science is transparent on the good and the bad so that other researchers don't waste time in wrong directions.
But self-esteem and the desire to build a reputation as an effective researcher disincentivizes researchers to be a purist when it comes to science.
Researchers have a subconscious feeling of "embarassment" to admit they tried something that doesn't work and often don't publish the bad with the good; their egos make them want to only publish the good results.
They'll sit on something for years until p eeks down to .049999 and then immediately publish a paper as a breakthrough that's statistically significant.
This is human nature.
This is academia.
This is not science.
Good science is transparent on the good and the bad so that other researchers don't waste time in wrong directions.
But self-esteem and the desire to build a reputation as an effective researcher disincentivizes researchers to be a purist when it comes to science.
Researchers have a subconscious feeling of "embarassment" to admit they tried something that doesn't work and often don't publish the bad with the good; their egos make them want to only publish the good results.
They'll sit on something for years until p eeks down to .049999 and then immediately publish a paper as a breakthrough that's statistically significant.
This is human nature.
This is academia.
This is not science.
There's a new AI ChatBot implementation called "RMT" which can remember a million words in the convo, instead of the typical 32,000 words. I just read the paper. It's very clever and will work and become the standard.
It's based on a paper published by AI researchers in Russia.
I fear that the elites with their proxy war in Ukraine are demonizing Russians which will disincentivize Russian AI researchers from engaging with NATO AI researchers.
We need papers like this published and willing to be shared across national boundaries. If we start becoming prejudiced against Russians, will Russian AI researchers stop collaborating with Western AI researchers?
It's all just abstractions and hypothesizing until you start using ChatGPT every day and really appreciate that it remembers what you both were discussing at the start of the convo a hundred messages ago, and gives you better answers. Then you'll really appreciate how the Russian AI researchers figured out how to let chatbots remember a million words instead of a few thousand words. You won't be able to live without it, and think chatbots which only remember a few messages are ridiculously outdated and unusable.
It's based on a paper published by AI researchers in Russia.
I fear that the elites with their proxy war in Ukraine are demonizing Russians which will disincentivize Russian AI researchers from engaging with NATO AI researchers.
We need papers like this published and willing to be shared across national boundaries. If we start becoming prejudiced against Russians, will Russian AI researchers stop collaborating with Western AI researchers?
It's all just abstractions and hypothesizing until you start using ChatGPT every day and really appreciate that it remembers what you both were discussing at the start of the convo a hundred messages ago, and gives you better answers. Then you'll really appreciate how the Russian AI researchers figured out how to let chatbots remember a million words instead of a few thousand words. You won't be able to live without it, and think chatbots which only remember a few messages are ridiculously outdated and unusable.
Forget about banning plastic straws for Americans to feel virtuous and then move on with their days, realize that Philippines produce way most plastic waste than anyone else. https://www.visualcapitalist.com/cp/visualized-ocean-plastic-waste-pollution-by-country/
Forget about making people feel guilty for their “carbon footprint” from being alive, realize we’re cutting down 3600 football fields worth of forests every hour. https://www.theworldcounts.com/challenges/planet-earth/forests-and-deserts/rate-of-deforestation
Forget about climate alarmism making individuals feel guilty for their CO2 effects, and realize that 15 food and beverage companies emit more greenhouse gases than Australia and just 100 energy companies cause 71% of industrial emissions. https://www.nrdc.org/bio/josh-axelrod/corporate-honesty-and-climate-change-time-own-and-act
Forget about what the elites are telling you about “climate change” consensus and try to actually find non-biased scientific publications with open data and open sourced code that admit how little we know about global temperature before 1900.
I love the environment. I love nature. Humans are not a plague on the earth. We need to live in harmony with nature. But instead of pretending that the governments and corporate leaders actually care, and instead of people thinking they’re saving the world, actually figure out what will move the needle and what’s mere virtue signaling.
Which countries are polluting the most? Which corporations? Which regions within the country and which specific industries are the worst offenders? Have the coastal towns gone underwater from rising sea levels yet or is that in another few years? Again I don’t want humans to be disgusting polluting bane on nature, but politics is making things worse not better by misaligned incentives and a hubris that we got it all figured out. The earth’s biosphere is complex, the ecosystem resilient, and we need to realize there are more data driven ways to address environmentalism that admits we don’t know it all and actually tries to empirically determine what’s the best way to improve our relationship with nature without being stupid fearmongerers that believe big daddy governments can fix the weather. There are better ways to go about this.
/rant
Forget about making people feel guilty for their “carbon footprint” from being alive, realize we’re cutting down 3600 football fields worth of forests every hour. https://www.theworldcounts.com/challenges/planet-earth/forests-and-deserts/rate-of-deforestation
Forget about climate alarmism making individuals feel guilty for their CO2 effects, and realize that 15 food and beverage companies emit more greenhouse gases than Australia and just 100 energy companies cause 71% of industrial emissions. https://www.nrdc.org/bio/josh-axelrod/corporate-honesty-and-climate-change-time-own-and-act
Forget about what the elites are telling you about “climate change” consensus and try to actually find non-biased scientific publications with open data and open sourced code that admit how little we know about global temperature before 1900.
I love the environment. I love nature. Humans are not a plague on the earth. We need to live in harmony with nature. But instead of pretending that the governments and corporate leaders actually care, and instead of people thinking they’re saving the world, actually figure out what will move the needle and what’s mere virtue signaling.
Which countries are polluting the most? Which corporations? Which regions within the country and which specific industries are the worst offenders? Have the coastal towns gone underwater from rising sea levels yet or is that in another few years? Again I don’t want humans to be disgusting polluting bane on nature, but politics is making things worse not better by misaligned incentives and a hubris that we got it all figured out. The earth’s biosphere is complex, the ecosystem resilient, and we need to realize there are more data driven ways to address environmentalism that admits we don’t know it all and actually tries to empirically determine what’s the best way to improve our relationship with nature without being stupid fearmongerers that believe big daddy governments can fix the weather. There are better ways to go about this.
/rant
Visual Capitalist
Which Countries Pollute the Most Ocean Plastic Waste?
This graphic visualizes the top 10 countries emitting plastic pollutants into our oceans.
Ironically, Alan Turing invented the Turing Test (he called it “Imitation Game”) but didn’t care too much if machines could technically think or not, rather what they could do. Detecting machines “thinking” not as conversation imitation but as finding novel solutions to problems?
Drill deeply into the subtleties of the sensations you’re experiencing in this present moment, and as time progresses delve deeper and deeper into the hearts of the sensations.
Relax, AI is gonna displace not replace jobs, like every other tech invention ever.
Oh wow cars will replace horse & carriage jobs.
Oh wow photoshop will replace photographers.
Oh wow excel will make finance jobs obsolete.
Literally just creates new jobs as it eats old ones.
If you personally don’t adapt to AI and don’t bother using it in your job, then when you personally get replaced it’s your own fault.
En-masse we’re gonna be fine.
I know, I know, “THiS tiMe iT’s diFFErEnT!”
Oh wow cars will replace horse & carriage jobs.
Oh wow photoshop will replace photographers.
Oh wow excel will make finance jobs obsolete.
Literally just creates new jobs as it eats old ones.
If you personally don’t adapt to AI and don’t bother using it in your job, then when you personally get replaced it’s your own fault.
En-masse we’re gonna be fine.
I know, I know, “THiS tiMe iT’s diFFErEnT!”