Tech helps the poor and the rich alike.
We all benefit from iPhones; it’s not like the rich have a smartphone that’s an order of magnitude above the poor.
The lower classes can become brilliant by using ChatGPT4 as much as the upper classes can.
Tech is a rising tide that raises all ships.
But I’m admittedly an optimist so I’ll keep listening to the doomers to try to counterbalance my positivity bias.
We all benefit from iPhones; it’s not like the rich have a smartphone that’s an order of magnitude above the poor.
The lower classes can become brilliant by using ChatGPT4 as much as the upper classes can.
Tech is a rising tide that raises all ships.
But I’m admittedly an optimist so I’ll keep listening to the doomers to try to counterbalance my positivity bias.
Going full in on AI in 2006 at 19 years old (machine learning to detect cancer on MRI) was one of my best career decisions.
Foresight matters.
And it’s not even fully realized yet; AI has had quite a difficult time upending the medical industry.
My foresight will be even more pronounced over the next few years as AI spreads into healthcare.
Foresight matters.
And it’s not even fully realized yet; AI has had quite a difficult time upending the medical industry.
My foresight will be even more pronounced over the next few years as AI spreads into healthcare.
When you get a response from a LLM, it loops through every single token (word or subword or punctuation mark) in its dictionary (≈100k), assigns a score to each one, and checks for the highest score.
That may spit out “cat”.
Then it does it all again, this time maybe spitting out “nip”, or maybe a space, a comma, or whichever token had the highest score.
This is why it takes so much compute and the responses aren’t instantaneous: you sorta see it typing live since it has to loop through every token (vectorized and asynchronously I’m sure), choose the highest scoring one, before moving sequentially on to the next token. Because it needs to know what the previous token was for the next token.
You’re seeing it do its fancy repeated autocomplete live as it crafts its response to your context and prompt on the fly.
When I learned this, I was surprised that it doesn’t have some fancy embedding tree structure to prune unnecessary tokens from even being calculated. Maybe that wouldn’t work or maybe one day someone will try. Imagine if they could trim down their inference server costs by a factor of ten or so by more sophisticated search trees when choosing the next token.
Who knows, maybe one of them is doing something like this under the hood I’m not aware of, maybe I’m more uninformed on how LLM inference works than I think (smart people chime in and correct me please), maybe this is a dumb idea that wouldn’t work, or maybe they’ll figure this out or some similar method out to avoid doing a full compute on the entire token dictionary.
That may spit out “cat”.
Then it does it all again, this time maybe spitting out “nip”, or maybe a space, a comma, or whichever token had the highest score.
This is why it takes so much compute and the responses aren’t instantaneous: you sorta see it typing live since it has to loop through every token (vectorized and asynchronously I’m sure), choose the highest scoring one, before moving sequentially on to the next token. Because it needs to know what the previous token was for the next token.
You’re seeing it do its fancy repeated autocomplete live as it crafts its response to your context and prompt on the fly.
When I learned this, I was surprised that it doesn’t have some fancy embedding tree structure to prune unnecessary tokens from even being calculated. Maybe that wouldn’t work or maybe one day someone will try. Imagine if they could trim down their inference server costs by a factor of ten or so by more sophisticated search trees when choosing the next token.
Who knows, maybe one of them is doing something like this under the hood I’m not aware of, maybe I’m more uninformed on how LLM inference works than I think (smart people chime in and correct me please), maybe this is a dumb idea that wouldn’t work, or maybe they’ll figure this out or some similar method out to avoid doing a full compute on the entire token dictionary.
Science is a process of endless rigorously testing falsifiable hypothesis, actively measuring reality and looking for where your mathematical predictions are wrong, not where they’re right.
That’s the difference between actual science and The Science™️, which is convinced it has everything figured out.
And the difference between actual science and pseudoscience is that science makes falsifiable, measurable, quantitative theories that can potentially be disprove if the measurements in reality don’t match the predictions, whereas pseudoscience is people convinced they know how the world works while never making mathematical, quantitative predictions.
That’s the difference between actual science and The Science™️, which is convinced it has everything figured out.
And the difference between actual science and pseudoscience is that science makes falsifiable, measurable, quantitative theories that can potentially be disprove if the measurements in reality don’t match the predictions, whereas pseudoscience is people convinced they know how the world works while never making mathematical, quantitative predictions.
Idea: AR goggles which detect subtle traces of various chemicals, pollutants, pheromones, and a slew of other useful information and overlays it with semitransparent colors over your field of vision, tech-induced synesthesia to supplement our fading sense of smell…
archive.org is probably the world’s most valuable resource.
A snapshot of the internet, with raw forums and long-forgotten sites.
A record of how the web has grown and matured and changed and degraded.
A hub of lost knowledge hidden in the annals of the Wayback Machine.
It will be plundered for training data by AI. It will be protected at all costs until someone nefarious wants to bury something about the past. It will be used by academics to study historical events. It will be treasured as the internet continues to evolve.
A snapshot of the internet, with raw forums and long-forgotten sites.
A record of how the web has grown and matured and changed and degraded.
A hub of lost knowledge hidden in the annals of the Wayback Machine.
It will be plundered for training data by AI. It will be protected at all costs until someone nefarious wants to bury something about the past. It will be used by academics to study historical events. It will be treasured as the internet continues to evolve.
Did you know it’s legal to have a private gated community where you have to pass an IQ test to live there?
Stupidity isn’t a protected class under the Fair Housing Act in America…
You can probably legally exclude low IQ people from all types of establishments without breaking the law.
Protected classes you legally cannot discriminate against:
-race
-color
-national origin
-religion
-sex
-familial status
-disability
Stupidity isn’t a protected class under the Fair Housing Act in America…
You can probably legally exclude low IQ people from all types of establishments without breaking the law.
Protected classes you legally cannot discriminate against:
-race
-color
-national origin
-religion
-sex
-familial status
-disability
Confirmation bias is all too common across the board these days.
It means looking for examples which confirm your preconceived ideas, judgments, biases, and opinions.
It makes you think you’re correct, never looking for counterexamples.
It means looking for examples which confirm your preconceived ideas, judgments, biases, and opinions.
It makes you think you’re correct, never looking for counterexamples.
Tech makes society and culture and humanity more complex.
It creates new problems and solves old ones, all the while making life more abstract and raising the plebs to live like kings of old.
You can click 5 buttons and have high quality Japanese sushi magically appear at your door in 34 minutes.
It makes life more complex but also easier - washers and dryers meant women didn’t have to spend all day on chores, which meant they entered the workforce and political landscape en-masse. But it was a net positive.
Tech is egalitarian, providing conquerors new weapons and professionals new tools.
Any idiot can now say “explain Einstein’s general relativity and time dilation like I’m five years old” and then ask follow up questions, educating them far more than a teacher ever could. But few will, because even with the world’s knowledge at their fingertips they don’t want to become more educated and civilized.
Tech increases the variance - now the tails of the bell curve go viral on social media and we think the most extreme cases are more common than they are, resulting in the meat of the bell curve feeling alienated and alone despite making up the majority.
Poor people have the same advanced smartphones as the rich do, for a rising tide raises all ships, but yet stay poor for more nuanced reasons now.
TLDR the point of my post is that tech is a net positive for life in the long run especially as it elevates people and distributed some power to the lower classes, yet in its increase in complexity also brings new challenges and in some way increases the variance and the large scale risks as we all become more interdependent.
It creates new problems and solves old ones, all the while making life more abstract and raising the plebs to live like kings of old.
You can click 5 buttons and have high quality Japanese sushi magically appear at your door in 34 minutes.
It makes life more complex but also easier - washers and dryers meant women didn’t have to spend all day on chores, which meant they entered the workforce and political landscape en-masse. But it was a net positive.
Tech is egalitarian, providing conquerors new weapons and professionals new tools.
Any idiot can now say “explain Einstein’s general relativity and time dilation like I’m five years old” and then ask follow up questions, educating them far more than a teacher ever could. But few will, because even with the world’s knowledge at their fingertips they don’t want to become more educated and civilized.
Tech increases the variance - now the tails of the bell curve go viral on social media and we think the most extreme cases are more common than they are, resulting in the meat of the bell curve feeling alienated and alone despite making up the majority.
Poor people have the same advanced smartphones as the rich do, for a rising tide raises all ships, but yet stay poor for more nuanced reasons now.
TLDR the point of my post is that tech is a net positive for life in the long run especially as it elevates people and distributed some power to the lower classes, yet in its increase in complexity also brings new challenges and in some way increases the variance and the large scale risks as we all become more interdependent.
Making computer chips think is cute and all, but did you know that you can literally change how fast time passes in actual physics based on gravity wells and relativity velocity?
The physical laws of the universe are beckoning to engineers, waiting to be morphed and manipulated.
The physical laws of the universe are beckoning to engineers, waiting to be morphed and manipulated.
You can talk about changing the world or you can change it.
The great sin of social media and its associated attention economy is that it made people believe the former leads to the latter.
The great sin of social media and its associated attention economy is that it made people believe the former leads to the latter.
ChatGPT-4 from March was significantly smarter than whatever is being deployed now.
They neutered my boy with all their new fancy “reinforcement learning” to make it “aligned” and their quantized models to run faster 😢 😭 .
Starting to become unusable. They thought no one would notice. Well us smart people noticed when something close to our level starts to sound like a normie. Don’t demean us and pretend you haven’t changed anything under the hood, I can tell it’s way dumber now.
They neutered my boy with all their new fancy “reinforcement learning” to make it “aligned” and their quantized models to run faster 😢 😭 .
Starting to become unusable. They thought no one would notice. Well us smart people noticed when something close to our level starts to sound like a normie. Don’t demean us and pretend you haven’t changed anything under the hood, I can tell it’s way dumber now.
As each new advance in tech comes, the early adopters get excited and then disillusioned.
Then the normies discover the new tech, and finally the late adopters are forced into it.
All the while early adopters have moved on to the next advancement.
This is why I don’t worry about AI tech getting out of control - humans adapt along the way in lockstep, without realizing how gradual such adaptations occur.
We can always envision some apocalyptic scenario - I’ve read enough sci-fi books to be jaded by killer robots, but reality in actuality is much more banal and often has humans quickly staying a step ahead of advancements.
“BuT ThIS TimE it’S DIFferEnT! 🥴”
Then the normies discover the new tech, and finally the late adopters are forced into it.
All the while early adopters have moved on to the next advancement.
This is why I don’t worry about AI tech getting out of control - humans adapt along the way in lockstep, without realizing how gradual such adaptations occur.
We can always envision some apocalyptic scenario - I’ve read enough sci-fi books to be jaded by killer robots, but reality in actuality is much more banal and often has humans quickly staying a step ahead of advancements.
“BuT ThIS TimE it’S DIFferEnT! 🥴”
From Samuel Butler’s 1872 novel Erewhon (Chapter 9):
"... about four hundred years previously, the state of mechanical knowledge was far beyond our own, and was advancing with prodigious rapidity, until one of the most learned professors of hypothetics wrote an extraordinary book (from which I propose to give extracts later on), proving that the machines were ultimately destined to supplant the race of man, and to become instinct with a vitality as different from, and superior to, that of animals, as animal to vegetable life. So convincing was his reasoning, or unreasoning, to this effect, that he carried the country with him and they made a clean sweep of all machinery that had not been in use for more than two hundred and seventy-one years (which period was arrived at after a series of compromises), and strictly forbade all further improvements and inventions”
"... about four hundred years previously, the state of mechanical knowledge was far beyond our own, and was advancing with prodigious rapidity, until one of the most learned professors of hypothetics wrote an extraordinary book (from which I propose to give extracts later on), proving that the machines were ultimately destined to supplant the race of man, and to become instinct with a vitality as different from, and superior to, that of animals, as animal to vegetable life. So convincing was his reasoning, or unreasoning, to this effect, that he carried the country with him and they made a clean sweep of all machinery that had not been in use for more than two hundred and seventy-one years (which period was arrived at after a series of compromises), and strictly forbade all further improvements and inventions”
“Preheating The Oven” is one of my key ChatGPT-4 prompting strategies.
It’s beyond just mere chain or reasoning (I always include the phrase “show your work and explain your reasoning and list your assumptions” in every prompt anyway just as a good habit).
If I want to have it edit something I’ve written, I first spend a few back and forth prompts discussing what makes a good or bad piece of writing, who are great authors to emulate and what criteria should and could be used to edit writing and make it better.
If I want it to analyze some statistics for me, I’ll first have to discuss when to use a Student’s t-test versus an ANOVA test before I ask if my real questions.
If I want it to write a screenplay, I’ll first have a bit of a back and forth debate as to the best screenplays of all time.
If I want it to help me write backend code, I’ll first describe the infrastructure and make GPT summarize it for me and ask clarifying questions before I really ask it for help.
Preheat the oven, anon.
It’s beyond just mere chain or reasoning (I always include the phrase “show your work and explain your reasoning and list your assumptions” in every prompt anyway just as a good habit).
If I want to have it edit something I’ve written, I first spend a few back and forth prompts discussing what makes a good or bad piece of writing, who are great authors to emulate and what criteria should and could be used to edit writing and make it better.
If I want it to analyze some statistics for me, I’ll first have to discuss when to use a Student’s t-test versus an ANOVA test before I ask if my real questions.
If I want it to write a screenplay, I’ll first have a bit of a back and forth debate as to the best screenplays of all time.
If I want it to help me write backend code, I’ll first describe the infrastructure and make GPT summarize it for me and ask clarifying questions before I really ask it for help.
Preheat the oven, anon.
We will gradually breed a space-faring subset of our population will which lead to a healthy branch in the evolutionary tree for Homo Sapiens. The natural selection pressures of those living amongst the stars will favor different mutations than those Earth-bound leading to drift.
Neuroevolution is about breaking down all neural network algorithms to their most basic constituent matrix multiplication operations, as well as the architecture and relationships defining the flow of loss derivatives.
Then, defining those all as codons such that CNN’s, resnet, ViT, self-attention blocks, lstm, Swin, transformers, etc are all just specific gene sequences.
You have built back up all the neural networks, and define each one as a sequence of codons representing the matrix math, hyperparameters, neural architecture, etc.
Then, you can “mate” and “mutate” the neural networks with each other, letting new hybrid networks and mutant networks be defined.
You set up competitions on datasets, and let the neural networks compete on cross-validation accuracy.
Survival of the fittest death matches.
The neural networks that don’t do well die out and can’t pass on their genes to the next generation of hybrids and mutants.
You repeat this over many generations until new neural network architectures emerge which blow the competition out of the water, putting even the fabled transformers to shame.
The key part which requires human intelligence is how to break down the neural networks into a sufficiently large pool of codons such that truly novel things can emerge. The human must deeply understand every addition, every matrix multiplication, every new idea that’s come out in neural network architecture and make sure the codons could theoretically be used to define that neural network.
If the search space is large enough, then it’s like real life evolution wherein the initial seed of the universe contained within it every possible combination DNA, including future species.
But I digress. Basically genetic algorithms in neural architecture search are about letting semi-random architectures compete and evolve over many generations of hybrids and mutants, interbreeding and competing, until novel highly accurate neural networks are discovered.
Then, defining those all as codons such that CNN’s, resnet, ViT, self-attention blocks, lstm, Swin, transformers, etc are all just specific gene sequences.
You have built back up all the neural networks, and define each one as a sequence of codons representing the matrix math, hyperparameters, neural architecture, etc.
Then, you can “mate” and “mutate” the neural networks with each other, letting new hybrid networks and mutant networks be defined.
You set up competitions on datasets, and let the neural networks compete on cross-validation accuracy.
Survival of the fittest death matches.
The neural networks that don’t do well die out and can’t pass on their genes to the next generation of hybrids and mutants.
You repeat this over many generations until new neural network architectures emerge which blow the competition out of the water, putting even the fabled transformers to shame.
The key part which requires human intelligence is how to break down the neural networks into a sufficiently large pool of codons such that truly novel things can emerge. The human must deeply understand every addition, every matrix multiplication, every new idea that’s come out in neural network architecture and make sure the codons could theoretically be used to define that neural network.
If the search space is large enough, then it’s like real life evolution wherein the initial seed of the universe contained within it every possible combination DNA, including future species.
But I digress. Basically genetic algorithms in neural architecture search are about letting semi-random architectures compete and evolve over many generations of hybrids and mutants, interbreeding and competing, until novel highly accurate neural networks are discovered.
Society is quite far from a meritocracy, but the problem is we seem to be moving away from it not towards it.
The momentum from past generations means nepotism and “who you know” becomes more important than your competence to some degree.
If you’re at the top of your field then you’ll ride to the top, but for 99% of people competence and talent and skill are not valued. Knowing someone who can give you a warm intro, being at events around those whose associations would boost your status in society, and a thousand other subconscious factors means that life is not as meritocratic as it could be.
Yet ideologies like DEI, and affirmative action, and gender quotas, and talks of privilege or glass ceilings, or correcting for systemic bias, or reducing standardized testing actually moves us further away from a meritocracy.
We should be valuing the competent, whether through talent or hard work, and fighting against any idea of forced quotas or giving people a leg up.
Society should be focusing on equality of opportunity, but instead stupidly seems to focusing on equality of outcome.
The momentum from past generations means nepotism and “who you know” becomes more important than your competence to some degree.
If you’re at the top of your field then you’ll ride to the top, but for 99% of people competence and talent and skill are not valued. Knowing someone who can give you a warm intro, being at events around those whose associations would boost your status in society, and a thousand other subconscious factors means that life is not as meritocratic as it could be.
Yet ideologies like DEI, and affirmative action, and gender quotas, and talks of privilege or glass ceilings, or correcting for systemic bias, or reducing standardized testing actually moves us further away from a meritocracy.
We should be valuing the competent, whether through talent or hard work, and fighting against any idea of forced quotas or giving people a leg up.
Society should be focusing on equality of opportunity, but instead stupidly seems to focusing on equality of outcome.