Links for 2023-11-26

1. “There's another whole route to scaling LLM capabilities. LLMs themselves might not need to scale up much more at all to enable AGI. Scaffolding LLMs to them to make language model cognitive architectures can fill at least some of the gaps in their abilities. Many groups are working on this.” https://www.lesswrong.com/posts/hg9SvxBQw6KFNEzrH/what-s-the-evidence-that-llms-will-scale-up-efficiently

2. “These results suggest that without any additional training, LLMs can serve as general sequence modelers, driven by in-context learning.” https://arxiv.org/abs/2307.04721

3. DROC: Distilling and Retrieving Generalizable Knowledge for Robot Manipulation via Language Corrections https://sites.google.com/stanford.edu/droc

4. LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenes https://luciddreamer-cvlab.github.io/

5. MetaDreamer: Efficient Text-to-3D Creation With Disentangling Geometry and Texture https://arxiv.org/abs/2311.10123

6. Jorge: Approximate Preconditioning for GPU-efficient Second-order Optimization https://arxiv.org/abs/2310.12298

7. “In this work, we propose a distributed optimization algorithm, Distributed Low-Communication (DiLoCo), that enables training of language models on islands of devices that are poorly connected.” https://arxiv.org/abs/2311.08105

8. Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers https://arxiv.org/abs/2311.10642

9. “…thanks to recent advances in artificial intelligence (AI), the latest electronic noses - high-tech sensors that can detect and report specific smells - are quickly improving their levels of speed and accuracy.” https://www.bbc.com/news/business-67354443

10. How Laws Evolved by Natural Selection https://www.psychologytoday.com/us/blog/202308/how-laws-evolved-by-natural-selection

11. “Cosmological natural selection… of universes capable of prolific reproduction. It was originally proposed that reproduction occurs through singularities resulting from supernovae, and subsequently argued that life may facilitate the production of the singularities that become offspring universes. Here I argue technology is necessary for production of singularities by living beings, and ask whether the physics of our universe has been selected to simultaneously enable stars, intelligent life, and technology capable of creating progeny. Specific technologies appear implausibly equipped to perform tasks necessary for production of singularities…” https://iopscience.iop.org/article/10.1088/1367-2630/ab9d9d/pdf

12. “How do elite groups form? The truth is that, in many cases, elite groups are genetically different from the rest of the population.” https://www.aporiamagazine.com/p/how-do-elite-groups-form

Computer scientist who had his bike stolen tried to explain binary search to the cops.

Source: https://archive.is/TgrKu

Links for 2023-11-27

1. A framework for robotic excavation and dry stone construction using on-site materials https://www.youtube.com/watch?v=TxpE5yryCTU

2. AI can figure out sewing patterns from a single photo of clothing https://www.newscientist.com/article/2404358-ai-can-figure-out-sewing-patterns-from-a-single-photo-of-clothing/ [https://archive.is/T2q7u]

3. Cell-free quest for new antibiotics: A new method combines synthetic biology with artificial intelligence https://www.nature.com/articles/s41467-023-42434-9

4. Introducing EUGENe: An Easy-to-Use Deep Learning Genomics Software https://today.ucsd.edu/story/introducing-eugene-an-easy-to-use-deep-learning-genomics-software

5. Does creatine supplementation make you smarter? The largest RCT to date. Weak Bayesian evidence for a small effect (not significant), still uncertain. Veg vs omni made no difference. Previous studies likely too small. https://www.lesswrong.com/posts/CbaznRo9fKpriw2mi/paper-out-now-on-creatine-and-cognitive-performance

6. Aaron Silverbook on anti-cavity bacteria https://www.lesswrong.com/posts/CdxdMLbBwHgEcGYCv/aaron-silverbook-on-anti-cavity-bacteria

7. Researchers 3D printed and deployed a communication satellite in 90 min https://www.uoc.edu/portal/en/news/actualitat/2023/279-nanosatellite-for-emergency-broadband.html

Links for 2023-11-28

1. GPQA, a graduate-level “Google-proof” Q&A benchmark designed for scalable oversight! GPQA is a dataset of *really hard* questions that PhDs with full access to Google can’t answer. Highly skilled non-expert humans spending 30+ mins with unrestricted web search: 34% accuracy; GPT-4: 39%; PhDs in corresponding domains: 65%. https://arxiv.org/abs/2311.12022

2. Rankitect: Ranking Architecture Search Battling World-class Engineers at Meta Scale https://arxiv.org/abs/2311.08430

3. Simplifying Transformer Blocks https://arxiv.org/abs/2311.01906

4. PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics https://xpandora.github.io/PhysGaussian/

5. Baidu Inc. reported a better-than-expected 6% increase in revenue, partly due to the early stages of monetizing its ChatGPT-style AI, Ernie bots https://www.bnnbloomberg.ca/baidu-s-revenue-beats-as-chatgpt-style-ai-starts-monetization-1.2001478

6. Doug Lenat's source code for AM and EURISKO (+Traveller?) found in public archives https://white-flame.com/am-eurisko.html

7. Northvolt in new sodium-ion battery breakthrough https://www.ft.com/content/826fb9bb-b27f-4c68-ad4b-a54ad74713c9 [https://archive.is/HBjrC]

8. Electric car battery prices are going back down faster than expected https://electrek.co/2023/11/20/electric-car-battery-prices-are-going-back-down-faster/

9. “Individuals with lower intelligence levels (typically measured as IQ) tend to be more likely to engage in criminal behavior," and there is "consistent evidence of a mostly linear" association between low IQ and criminality. https://psycnet.apa.org/record/2015-31412-011

10. “Caring about race differences is not a fetish: Critics accuse race realists of obsessing over race. But refuting widespread lies about race is not an intellectual failure or a fetish. It is a duty.” https://www.aporiamagazine.com/p/caring-about-race-differences-is

All SpaceX launches from Cape Canaveral in 13 years:

https://twitter.com/AlexSvanArt/status/1630896377176702977

Pika 1.0, the idea-to-video platform that brings your creativity to life: https://pika.art/

"You know how image generation went from blurry 32x32 texture patches to high-resolution images that are difficult to distinguish from real in roughly a snap of a finger? The same is now happening along the time axis (extending to video) and the repercussions boggle the mind just a bit. Every human becomes a director of multi-modal dreams, like the architect in Inception." -- Andrej Karpathy

Links for 2023-11-29

1. GPT-4 for radiology. Far from perfect, but state-of-the-art performance on some tasks: “Surprisingly, we found radiology report summaries generated by GPT-4 to be comparable and, in some cases, even preferred over those written by experienced radiologists” https://www.microsoft.com/en-us/research/blog/gpt-4s-potential-in-shaping-the-future-of-radiology/

2. NVIDIA Powers Training for Some of the Largest Amazon Titan Foundation Models https://blogs.nvidia.com/blog/nemo-amazon-titan/

3. An Embodied Generalist Agent in 3D World https://embodied-generalist.github.io/

4. Bard can now watch YouTube videos for you https://www.theverge.com/2023/11/22/23972636/bard-youtube-extension-update-search-video-content

5. Vitalik Buterin: "AI risk 1: existential risk. My p(doom) is around 0.1." https://vitalik.eth.limo/general/2023/11/27/techno_optimism.html (thread: https://twitter.com/VitalikButerin/status/1729251822391447904)

6. Apocalypse insurance, and the hardline libertarian take on AI risk https://www.lesswrong.com/posts/mSeesg7i4d9scWAet/apocalypse-insurance-and-the-hardline-libertarian-take-on-ai

7. 5 more brain-computer interface companies you need to know https://www.massdevice.com/brain-computer-interface-companies-need-to-know-2023/

8. “Hence at the macroevolutionary scale, there can be said to have been a “ten-million-year explosion” in primate G leading up to modern humans.” https://www.sciencedirect.com/science/article/abs/pii/S0160289623000764

9. The prefrontal cortex: from monkey to man https://academic.oup.com/brain/advance-article/doi/10.1093/brain/awad389/7424860

10. First of a 3-part series on advances in the aging biotech space. Stay tuned for future posts focusing on reprogramming and cell/organ replacement! https://medium.com/prime-movers-lab/aging-biotech-moving-beyond-the-hallmarks-b2c417ec59e8 [https://archive.is/wip/ICXyz]

11. No Digging Required: Space Mining on the Moon and Beyond May Be Solar Powered https://www.space.com/moon-asteroid-space-mining-with-concentrated-sunlight.html

Links for 2023-11-30

1. AI tool GNoME finds 2.2 million new crystals, including 380,000 stable materials that could power future technologies 💎 https://deepmind.google/discover/blog/millions-of-new-materials-discovered-with-deep-learning/

2. “We trained a small transformer (100M params) for basic arithmetic. W. the right training data it nails 12x12 digits multiplication w/o CoT (that's 10^24 possibilities, so no it's not memorization🤣).” https://arxiv.org/abs/2311.14737

3. ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Search — Uses algorithms like A* to improve LLM answers, improving sota on both planning and reasoning tasks https://arxiv.org/abs/2310.13227

4. “Google DeepMind work creating agents that can mimic experts in goal-driven behaviour in real-time 3D environments. What sets their AI agent apart is its ability to learn from humans in real-time, adopting a third-person perspective. It mirrors the way we learn by observing others – quick and efficient!” [Note: The preprint was already published in March 2022] https://www.nature.com/articles/s41467-023-42875-2

5. Goodhart's law Example: Training Verifiers to Solve Math Word Problems https://www.lesswrong.com/posts/euwMMxwuBeS5QZkC4/goodheart-s-law-example-training-verifiers-to-solve-math

6. The Surprising Genius of Sewing Machines https://www.youtube.com/watch?v=RQYuyHNLPTQ

7. Nano-Sized Powerhouses: Ultrafast Laser Technology Miniaturized on Tiny Photonic Chips https://www.caltech.edu/about/news/ultrafast-lasers-on-ultra-tiny-chips

8. Antares is “building micro-sized nuclear reactors to provide power to remote off-grid locations,” with a vision of “abundant clean energy for all, from Earth to the asteroid belt” https://twitter.com/juliadewahl/status/1719448793027109114

9. FDA has agreed that Loyal’s first drug supports a reasonable expectation of large dog lifespan extension. https://loyalfordogs.com/posts/loyal-announces-historic-fda-milestone-for-large-dog-lifespan-extension-drug

10. In Continued Defense Of Effective Altruism https://www.astralcodexten.com/p/in-continued-defense-of-effective

11. In a sample of 259 extremely high IQ adolescents (from the top 0.01% of the population in intelligence), 37% had doctorates, 9% had patents, 39% had peer-reviewed publications, along with various other accomplishments. https://journals.sagepub.com/doi/abs/10.1177/0956797616644735

Links for 2023-12-01

1. Large-scale pancreatic cancer detection via non-contrast CT and deep learning https://www.nature.com/articles/s41591-023-02640-w

2. “Welcome to the future. ‘Over 17 days of continuous operation, the A-Lab realized 41 novel compounds from a set of 58 targets… identified using large-scale ab initio phase-stability data from the Materials Project and Google DeepMind.” https://www.nature.com/articles/s41586-023-06734-w

3. “Machine learning for neuroscience: We build interpretable biological network reconstructions from electrode recordings with ML and optimal transport.” https://arxiv.org/abs/2306.03603

4. Q: “The reports about the Q* model breakthrough that you all recently made, what’s going on there?” Altman: “No particular comment on that unfortunate leak. But what we have been saying — two weeks ago, what we are saying today, what we’ve been saying a year ago, what we were saying earlier on — is that we expect progress in this technology to continue to be rapid...You can always hit a wall, but we expect that progress will continue to be significant.” https://www.theverge.com/2023/11/29/23982046/sam-altman-interview-openai-ceo-rehired

5. Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models https://sliders.baulab.info/

6. P(doom) roundup: what probability do people put on AI killing everyone? https://twitter.com/AISafetyMemes/status/1729892336782524676

7. "E. coli bacteria use iron levels as a way to store information about different behaviors that can then be activated in response to certain stimuli... These iron memories persist for at least four generations and disappear by the seventh generation." https://news.utexas.edu/2023/11/21/bacteria-store-memories-and-pass-them-on-for-generations/

8. A combinatorially complete fitness landscape with >260.000 E.coli genotypes shows unexpected 🤯 properties. Despite ~500 fitness peaks in this landscape, adaptive evolution can easily navigate to highest fitness peaks https://www.science.org/doi/10.1126/science.adh3860

9. Genes that increase the chances of getting addicted to alcohol increase the risk of feeling lonely. Genes that increase the risk for autism increase IQ. Genes that increase the risk for anorexia increase the chances of getting a higher education 🧬🧠 [published in 2021] https://bigthink.com/health/tiny-genetic-differences-behavioral-effects/

10. Births in South Korea are down by 11.5% in Q3 2023 compared to Q3 2022. Seoul TFR: 0.54 https://twitter.com/nonebusinesshey/status/1729710339598569513

Astronomers have detected over 5,300 planets near other stars.

Here's a beautiful visualization of some of them: https://www.halcyonmaps.com/terrestrial-exoplanets/

"A price is a signal wrapped up in an incentive."

https://mru.org/courses/principles-economics-microeconomics/price-system-spontaneous-order

Links for 2023-12-02

1. “We just made speech translation a whole lot better! Introducing Seamless, an AI model that translates speech in real-time while also maintaining similar vocal style.” https://ai.meta.com/research/seamless-communication/

2. What if the missing ingredient for AGI is not more compute, or more parameters, or even a neat new algorithm, but rather social learning? A social path to human-like artificial intelligence https://rdcu.be/drm0j

3. “This exciting paper shows AI design of materials, robotic synthesis. 10s of new compounds in 17 days. But did they? This paper has very serious problems in materials characterisation. In my view it should never have got near publication. Hold on tight let's take a look...” https://twitter.com/Robert_Palgrave/status/1730358675523424344

4. DeepMind has formalized a theoretical result related to AI safety in Lean. https://twitter.com/davidad/status/1729461156618637502

5. “We are excited to open-source the self-operating computer framework that enables multimodal models, including gpt-4-vision to simulate human-like mouse clicks and keyboard inputs on a computer.” https://github.com/OthersideAI/self-operating-computer

6. On possible cross-fertilization between AI and neuroscience https://www.lesswrong.com/posts/pL7CT2iJYaA4ne9Yd/on-possible-cross-fertilization-between-ai-and-neuroscience

7. Groq Says It Can Deploy 1 Million AI Inference Chips In Two Years https://www.nextplatform.com/2023/11/27/groq-says-it-can-deploy-1-million-ai-inference-chips-in-two-years/

8. “XTX Markets has just launched the "Artificial Intelligence-Mathematical Olympiad (AI-MO)" prize fund, totaling USD 10 million” https://mathstodon.xyz/@tao/111485815228734243 (“A strong IMO competitor is not unlike an athlete. It takes years of problem solving for multiple hours a day to develop the background and skills to compete at this level.” https://twitter.com/rm_rafailov/status/1729670630495145984)

9. An open source AI that proves theorems https://learnandburn.ai/p/an-open-source-ai-that-proves-theorems

10. Could a single alien message destroy us? https://www.youtube.com/watch?v=st9EJg_t6yc

"Microsoft's new Florence 2 is big for Computer Vision.
It's a merge between Text and Vision.

With a single prompt you can instruct the model to do CV tasks like captioning, object detection, grounding, and segmentation.

The best part, it only uses a single backbone to handle everything.

▸ Excels in zero-shot performance
▸ Unified model for detection, captioning, etc.
▸ FLD-5B dataset: 5B+ annotations, 126M images
▸ New benchmarks (>5.5+) on COCO, ADE20K"

https://arxiv.org/abs/2311.06242

via AlphaSignalAI

Links for 2023-12-03

1. Using AI to control an electron microscope to rearrange individual atoms! https://arxiv.org/abs/2311.17894v1

2. CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generation https://codi-2.github.io/

3. “Meet Dobb·E: a home robot system that needs just 5 minutes of human teaching to learn new tasks. Dobb·E has visited 10 homes, learned 100+ tasks, and we are just getting started!” https://dobb-e.com/

4. Calibrated LLMs Must Hallucinate: For facts whose veracity cannot be determined from training data, hallucination is necessary for calibrated LLMs; Probability of generating a hallucination is close to fraction of facts that occur exactly once in data https://arxiv.org/abs/2311.14648

5. Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine https://arxiv.org/abs/2311.16452

6. “Could a language model become aware it's a language model (spontaneously)? Could it be aware it’s deployed publicly vs in training? Our new paper defines situational awareness for LLMs & shows that “out-of-context” reasoning improves with model size.” https://www.lesswrong.com/posts/mLfPHv4QjmeQrsSva/paper-on-measuring-situational-awareness-in-llms

7. SDXL Turbo: A real-time text-to-image generation model. https://stability.ai/news/stability-ai-sdxl-turbo

8. New work analysing how fine-tuning works mechanistically https://arxiv.org/abs/2311.12786

9. OpenAI's Bet on a Cognitive Architecture https://blog.langchain.dev/openais-bet-on-a-cognitive-architecture/

10. A first-of-its-kind geothermal project is now operational https://blog.google/outreach-initiatives/sustainability/google-fervo-geothermal-energy-partnership/

11. “A part of the male sparrow’s brain called the HVC grows from around 100,000 neurons to about 170,000 — nearly doubling in size — during the bird’s mating season.” https://www.nature.com/articles/d41586-023-03746-4

12. Penguins obtain large quantities of sleep via seconds-long microsleeps https://www.mpg.de/21169426/1127-orni-penguins-nesting-in-a-dangerous-environment-obtain-large-quantities-of-sleep-via-seconds-long-microsleeps-154562-x

Netflix doesn’t know how chess works: https://www.reddit.com/r/chess/comments/189klzb/netflix_doesnt_know_how_chess_works/

A lot of people jump on this, but if you think about it, it is actually a consistent definition. Sure, you could argue that it is conceptually ugly and misleading. But defining 1/0 as equal to 0 does not imply a contradiction, because it is different from saying that 0 has a multiplicative inverse.