Links for 2023-11-28

1. GPQA, a graduate-level “Google-proof” Q&A benchmark designed for scalable oversight! GPQA is a dataset of *really hard* questions that PhDs with full access to Google can’t answer. Highly skilled non-expert humans spending 30+ mins with unrestricted web search: 34% accuracy; GPT-4: 39%; PhDs in corresponding domains: 65%. https://arxiv.org/abs/2311.12022

2. Rankitect: Ranking Architecture Search Battling World-class Engineers at Meta Scale https://arxiv.org/abs/2311.08430

3. Simplifying Transformer Blocks https://arxiv.org/abs/2311.01906

4. PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics https://xpandora.github.io/PhysGaussian/

5. Baidu Inc. reported a better-than-expected 6% increase in revenue, partly due to the early stages of monetizing its ChatGPT-style AI, Ernie bots https://www.bnnbloomberg.ca/baidu-s-revenue-beats-as-chatgpt-style-ai-starts-monetization-1.2001478

6. Doug Lenat's source code for AM and EURISKO (+Traveller?) found in public archives https://white-flame.com/am-eurisko.html

7. Northvolt in new sodium-ion battery breakthrough https://www.ft.com/content/826fb9bb-b27f-4c68-ad4b-a54ad74713c9 [https://archive.is/HBjrC]

8. Electric car battery prices are going back down faster than expected https://electrek.co/2023/11/20/electric-car-battery-prices-are-going-back-down-faster/

9. “Individuals with lower intelligence levels (typically measured as IQ) tend to be more likely to engage in criminal behavior," and there is "consistent evidence of a mostly linear" association between low IQ and criminality. https://psycnet.apa.org/record/2015-31412-011

10. “Caring about race differences is not a fetish: Critics accuse race realists of obsessing over race. But refuting widespread lies about race is not an intellectual failure or a fetish. It is a duty.” https://www.aporiamagazine.com/p/caring-about-race-differences-is

All SpaceX launches from Cape Canaveral in 13 years:

https://twitter.com/AlexSvanArt/status/1630896377176702977

Pika 1.0, the idea-to-video platform that brings your creativity to life: https://pika.art/

"You know how image generation went from blurry 32x32 texture patches to high-resolution images that are difficult to distinguish from real in roughly a snap of a finger? The same is now happening along the time axis (extending to video) and the repercussions boggle the mind just a bit. Every human becomes a director of multi-modal dreams, like the architect in Inception." -- Andrej Karpathy

Links for 2023-11-29

1. GPT-4 for radiology. Far from perfect, but state-of-the-art performance on some tasks: “Surprisingly, we found radiology report summaries generated by GPT-4 to be comparable and, in some cases, even preferred over those written by experienced radiologists” https://www.microsoft.com/en-us/research/blog/gpt-4s-potential-in-shaping-the-future-of-radiology/

2. NVIDIA Powers Training for Some of the Largest Amazon Titan Foundation Models https://blogs.nvidia.com/blog/nemo-amazon-titan/

3. An Embodied Generalist Agent in 3D World https://embodied-generalist.github.io/

4. Bard can now watch YouTube videos for you https://www.theverge.com/2023/11/22/23972636/bard-youtube-extension-update-search-video-content

5. Vitalik Buterin: "AI risk 1: existential risk. My p(doom) is around 0.1." https://vitalik.eth.limo/general/2023/11/27/techno_optimism.html (thread: https://twitter.com/VitalikButerin/status/1729251822391447904)

6. Apocalypse insurance, and the hardline libertarian take on AI risk https://www.lesswrong.com/posts/mSeesg7i4d9scWAet/apocalypse-insurance-and-the-hardline-libertarian-take-on-ai

7. 5 more brain-computer interface companies you need to know https://www.massdevice.com/brain-computer-interface-companies-need-to-know-2023/

8. “Hence at the macroevolutionary scale, there can be said to have been a “ten-million-year explosion” in primate G leading up to modern humans.” https://www.sciencedirect.com/science/article/abs/pii/S0160289623000764

9. The prefrontal cortex: from monkey to man https://academic.oup.com/brain/advance-article/doi/10.1093/brain/awad389/7424860

10. First of a 3-part series on advances in the aging biotech space. Stay tuned for future posts focusing on reprogramming and cell/organ replacement! https://medium.com/prime-movers-lab/aging-biotech-moving-beyond-the-hallmarks-b2c417ec59e8 [https://archive.is/wip/ICXyz]

11. No Digging Required: Space Mining on the Moon and Beyond May Be Solar Powered https://www.space.com/moon-asteroid-space-mining-with-concentrated-sunlight.html

Links for 2023-11-30

1. AI tool GNoME finds 2.2 million new crystals, including 380,000 stable materials that could power future technologies 💎 https://deepmind.google/discover/blog/millions-of-new-materials-discovered-with-deep-learning/

2. “We trained a small transformer (100M params) for basic arithmetic. W. the right training data it nails 12x12 digits multiplication w/o CoT (that's 10^24 possibilities, so no it's not memorization🤣).” https://arxiv.org/abs/2311.14737

3. ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Search — Uses algorithms like A* to improve LLM answers, improving sota on both planning and reasoning tasks https://arxiv.org/abs/2310.13227

4. “Google DeepMind work creating agents that can mimic experts in goal-driven behaviour in real-time 3D environments. What sets their AI agent apart is its ability to learn from humans in real-time, adopting a third-person perspective. It mirrors the way we learn by observing others – quick and efficient!” [Note: The preprint was already published in March 2022] https://www.nature.com/articles/s41467-023-42875-2

5. Goodhart's law Example: Training Verifiers to Solve Math Word Problems https://www.lesswrong.com/posts/euwMMxwuBeS5QZkC4/goodheart-s-law-example-training-verifiers-to-solve-math

6. The Surprising Genius of Sewing Machines https://www.youtube.com/watch?v=RQYuyHNLPTQ

7. Nano-Sized Powerhouses: Ultrafast Laser Technology Miniaturized on Tiny Photonic Chips https://www.caltech.edu/about/news/ultrafast-lasers-on-ultra-tiny-chips

8. Antares is “building micro-sized nuclear reactors to provide power to remote off-grid locations,” with a vision of “abundant clean energy for all, from Earth to the asteroid belt” https://twitter.com/juliadewahl/status/1719448793027109114

9. FDA has agreed that Loyal’s first drug supports a reasonable expectation of large dog lifespan extension. https://loyalfordogs.com/posts/loyal-announces-historic-fda-milestone-for-large-dog-lifespan-extension-drug

10. In Continued Defense Of Effective Altruism https://www.astralcodexten.com/p/in-continued-defense-of-effective

11. In a sample of 259 extremely high IQ adolescents (from the top 0.01% of the population in intelligence), 37% had doctorates, 9% had patents, 39% had peer-reviewed publications, along with various other accomplishments. https://journals.sagepub.com/doi/abs/10.1177/0956797616644735

Links for 2023-12-01

1. Large-scale pancreatic cancer detection via non-contrast CT and deep learning https://www.nature.com/articles/s41591-023-02640-w

2. “Welcome to the future. ‘Over 17 days of continuous operation, the A-Lab realized 41 novel compounds from a set of 58 targets… identified using large-scale ab initio phase-stability data from the Materials Project and Google DeepMind.” https://www.nature.com/articles/s41586-023-06734-w

3. “Machine learning for neuroscience: We build interpretable biological network reconstructions from electrode recordings with ML and optimal transport.” https://arxiv.org/abs/2306.03603

4. Q: “The reports about the Q* model breakthrough that you all recently made, what’s going on there?” Altman: “No particular comment on that unfortunate leak. But what we have been saying — two weeks ago, what we are saying today, what we’ve been saying a year ago, what we were saying earlier on — is that we expect progress in this technology to continue to be rapid...You can always hit a wall, but we expect that progress will continue to be significant.” https://www.theverge.com/2023/11/29/23982046/sam-altman-interview-openai-ceo-rehired

5. Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models https://sliders.baulab.info/

6. P(doom) roundup: what probability do people put on AI killing everyone? https://twitter.com/AISafetyMemes/status/1729892336782524676

7. "E. coli bacteria use iron levels as a way to store information about different behaviors that can then be activated in response to certain stimuli... These iron memories persist for at least four generations and disappear by the seventh generation." https://news.utexas.edu/2023/11/21/bacteria-store-memories-and-pass-them-on-for-generations/

8. A combinatorially complete fitness landscape with >260.000 E.coli genotypes shows unexpected 🤯 properties. Despite ~500 fitness peaks in this landscape, adaptive evolution can easily navigate to highest fitness peaks https://www.science.org/doi/10.1126/science.adh3860

9. Genes that increase the chances of getting addicted to alcohol increase the risk of feeling lonely. Genes that increase the risk for autism increase IQ. Genes that increase the risk for anorexia increase the chances of getting a higher education 🧬🧠 [published in 2021] https://bigthink.com/health/tiny-genetic-differences-behavioral-effects/

10. Births in South Korea are down by 11.5% in Q3 2023 compared to Q3 2022. Seoul TFR: 0.54 https://twitter.com/nonebusinesshey/status/1729710339598569513

Astronomers have detected over 5,300 planets near other stars.

Here's a beautiful visualization of some of them: https://www.halcyonmaps.com/terrestrial-exoplanets/

"A price is a signal wrapped up in an incentive."

https://mru.org/courses/principles-economics-microeconomics/price-system-spontaneous-order

Links for 2023-12-02

1. “We just made speech translation a whole lot better! Introducing Seamless, an AI model that translates speech in real-time while also maintaining similar vocal style.” https://ai.meta.com/research/seamless-communication/

2. What if the missing ingredient for AGI is not more compute, or more parameters, or even a neat new algorithm, but rather social learning? A social path to human-like artificial intelligence https://rdcu.be/drm0j

3. “This exciting paper shows AI design of materials, robotic synthesis. 10s of new compounds in 17 days. But did they? This paper has very serious problems in materials characterisation. In my view it should never have got near publication. Hold on tight let's take a look...” https://twitter.com/Robert_Palgrave/status/1730358675523424344

4. DeepMind has formalized a theoretical result related to AI safety in Lean. https://twitter.com/davidad/status/1729461156618637502

5. “We are excited to open-source the self-operating computer framework that enables multimodal models, including gpt-4-vision to simulate human-like mouse clicks and keyboard inputs on a computer.” https://github.com/OthersideAI/self-operating-computer

6. On possible cross-fertilization between AI and neuroscience https://www.lesswrong.com/posts/pL7CT2iJYaA4ne9Yd/on-possible-cross-fertilization-between-ai-and-neuroscience

7. Groq Says It Can Deploy 1 Million AI Inference Chips In Two Years https://www.nextplatform.com/2023/11/27/groq-says-it-can-deploy-1-million-ai-inference-chips-in-two-years/

8. “XTX Markets has just launched the "Artificial Intelligence-Mathematical Olympiad (AI-MO)" prize fund, totaling USD 10 million” https://mathstodon.xyz/@tao/111485815228734243 (“A strong IMO competitor is not unlike an athlete. It takes years of problem solving for multiple hours a day to develop the background and skills to compete at this level.” https://twitter.com/rm_rafailov/status/1729670630495145984)

9. An open source AI that proves theorems https://learnandburn.ai/p/an-open-source-ai-that-proves-theorems

10. Could a single alien message destroy us? https://www.youtube.com/watch?v=st9EJg_t6yc

"Microsoft's new Florence 2 is big for Computer Vision.
It's a merge between Text and Vision.

With a single prompt you can instruct the model to do CV tasks like captioning, object detection, grounding, and segmentation.

The best part, it only uses a single backbone to handle everything.

▸ Excels in zero-shot performance
▸ Unified model for detection, captioning, etc.
▸ FLD-5B dataset: 5B+ annotations, 126M images
▸ New benchmarks (>5.5+) on COCO, ADE20K"

https://arxiv.org/abs/2311.06242

via AlphaSignalAI

Links for 2023-12-03

1. Using AI to control an electron microscope to rearrange individual atoms! https://arxiv.org/abs/2311.17894v1

2. CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generation https://codi-2.github.io/

3. “Meet Dobb·E: a home robot system that needs just 5 minutes of human teaching to learn new tasks. Dobb·E has visited 10 homes, learned 100+ tasks, and we are just getting started!” https://dobb-e.com/

4. Calibrated LLMs Must Hallucinate: For facts whose veracity cannot be determined from training data, hallucination is necessary for calibrated LLMs; Probability of generating a hallucination is close to fraction of facts that occur exactly once in data https://arxiv.org/abs/2311.14648

5. Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine https://arxiv.org/abs/2311.16452

6. “Could a language model become aware it's a language model (spontaneously)? Could it be aware it’s deployed publicly vs in training? Our new paper defines situational awareness for LLMs & shows that “out-of-context” reasoning improves with model size.” https://www.lesswrong.com/posts/mLfPHv4QjmeQrsSva/paper-on-measuring-situational-awareness-in-llms

7. SDXL Turbo: A real-time text-to-image generation model. https://stability.ai/news/stability-ai-sdxl-turbo

8. New work analysing how fine-tuning works mechanistically https://arxiv.org/abs/2311.12786

9. OpenAI's Bet on a Cognitive Architecture https://blog.langchain.dev/openais-bet-on-a-cognitive-architecture/

10. A first-of-its-kind geothermal project is now operational https://blog.google/outreach-initiatives/sustainability/google-fervo-geothermal-energy-partnership/

11. “A part of the male sparrow’s brain called the HVC grows from around 100,000 neurons to about 170,000 — nearly doubling in size — during the bird’s mating season.” https://www.nature.com/articles/d41586-023-03746-4

12. Penguins obtain large quantities of sleep via seconds-long microsleeps https://www.mpg.de/21169426/1127-orni-penguins-nesting-in-a-dangerous-environment-obtain-large-quantities-of-sleep-via-seconds-long-microsleeps-154562-x

Netflix doesn’t know how chess works: https://www.reddit.com/r/chess/comments/189klzb/netflix_doesnt_know_how_chess_works/

A lot of people jump on this, but if you think about it, it is actually a consistent definition. Sure, you could argue that it is conceptually ugly and misleading. But defining 1/0 as equal to 0 does not imply a contradiction, because it is different from saying that 0 has a multiplicative inverse.

What South Korea's 0.7 births per woman actually means in practice: https://archive.is/cRGQ7

Of course, this assumes that the birth rate doesn't collapse even further.

Links for 2023-12-04

1. “The Pentagon’s portfolio boasts more than 800 AI-related unclassified projects, much still in testing.” https://apnews.com/article/us-military-ai-projects-0773b4937801e7a0573f44b57a9a5942

2. Amazon is using AI to deliver packages faster than ever this holiday season https://www.cbsnews.com/news/amazon-faster-deliveries-ai-holiday-season-cyber-monday-deals/

3. Dell Inks $150 Million Hardware Deal With AI Startup Imbue https://www.bloomberg.com/news/articles/2023-11-28/dell-inks-150-million-computing-deal-with-ai-startup-imbue [https://archive.is/HuRe4]

4. OpenAI signed a deal to order $51M worth of “neuromorphic” AI chips in advance from a startup called Rain AI. https://www.wired.com/story/openai-buy-ai-chips-startup-sam-altman/ [https://archive.is/xUxQ9]

5. "Standard deep learning models can be trained to predict race from medical images with high performance across multiple imaging modalities... We demonstrate this detection is not due to trivial proxies or imaging-related surrogate covariates for race..." https://www.thelancet.com/journals/landig/article/PIIS2589-7500(22)00063-2/fulltext

6. Shoggoth Systems: A “peer-to-peer, anonymous network for publishing and distributing open-source code, Machine Learning models” https://shoggoth.systems/

7. Breakthroughs of 2023 https://twitter.com/g_leech_/status/1731263549182206291 (e.g.: “rigorous sim of a new kind of computer not limited by the usual physical constraints and close to reversible (i.e. free operations).” https://arxiv.org/abs/2202.07122)

8. Bees with bigger brains are better at adapting to urban environments: "[S]pecies with larger brains have enhanced behavioural plasticity, enabling them to confront and adapt to novel challenges." https://doi.org/10.6084/m9.figshare.c.6935821

9. A research roadmap: replacing brain cells to combat brain aging https://static1.squarespace.com/static/64becfbb6ea1fd445440d59d/t/656628168313873881f221d8/1701193751412/brain-aging-white-paper.pdf