Hacker News
@hacker_news_feed
24.1K subscribers
118K links
Top stories from https://news.ycombinator.com (with 100+ score)
Contribute to the development here: https://github.com/phil-r/hackernewsbot
Also check https://t.me/designer_news

Contacts: @philr
Download Telegram
About
Blog
Apps
Platform
Join
Hacker News
24.1K subscribers
Hacker News
Show HN: KVSplit – Run 2-3x longer contexts on Apple Silicon (🔥 Score: 150+ in 2 hours)

Link: https://readhacker.news/s/6uBAK
Comments: https://readhacker.news/c/6uBAK

I discovered that in LLM inference, keys and values in the KV cache have very different quantization sensitivities. Keys need higher precision than values to maintain quality.
I patched llama.cpp to enable different bit-widths for keys vs. values on Apple Silicon. The results are surprising:
- K8V4 (8-bit keys, 4-bit values): 59% memory reduction with only 0.86% perplexity loss
- K4V8 (4-bit keys, 8-bit values): 59% memory reduction but 6.06% perplexity loss
- The configurations use the same number of bits, but K8V4 is 7× better for quality
This means you can run LLMs with 2-3× longer context on the same Mac. Memory usage scales with sequence length, so savings compound as context grows.
Implementation was straightforward:
1. Added --kvq-key and --kvq-val flags to llama.cpp
2. Applied existing quantization logic separately to K and V tensors
3. Validated with perplexity metrics across context lengths
4. Used Metal for acceleration (with -mlong-calls flag to avoid vectorization issues)
Benchmarked on an M4 MacBook Pro running TinyLlama with 8K context windows. Compatible with Metal/MPS and optimized for Apple Silicon.
GitHub: https://github.com/dipampaul17/KVSplit
GitHub
GitHub - dipampaul17/KVSplit: Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache…
Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with &am...
2.5K views
Read13+ Comments
Hacker News
Moody’s strips U.S. of triple-A credit rating (🔥 Score: 153+ in 2 hours)

Link: https://readhacker.news/s/6uBNR
Comments: https://readhacker.news/c/6uBNR
Ft
Moody’s strips US of top-notch triple-A credit rating
Agency warns of strains caused by rising government debt and a widening budget deficit
2.5K views
Read76+ Comments
Hacker News
X X^t can be faster (Score: 150+ in 8 hours)

Link: https://readhacker.news/s/6uANa
Comments: https://readhacker.news/c/6uANa
arXiv.org
$XX^{t}$ Can Be Faster
We present a new algorithm RXTX that computes product of matrix by its transpose $XX^{t}$. RXTX uses $5\%$ less multiplications and additions than State-of-the-Art and achieves accelerations even...
3.7K views
Read45+ Comments
Hacker News
Java at 30: Interview with James Gosling (Score: 150+ in 11 hours)

Link: https://readhacker.news/s/6uAdJ
Comments: https://readhacker.news/c/6uAdJ
The New Stack
Java at 30: The Genius Behind the Code That Changed Tech
From trash-diving teen to tech pioneer, James Gosling's pragmatic genius shaped three decades of Java and modern computing.
2.7K views
Read205+ Comments
Hacker News
Getting AI to write good SQL (Score: 154+ in 4 hours)

Link: https://readhacker.news/s/6uBLa
Comments: https://readhacker.news/c/6uBLa
Google Cloud Blog
Techniques for improving text-to-SQL | Google Cloud Blog
Learn about text-to-SQL techniques like context building and table retrieval, LLM-as-a-judge, and LLM prompting and post-processing.
2.8K views
Read81+ Comments
Hacker News
ClojureScript 1.12.42 (Score: 151+ in 11 hours)

Link: https://readhacker.news/s/6uBDi
Comments: https://readhacker.news/c/6uBDi
2.6K views
Read26+ Comments
Hacker News
Pathfinding (Score: 150+ in 1 day)

Link: https://readhacker.news/s/6uwP7
Comments: https://readhacker.news/c/6uwP7
itch.io
#9 - Pathfinding
Hello! I've recently been working on the pathfinding for NPCs in my game, which is something I've been looking forward to for a while now since it's a nice chunky problem to solve. I thought I'd write...
2.6K views
Read44+ Comments
Hacker News
Náhuatl and Mayan Language Renaissance Occurring in Mexico (❄️ Score: 150+ in 3 days)

Link: https://readhacker.news/s/6uquf
Comments: https://readhacker.news/c/6uquf
Yucatán Magazine
Náhuatl and Mayan Language Renaissance Occurring in Mexico
México is home to 68 officially recognized Indigenous languages and is experiencing a remarkable Mayan language renaissance
2.5K views
Read65+ Comments
Hacker News
JavaScript's New Superpower: Explicit Resource Management (Score: 150+ in 8 hours)

Link: https://readhacker.news/s/6uCwD
Comments: https://readhacker.news/c/6uCwD
v8.dev
JavaScript's New Superpower: Explicit Resource Management · V8
The Explicit Resource Management proposal empowers developers to explicitly manage the lifecycle of resources.
2.5K views
Read94+ Comments
Hacker News
Evolution of Rust Compiler Errors (Score: 150+ in 1 day)

Link: https://readhacker.news/s/6uAh5
Comments: https://readhacker.news/c/6uAh5
Kobzol’s blog
Evolution of Rust compiler errors
Blog about programming stuff.
2.5K views
Read32+ Comments
Hacker News
Wow@Home – Network of Amateur Radio Telescopes (Score: 150+ in 12 hours)

Link: https://readhacker.news/s/6uCht
Comments: https://readhacker.news/c/6uCht
phl.upr.edu
PHL @ UPR Arecibo - outreach
Introduction
2.6K views
Read18+ Comments
Hacker News
Japan's IC cards are weird and wonderful (❄️ Score: 154+ in 2 days)

Link: https://readhacker.news/s/6uwBZ
Comments: https://readhacker.news/c/6uwBZ
aruarian.dance
Japan's IC cards are weird and wonderful
Exploring what makes Japan's transit cards so unique compared to the West.
2.5K views
Read129+ Comments
Hacker News
XTool – Cross-platform Xcode replacement (Score: 150+ in 12 hours)

Link: https://readhacker.news/s/6uChV
Comments: https://readhacker.news/c/6uChV
GitHub
GitHub - xtool-org/xtool: Cross-platform Xcode replacement. Build and deploy iOS apps with SwiftPM on Linux, Windows, macOS.
Cross-platform Xcode replacement. Build and deploy iOS apps with SwiftPM on Linux, Windows, macOS. - xtool-org/xtool
3.3K views
Read38+ Comments
Hacker News
Push Ifs Up and Fors Down (Score: 152+ in 6 hours)

Link: https://readhacker.news/s/6uCPf
Comments: https://readhacker.news/c/6uCPf
matklad.github.io
Push Ifs Up And Fors Down
A short note on two related rules of thumb.
2.6K views
Read69+ Comments
Hacker News
Rustls Server-Side Performance (❄️ Score: 150+ in 4 days)

Link: https://readhacker.news/s/6upUf
Comments: https://readhacker.news/c/6upUf
Prossimo
Rustls Server-Side Performance
In past years, the Rustls project has been happy to receive substantial investments from the ISRG. One of our goals has been to improve performance without compromising on safety. We last posted about our performance improvements in October of 2024, and we're…
2.6K views
Read46+ Comments
Hacker News
MCP: An in-depth introduction (❄️ Score: 150+ in 4 days)

Link: https://readhacker.news/s/6upNg
Comments: https://readhacker.news/c/6upNg
Speakeasy
MCP: an in-depth introduction | Speakeasy
The API Platform. Generate, test, monitor, and document idiomatic SDKs directly from your OpenAPI spec.
2.8K views
Read64+ Comments
Hacker News
Palette lighting tricks on the Nintendo 64 (Score: 151+ in 5 hours)

Link: https://readhacker.news/s/6uDgM
Comments: https://readhacker.news/c/6uDgM
2.6K views
Read25+ Comments
Hacker News
"We would be less confidential than Google" Proton threatens to quit Switzerland (Score: 150+ in 4 hours)

Link: https://readhacker.news/s/6uDkJ
Comments: https://readhacker.news/c/6uDkJ
TechRadar
"We would be less confidential than Google" – Proton threatens to quit Switzerland over new surveillance law
An amendment to the Swiss surveillance law would require VPNs and messaging apps to identify and retain user data, undermining their privacy and security framework.
2.8K views
Read74+ Comments
Hacker News
A kernel developer plays with Home Assistant (Score: 151+ in 19 hours)

Link: https://readhacker.news/s/6uCfx
Comments: https://readhacker.news/c/6uCfx
LWN.net
A kernel developer plays with Home Assistant: general impressions
Those of us who have spent our lives playing with computers naturally see the appeal of deployi [...]
2.5K views
Read168+ Comments
Hacker News
Coding Without a Laptop – Two Weeks with AR Glasses and Linux on Android (❄️ Score: 151+ in 3 days)

Link: https://readhacker.news/s/6utZB
Comments: https://readhacker.news/c/6utZB
Holdtherobot
Coding Without a Laptop - Two Weeks with AR Glasses and Linux on Android | Hold The Robot
I recently learned something that blew my mind;
2.6K views
Read39+ Comments