KahaDB is a file based persistence database that is local to the message broker that is using it. It has been optimized for fast persistence. It is the the default storage mechanism since ActiveMQ Classic 5.4. KahaDB uses less file descriptors and provides faster recovery than its predecessor, the AMQ Message Store.

In order to facilitate rapid retrieval of messages from the data logs, a B-tree index is created, which contains pointers to the locations of all the messages embedded in the data log files. The complete B-tree index is stored on disk and part or all of the B-tree index is held in a cache in memory. Evidently, the B-tree index can work more efficiently, if the complete index fits into the cache.

https://github.com/apache/activemq/tree/main/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb

Interesting article about Kafka latency issue in scale due to ext4 filesystem, the sciencific approach they followed is really beuatiful.

https://blog.allegro.tech/2024/03/kafka-performance-analysis.html

Slim allows developers to inspect, optimize and debug their containers .

#docker #optimization
https://github.com/slimtoolkit/slim

Kafka Tierd storage feature, a novel approach for archiving kafka messages. IMO NOT READY FOR PRODUCTION BUT KEEP EYE ON IT

https://developers.redhat.com/articles/2024/03/13/kafka-tiered-storage-deep-dive

https://aws.amazon.com/blogs/big-data/deep-dive-on-amazon-msk-tiered-storage/

How neural network works under the hood? explaining about vector embeddings and it's use cases.
https://www.datastax.com/guides/what-is-a-vector-embedding

We've all heard about RocksDB, but there's also Speedb, which may not be as famous as RocksDB, but it appears to be the fastest key-value storage engine in the world. Redis recently announced that they own Speedb and have included it in Redis's core. You can find more information about it here:
https://www.speedb.io/

"I wrote this Format dialog back on a rainy Thursday morning at Microsoft in late 1994, I think it was.

We were porting the bajillion lines of code from the Windows95 user interface over to NT, and Format was just one of those areas where WindowsNT was different enough from Windows95 that we had to come up with some custom UI.

I got out a piece of paper and wrote down all the options and choices you could make with respect to formatting a disk, like filesystem, label, cluster size, compression, encryption, and so on.

Then I busted out VC++2.0 and used the Resource Editor to lay out a simple vertical stack of all the choices you had to make, in the approximate order you had to make. It wasn't elegant, but it would do until the elegant UI arrived.

That was some 30 years ago, and the dialog is still my temporary one from that Thursday morning, so be careful about checking in "temporary" solutions!

I also had to decide how much "cluster slack" would be too much, and that wound up constraining the format size of a FAT volume to 32GB. That limit was also an arbitrary choice that morning, and one that has stuck with us as a permanent side effect.

So remember... there are no "temporary" checkins :)

Follow me for more random code musings!"

From Dave W Plummer, Developer of many famous Windows components such as Task Manager, Windows Pinball, Calc, ZIPFolders, Product Activation, etc.

https://twitter.com/davepl1968/status/1772042158046146792

https://people.freebsd.org/~phk/
Poul-Henning Kamp (a.k.a phk) is one of the most well-known software engineers in the world, known for developing many parts of the FreeBSD kernel and Varnish Cache.

This is one of his notes titled 'Notes from the Architect' on the Varnish Cache website.
https://varnish-cache.org/docs/trunk/phk/notes.html

—
"Well, today computers really only have one kind of storage, and it is usually some sort of disk, the operating system and the virtual memory management hardware has converted the RAM to a cache for the disk storage.

So what happens with squids elaborate memory management is that it gets into fights with the kernels elaborate memory management, and like any civil war, that never gets anything done."

—-
Multi-CPU systems is nothing new, but writing programs that use more than one CPU at a time has always been tricky and it still is.
—-

https://google.github.io/pytype/
https://github.com/facebook/pyre-check/
https://github.com/google/importlab

python static analysis code, especially type checking , and modules dependencies on each other

Do not forget these two rules:

Everything in software architecture is a trade off,
and always why is more important than how.

Anderson's law:
I have yet to see any problem, however complicated, which, when you looked at it in the right way, did not become still more complicated.

so when initially examining a problem, it's common to find that it becomes more complex as you go deeper into it. This complexity often requires a deeper level of investigation to determine the best solution.

#architecture

human interaction could be a bottleneck in most software projects, and should be managed very carefully.
there is a law around it that says "Adding manpower to a late software project makes it later."
it called Brooks's law

#architecture

Whenever you find yourself implementing distributed transactions within a system, keep the meaning of saga word in your mind.

a long story of heroic achievement;
a story of quasi-legendary events; colloquially, a long tale.
- Oxford English Dictionary

https://huly.io/

#slack #notion #jira #aletrnative

Using Postgres WAL trade-offs, rules you should know before making any decision in this area.

https://www.enterprisedb.com/blog/postgresql-wal-write-ahead-logging-management-strategy-tradeoffs

Laws of UX that worth to read for everyone

https://lawsofux.com/

The Unanimous 2PC (U2PC) protocol introduced in this paper is a fault-tolerant variant of the traditional Two-Phase Commit (2PC) designed to combine efficiency, simplicity, and fault tolerance in distributed transactional systems.

https://dl.acm.org/doi/pdf/10.1145/3642976.3653035

The practice makes permenant, but why?
https://www.sec-ed.co.uk/content/best-practice/developing-skills-the-magic-of-myelin/

When it comes to protecting against credentials theft on Windows, enabling LSA Protection (a.k.a. RunAsPPL) on LSASS may be considered as the very first recommendation to implement. But do you really know what a PPL is? https://itm4n.github.io/lsass-runasp
anonymous poll

No, it's the first time I've ever heard about that. – 3
👍👍👍👍👍👍👍 100%

Yes, I was familiar with that.
▫️ 0%

👥 3 people voted so far.

Introduction to RunAsPPL or LSA protection
https://itm4n.github.io/lsass-runasppl/

What will happen in real world if you don't enable that?
"LSASS has the ability to execute arbitrary kernel-mode addresses? I wrote a small proof of concept that allows administrators to execute unsigned code in the kernel if LSA Protection is disabled."
https://tierzerosecurity.co.nz/2024/04/29/kexecdd.html