I'm looking into setting up open source reproducible blueprints / performance tests. So that they can be run on the hardware paid by the company, but the code remains open & reusable. Is this a crazy idea?

My post for Inteviewing.io about the differences in various interview scores.

New episode, Leveraging the Cores: from CPU and the OS kernel to coroutines, green threads, and actor models.

And the Distributed Task Queue episode is now released!

Also, we are now at 1K YouTube subscribers, yay!

Nothing extraordinary, just the SysDesign Meetup creating time loops.

Our OPA C++ Transpilation blog post, thanks to Max.

I keep reading more and more about GraphQL [1]. It's useful both for my meetup and for my work, so keeping up is more and more of my professional duties than intellectual curiosity these days.

Originally — say, ~three years ago — I was quite skeptical about the prospects of this "language". Back then, it didn't even have mutations, after all, and "JOIN"-ing data between, say, PostgreSQL and Redis was remarkably inefficient. Besides, the query language is not standard and not mapped 1:1 to JavaScript or anything else popular, and type checking was semi-strict. So it did look like an interesting pet project, but I was not convinced it would have a future.

Today, after hearing more and more about GraphQL's power, and after doing more research, I'm ready to concede I was largely wrong. With @defer, @stream and `@live`GraphQL may well be the future Data API language. Yes, some of these features are still deeply experimental, and I've got quite a few concerns about their theoretical performance.

In a way, the role of a "GraphQL DBA" should be emerging as we speak.

And then it clicked. I blogged about such a future back in 2014. My prediction back then [2] was that SQL would:

• Exist in 5 years,
• Exist as legacy in 10 years, and
• Go extinct in 15 years.

Well, I have to be rooting for GraphQL now. Because it may well be the force that proves my vision to be right 😊

[1]: New features in GraphQL: Batch, defer, stream, live, and subscribe. 2021.
[2]: The Future of Data Engineering, What would happen after SQL is no more. 2014.

Very proud of this episode we’ve recorded with Interviewing.io.

May well be the best SysDesign interview prep video out there today.

Enjoy responsibly!

https://www.youtube.com/watch?v=IJSVmyhq2i0

Another video with Interviewing.io, codename "two experts design the system". Extremely well moderated by both Kevin & Mike, and pleasure working with Bruno & the broader team on this one.

Folks, it's time to get back on track with SysDesign Meetup!

We've been gathering off the record, but haven't released an episode in a while. We even recorded one in December, but there was a fluke with recording, so it's only released as "unlisted", as our faces are half-in. (Thx Maxim for cropping them away back then!)

Let's make the next event happen. Topic: Benchmarking Production Systems.

We're shooting this Saturday, March the 25th. It turns out the best time is Pacific morning, Eastern afternoon, Europe evening. Please vote in #general in Slack, ref. https://tinyurl.com/sdm-slack-invite.

Benchmarking Production Systems.

We open the can of worms of how much is there to cover on the topic of performance testing beyond measuring the peak RPS that a service can handle.

Video: https://tinyurl.com/sdm-benchmarking
Slides: https://tinyurl.com/sdm-benchmarking-slides

The event dedicated to interviewing.io's SysDesign guide, of which I'm one of the authors, is happening!

San Francisco, USA Fri, 7 Apr 2023 at 11:00 PDT
New York, USA Fri, 7 Apr 2023 at 14:00 EDT
London, United Kingdom Fri, 7 Apr 2023 at 19:00 BST
Amsterdam, Netherlands Fri, 7 Apr 2023 at 20:00 CEST
Tbilisi, Georgia Fri, 7 Apr 2023 at 22:00 GET
Kuala Lumpur, Malaysia Sat, 8 Apr 2023 at 02:00 MYT

Pls join #iio-sd-guide on our Slack if you're interested, and please ask your questions in Slido, regardless of whether you plan to be there live. And here's a Google Calendar link.

Very much looking forward to!

Blockchain and Web3: just released this new episode!

• https://tinyurl.com/sdm-blockchain
• https://tinyurl.com/sdm-blockchain-slides

The history and the state of the art of blockchain and Web3 from first principles.

One of the ideas we are flirting with is to sketch out some “open source design” of a nontrivial architectural component.

If done well, not only it could serve as a reference document on how to prepare such designs, but also a blueprint based on which we could implement the prototype.

I think there’s room for such a design out there today, from contracts and protocol specifications (HTTP JSON to gRPC to binary) all the way to components’ implementation and unit/integration/load testing.

As an example problem I keep coming back to either the Total Order Broadcast engine or a simple De-Duplicator.

The former can be viewed as an extension of a service mesh proxy, which inserts a sequence ID into every requests, guaranteeing that each and every ID would be unique.

The latter can be viewed as something similar, but less so on the transport level, focusing on respecting the logic of idempotency tokens instead.

Do you think this would be a good idea? Would you be interested in participating in parts of such “open designs” one way or another? Let me know pls.

https://adamwalker.github.io/Building-FPGA-KVS/ -- there's always something fun to be done with FPGAs!

In other news, I've created a Github repo dedicated to our meetup.

Besides all the relevant links, there is now an open pull request into it, in which I outline, in an RFC form, how I envision the future "open design" work.

It should expand the markdown nicely if you click "Display the rich diff" next to the only changed file on Github UI, or try this link to get to the respective branch and read it in rich format right away.

Comments are more than welcome, please and thank you. And cheers to a good start, I sure hope this opens the next big chapter in this venture we are enjoying together!