Video of "Where do Microservices go?" with Andrey Rebrove is publicly available!
https://youtube.com/live/PboH8FFqq5c

Time for the new announcement! What should Engineering Manager know about Software Architecture with @MagdaMiu
March, 7th. 10 AM. GMT+2(EET)

https://youtube.com/live/4jjcU9JD6Pc?feature=share

Architecture Weekly #57 Highlights

Bitwarden design flaw 🤟
After LastPass breach, the alternatives left to the public are 1Password and Bitwarden. And looking at the latter, it is not perfect indeed. The problems include issues with low number of PBKDF2 - significantly lower than OWASP recommends - along with the fact that in case of the breach those iterations does not mean anything, as the attackers will try to decrypt the data right away, instead of guessing the master key hash. Are there any products without that problem? How to design that properly? Find in the article!

#security

Architecture Modernization Teams 🍼

The software architecture is not wine by no stretch of the imagination: it only becomes worse with time. Although organization typically recognize it, they still struggle with modernization of solutions architecture. One of the ways to resolve this struggle is a creation of an Enabling team for architecture modernization. The goals of this team includes upskilling the people, keeping the architecture momentum and facilitation of better design. Grab a long read on when to introduce AMET, how it's work looks like when you don't need that team anymore.

#architecture #strategy

Fixing Go's Linker: ARM64, DWARF and linker internals 🤟
I know, not really an architecture topic, but exciting read. So Uber's team discovered they can't debug the Golang binaries for ARM64, while the AMD64 binaries are debuggable perfectly fine. So they decided to figure out what is the issue. They discovered that due to the usage of a smaller type for function references, the linked inserts additional jumps making the functions external; thus breaking the adding of the debug symbols. The fix is a one-liner; but the journey!...

#go #compiler #investigation

Architecture Weekly #57 Follow-Up

Multitask learning at LinkedIn 👷‍♂️
When you have several learning tasks that are somehow similar, it makes sense to leverage multitask learning to increase performance and quality of learning. However, with such an approach new difficulties emerge like a different set of features for tasks, different input datasets etc. LinkedIn shared a post explaining how they handle those issues with their multitask learning framework.

#ml

Types of CQRS 👷‍♂️
When you learn about CQRS, you typically think binary: either you have one, or you don't. Vladimir Khorikov reasons about CQRS in a gradual manner: you can indeed have no CQRS, but then you can have stage 1, 2 and three. Read the article to know the difference and when you need to apply a particular stage.

#cqrs #patterns #architecture

Scalable Annotation Service at Netflix 👷‍♂️
Netflix wanted to store the annotations - some metadata - for any objects of theirs: from whole movies to particular timeframes. They wanted to do that in a unified, performant and scalable fashion. So they created Marken. Read how they employed Cassandra and ElasticSearch to accomplish the goal.

#netflix #microservices #architecture #casestudy

Consumer IoT Device Cybersecurity Standards and Policies 🍼
As you know, "S" in IoT stands for Security. That's why it is so mandatory to bring some clarity to the field and introduce some standardization. Please find a report on the policies, certifications and standartization procedures in different parts of the Globe in IoT field.

#security #iot

Software Build Systems 🍼
This week I took an interview with Artem Zinnatullin about Software Build Systems. We discussed what you should expect from a good build system - performance, scalability, learning curve - and what you, as an architect of a software architect, should know about them. We also covered the evolution of build systems from Ant to Bazel and speculated about their future.

#interview #video #buildsystem

ChatGPT is a Blurry JPEG of the Web 🍼
Did you know that Xerox copying machines are not doing the direct copy? Instead they scan the document, analyze it, compress and then decompress for printing. ChatGPT is simingly doing the same - analyzing and compressing the data from the web and giving us it's own - and blurry - image of the web. But what we prefer better?

#ai

Archtiecture Weekly #58 Highlights

Video

Platform Engineering Teams Done Right 👷‍♂️
Platform Engineering term is on hype now due to the popularity of Team Topologies and overall discussion on how to apply DevOps practices in a right way. Adrian Cockcroft is sharing his ideas about the principles you should build the Platform Teams. And the plural here is no wonder, as it appears you should have several platform teams covering different layers of a platform - and it's no wonder - due to the level of complexity we now have. Find a great article and example of organization structuring in the end of the article.

#platform #devops

Design Considerations for High Throughput Cloud Native Relational Database 🤟
Amazon Aurora is built with an idea of separation compute from storage for the resilience. The way how they commit data in multinode system is through redo log associated witha a database. The paper describes how the Network IO looks like in Aurora, how the traffic goes through it's nodes and finally shares the learning from building a complex cloud based database system.

#aws #cloud #aurora #database

Ongoing Security: a step-by-step guide to a secure app development process 👷‍♂️
Some companies think that they can add security as some step in the CI/CD pipeline, but that obviously does not work. Security impacts the application architecture, the threats have significant impact on the data formats and protocols and many other reasons. This week I published the article explaining those and providing receipts what to do at every step of the development lifecycle. This is also just a first article in the series, so make sure to subscribe!

#security #mobile #ongoingsecurity

Architecture Weekly #58 Follow-Up

Design Considerations fpr Platform Engineering Teams 🍼
Following up the topic of Platform Teams, please find an article on the teams that might together represent the platform. Platform User Interface, Infrastructure, Services, Support and others inside.

#platform #bestpractices #teams

ACID Transactions for Cassandra 👷‍♂️
Cassandra is having lightweight transactions since 2013, but they are not enough as the users would like ot have the performant commits across data centers. Now it becomes available with the new breakthrough called Accord. Now they promise to guarantee an isolation for the transactions through a concensus reached in a single roundtrip. Follow the post for the details.

#databases #distributedsystem #cassandra

Sustainability Cloud Native Way 🍼
Energy consumption can be one of the quality attributes due to the regulations or just because electricity is expensive. RedHat suggests a solution to monitor the power consumption of Kubernetes workloads. Read more inside.

#cloudnative #cloud #sustainability

Secure AWS Organizations with Service Control policies 👷‍♂️
When you start a new account in AWS, you start with the root account which in turns creates the first admin account. But what do you do next? How do you separate dev and prod environment? How do you manage the user permissions? Find the article exactly about that.


#aws #security #cloud

Big Ball of Mud 🍼
The phenomena of big ball of mud is known to everyone: tons of global state, everything talks to everything else, no tests, unmantainable etc. Typically the solution is to impose a design from some kind of an architect and rewrite the whole thing, but it rarely works well, and the question is how to avoid the bbom in the first place. Find a nice article which tries to analyze how we end up with such systems and what to do to make better systems.


#architecture

The Ultimate Guide to Software Architecture Documentation 🍼
And the last one of today is about documenting software architecture. We highlighted the topic many times, and I found a good piece again. They cover the goals of documentation and the reasons why having just code is not enough, like the code won't tell you the non-functional requirements ever. So grab a 12-minute read!

#documentation

And before you go, next week I am interviewing Magda Miu about what an Engineering Manager should know about Software Architecture, so don't forget to join the stream!

Like the newsletter? Wanna receive new content earlier, than everybody else? Consider helping to run it at Patreon or Boosty. The funds go to pay for the hosting and some software like a Camo Studio license. Patrons and Boosty subscribers of a certain level also get access to a private Architecture Community and of course every supporter gets early access. Big thanks to Nikita, Anatoly, Oleksandr, Dima, Pavel B, Pavel, Robert, Roman, Iyri, Andrey, Lidia, Vladimir, August and Roman for already supporting the newsletter. Join them as well!

Tomorrow at 10 AM Tallinn time(GMT+2) I will be hosting an interview with EM from Adobe - Magda Miu - about what an Engineering Manager should know about software architecture. Come for the broadcast to ask questions!
https://youtube.com/live/4jjcU9JD6Pc?feature=share

Architecture Weekly #59 Highlights

Uber's Move to the Cloud 👷‍♂️
When you start a new product or a company, you always ask yourself: should I go with the cloud or do something on-prem? Of course, it takes a lot of considerations, starting with a budget and ending with the technologies and available talent. From this perspective, it is fascinating how Uber operating on-prem for more than 12 years decided to move to the cloud. The Pragmatic Engineer newsletter delivers 2 parts of this journey and covers the story of data centres at Uber, the reasoning for the migration and of course the cloud migration problems.

#cloud #migration #onprem

Roblox 3-days long outage 🤟
When a known company's services with millions of users are offline for 15 minutes it is already a tragedy. Facebook was down for several hours last year, and it caused a global panic. Roblox - a gaming company - was down for 3 days which is an infinite timespan. Roblox relies heavily on HashiCorp products like Vault and Consul, and the incident was connected to the usage of those - especially the usage of a new feature in Consul that triggered a pathological issue in its open-source engine - BoltDB. Find a summary on protocol.com, as well as a detailed Post Mortem from Roblox themselves.

#incident #hashicorp

Guide to the system design interview 👷‍♂️
Aaand system design interviews again! This time I've got for you a very detailed guide which covers the structure of system design interview, the expectations from you as a candidate and of course the principles you need to be aware of in the majority of the system design interviews. The ways of thinking, the ways interviewers look at the candidate and real-life stories from FAANG(or MANGA?) - all inside the guide to the system design interview.

And if you need help preparing for one - you can always request a consultation here.

#systemdesign #interview

Architecture Weekly #59 Follow-Up

11 Reasons to use Lambdas 🍼
There are a lot of talks about serverless nowadays, and AWS Lambda is probably the most known, maybe after Firebase Functions. Is it just a fancy toy or you can really leverage it in the production environment? Find an article from SentinelOne, which explains the advantages behind Lambdas.

#aws #cloud #lambda #serverless

Comparing Security Frameworks 🍼
How do you know which security measures you need to apply in your organization? To answer this question security frameworks exists. Grab an article to understand the detailed comparison of four major cybersecurity frameworks, namely NIST 800-53, ISO 27002, NIST CSF, and SCF. The author highlights the strengths and weaknesses of each framework and explains how they differ in terms of scope, focus, and applicability. The article also offers practical insights and recommendations for organizations looking to adopt or integrate these frameworks into their cybersecurity programs.

#security #standard #csf

The Journey to Server Driven UI at Lyft Rentals 👷‍♂️
One day, a ride-hailing company found that its mobile app's UI was becoming too complex and difficult to maintain, so they decided to adopt a server-driven UI approach to simplify the app and improve the user experience. The article from Lyft's engineering team details their journey towards implementing server-driven UI in their bikes and scooters app, the challenges they faced, and the benefits they achieved in terms of faster development, reduced app size, and improved customer satisfaction.

#mobile #serverdrivenui

Moving back end closer to the users with Edge functions 👷‍♂️
Edge functions allow developers to improve website performance and user experience by moving the back end closer to the front end. This article from the StackOverflow blog explores the concept of edge functions and their ability to move the back end closer to the front end, resulting in improved website performance and user experience. The author highlights the advantages of edge functions, including faster load times, improved scalability, and reduced costs. The article also delves into the use cases of edge functions, such as caching and content delivery, and provides examples of popular edge function platforms, so grab a read!

#serverless #edgefunctions #lambda

Load Balancing Algorithms 👷‍♂️
Large-scale applications and websites often receive a high volume of traffic that cannot be handled by a single server. Load balancing distributes traffic across multiple servers, enabling efficient resource utilization, reducing downtime, and improving application performance. This article discusses various load balancing techniques, such as round-robin, weighted round-robin, and least connections. The author also provides an overview of advanced techniques such as SSL offloading and content-based routing.

#loadbalancing #highload

Anti-Corruption Layer Pattern 🍼
If you read anything about DDD, you might have heard about the Anti-Corruption layer: a pattern useful when you have two subdomains which don't share the same semantics. Basically, you introduce a layer which will translates the messages from one system into the messages readable by the another. Find a detailed description of the pattern it's drawbacks in the article by Microsoft.

#ddd #patterns

Speaking about the DDD - this Wednesday I will be holding an interview the the author of "Learning DDD" - Vlad Khononov! Press the "remind me" button in order not to miss!

And of course, this week I published an interview with Magda Miu about "What should EM know about architecture". Find it here!

Like the newsletter? Wanna receive new content earlier, than everybody else? Consider helping to run it at Patreon or Boosty. The funds go to pay for the hosting and some software like a Camo Studio license. Patrons and Boosty subscribers of a certain level also get access to a private Architecture Community and of course every supporter gets early access. Big thanks to Nikita, Anatoly, Oleksandr, Dima, Pavel B, Pavel, Robert, Roman, Iyri, Andrey, Lidia, Vladimir, August and Roman for already supporting the newsletter. Join them too!

Learning DDD, Software Coupling... Who you are thinking about in context of those terms? Yes, Vlad Khononov!

15.03 18.30 GMT+2 we will discuss Software Coupling and what are the biggest problems in DDD adoption. Join the live session to ask questions!

https://youtube.com/live/sCR-5s-ULgA?feature=share

Live in 10 minutes!

https://youtube.com/live/sCR-5s-ULgA?feature=share

Video

Architecture Weekly #60 Highlights

Cloud-Bound Applications 🍼
If you consider the whole environment of the application running in a modern cloud, be it a container, a VM or a lambda function, you will discover two sets of bindings: compute bindings and integration bindings. While the first set is transparent to developers and basically means a form of delivery(docker container or just a binary), the second one includes all sorts of APIs and protocols to leverage the platform resources and external services. If you add their stateful and asynchronous binding you will get a landscape of a Cloud-Bound application. To know more, follow a long-read which is not poor with the illustrations though.

#cloud #architecture #vision

Container Security Fundamentals part 2 🤟
Containers are used everywhere, and there are tons of best practices around container security. But suppose you need to understand why those practices are in place, or you just want to understand container security better. In that case, I recommend reading a series of articles on container security fundamentals by Datadog. They explain how the containers are mostly Linux processes and continue to show that Linux applies the namespaces across multiple parameters to secure those. Follow the details inside.

#security #linux

Master the Art of Caching for System Design Interviews 👷‍♂️
Caching is an essential technique in system design, which can help improve the performance of applications, reduce latency, and save resources. However, caching can be a double-edged sword, as it can also introduce some challenges, such as consistency and invalidation. This article provides a comprehensive guide on caching, covering everything from the basics to the advanced topics. It starts with an introduction to caching, including its definition, benefits, and types, such as in-memory, disk-based, and distributed caching. It then dives into the different caching strategies and algorithms, such as Least Recently Used (LRU), Least Frequently Used (LFU), and Random Replacement.

#systemdesign #caching #performance

Architecture Weekly #60 Follow-Up

Database Versioning Best Practices 👷‍♂️
You can think that all the industry switched to versioning of the database schemes long ago and everybody is using something like Flyway or Liquibase for that. The reality is that it is far from true. That's why I decided to highlight a several-years old, but still very actual post by Vladimir Khorikov on Database Versioning. Grab 6 best practices on storing version inside the db itself, having the immutable SQL files, keeping them in the source control and some others.

#database #db #bestpractices

Large Scale Ad Data Systems at Booking.com using GCP 🍼
How much money Booking.com spends on advertising yearly you think? It's almost 5 billions! With such a budget, you definitely want to spend it wisely. In this article in their Engineering Blog, Warren Qi shares how Booking.com is using BigQuery to analyze the efficiency of their ad-campaigns, and how the Spanner is leveraged to combine different data into useful views.

#cloud #gcp #booking

How to do distributed locking 🤟
Locking is a typical approach for consistency: for example, you lock a row in a database so that only one process can modify it. However, in a distributed system it becomes much harder due to network delays. I want to share another old article on how to handle distributed locks by Martin Kleppmann - with an example of Redis. Grab a read.

#distributedsystem #locking

Zachman Framework explained in 12 minutes 🍼
Imagine you're looking into positions of Enterprise Architect, Chief Architect or a CTO. In any case you would have to understand a lot about the company. Zachman matrix is one of the enterprise frameworks that will help you to ask the most important questions on all the levels that matter. Follow the video by Mark Richards - a co-author of "Fundamentals of Software Architecture" and its sequel.

#ea #enterprisearchitecture

Transactional Data Lake with Incremental ETL at Uber 👷‍♂️
Batch processing is a powerful tool able to process tons of data. However, if the major part of this data does not change much, you can lose on the efficiency and the latency. Uber's blog features an article by Sara Wilmes-Reitz on the idea of incremental ETL processes, allowing them to update the information much quicker and with lower latency. Follow the article for details.

#bigdata #etl #datawarehouse

Domain-Driven Design with Vlad Khononov 🍼
And this week I recorded an interview with Vlad Khononov - an author of "Learning Domain-Driven Design" and other O'Reilly-published books. Watch a video to find out how the DDD can help you be a better architect.

#video #ddd #interview

Like the newsletter? Wanna receive new content earlier, than everybody else? Consider helping to run it at Patreon or Boosty. The funds go to pay for the hosting and some software like a Camo Studio license. Patrons and Boosty subscribers of a certain level also get access to a private Architecture Community and of course every supporter gets early access. Big thanks to Nikita, Anatoly, Oleksandr, Dima, Pavel B, Pavel, Robert, Roman, Iyri, Andrey, Lidia, Vladimir, August and Roman for already supporting the newsletter. Join them as well!

New announcement! Will ChatGPT render the developer job obsolete?

https://www.youtube.com/watch?v=fxQ5nfXJauY

Architecture Weeekly #61

Video

Highlights
Google Sheets as backend 👷‍♂️
When you start a new project you probably think, hey, I would pick MongoDB, or I want a PostgreSQL installation. But some successful projects start with none other than Google Sheets as their storage. It appears it is scalable, simple, and it's easy to integrate with. Find a great post by Levels.fyi - a site with literally salary bands of IT industry - where they show the evolution of their system design, which started with Google Forms and Google Sheets.

#casestudy

Reality Check Pattern 🍼
Have you ever encountered your navigation software telling you you're in a river while you were driving a highway? Well, Uwe Friedrechson just did, and a resiliency architect, he immediately came up with the pattern which he called "A reality check". It means that your system should check the signals from the outside world, and, if they stop making sense, change the state to something like safe mode, where it could not mislead the user or harm them. Grab more examples and conclusions inside 🙂

#architecture #pattern #resilience

Distributed Machine Learning at Instacart 👷‍♂️
Several issues ago I mentioned the KubeRay - a platform for running ML workloads on Kubernetes. And this week Instacart published a piece on building the distributed ML platform using exactly that. Find out, how they managed to build a scalable, efficient, diverse platform which allows for development and maintenance simplicity, as well as seamless integration with the previous platform called Griffin.

#ml #casestudy #k8s #kubernetes #kuberay

Architecture Weekly #61 Follow Up

AWS Aurora receives Write Through Cache 👷‍♂️
Replication lag appears when the replication stream is generated too fast for a replica to keep up. To improve this situation, Write-Through Cache was introduced which decreases the replication lag by 44%. More details in the article.
#database #cloud #aws

Architects, Anti-Patterns and Organizational Fuckery 🍼
What is the difference between Principal Engineer and Software Architect? Should Architect be a Role or a Title? How to avoid screwing up your organization by introducing the Architects rotting up in their Ivory Tower? The person named mipsytipsi wrote a long read after a discussion in Twitter where the attitude toward the title was collected. Funny, that the post confuses the title and the role... but it's still an entertaining read nonetheless.

#architect #philosofy

Cloud Native Landscape 👷‍♂️
The amount of tools for developing, deploying, running and monitoring cloud-native apps is over 1000. So you face a problem: what tools you would like to solve the expected issues? Worry no more: Cloud-Native Foundation has a tool called landscape, where you can find all the tools conveniently laid out by the area of applicability.

#cloud #cloudnative

Workflow Orchestration 👷‍♂️
The common approach to a distributed transaction is a saga either orchestrated or via choreography. However, sagas compensation can not be always possible or at least expensive, like refunding the money sent for payment due to commissions. One of the ideas to solve the problem is the workflow approach, which camunda or temporal.io is implementing. This way you have a separate component which responsibility is only to manage the workflows and ensure they finish or complete gracefully. Follow the article for details.

#patterns #distributedsystems

Contract-Driven Development 👷‍♂️
The "Contract-Driven Adoption" article on InfoQ explores how contract-driven development can improve the adoption of new technologies and tools in organizations. It discusses the benefits of using contracts to define expected behaviours and outcomes, reduce ambiguity, and enable faster feedback and iteration. The article covers different types of contracts, provides examples of real-world projects, and offers practical tips for using contract-driven development. Overall, it provides a useful perspective on how contract-driven development can facilitate technology adoption and improve collaboration among teams.

#microservices #contract #api

Microservice Dependency Graphs 🤟
The "Characterizing Microservice Dependency Graphs" article on MuratBuffalo's blog discusses a study on analyzing over 100 microservice systems using network science techniques to identify common patterns and properties. The article highlights key findings, such as the prevalence of power-law distributions, clustering, and hierarchical structures in microservice dependency graphs. The authors discuss the implications of these findings on the design and management of microservice architectures, such as the need for monitoring, fault tolerance, and modularization. Overall, the article offers insights for improving the scalability, reliability, and maintainability of microservice architectures.

#microservice #paper #tracing

Like the newsletter? Wanna receive new content earlier, than everybody else? Consider helping to run it at Patreon or Boosty. The funds go to pay for the hosting and some software like a Camo Studio license. Patrons and Boosty subscribers of a certain level also get access to a private Architecture Community and of course every supporter gets early access. Big thanks to Nikita, Anatoly, Oleksandr, Dima, Pavel B, Pavel, Robert, Roman, Iyri, Andrey, Lidia, Vladimir, August, Roman and Egor for already supporting the newsletter. Join them as well!

Why do people fail system design interviews?
https://youtube.com/shorts/KgxTv48mzK0

The biggest lie of IT industry
https://youtube.com/shorts/5lAIt_L6Hps

Why ChatGPT won't replace developers. Post for patrons and boosty subscribers

Patreon.
Boosty.