Today I am interviewing Andrey Rebrov - a co-founder & CTO at Scentbird - about microservices. How we came to them, what are the microservices problems, what's the ideal size of a microservice and where do we go with them. Join the broadcast at 4 PM GMT+2! Make sure to attend, as the recording will be taken private for a couple of weeks
https://youtube.com/live/PboH8FFqq5c
https://youtube.com/live/PboH8FFqq5c
YouTube
Where do microservices go - lambdas or back to monolith?
We came to microservices through an evolution of deployment and operation from the one hand side, and through splitting up the responsibility from the other hand side. Where we will go next though? To the distributed monolith? Or to lambdas? What technologies…
🔥5👍1
Architecture Weekly #55
I will publish the video for this issue tomorrow. However, last week I conducted not 1, but 2 interviews. The first one is with Uwe Friedrechsen, where I ask him about where software architecture is going in 2023. The second one is where Andrey Rebrov, a CTO from ScentBird, tells us about the microservices, where we go with them, how to pick the right size for a microservice and how they transitioned from a Grails monolith to Java microservices. Both videos are available for Patrons and Boosty subscribers.
Highlights
Breaking up a Monolith by Khan Academy 👷♂️
The question "How to do you transition from a monolith to microservices?" is very real. And fortunate you are, Khan Academy just completed a project of migrating from a Python 2 monolith to a set of Go microservices across the timespan of 3,5 years. Such projects are challenging from so many perspectives: how do you start? Do you apply Agile practices or handle the project as a waterfall one? How do you make the transition: per feature, per domain, per function? How to retain people over this timespan? Khan Academy shared what went well, and what choices they would reconsider. A very insightful article by Gergely Orosz.
#microservices #refactoring #casestudy
How to fix a $1 Million Query 👷♂️
Shopify decided to use BigQuery as its Data Warehouse solution. They had to process the data of their customers. But once they estimated the cost of a single query type running for their volumes for a month, they got a walloping value of approx 1 million dollars. That definitely rang a bell, so the engineers went to find a way to optimize that. It appears that a where clause and a cluster feature of BigQuery led them to the solution. Check out which one.
#bigquery #cloud #gcp
Is software getting worse? 🍼
And the last highlight for today is a post in StackOverflow blog on the software getting worse meaning it's heavy, slow and buggy. Ryan Donovan argues that's wrong to blame the programmers: they really want to make fast, compact software; but the market just does not care! The very interesting business reasoning inside.
I will publish the video for this issue tomorrow. However, last week I conducted not 1, but 2 interviews. The first one is with Uwe Friedrechsen, where I ask him about where software architecture is going in 2023. The second one is where Andrey Rebrov, a CTO from ScentBird, tells us about the microservices, where we go with them, how to pick the right size for a microservice and how they transitioned from a Grails monolith to Java microservices. Both videos are available for Patrons and Boosty subscribers.
Highlights
Breaking up a Monolith by Khan Academy 👷♂️
The question "How to do you transition from a monolith to microservices?" is very real. And fortunate you are, Khan Academy just completed a project of migrating from a Python 2 monolith to a set of Go microservices across the timespan of 3,5 years. Such projects are challenging from so many perspectives: how do you start? Do you apply Agile practices or handle the project as a waterfall one? How do you make the transition: per feature, per domain, per function? How to retain people over this timespan? Khan Academy shared what went well, and what choices they would reconsider. A very insightful article by Gergely Orosz.
#microservices #refactoring #casestudy
How to fix a $1 Million Query 👷♂️
Shopify decided to use BigQuery as its Data Warehouse solution. They had to process the data of their customers. But once they estimated the cost of a single query type running for their volumes for a month, they got a walloping value of approx 1 million dollars. That definitely rang a bell, so the engineers went to find a way to optimize that. It appears that a where clause and a cluster feature of BigQuery led them to the solution. Check out which one.
#bigquery #cloud #gcp
Is software getting worse? 🍼
And the last highlight for today is a post in StackOverflow blog on the software getting worse meaning it's heavy, slow and buggy. Ryan Donovan argues that's wrong to blame the programmers: they really want to make fast, compact software; but the market just does not care! The very interesting business reasoning inside.
Patreon
Get more from Vladimir Ivanov on Patreon
создаёт Architecture Weekly Newsletter
🔥1
Architecture Weekly #55 - Follow-Up
Investing Wisely in Architecture Through Refactoring 🍼
Simon Brown said that industry was transitioning from big upfront design to no design at all. The same might happen at an application level: investing in the design too much makes it a bit overengineered; once you learn you will start doing less designing until you find the right balance. J. B. Rainsberger reason this tendency and finds that it is an essential part of learning. You can't predict if you're overdesigning or under-designing, but that's not an issue if you can get out of it via refactoring. And exactly at this point, you might need the architecture patterns.
#architecture #refactoring
Kubernetes Patterns e-book 👷♂️
Speaking about the patterns, RedHat just shared an online book published with O'Reilly on the patterns. So you will find foundation patterns like declarative deployment and managed lifecycle, behavioural patterns like batch and periodic jobs, and structural patterns like sidecard and adapter and many more inside.
#patterns #k8s #kubernetes
Designing modern frontend app architecture 👷♂️
So you decide you want a new big frontend application - maybe you're building something from scratch, or you're rewriting an old big ball of mud into something more appropriate. How would you approach the front-end architecture? What quality attributes do you need to take into account? Find a well-structured and illustrated article on what you should be aware of to build a good solution.
#architecture #frontend
Modern Cloud for Multiplayer Games 👷♂️
I am a personal fan of StarCraft 2 - an online RTS game. But as a software architect I always wondered how you design such a system given the requirement of synchronization of the game data between multiple players simultaneously supporting low latency requirements. This week I stumbled upon an article which digs into the technical challenges of multiplayer games and explains how they can be solved within a cloud environment.
#architecture #gaming #multiplayer
Raft visualized 🤟
Raft is a distributed consensus algorithm. They are typically hard to comprehend, and that's the reason good visualization can be highly valuable. Grab one for Raft and go step by step through the concepts of consensus, leader, follower, leader elections and other related stuff.
#distributedsystem #consensus #raft
Using AI for system design and diagram creation 👷♂️
Talks about ChatGPT are all over the internet right now. Some people use them for writing cover letters, some create manuals, and some even suggest building YouTube video scenarios with it. Software architects though are experimenting with building system design with it. Find a receipt for formulating the data model and sequence diagrams for a wealth management system using ChatGPT below.
#systemdesign #ai #chatgpt
Like the newsletter? Wanna receive new content earlier, than everybody else? Consider helping to run it at Patreon or Boosty. The funds go to pay for the hosting and some software like a Camo Studio license. Patrons and Boosty subscribers of a certain level also get access to a private Architecture Community and of course every supporter gets early access. Big thanks to Nikita, Anatoly, Oleksandr, Dima, Pavel B, Pavel, Robert, Roman, Iyri, Andrey, Lidia, Vladimir, August and Roman for already supporting the newsletter. Join them as well!
Investing Wisely in Architecture Through Refactoring 🍼
Simon Brown said that industry was transitioning from big upfront design to no design at all. The same might happen at an application level: investing in the design too much makes it a bit overengineered; once you learn you will start doing less designing until you find the right balance. J. B. Rainsberger reason this tendency and finds that it is an essential part of learning. You can't predict if you're overdesigning or under-designing, but that's not an issue if you can get out of it via refactoring. And exactly at this point, you might need the architecture patterns.
#architecture #refactoring
Kubernetes Patterns e-book 👷♂️
Speaking about the patterns, RedHat just shared an online book published with O'Reilly on the patterns. So you will find foundation patterns like declarative deployment and managed lifecycle, behavioural patterns like batch and periodic jobs, and structural patterns like sidecard and adapter and many more inside.
#patterns #k8s #kubernetes
Designing modern frontend app architecture 👷♂️
So you decide you want a new big frontend application - maybe you're building something from scratch, or you're rewriting an old big ball of mud into something more appropriate. How would you approach the front-end architecture? What quality attributes do you need to take into account? Find a well-structured and illustrated article on what you should be aware of to build a good solution.
#architecture #frontend
Modern Cloud for Multiplayer Games 👷♂️
I am a personal fan of StarCraft 2 - an online RTS game. But as a software architect I always wondered how you design such a system given the requirement of synchronization of the game data between multiple players simultaneously supporting low latency requirements. This week I stumbled upon an article which digs into the technical challenges of multiplayer games and explains how they can be solved within a cloud environment.
#architecture #gaming #multiplayer
Raft visualized 🤟
Raft is a distributed consensus algorithm. They are typically hard to comprehend, and that's the reason good visualization can be highly valuable. Grab one for Raft and go step by step through the concepts of consensus, leader, follower, leader elections and other related stuff.
#distributedsystem #consensus #raft
Using AI for system design and diagram creation 👷♂️
Talks about ChatGPT are all over the internet right now. Some people use them for writing cover letters, some create manuals, and some even suggest building YouTube video scenarios with it. Software architects though are experimenting with building system design with it. Find a receipt for formulating the data model and sequence diagrams for a wealth management system using ChatGPT below.
#systemdesign #ai #chatgpt
Like the newsletter? Wanna receive new content earlier, than everybody else? Consider helping to run it at Patreon or Boosty. The funds go to pay for the hosting and some software like a Camo Studio license. Patrons and Boosty subscribers of a certain level also get access to a private Architecture Community and of course every supporter gets early access. Big thanks to Nikita, Anatoly, Oleksandr, Dima, Pavel B, Pavel, Robert, Roman, Iyri, Andrey, Lidia, Vladimir, August and Roman for already supporting the newsletter. Join them as well!
Thecodewhisperer
Investing Wisely in Architecture Through Refactoring
The stronger your refactoring skill, the more easily you can use architecture advice as guidelines instead of as rules to enforce. This makes it significantly more likely that you'll invest wisely in architecture, rather than over- or under-engineer.
👍4
Recorded a new video - this time on task delegation. Tried to make it fun, tell me if I managed to do so 🙂 https://youtu.be/_Ca9zHMT8AA
YouTube
Delegation done right. Teach, inspire and control the result.
The majority of people are afraid of delegating tasks. But if you follow the video you'll know how to choose a proper person for a task and what to do to ensure the result
Telegram channel: https://t.me/architectureweekly
Patreon: https://www.patreon.co…
Telegram channel: https://t.me/architectureweekly
Patreon: https://www.patreon.co…
🤓1💅1
Architecture Weekly #56 Highlights
Will my cloud bill go up with looser coupling? 👷♂️
Coupling is considered a bad thing, so architects usually try to decrease it for better flexibility and maintainability. But what about the cost of infrastructure? Gregor Hohpe picks up an example of a Lambda function writing data to DynamoDB and sending an event to EventBridge to see if moving the event handling to DynamoDB streams will increase the cost. But the main point of the article is in its end: whatever the infrastructure increase would be, you will probably save a couple of orders of magnitude in value brought by your programmers.
#architecture #cloud #cost
Big Data is Dead 🍼
Big Data was on the hype for over a decade. The reason for that was the expectation of the neverending growth of data and the growth of the companies handling this data. But it turns out that really large volumes are only handled by a handful of companies, and even for them the workloads are much less than the total amount of data. What does that mean for you? Find out in the article below.
#bigdata
Distributed Reset 🤟
Last time we shared a visualization of Raft consensus protocol. But it is a relatively recent development. But the problems of distributed systems are much older. For example, we had a problem of distributed reset: setting a default value for all the nodes in the system. The original paper was written in 1994, but this week Jesse Jiryu Davis wrote a review for it. The basic idea is close to the Deijksta path search algorithm with the difference that here we need to traverse all the nodes in DAG. Grab the details inside.
#distributedsystem
Will my cloud bill go up with looser coupling? 👷♂️
Coupling is considered a bad thing, so architects usually try to decrease it for better flexibility and maintainability. But what about the cost of infrastructure? Gregor Hohpe picks up an example of a Lambda function writing data to DynamoDB and sending an event to EventBridge to see if moving the event handling to DynamoDB streams will increase the cost. But the main point of the article is in its end: whatever the infrastructure increase would be, you will probably save a couple of orders of magnitude in value brought by your programmers.
#architecture #cloud #cost
Big Data is Dead 🍼
Big Data was on the hype for over a decade. The reason for that was the expectation of the neverending growth of data and the growth of the companies handling this data. But it turns out that really large volumes are only handled by a handful of companies, and even for them the workloads are much less than the total amount of data. What does that mean for you? Find out in the article below.
#bigdata
Distributed Reset 🤟
Last time we shared a visualization of Raft consensus protocol. But it is a relatively recent development. But the problems of distributed systems are much older. For example, we had a problem of distributed reset: setting a default value for all the nodes in the system. The original paper was written in 1994, but this week Jesse Jiryu Davis wrote a review for it. The basic idea is close to the Deijksta path search algorithm with the difference that here we need to traverse all the nodes in DAG. Grab the details inside.
#distributedsystem
The Architect Elevator
I made everything loosely coupled. Will my cloud bill go up?
Loose coupling is an essential property of fine-grained, event-driven systems. However, cloud resources that decouple event producers and consumers incur a run-time cost. How to weigh the trade-offs?
👍3
Architecture Weekly #56 Follow-Up
Evolution of a React application 👷♂️
React is usually perceived as a framework, but it's rather a library for building UI. And the application architecture is still up to you. Juntao QIU from ThoughtWorks wrote an article in Martin Fowler's blog on how you can evolve your frontend application which uses React from a single component to a sophisticated multilayer application. Grab a great read.
#frontend #react #architecture
How the Istio Service Mesh became the Critical Infrastructure for CN apps 👷♂️
We spoke about service meshes in previous articles; this time I would like to share an article which covers the history of evolution from plain Kubernetes systems to using Services Meshes and the place of Istio there. The article also covers the evolution of proxies from locally configurable to cloud-native solutions at scale.
#servicemesh #istio #cloudnative
DevOps Roadmap 🍼
It's impossible to imagine a high performing organization without DevOps practices nowadays. So you might end up in situation where you need to hire SREs and other relevant specialist or grow them inside the company. The roadmap I found on the internet does a good job depicting the areas of expertise required nowadays from git and programming languages to Clouds, CI/CD and Monitoring and Observability.
#devops
Guide to Projections and Read Models in Event-Driven Architecture
With event sourcing, you typically have a fixation of facts or event happened in the system. However, we frequently need to understand the current state. Projections can help to build up this state from the recorded events. Oskar Dudycz shares the explanations of what exactly projections are in his event-driven architecture blog.
#eventdriven
Postgres WAL Files and Sequence Numbers 🤟
WAL files are used for the recovery and replication purposes. But if you need to dig deep into some issue, you might find useful the understanding of the WAL files naming conventions, log sequence numbers and segment positions. This is why I am sharing an article by Crunchy Data with you.
#postgresql #db #wal
Reference Architecture and Implementation for Deployment Pipelines 👷♂️
If you ever needed to build a deployment pipeline, you know how many details are there. Last week AWS published the reference architecture for the deployment pipeline and InfoQ blog has a nice article covering them. Find out how you can build a pipeline using AWS tools and services and how to follow the best practices.
Like the newsletter? Wanna receive new content earlier, than everybody else? Consider helping to run it at Patreon or Boosty. The funds go to pay for the hosting and some software like a Camo Studio license. Patrons and Boosty subscribers of a certain level also get access to a private Architecture Community and of course every supporter gets early access. Big thanks to Nikita, Anatoly, Oleksandr, Dima, Pavel B, Pavel, Robert, Roman, Iyri, Andrey, Lidia, Vladimir, August and Roman for already supporting the newsletter. Join them as well!
Evolution of a React application 👷♂️
React is usually perceived as a framework, but it's rather a library for building UI. And the application architecture is still up to you. Juntao QIU from ThoughtWorks wrote an article in Martin Fowler's blog on how you can evolve your frontend application which uses React from a single component to a sophisticated multilayer application. Grab a great read.
#frontend #react #architecture
How the Istio Service Mesh became the Critical Infrastructure for CN apps 👷♂️
We spoke about service meshes in previous articles; this time I would like to share an article which covers the history of evolution from plain Kubernetes systems to using Services Meshes and the place of Istio there. The article also covers the evolution of proxies from locally configurable to cloud-native solutions at scale.
#servicemesh #istio #cloudnative
DevOps Roadmap 🍼
It's impossible to imagine a high performing organization without DevOps practices nowadays. So you might end up in situation where you need to hire SREs and other relevant specialist or grow them inside the company. The roadmap I found on the internet does a good job depicting the areas of expertise required nowadays from git and programming languages to Clouds, CI/CD and Monitoring and Observability.
#devops
Guide to Projections and Read Models in Event-Driven Architecture
With event sourcing, you typically have a fixation of facts or event happened in the system. However, we frequently need to understand the current state. Projections can help to build up this state from the recorded events. Oskar Dudycz shares the explanations of what exactly projections are in his event-driven architecture blog.
#eventdriven
Postgres WAL Files and Sequence Numbers 🤟
WAL files are used for the recovery and replication purposes. But if you need to dig deep into some issue, you might find useful the understanding of the WAL files naming conventions, log sequence numbers and segment positions. This is why I am sharing an article by Crunchy Data with you.
#postgresql #db #wal
Reference Architecture and Implementation for Deployment Pipelines 👷♂️
If you ever needed to build a deployment pipeline, you know how many details are there. Last week AWS published the reference architecture for the deployment pipeline and InfoQ blog has a nice article covering them. Find out how you can build a pipeline using AWS tools and services and how to follow the best practices.
Like the newsletter? Wanna receive new content earlier, than everybody else? Consider helping to run it at Patreon or Boosty. The funds go to pay for the hosting and some software like a Camo Studio license. Patrons and Boosty subscribers of a certain level also get access to a private Architecture Community and of course every supporter gets early access. Big thanks to Nikita, Anatoly, Oleksandr, Dima, Pavel B, Pavel, Robert, Roman, Iyri, Andrey, Lidia, Vladimir, August and Roman for already supporting the newsletter. Join them as well!
martinfowler.com
Modularizing React Applications with Established UI Patterns
Learn how to apply established UI patterns for a more organized and maintainable codebase and discover the benefits of layering architecture in React development
👍3❤1
Good news, everyone! Artem Zinatullin agreed to an interview about build systems and what a software architect or staff engineer should know about them. Join this Monday at 6 PM EET. While you wait, the video with Uwe Friedrechsen became publicly available here.
YouTube
Build systems
With Artem Zinatullin we are talking about Build Systems and what an Architect or a Staff Engineer should know about them. We are touching different systems from Makefile to Gradle and Bazel.
The proper stream is here: https://www.youtube.com/watch?v=I5qLNIFsywQ
The proper stream is here: https://www.youtube.com/watch?v=I5qLNIFsywQ
🔥6
Architecture Weekly
Good news, everyone! Artem Zinatullin agreed to an interview about build systems and what a software architect or staff engineer should know about them. Join this Monday at 6 PM EET. While you wait, the video with Uwe Friedrechsen became publicly available…
Artem didn't join, will reschedule
😢10
Architecture Weekly
Artem didn't join, will reschedule
New stream today 6 PM EET(GMT+2)! Same time, but today 🙂
https://youtube.com/live/I5qLNIFsywQ?feature=share
https://youtube.com/live/I5qLNIFsywQ?feature=share
YouTube
Software Build systems
With Artem Zinatullin we are talking about Software Build Systems and what an Architect or a Staff Engineer should know about them. We are touching different systems from Makefile to Gradle and Bazel.
Fundamental issues of Gradle Build System: https://a…
Fundamental issues of Gradle Build System: https://a…
🔥2
Video of "Where do Microservices go?" with Andrey Rebrove is publicly available!
https://youtube.com/live/PboH8FFqq5c
https://youtube.com/live/PboH8FFqq5c
YouTube
Where do microservices go - lambdas or back to monolith?
We came to microservices through an evolution of deployment and operation from the one hand side, and through splitting up the responsibility from the other hand side. Where we will go next though? To the distributed monolith? Or to lambdas? What technologies…
Time for the new announcement! What should Engineering Manager know about Software Architecture with @MagdaMiu
March, 7th. 10 AM. GMT+2(EET)
https://youtube.com/live/4jjcU9JD6Pc?feature=share
March, 7th. 10 AM. GMT+2(EET)
https://youtube.com/live/4jjcU9JD6Pc?feature=share
YouTube
What should Engineering Manager know about Software Architecture with Magda Miu
There are different controversial opinions on what Engineering Manager should do. Should they code? Should they make system design? What should EMs know about Software Architecture? In this session we are going to figure that out with Magda Miu - a GDE and…
Architecture Weekly #57 Highlights
Bitwarden design flaw 🤟
After LastPass breach, the alternatives left to the public are 1Password and Bitwarden. And looking at the latter, it is not perfect indeed. The problems include issues with low number of PBKDF2 - significantly lower than OWASP recommends - along with the fact that in case of the breach those iterations does not mean anything, as the attackers will try to decrypt the data right away, instead of guessing the master key hash. Are there any products without that problem? How to design that properly? Find in the article!
#security
Architecture Modernization Teams 🍼
The software architecture is not wine by no stretch of the imagination: it only becomes worse with time. Although organization typically recognize it, they still struggle with modernization of solutions architecture. One of the ways to resolve this struggle is a creation of an Enabling team for architecture modernization. The goals of this team includes upskilling the people, keeping the architecture momentum and facilitation of better design. Grab a long read on when to introduce AMET, how it's work looks like when you don't need that team anymore.
#architecture #strategy
Fixing Go's Linker: ARM64, DWARF and linker internals 🤟
I know, not really an architecture topic, but exciting read. So Uber's team discovered they can't debug the Golang binaries for ARM64, while the AMD64 binaries are debuggable perfectly fine. So they decided to figure out what is the issue. They discovered that due to the usage of a smaller type for function references, the linked inserts additional jumps making the functions external; thus breaking the adding of the debug symbols. The fix is a one-liner; but the journey!...
#go #compiler #investigation
Bitwarden design flaw 🤟
After LastPass breach, the alternatives left to the public are 1Password and Bitwarden. And looking at the latter, it is not perfect indeed. The problems include issues with low number of PBKDF2 - significantly lower than OWASP recommends - along with the fact that in case of the breach those iterations does not mean anything, as the attackers will try to decrypt the data right away, instead of guessing the master key hash. Are there any products without that problem? How to design that properly? Find in the article!
#security
Architecture Modernization Teams 🍼
The software architecture is not wine by no stretch of the imagination: it only becomes worse with time. Although organization typically recognize it, they still struggle with modernization of solutions architecture. One of the ways to resolve this struggle is a creation of an Enabling team for architecture modernization. The goals of this team includes upskilling the people, keeping the architecture momentum and facilitation of better design. Grab a long read on when to introduce AMET, how it's work looks like when you don't need that team anymore.
#architecture #strategy
Fixing Go's Linker: ARM64, DWARF and linker internals 🤟
I know, not really an architecture topic, but exciting read. So Uber's team discovered they can't debug the Golang binaries for ARM64, while the AMD64 binaries are debuggable perfectly fine. So they decided to figure out what is the issue. They discovered that due to the usage of a smaller type for function references, the linked inserts additional jumps making the functions external; thus breaking the adding of the debug symbols. The fix is a one-liner; but the journey!...
#go #compiler #investigation
Almost Secure
Bitwarden design flaw: Server side iterations
Bitwarden is a hot candidate for a LastPass replacement. Looking into how they encrypt data, it doesn’t do things that much better however.
👍1🔥1
Architecture Weekly #57 Follow-Up
Multitask learning at LinkedIn 👷♂️
When you have several learning tasks that are somehow similar, it makes sense to leverage multitask learning to increase performance and quality of learning. However, with such an approach new difficulties emerge like a different set of features for tasks, different input datasets etc. LinkedIn shared a post explaining how they handle those issues with their multitask learning framework.
#ml
Types of CQRS 👷♂️
When you learn about CQRS, you typically think binary: either you have one, or you don't. Vladimir Khorikov reasons about CQRS in a gradual manner: you can indeed have no CQRS, but then you can have stage 1, 2 and three. Read the article to know the difference and when you need to apply a particular stage.
#cqrs #patterns #architecture
Scalable Annotation Service at Netflix 👷♂️
Netflix wanted to store the annotations - some metadata - for any objects of theirs: from whole movies to particular timeframes. They wanted to do that in a unified, performant and scalable fashion. So they created Marken. Read how they employed Cassandra and ElasticSearch to accomplish the goal.
#netflix #microservices #architecture #casestudy
Consumer IoT Device Cybersecurity Standards and Policies 🍼
As you know, "S" in IoT stands for Security. That's why it is so mandatory to bring some clarity to the field and introduce some standardization. Please find a report on the policies, certifications and standartization procedures in different parts of the Globe in IoT field.
#security #iot
Software Build Systems 🍼
This week I took an interview with Artem Zinnatullin about Software Build Systems. We discussed what you should expect from a good build system - performance, scalability, learning curve - and what you, as an architect of a software architect, should know about them. We also covered the evolution of build systems from Ant to Bazel and speculated about their future.
#interview #video #buildsystem
ChatGPT is a Blurry JPEG of the Web 🍼
Did you know that Xerox copying machines are not doing the direct copy? Instead they scan the document, analyze it, compress and then decompress for printing. ChatGPT is simingly doing the same - analyzing and compressing the data from the web and giving us it's own - and blurry - image of the web. But what we prefer better?
#ai
Multitask learning at LinkedIn 👷♂️
When you have several learning tasks that are somehow similar, it makes sense to leverage multitask learning to increase performance and quality of learning. However, with such an approach new difficulties emerge like a different set of features for tasks, different input datasets etc. LinkedIn shared a post explaining how they handle those issues with their multitask learning framework.
#ml
Types of CQRS 👷♂️
When you learn about CQRS, you typically think binary: either you have one, or you don't. Vladimir Khorikov reasons about CQRS in a gradual manner: you can indeed have no CQRS, but then you can have stage 1, 2 and three. Read the article to know the difference and when you need to apply a particular stage.
#cqrs #patterns #architecture
Scalable Annotation Service at Netflix 👷♂️
Netflix wanted to store the annotations - some metadata - for any objects of theirs: from whole movies to particular timeframes. They wanted to do that in a unified, performant and scalable fashion. So they created Marken. Read how they employed Cassandra and ElasticSearch to accomplish the goal.
#netflix #microservices #architecture #casestudy
Consumer IoT Device Cybersecurity Standards and Policies 🍼
As you know, "S" in IoT stands for Security. That's why it is so mandatory to bring some clarity to the field and introduce some standardization. Please find a report on the policies, certifications and standartization procedures in different parts of the Globe in IoT field.
#security #iot
Software Build Systems 🍼
This week I took an interview with Artem Zinnatullin about Software Build Systems. We discussed what you should expect from a good build system - performance, scalability, learning curve - and what you, as an architect of a software architect, should know about them. We also covered the evolution of build systems from Ant to Bazel and speculated about their future.
#interview #video #buildsystem
ChatGPT is a Blurry JPEG of the Web 🍼
Did you know that Xerox copying machines are not doing the direct copy? Instead they scan the document, analyze it, compress and then decompress for printing. ChatGPT is simingly doing the same - analyzing and compressing the data from the web and giving us it's own - and blurry - image of the web. But what we prefer better?
#ai
Linkedin
Applying multitask learning to AI models at LinkedIn
👍4
Archtiecture Weekly #58 Highlights
Video
Platform Engineering Teams Done Right 👷♂️
Platform Engineering term is on hype now due to the popularity of Team Topologies and overall discussion on how to apply DevOps practices in a right way. Adrian Cockcroft is sharing his ideas about the principles you should build the Platform Teams. And the plural here is no wonder, as it appears you should have several platform teams covering different layers of a platform - and it's no wonder - due to the level of complexity we now have. Find a great article and example of organization structuring in the end of the article.
#platform #devops
Design Considerations for High Throughput Cloud Native Relational Database 🤟
Amazon Aurora is built with an idea of separation compute from storage for the resilience. The way how they commit data in multinode system is through redo log associated witha a database. The paper describes how the Network IO looks like in Aurora, how the traffic goes through it's nodes and finally shares the learning from building a complex cloud based database system.
#aws #cloud #aurora #database
Ongoing Security: a step-by-step guide to a secure app development process 👷♂️
Some companies think that they can add security as some step in the CI/CD pipeline, but that obviously does not work. Security impacts the application architecture, the threats have significant impact on the data formats and protocols and many other reasons. This week I published the article explaining those and providing receipts what to do at every step of the development lifecycle. This is also just a first article in the series, so make sure to subscribe!
#security #mobile #ongoingsecurity
Video
Platform Engineering Teams Done Right 👷♂️
Platform Engineering term is on hype now due to the popularity of Team Topologies and overall discussion on how to apply DevOps practices in a right way. Adrian Cockcroft is sharing his ideas about the principles you should build the Platform Teams. And the plural here is no wonder, as it appears you should have several platform teams covering different layers of a platform - and it's no wonder - due to the level of complexity we now have. Find a great article and example of organization structuring in the end of the article.
#platform #devops
Design Considerations for High Throughput Cloud Native Relational Database 🤟
Amazon Aurora is built with an idea of separation compute from storage for the resilience. The way how they commit data in multinode system is through redo log associated witha a database. The paper describes how the Network IO looks like in Aurora, how the traffic goes through it's nodes and finally shares the learning from building a complex cloud based database system.
#aws #cloud #aurora #database
Ongoing Security: a step-by-step guide to a secure app development process 👷♂️
Some companies think that they can add security as some step in the CI/CD pipeline, but that obviously does not work. Security impacts the application architecture, the threats have significant impact on the data formats and protocols and many other reasons. This week I published the article explaining those and providing receipts what to do at every step of the development lifecycle. This is also just a first article in the series, so make sure to subscribe!
#security #mobile #ongoingsecurity
YouTube
Platform Engineering done right, Design Considerations behind Amazon Aurora and Ongoing Security
Architecture Weekly #58
#distributedsystems #aws #cloud #architecture #security #documentation #aurora
Text version of the issue: https://vvsevolodovich.dev/architecture-weekly-58/
Telegram channel: https://t.me/architectureweekly
Patreon: https://www…
#distributedsystems #aws #cloud #architecture #security #documentation #aurora
Text version of the issue: https://vvsevolodovich.dev/architecture-weekly-58/
Telegram channel: https://t.me/architectureweekly
Patreon: https://www…
🔥1
Architecture Weekly #58 Follow-Up
Design Considerations fpr Platform Engineering Teams 🍼
Following up the topic of Platform Teams, please find an article on the teams that might together represent the platform. Platform User Interface, Infrastructure, Services, Support and others inside.
#platform #bestpractices #teams
ACID Transactions for Cassandra 👷♂️
Cassandra is having lightweight transactions since 2013, but they are not enough as the users would like ot have the performant commits across data centers. Now it becomes available with the new breakthrough called Accord. Now they promise to guarantee an isolation for the transactions through a concensus reached in a single roundtrip. Follow the post for the details.
#databases #distributedsystem #cassandra
Sustainability Cloud Native Way 🍼
Energy consumption can be one of the quality attributes due to the regulations or just because electricity is expensive. RedHat suggests a solution to monitor the power consumption of Kubernetes workloads. Read more inside.
#cloudnative #cloud #sustainability
Secure AWS Organizations with Service Control policies 👷♂️
When you start a new account in AWS, you start with the root account which in turns creates the first admin account. But what do you do next? How do you separate dev and prod environment? How do you manage the user permissions? Find the article exactly about that.
#aws #security #cloud
Big Ball of Mud 🍼
The phenomena of big ball of mud is known to everyone: tons of global state, everything talks to everything else, no tests, unmantainable etc. Typically the solution is to impose a design from some kind of an architect and rewrite the whole thing, but it rarely works well, and the question is how to avoid the bbom in the first place. Find a nice article which tries to analyze how we end up with such systems and what to do to make better systems.
#architecture
The Ultimate Guide to Software Architecture Documentation 🍼
And the last one of today is about documenting software architecture. We highlighted the topic many times, and I found a good piece again. They cover the goals of documentation and the reasons why having just code is not enough, like the code won't tell you the non-functional requirements ever. So grab a 12-minute read!
#documentation
And before you go, next week I am interviewing Magda Miu about what an Engineering Manager should know about Software Architecture, so don't forget to join the stream!
Like the newsletter? Wanna receive new content earlier, than everybody else? Consider helping to run it at Patreon or Boosty. The funds go to pay for the hosting and some software like a Camo Studio license. Patrons and Boosty subscribers of a certain level also get access to a private Architecture Community and of course every supporter gets early access. Big thanks to Nikita, Anatoly, Oleksandr, Dima, Pavel B, Pavel, Robert, Roman, Iyri, Andrey, Lidia, Vladimir, August and Roman for already supporting the newsletter. Join them as well!
Design Considerations fpr Platform Engineering Teams 🍼
Following up the topic of Platform Teams, please find an article on the teams that might together represent the platform. Platform User Interface, Infrastructure, Services, Support and others inside.
#platform #bestpractices #teams
ACID Transactions for Cassandra 👷♂️
Cassandra is having lightweight transactions since 2013, but they are not enough as the users would like ot have the performant commits across data centers. Now it becomes available with the new breakthrough called Accord. Now they promise to guarantee an isolation for the transactions through a concensus reached in a single roundtrip. Follow the post for the details.
#databases #distributedsystem #cassandra
Sustainability Cloud Native Way 🍼
Energy consumption can be one of the quality attributes due to the regulations or just because electricity is expensive. RedHat suggests a solution to monitor the power consumption of Kubernetes workloads. Read more inside.
#cloudnative #cloud #sustainability
Secure AWS Organizations with Service Control policies 👷♂️
When you start a new account in AWS, you start with the root account which in turns creates the first admin account. But what do you do next? How do you separate dev and prod environment? How do you manage the user permissions? Find the article exactly about that.
#aws #security #cloud
Big Ball of Mud 🍼
The phenomena of big ball of mud is known to everyone: tons of global state, everything talks to everything else, no tests, unmantainable etc. Typically the solution is to impose a design from some kind of an architect and rewrite the whole thing, but it rarely works well, and the question is how to avoid the bbom in the first place. Find a nice article which tries to analyze how we end up with such systems and what to do to make better systems.
#architecture
The Ultimate Guide to Software Architecture Documentation 🍼
And the last one of today is about documenting software architecture. We highlighted the topic many times, and I found a good piece again. They cover the goals of documentation and the reasons why having just code is not enough, like the code won't tell you the non-functional requirements ever. So grab a 12-minute read!
#documentation
And before you go, next week I am interviewing Magda Miu about what an Engineering Manager should know about Software Architecture, so don't forget to join the stream!
Like the newsletter? Wanna receive new content earlier, than everybody else? Consider helping to run it at Patreon or Boosty. The funds go to pay for the hosting and some software like a Camo Studio license. Patrons and Boosty subscribers of a certain level also get access to a private Architecture Community and of course every supporter gets early access. Big thanks to Nikita, Anatoly, Oleksandr, Dima, Pavel B, Pavel, Robert, Roman, Iyri, Andrey, Lidia, Vladimir, August and Roman for already supporting the newsletter. Join them as well!
The New Stack
Architecture and Design Considerations for Platform Engineering Teams
What exactly is a platform? Is it an internal developer platform, a developer self-service portal or simply a developer onboarding tool?
👍5
Tomorrow at 10 AM Tallinn time(GMT+2) I will be hosting an interview with EM from Adobe - Magda Miu - about what an Engineering Manager should know about software architecture. Come for the broadcast to ask questions!
https://youtube.com/live/4jjcU9JD6Pc?feature=share
https://youtube.com/live/4jjcU9JD6Pc?feature=share
YouTube
What should Engineering Manager know about Software Architecture with Magda Miu
There are different controversial opinions on what Engineering Manager should do. Should they code? Should they make system design? What should EMs know about Software Architecture? In this session we are going to figure that out with Magda Miu - a GDE and…
Architecture Weekly #59 Highlights
Uber's Move to the Cloud 👷♂️
When you start a new product or a company, you always ask yourself: should I go with the cloud or do something on-prem? Of course, it takes a lot of considerations, starting with a budget and ending with the technologies and available talent. From this perspective, it is fascinating how Uber operating on-prem for more than 12 years decided to move to the cloud. The Pragmatic Engineer newsletter delivers 2 parts of this journey and covers the story of data centres at Uber, the reasoning for the migration and of course the cloud migration problems.
#cloud #migration #onprem
Roblox 3-days long outage 🤟
When a known company's services with millions of users are offline for 15 minutes it is already a tragedy. Facebook was down for several hours last year, and it caused a global panic. Roblox - a gaming company - was down for 3 days which is an infinite timespan. Roblox relies heavily on HashiCorp products like Vault and Consul, and the incident was connected to the usage of those - especially the usage of a new feature in Consul that triggered a pathological issue in its open-source engine - BoltDB. Find a summary on protocol.com, as well as a detailed Post Mortem from Roblox themselves.
#incident #hashicorp
Guide to the system design interview 👷♂️
Aaand system design interviews again! This time I've got for you a very detailed guide which covers the structure of system design interview, the expectations from you as a candidate and of course the principles you need to be aware of in the majority of the system design interviews. The ways of thinking, the ways interviewers look at the candidate and real-life stories from FAANG(or MANGA?) - all inside the guide to the system design interview.
And if you need help preparing for one - you can always request a consultation here.
#systemdesign #interview
Uber's Move to the Cloud 👷♂️
When you start a new product or a company, you always ask yourself: should I go with the cloud or do something on-prem? Of course, it takes a lot of considerations, starting with a budget and ending with the technologies and available talent. From this perspective, it is fascinating how Uber operating on-prem for more than 12 years decided to move to the cloud. The Pragmatic Engineer newsletter delivers 2 parts of this journey and covers the story of data centres at Uber, the reasoning for the migration and of course the cloud migration problems.
#cloud #migration #onprem
Roblox 3-days long outage 🤟
When a known company's services with millions of users are offline for 15 minutes it is already a tragedy. Facebook was down for several hours last year, and it caused a global panic. Roblox - a gaming company - was down for 3 days which is an infinite timespan. Roblox relies heavily on HashiCorp products like Vault and Consul, and the incident was connected to the usage of those - especially the usage of a new feature in Consul that triggered a pathological issue in its open-source engine - BoltDB. Find a summary on protocol.com, as well as a detailed Post Mortem from Roblox themselves.
#incident #hashicorp
Guide to the system design interview 👷♂️
Aaand system design interviews again! This time I've got for you a very detailed guide which covers the structure of system design interview, the expectations from you as a candidate and of course the principles you need to be aware of in the majority of the system design interviews. The ways of thinking, the ways interviewers look at the candidate and real-life stories from FAANG(or MANGA?) - all inside the guide to the system design interview.
And if you need help preparing for one - you can always request a consultation here.
#systemdesign #interview
Pragmaticengineer
Inside Uber’s move to the Cloud: Part 1
Uber has operated its own data centers for 9 years. What challenges did the company face, and why is it considering moving to the Cloud? Part 1.
👍1🔥1