#System #Design #Study #Guide

Amazing study guide for systems design interviews

https://github.com/donnemartin/system-design-primer

#Book #Microservices #Design

https://www.nginx.com/wp-content/uploads/2015/01/Building_Microservices_Nginx.pdf

#gzip #compression

Bandwidth optimization

Web servers use gzip to reduce the total amount of data transferred to clients. When a browser with gzip support sends a request, it adds “gzip” to its Accept-Encoding header. When the web server receives the request, it generates the response as normal, then checks the Accept-Encoding header to determine how to encode the response. If the server supports gzip, it uses gzip to compress each resource. It then delivers the compressed copies of each resource with an added Content-Encoding header, specifying that the resource is encoded using gzip. The browser then decompresses the content into its original uncompressed version before rendering it to the user.

However, this comes at a cost. Compression is a CPU-intensive process, and the more you compress a file, the longer it takes. Because of this, gzip offers a range of compression levels from 1 to 9; 1 offers the fastest compression speed but at a lower ratio, and 9 offers the highest compression ratio but at a lower speed. The gzip application uses level 6 by default, favoring higher compression over speed. Nginx, on the other hand, uses level 1, favoring higher speeds over file size savings.

https://www.pingdom.com/blog/can-gzip-compression-really-improve-web-performance/

#python #memory #management

https://realpython.com/python-memory-management/

#images #ML #paper #video
https://www.youtube.com/watch?v=TrdevFK_am4

#python

This article series is a guide to modern Python tooling with a focus on simplicity and minimalism.1 It walks you through the creation of a complete and up-to-date Python project structure, with unit tests, static analysis, type-checking, documentation, and continuous integration and delivery.

Here is a list of the articles in this series:

Chapter 1: Setup (this article)
Chapter 2: Testing
Chapter 3: Linting
Chapter 4: Typing
Chapter 5: Documentation
Chapter 6: CI/CD

https://cjolowicz.github.io/posts/hypermodern-python-01-setup/

#consistentHashing #cache

Removing a server results in its object keys being randomly reassigned to the rest of the servers, leaving all other keys untouched

https://www.toptal.com/big-data/consistent-hashing

#indexing #postgres #sql

A good and short video about indexing:

https://www.youtube.com/watch?v=-qNSXK7s7_w

#go #goroutines

Go uses goroutines while a language like Java uses threads.

The creation of a goroutine does not require much memory - only 2kB of stack space. They grow by allocating and freeing heap storage as required. Threads on the other hand start out at 1Mb (500 times more), along with a region of memory called a guard page that acts as a guard between one thread’s memory and another.

A server handling incoming requests can therefore create one goroutine per request without a problem, but one thread per request will eventually lead to the dreaded OutOfMemoryError. This isn’t limited to Java - any language that uses OS threads as the primary means of concurrency will face this issue.

Threads have significant setup and teardown costs because it has to request resources from the OS and return it once its done. The workaround to this problem is to maintain a pool of threads. In contrast, goroutines are created and destroyed by the runtime and those operations are pretty cheap. The language doesn’t support manual management of goroutines.

https://blog.nindalf.com/posts/how-goroutines-work/

#go #architecture #microservice #banking

Brilliant talk about Go used in Banking (Monzo)

https://www.youtube.com/watch?v=y2j_TB3NsRc

#DB #ACID #Video

So far the best explanation of ACID transactions. Highly recommend you to watch it if you have not worked with DBs before and do not know what this is.

This might also be useful if you prepare for an interview

https://www.youtube.com/watch?v=pomxJOFVcQs&list=PLQnljOFTspQXjD0HOzN7P2tgzu7scWpl2

#DB #Sharding
I am currently reading a lot about systems design for distributed systems. Data management is one of the most complex parts (at least for me, a person, who did not work with it much).

I looked through different "non-distributed" terms like indexing, views, materialised views. The other part is about partitilning/sharding,data replication, leader election algorithms, and how all that correlates with CAP theorem. One should carefully choose between relational and non-relational DBs.

In this article, it is written very well about scaling and sharding for relational DBs.

I highly recommend this, if you also struggle with the concept of Sharding for Relational DBs


Sharding with Amazon Relational Database Service | AWS Database Blog
https://aws.amazon.com/blogs/database/sharding-with-amazon-relational-database-service/

#LoadBalancer #L4 #L7

“L4 vs L7 Load Balancing” by Mohak Puri https://link.medium.com/k7BQvsUHMab

#gRPC

Крутой доклад по gRPC

- High Speed
- Language agnostic API and Definitions
- Two-ways communication
- Deadlines
- Cancellation Propagation
- Flow Control

https://www.youtube.com/watch?v=zPbaKUIcFx0

“System Design — Storage” by Peng Yang https://link.medium.com/zhoN5AJI1ab

“GraphQL Search Indexing” by Netflix Technology Blog https://link.medium.com/ASN0y7e14ab

На прошлой неделе весь день провёл на интервью. 5 часов интервью, с 5 людьми. Половину времени знакомились и общались по ценностям, оставшееся время по технической части: решение задачек, алгоритмы, ООП, системный дизайн

В итоге залетаю в одну крутую команду Амазона!

Системный дизайном никогда не занимался и интервью не проходил. Но имея прошлый опыт подготовок к разным конкурсам, я нашел свою модель подготовки.

Моя формула успеха - систмность, от общего к частному, и калибровка с последующей работой над ошибками.

В итоге, полторы-две недели и успешное прохождение! Чуть позже постараюсь раскрыть в больших деталях

Youtube - главный учитель нашего времени! По крайней мере в моем случае.

У всех разные подходы к развитию, в моем же случае это через видео. Иногда конечно читаю статьи, но в целом это не самая первая стадия. Я читаю что для статей мне нужно дорасти, а для этого сперва мне нужно понять общий контекст и услышать ряд точек зрения. Фильтруя по количеству лайков/дизлайков я могу предоложить, какой контент стоит смотреть, а какой можно скипнуть.

Дополнительный бустр - я практически всегда смотрю все на скорости 1.5-2х