💻 The byte is not enough
291 subscribers
1 photo
10 files
64 links
Personal hand-picked collection of articles and tutorials on the matter of software engineering and computer science. Occasionally on science, history, or linguistics.

@virtyaluk for any inquiries.

https://modern-dev.com/
https://github.com/virtyaluk
Download Telegram
💡 System Design Interview: mini Twitter

How would you design Twitter? It’s essentially the same question as designing Facebook, Linkedin, or many other social media services. The key scenario in question is usually that a user can follow or unfollow other users. Any user can tweet stuff. A user should be able to see tweets, displayed in a certain order, from the users she is following — the so-called timeline. And the core discussion point is designing for scale.

https://eileen-code4fun.medium.com/system-design-interview-mini-twitter-1e0e99bd7377

#systemdesign #sdi #design #infrastructure #twitter #interview #prep #fanout

💻 The byte is not enough
System Design Interview: Replicated and Strongly Consistent Key-Value Store

Designing a replicated and strongly consistent key-value store, which sounds seemingly simple, is actually quite a tricky interview question. You’d need to have a good understanding of some of the distributed systems concepts to be able to answer this question well. If you’re not in a domain specific interview, for this kind of questions, chances are that the interviewer just wants to observe how well you can navigate through a complex design problem. In that case, the discussion itself is usually more important than the final solution.

https://levelup.gitconnected.com/system-design-interview-replicated-and-strongly-consistent-key-value-store-b690d8e15c9a

#systemdesign #infrastructure #database #dht #keyvalue #store #replication #strong #consistency #design #faulttolerance #leader #election #quorum #raft

💻 The byte is not enough
🔐 The Next Gen Database Servers Powering Let's Encrypt

Let’s Encrypt helps to protect a huge portion of the Web by providing TLS certificates to more than 235 million websites. A database is at the heart of how Let’s Encrypt manages certificate issuance. If this database isn’t performing well enough, it can cause API errors and timeouts for our subscribers. Database performance is the single most critical factor in our ability to scale while meeting service level objectives. In late 2020, Let's Encrypt upgraded the database servers and they’ve been very happy with the results.

https://letsencrypt.org/2021/01/21/next-gen-database-servers.html

#infrastructure #letsencrypt #ssl #tls #security #hardware #database #amd #intel #nvme #performance

💻 The byte is not enough
🕰 Blast from the past: The Architecture Twitter Uses To Deal With 150M Active Users, 300K QPS, A 22 MB/S Firehose, And Send Tweets In Under 5 Seconds

Toy solutions solving Twitter’s “problems” are a favorite scalability trope. Everybody has this idea that Twitter is easy. With a little architectural hand waving we have a scalable Twitter, just that simple. Well, it’s not that simple as Raffi Krikorian, VP of Engineering at Twitter, describes in his superb and very detailed presentation on Timelines at Scale. If you want to know how Twitter works - then start here.

http://highscalability.com/blog/2013/7/8/the-architecture-twitter-uses-to-deal-with-150m-active-users.html

#systemdesign #twitter #infrastructure #social #graph #highload #performance #scalability #fanout #timeline #availability #reliability #monitoring

💻 The byte is not enough
How to Design a Scalable Rate Limiting Algorithm

Rate limiting protects your APIs from inadvertent or malicious overuse by limiting how often each user can call the API. Without rate limiting, each user may make a request as often as they like, leading to “spikes” of requests that starve other consumers. Once enabled, rate limiting can only perform a fixed number of requests per second. A rate limiting algorithm helps automate the process.

https://konghq.com/blog/how-to-design-a-scalable-rate-limiting-algorithm/

#algorithms #design #systemdesign #rate #limiting #slidingwindow #slidinglog #fixedwindow #leakybucket #performance #faulttolerance #scalability

💻 The byte is not enough
A Scalable Prefix Search Service for Powering Autocomplete

If you’ve ever googled something, you probably take it for granted that suggestions appear based on what you’ve typed so far. Yet, these suggestions are an essential part of the googling experience. It would almost feel more weird if you were typing out a search query and autocomplete suggestions did not show up.

Indeed, Google’s autocomplete is powerful and useful, so it’s no surprise that there are many open source implementations of autocomplete widely available.

https://medium.com/@prefixyteam/how-we-built-prefixy-a-scalable-prefix-search-service-for-powering-autocomplete-c20f98e2eff1

#systemdesign #design #autoprefix #trie #algorithms #redis #mongodb #google #search #prefixhashtree #prefixy #availability #performance #scalability

💻 The byte is not enough
💡 System Design Interview: Online Judge with Data Modelling

A good system design question can be to design an online coding challenge judge.

Problem: A coding challenge is a competition where a set of coding questions gets released for a period of few hours/days and a list of pre-registered competitors participate by solving these questions and submitting the solutions which gets run against some hidden test cases and competitors are scored based on the test results. A platform hosting such challenges is called an online judge. ex: SPOJ, topcoder etc.

https://medium.com/@saisandeepmopuri/system-design-online-judge-with-data-modelling-40cb2b53bfeb

#systemdesign #sdi #design #infrastructure #onlinejudge #interview #prep #SPOJ #topcoder #modeling #estimation #python

💻 The byte is not enough
💡 System Design Interview: Rate limiter and Data modelling

More often than not, the design interview rounds start with basic questions such as “design a rate limiter” or “design a circuit breaker” and can go to much more depth based on your experience. The questions posed can be to design an existing technology or any specific use case of the company you’re interviewing for. Sometimes these questions get asked in telephonic screening which is an elimination round and decides whether a person’s candidature should be taken forward or not; Along with the design, one might be asked for data model to the solution or even code it up.

https://medium.com/@saisandeepmopuri/system-design-rate-limiter-and-data-modelling-9304b0d18250

And, blast from the past, Cloudlfare's story on building rate limiter capable of scaling to millions of domains:
https://blog.cloudflare.com/counting-things-a-lot-of-different-things/

#systemdesign #design #infrastructure #ratelimiter #performance #scalability #cloudflare #python #sdi #estimation #algorithms

💻 The byte is not enough
H3: Uber’s Hexagonal Hierarchical Spatial Index

Grid systems are critical to analyzing large spatial data sets, partitioning areas of the Earth into identifiable grid cells.

With this in mind, Uber developed H3, the grid system for efficiently optimizing ride pricing and dispatch, for visualizing and exploring spatial data. H3 enables Uber to analyze geographic information to set dynamic prices and make other decisions on a city-wide level.

https://eng.uber.com/h3/

#systemdesign #algorithms #uber #h3 #data #visualization #geospatial #hexagonal #maps #s2 #geofencing

💻 The byte is not enough
Asynchronous computing at Facebook: Driving efficiency and developer productivity at Facebook scale

Async - is a widely used general purpose asynchronous computing platform at Facebook. It powers a variety of use cases, ranging from notifications to integrity checks to scheduling posts for a future time, and leverages the ability of FOQS (Facebook Ordered Queueing Service) to hold large backlogs of work items to defer running use cases that can tolerate delays to off-peak hours.

https://engineering.fb.com/2020/08/17/production-engineering/async/

#systemdesign #design #infrastructure #facebook #async #scheduler #dispatcher #scalability #reliability #faulttolerance #distributed #priority #queue #dequeue #enqueue #replication

💻 The byte is not enough
Scaling services with Shard Manager

A comprehensive read on one of the key components of Facebook's underlying infrastructure — Shard Manager. Shard Manager — is a control plane that enables engineers at Facebook to scale their services in a quick and efficient fashion. It is built on top of Twine, and in fact, complements the Twine system providing a wide variety of helpful features like fault tolerance, load balancing, and shard scaling.

https://engineering.fb.com/2020/08/24/production-engineering/scaling-services-with-shard-manager/

#systemdesign #design #infrastructure #facebook #shardmanager #twine #controlplane #loadbalancing #scalability #faulttolerance #sharding #api #performance #reliability

💻 The byte is not enough
👍1
Scribe: Transporting petabytes per hour via a distributed, buffered queueing system

Scribe - is a unified Facebook infrastructure that enables the users to collect, aggregate, and deliver application logs from millions of machines achieving enormous input/output rates (over 2.5 terabytes per second). Scribe has recently undergone a major architectural revamp and simplification, and its new architecture is currently in production.

https://engineering.fb.com/2019/10/07/data-infrastructure/scribe/

#systemdesign #design #infrastructure #facebook #scribe #queue #scribed #logdevice #logmanagement #availability #scalability #performance #faulttolerance

💻 The byte is not enough
👍4