πŸ’» The byte is not enough
291 subscribers
1 photo
10 files
64 links
Personal hand-picked collection of articles and tutorials on the matter of software engineering and computer science. Occasionally on science, history, or linguistics.

@virtyaluk for any inquiries.

https://modern-dev.com/
https://github.com/virtyaluk
Download Telegram
Interesting article on how an SSD may lose all the data while being powered off for a while.

https://blog.korelogic.com/blog/2015/03/24

#hardware #tech #ssd

πŸ’» The byte is not enough
The Netflix Simian Army

Chaos Monkey, a tool that randomly disables production instances to make sure [we] can survive this common type of failure without any customer impact.

https://netflixtechblog.com/the-netflix-simian-army-16e57fbab116

#netflix #chaosmonkey #systemdesign #faulttolerance #cloud #computation #security

πŸ’» The byte is not enough
gfs-sosp2003.pdf
269.5 KB
πŸ§‘β€πŸ”¬ Google File System (GFS)

A classical paper on Google File System dating back to 2003 devoted to the distributed file system developed by Google with the main aim to store and process enormous amounts of data at scale effectively.

The work influenced the creation of two other well-known technologies like HDFS and Google's BigTable.

https://www.youtube.com/watch?v=eRgFNW4QFDc

#systemdesign #google #distributed #filesystems #design #reliability #performance #scalability #faulttolerance #research #paper

πŸ’» The byte is not enough
43438.pdf
836.5 KB
πŸ§‘β€πŸ”¬ Large-scale cluster management at Google with Borg

An incredible paper on Google's Borg, a Kubernetes predecessor, and how Google successfully managed tens of thousands of machine clusters for over a decade.

Google's Borg system is a cluster manager that runs hundreds of thousands of jobs, from many thousands of different applications, across a number of clusters each with up to tens of thousands of machines.

Watch the Borg presentation at EuroSys 2015:
https://www.youtube.com/watch?v=7MwxA4Fj2l4

#systemdesign #google #borg #distributed #orchestrator #design #reliability #performance #scalability #faulttolerance #research #paper

πŸ’» The byte is not enough
consistent_hashing_and_random_trees_distributed_caching_protocols.pdf
179.9 KB
πŸ§‘β€πŸ”¬ Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web

Another fundamental work on distributed hashing protocols that influenced distributed systems' evolution. Nowadays, consistent hashing is being used in many software distributions like Amazon Dynamo, Apache Cassandra, Riak, Voldemort to name a few. Few big online platforms are known to implement consistent hashing algorithms to scale for performance, availability, and reliability.

More concise take on the matter:

https://www.toptal.com/big-data/consistent-hashing

#systemsdesign #distributed #design #reliability #performance #availability #scalability #research #paper #consistent #hashing

πŸ’» The byte is not enough
Twine_A_Unified_Cluster_Management_System_for_Shared_Infrastructure.pdf
1.1 MB
πŸ§‘β€πŸ”¬ Twine: A Unified Cluster Management System for Shared Infrastructure

If you as me were impressed by the impressive work Google done in their Borg system, and it's successor Kubernetes, then you will definitely enjoy learning how Facebook makes use of their infrastructure in an astonishing paper on Facebook Twine. This tremendous work benefits from experience gained through developing and managing other popular systems like Borg, Kubernetes or Mesos, and aims to scale to more than a million of machines.

https://engineering.fb.com/2019/06/06/data-center-engineering/twine/

#systemdesign #facebook #twine #distributed #orchestrator #design #reliability #performance #scalability #faulttolerance #research #paper #borg #kubernetes #mesos

πŸ’» The byte is not enough