In data centers, failures are inevitable. The jobs are relatively easy to restore upon failures - as binaries can simply be executed on different machines. However, protecting data is more complex. How do we ensure data availability during outages, such as a single machine becoming unavailable or, in worse cases, an entire region becoming unreachable due to network faults? This is extremely crucial for organizations like cloud providers, as maintaining user trust depends heavily on the high availability of data. This insight will address these challenges and the solutions used to ensure data resilience. This paper, published by Google in 2010 and presented at Operating System Design and Implementation (OSDI) 2010, is quite challenging due to its highly mathematical nature. This paper covers two independent main topics: The first section examines the availability and Mean Time To Failure of individual hardware components like disks and nodes . It also delves into failure bursts ...
In the previous insights, we explored several consensus algorithms; however, they were all designed for non-byzantine faults. In this insight, we will focus on protocols designed for public or permissioned environments where Byzantine faults can occur. This problem is exceptionally complex, with researchers publishing thousands of papers exploring this domain alone. This intense focus is driven by the fact that these protocols power blockchains and the economics of cryptocurrency. Developing these solutions is a significant challenge, as blockchain research involves rigorous logical and mathematical constructs. Additionally, the use of advanced cryptography often makes these papers challenging to read. This paper was presented at ACM Special Interest Group on Security, Audit and Control (SIGSAC) 2016, a premier conference in computer security. Authored by Andrew Miller et al. and affiliated with leading global universities, the paper is inherently technical and academically focuse...