Revision as of 00:57, 8 November 2021

External

Episode 06 Unqualified Engineer - Jackson Gabbard: Intro to Architecture and Systems Design Interviews https://www.youtube.com/watch?v=ZgdS0EUmn70

Internal

Software Engineering

Overview

The goals of system design is to build software systems that first and foremost are correct, in that the system correctly implements the functions it was built to implement. Additionally, they should aim to maximize reliability, scalability and maintainability.

A Typical System

Many web and mobile applications in use today are conceptually similar to the generic system described below. The functionality differs, as well as reliability, scalability and maintainability requirements, but most systems share at least several of the elements described here.

The applications have mobile and browser clients that communicate with the backend via HTTP or WebSocket protocols. They include a business logic layer deployed as a monolith or a set of microservices, in most cases in containers managed by a container orchestration system like Kubernetes. They persist their data in databases, either relational or NoSQL. They may need to remember results of expensive operations to speed up reads, so they can use caches for that. In case they need to allow users to search data by keyword, search indexes are available and can be integrated. They may need to rely on message brokers or streaming systems for asynchronous processing and better decoupling. If they need to periodically process large amounts of accumulated data, they can use batch processing tools.

Depending on the actual requirements on the system, these bits and pieces are implemented by different products, which are integrated into the system and stitched together with application code. The exercise of developing the system consists in combining standard building blocks into a structure that provides custom functionality.

Clients

The clients can be web applications or mobile applications. A web application uses a combination of server-side logic written in a language as Java or Python and packaged and deployed as containerized services to handle business logic and storage, and a client-side language as HTML or JavaScript for presentation. A mobile application uses the same type of backend infrastructure, but a device-specific language to implement the client.

Frontend/Backend Communication

In both cases, the clients send requests over HTTP. The server returns the response as part of the same HTTP request/response pair, either a HTML page to be rendered or JSON-serialized data.

Applications

Database

The database can be relational or NoSQL. The NoSQL article describes why a NoSQL database may be preferable over a relational database, in specific cases.

Cache

Search Index

Full-text search servers: Elasticsearch and Solr.

Stream Processing

Message_Brokers_and_Stream_Processing_TO_REFACTOR

Batch Processing

Dataflow

Request flow. Write path. Read path.

System Design Goals

Correctness

Ensured via testing.

Reliability

Discuss difference between reliability and high availability.

Scalability

Maintainability

Operations

Logs

Monitoring

Deployment

Capacity Estimation

Organizatorium

Papers and Articles Referred from "Designing Data-intensive Applications"

“One Size Fits All”: An Idea Whose Time Has Come and Gone https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.68.9136&rep=rep1&type=pdf

@@ Line 13: / Line 13: @@
 Depending on the actual requirements on the system, these bits and pieces are implemented by different products, which are integrated into the system and stitched together with application code. The exercise of developing the system consists in combining standard building blocks into a structure that provides custom functionality.
-::::[[File:ATypicalSystem.png|397px]]
+::::[[File:ATypicalSystem.png|672px]]
 ==Clients==

System Design: Difference between revisions