matytan

Metadata#

[!abstract] Introduction to Operating Systems

• 
• Book Title: Introduction to Operating Systems
• Author: Remzi H. Arpaci-Dusseau Andrea C. Arpaci-Dusseau
• Summary: This is a book about modern operating systems. The book revolves around three main concepts: virtualization, concurrency, and persistence, introducing the main components of all modern systems (including scheduling, virtual memory management, disk and I/O subsystems, and file systems). The book consists of 50 chapters divided into three parts, covering topics related to virtualization, concurrency, and persistence. Most chapters present specific problems and then solve them using the techniques, algorithms, and ideas introduced in the book. The author introduces the conceptual themes in a conversational style, with humor and insight, aiming to help readers understand the principles of virtualization, concurrency, and persistence in operating systems. The content is comprehensive and provides real runnable code (rather than pseudocode), along with corresponding exercises, making it suitable for teaching by faculty in related fields at higher education institutions and for self-study by university students.
• Publication Date: 2019-06-01 00:00:00
• ISBN: 9787115508232
• Category: Computer - Comprehensive Computer
• Publisher: People's Posts and Telecommunications Press

Highlights#

Preface#

• 📌 Virtualization, concurrency, and persistence. ^30179184-6-1240-1293
  • ⏱ 2021-11-07 09:36:51

Chapter 1 Dialogue About This Book#

• 📌 Professor: Welcome to this book, which is titled "Operating Systems: Three Easy Pieces" in English, and I will be teaching you about operating systems. Please introduce yourself. Student: Hello, Professor, I am a student, and you may have guessed, I am ready to start learning! Professor: Great. Any questions? Student: Yes! Why does this book talk about "three easy pieces"? Professor: That's simple. Richard Feynman has a few lectures on physics that are quite good... Student: Ah, is he the author of "Surely You're Joking, Mr. Feynman"? That book is great! Will this book be as funny as that one? Professor: Uh... no. That book is indeed great, and I'm glad you've read it. I hope this book is more like his lectures on physics. It gathers some basic content into a book called "Six Easy Pieces." He talks about physics, and the topics we will explore are the three easy pieces of operating systems. This is fitting because the difficulty of operating systems is about half that of physics. Student: Got it, I like physics. What are the three pieces? Professor: Virtualization, concurrency, and persistence. These are the three key concepts we will learn. By studying these three concepts, we will understand how operating systems work, including how they decide which program gets to use the CPU next, how they handle memory overload in a virtual memory system, how virtual machine monitors work, how they manage data on disks, and a bit about how to build distributed systems that can still function when some nodes fail. Student: I have no concept of what you just said. Professor: Excellent, that means you are in the right place. ^30179184-6-359

  • ⏱ 2021-10-13 09:10:43

• 📌 First, attend the class, listen to the teacher's explanations, and take notes, then read the notes every weekend to better understand these concepts. ^30179184-6-1738-1776

  • ⏱ 2021-11-07 10:17:48

• 📌 Especially when doing projects, you will write real code to solve real problems, which is a way to apply the concepts from the notes. As Confucius said... ^30179184-6-1830-1879

  • ⏱ 2021-11-07 10:17:55

• 📌 "To hear something and not understand it is not as good as hearing it; to hear it is not as good as seeing it; to see it is not as good as knowing it; to know it is not as good as doing it." ^30179184-6-1941-1971

  • ⏱ 2021-11-07 10:18:11

• 📌 Good question! I think sometimes it is more useful to step back from the narrative and do some thinking. These dialogues are thinking. ^30179184-6-2532-2577

  • ⏱ 2021-11-07 10:19:27

Chapter 2 Introduction to Operating Systems#

• 📌 Virtualization allows many programs to run (thus sharing the CPU) and allows many programs to access their own instructions and data simultaneously (thus sharing memory). ^30179184-7-2403-2450

  • ⏱ 2021-10-13 09:11:38

• 📌 Adding two numbers, accessing memory, checking conditions, jumping to functions, etc. ^30179184-7-734-757

  • ⏱ 2021-11-07 10:18:57

• 📌 Through what mechanisms and strategies does the operating system achieve virtualization? How does the operating system effectively implement virtualization? What hardware support is needed? ^30179184-7-1754-1798

  • ⏱ 2021-11-07 10:19:11

• 📌 If you are taking an undergraduate operating systems course, you should already have a basic understanding of what happens when a computer program runs. If not, this book (and the corresponding course) will be difficult for you: you should stop reading this book or run to the nearest bookstore to quickly learn the necessary background knowledge (including Patt / Patel [PP03], especially Bryant / O’Hallaron's book [BOH10], both of which are quite good). What happens when a program runs? A running program does one very simple thing: it executes instructions. The processor fetches an instruction from memory, decodes it (figures out what instruction it is), and then executes it (does what it is supposed to do, such as adding two numbers, accessing memory, checking conditions, jumping to functions, etc.). After completing that instruction, the processor continues to execute the next instruction, and so on, until the program eventually finishes [illustration]. This describes the basic concept of the Von Neumann computing model [illustration]. Sounds simple, right? But in this course, we will learn that while one program is running, many other crazy things are happening simultaneously—mainly to make the system easy to use. In fact, there is a class of software responsible for making program execution easy (even allowing you to run multiple programs at the same time), allowing programs to share memory, enabling programs to interact with devices, and other similar interesting tasks. This software is called the operating system (OS) [illustration], because it is responsible for ensuring that the system is both easy to use and operates correctly and efficiently. The key question: how to virtualize resources? We will answer a core question in this book: how does the operating system virtualize resources? This is the key question. Why does the operating system do this? That is not the main question, as the answer should be obvious: it makes the system easier to use. Therefore, we focus on how to virtualize: what mechanisms and strategies does the operating system use to achieve virtualization? How does the operating system effectively implement virtualization? What hardware support is needed? ^30179184-7-359

  • ⏱ 2021-11-07 10:21:48

• 📌 Execute instructions. ^30179184-7-657-661

  • ⏱ 2021-11-07 10:23:46

• 📌 Fetch an instruction, decode it (figure out what instruction it is), and then execute it (do what it is supposed to do, such as ^30179184-7-669-734

  • ⏱ 2021-11-07 10:23:53

• 📌 How to virtualize resources. ^30179184-7-1626-1634

  • ⏱ 2021-11-07 10:32:53

• 📌 Even though we have only one processor, these four programs seem to be running simultaneously! How does this magic happen? ^30179184-7-4049-4314

  • ⏱ 2021-11-07 10:35:34

• 📌 Transforming a single CPU (or a small part of it) into what appears to be an infinite number of CPUs, allowing many programs to seem to run simultaneously, is called virtualizing the CPU, which is the focus of the first part of this book. ^30179184-7-4399-4491

  • ⏱ 2021-11-07 10:35:57

• 📌 If two programs want to run at a specific time, which one should run? This question is answered by the operating system's policy. ^30179184-7-4680-4727

  • ⏱ 2021-11-07 10:36:15

• 📌 The operating system acts as a resource manager. ^30179184-7-4812-4846

  • ⏱ 2021-11-07 10:36:21

• 📌 To read memory, you must specify an address to access the data stored there. To write or update memory, you must also specify the data to be written to the given address. ^30179184-7-4996-5078

  • ⏱ 2021-11-07 10:50:35

• 📌 Don't forget that every instruction of a program is in memory, so every time an instruction is read, memory is accessed. ^30179184-7-5178-5211

  • ⏱ 2021-11-07 10:50:51

• 📌 Each running program is allocated memory at the same address (00200000), but each seems to independently update the value at 00200000! It's as if each running program has its own private memory, rather than sharing the same physical memory with other running programs. ^30179184-7-6454-6723

  • ⏱ 2021-11-07 10:54:54

• 📌 Private virtual address space (sometimes called address space), which the operating system maps to the machine's physical memory. ^30179184-7-7116-7191

  • ⏱ 2021-11-07 11:00:47

• 📌 Physical memory is a shared resource managed by the operating system. ^30179184-7-7261-7279

  • ⏱ 2021-11-07 11:00:54

• 📌 The program creates two threads using Pthread_create(). You can think of a thread as a function running in the same memory space as other functions. ^30179184-7-8694-8907

  • ⏱ 2021-11-07 11:01:38

• 📌 Instructions are executed one at a time. Unfortunately, the critical part of the program above is where the shared counter is incremented, which requires three instructions. ^30179184-7-10069-10113

  • ⏱ 2021-11-07 11:03:54

• 📌 What technologies are needed to correctly implement persistence? What mechanisms and strategies are needed for high-performance implementation? How is reliability achieved in the face of hardware and software failures? ^30179184-7-11947-12000

  • ⏱ 2021-11-07 11:06:49

• 📌 The call to open() opens a file and creates it. The second is write(), which writes some data to the file. The third is close(). ^30179184-7-12068-12121

  • ⏱ 2021-11-07 11:07:11

• 📌 Hardware appears in the form of some input/output (I/O) devices. ^30179184-7-10527-10563

  • ⏱ 2021-11-07 11:07:20

• 📌 Most file systems include some complex write protocols, such as journaling or copy-on-write. ^30179184-7-12788-12843

  • ⏱ 2021-11-07 11:08:52

• 📌 Physical resources (such as CPU, memory, or disk) are virtualized. It deals with the troublesome and tricky issues related to concurrency. It persistently stores files, ensuring their long-term safety. ^30179184-7-13143-13255

  • ⏱ 2021-11-07 11:09:26

• 📌 It achieves. ^30179184-7-13140-13143

  • ⏱ 2021-11-07 11:09:29

• 📌 Abstraction makes it possible to write a large program by breaking it down into small, easily understandable parts, allowing programs written in high-level languages like C without considering assembly, and writing assembly code without considering logic gates, and building processors with logic gates without too much consideration of transistors. ^30179184-7-13411-13623

  • ⏱ 2021-11-07 11:10:45

• 📌 One goal of designing and implementing an operating system is to provide high performance. ^30179184-7-13736-13757

  • ⏱ 2021-11-07 11:11:02

• 📌 Another goal is to provide protection between applications and between the OS and applications. ^30179184-7-14007-14047

  • ⏱ 2021-11-07 11:13:00

• 📌 Protection is at the core of one of the fundamental principles of operating systems, which is isolation. Isolating processes from each other is key to protection, thus determining most of the tasks the OS must perform. ^30179184-7-14134-14200

  • ⏱ 2021-11-07 11:13:19

• 📌 These overheads can appear in various forms: extra time (more instructions) and extra space (in memory or on disk). If possible, we will seek solutions to minimize one or both. However, perfection is not always achievable, and we will note this and (where appropriate) tolerate it. ^30179184-7-13875-13978

  • ⏱ 2021-11-07 11:15:03

• 📌 Virtualization and making the system easy to use are very worthwhile, but not at any cost. Therefore, we must strive to provide virtualization and other operating system features without excessive overhead. ^30179184-7-13815-13875

  • ⏱ 2021-11-07 11:15:34

• 📌 Users sit in front of the computer using it, and most of the time it is idle, leading to facilities wasting thousands of dollars per hour. ^30179184-7-15348-15392

  • ⏱ 2021-11-07 11:16:27

• 📌 On these old mainframe systems, one program runs at a time, controlled by an operator. This operator did many things you think modern operating systems would do (for example, deciding the order in which jobs run). If you are a smart developer, you will treat this operator well so they can move your work to the front of the queue. ^30179184-7-15130-15243

  • ⏱ 2021-11-07 11:16:45

• 📌 The key difference between system calls and procedure calls is that system calls transfer control (jump) to the OS while raising the hardware privilege level. ^30179184-7-15928-16002

  • ⏱ 2021-11-07 11:17:14

• 📌 In kernel mode, the operating system has complete access to the system's hardware, allowing it to perform functions such as initiating I/O requests or providing more memory for programs. When ^30179184-7-16210-16263

  • ⏱ 2021-11-07 11:17:40

• 📌 Early operating systems, such as DOS (Microsoft's Disk Operating System), did not consider memory protection to be important. Therefore, malicious programs (or just poorly programmed applications) could write garbage all over memory. ^30179184-7-17795-17876

  • ⏱ 2021-11-07 11:18:07

• 📌 Another part of the machine state of a process is its registers. ^30179184-9-2248-2265

  • ⏱ 2021-11-08 13:54:05

Part 1 Virtualization#

• 📌 Chapter 3 Dialogue About Virtualization Professor: Now we begin the first part of the three parts of operating systems—virtualization. Student: Dear Professor, what is virtualization? Professor: Imagine we have a peach. Student: A peach? (Incredulous) Professor: Yes, a peach, which we call the physical peach. But there are many people who want to eat this peach, and we want to provide each person who wants to eat one that belongs to them, so everyone is happy. We call the peach given to each person a virtual peach. We somehow create many virtual peaches from this physical peach. Importantly, in this illusion, everyone seems to have a physical peach, but in reality, they do not. Student: So everyone doesn't know they are sharing one peach with others? Professor: Yes. Student: But the reality is that there is only one peach. Professor: Yes, so what? Student: So if I share the same peach with others, I will definitely notice the problem. Professor: Yes! You are right. But the more people eat, the more likely this problem is. Most of the time, they are dozing off or doing other things, so you can take the peach from their hands while they are dozing and share it with others, thus creating the illusion of many virtual peaches, one for each person! ^30179184-8-375
  • ⏱ 2021-11-07 11:18:54

Chapter 4 Abstraction: Processes#

• 📌 A process is a running program. The program itself has no lifecycle; it is just some instructions (and possibly some static data) that exist on the disk. ^30179184-9-433-485

  • ⏱ 2021-11-08 13:50:32

• 📌 The operating system makes these bytes run, allowing the program to function. ^30179184-9-485-507

  • ⏱ 2021-11-08 13:50:38

• 📌 A significant component of a process's machine state is its memory. ^30179184-9-2124-2147

  • ⏱ 2021-11-08 13:54:03

• 📌 Similarly, the stack pointer and related frame pointer are used to manage function parameter stacks, local variables, and return addresses. ^30179184-9-2434-2498

  • ⏱ 2021-11-08 13:54:34

• 📌 Mechanisms answer the "how" questions of the system. For example, how does the operating system perform context switching? Policies answer the "which" questions. ^30179184-9-2644-2706

  • ⏱ 2021-11-08 13:56:30

• 📌 Separating the two allows for easy changes to policies without reconsidering mechanisms. ^30179184-9-2724-2748

  • ⏱ 2021-11-08 13:56:48

• 📌 All modern operating systems provide these APIs in some form. ^30179184-9-3002-3023

  • ⏱ 2021-11-08 13:57:19

• 📌 This is how a program is transformed into a process. ^30179184-9-3970-3982

  • ⏱ 2021-11-08 13:57:59

• 📌 Some memory must be allocated for the program's runtime stack. You may already know that C programs use the stack to store local variables, function parameters, and return addresses. ^30179184-9-4753-4823

  • ⏱ 2021-11-08 13:59:20

• 📌 The operating system may also allocate some memory for the program's heap. In C programs, the heap is used for dynamically requested memory. ^30179184-9-4920-4961

  • ⏱ 2021-11-08 13:59:27

• 📌 By loading code and static data into memory, creating and initializing the stack, and performing other I/O-related tasks, the OS now (finally) sets the stage for program execution. ^30179184-9-5337-5399

  • ⏱ 2021-11-08 13:59:52

• 📌 Doing so can improve resource utilization by keeping the CPU busy. ^30179184-9-7648-7671

  • ⏱ 2021-11-09 09:27:18

• 📌 At any moment, we can count the different parts of the system it accesses or affects during execution, thus summarizing a process. ^30179184-9-1943-1985

  • ⏱ 2021-11-09 09:57:14

• 📌 Program Counter (PC). ^30179184-9-2359-2384

  • ⏱ 2021-11-09 09:58:15

• 📌 Code and all static data (such as initialized variables) are loaded into memory, into the process's address space. ^30179184-9-4061-4104

  • ⏱ 2021-11-09 09:59:03

• 📌 The program starts running at the entry point, which is main(). By jumping to the main() routine (a specialized mechanism discussed in Chapter 5), the OS transfers control of the CPU to the newly created process, thus starting program execution. ^30179184-9-5411-5487

  • ⏱ 2021-11-09 09:59:53

• 📌 There are some key data structures to track various relevant information. For example, to track the state of each process, the operating system may maintain some kind of process list for all ready processes. ^30179184-9-7865-7938

  • ⏱ 2021-11-09 10:01:11

• 📌 And track some additional information about the currently running process. The operating system must also track blocked processes in some way. ^30179184-9-7939-7981

  • ⏱ 2021-11-09 10:01:18

• 📌 When a process stops, its registers will be saved to this memory location. By restoring these registers (putting their values back into the actual physical registers), the operating system can resume running that process. ^30179184-9-8217-8285

  • ⏱ 2021-11-09 10:01:40

• 📌 This is a relatively simple one, but any operating system capable of running multiple programs simultaneously will certainly have something like this structure to track all programs running in the system. Sometimes people refer to the individual structures that store information about processes as Process Control Blocks (PCBs), which is a way of talking about C structures that contain information for each process. ^30179184-9-10235-10375

  • ⏱ 2021-11-09 10:02:23

• 📌 Simulated jobs appear in the form of simulators. ^30179184-9-12159-12172

  • ⏱ 2021-11-09 10:05:10

Chapter 5 Interlude: Process API#

• 📌 The separation of fork() and exec() is very useful when building a UNIX shell, as it gives the shell the opportunity to run code between fork and exec, allowing for a series of interesting functionalities to be easily implemented. ^30179184-10-7408-7515

  • ⏱ 2021-11-09 10:06:46

• 📌 Let's understand in more detail what exactly happens in p1.c. ^30179184-10-2186-2208

  • ⏱ 2021-11-09 10:08:55

• 📌 UNIX systems use a very interesting way to create new processes, namely through a pair of system calls: fork() and exec(). ^30179184-10-631-679

  • ⏱ 2021-11-09 10:09:06

Chapter 6 Mechanism: Restricted Direct Execution#

• 📌 Run one process for a while, then run another process, and so on in rotation. By time-sharing the CPU in this way, virtualization is achieved. ^30179184-11-456-519

  • ⏱ 2021-11-09 10:09:44

• 📌 The first is performance: how to achieve virtualization without increasing system overhead? The second is control: how to run processes effectively while retaining control of the CPU? Control is particularly important for operating systems because they are responsible for resource management. Without control, a process could simply run indefinitely and take over the machine or access unauthorized information. Therefore, achieving high performance while maintaining control is one of the main challenges in building operating systems. ^30179184-11-570-727

  • ⏱ 2021-11-09 10:09:55

Chapter 15 Mechanism: Address Translation#

• 📌 1. Run with seeds 1, 2, and 3, and determine whether each virtual address generated by the process is within bounds or out of bounds? If within bounds, calculate the address translation. 2. Run with the following flags: -s 0 -n 10. What value should the -l (limit register) be set to ensure all generated virtual addresses are within bounds? 3. Run with the following flags: -s 1 -n 10 -l 100. What is the maximum value that can be set for the limit so that the address space still fits entirely in physical memory? 4. Perform the same operation as in question 3, but with a larger address space (-a) and physical memory (-p). 5. As a function of the value of the limit register, which part of the randomly generated virtual addresses is valid? Draw a graph, running with different random seeds, limiting values from 0 to the maximum address space size. ^30179184-20-14592
  • ⏱ 2021-10-13 09:08:27

Chapter 16 Segmentation#

• 📌 If we put the entire address space into physical memory, then the space between the stack and the heap is not used by the process but still occupies actual physical memory. ^30179184-21-548-599

  • ⏱ 2021-10-13 09:14:00

• 📌 How to support large address spaces. ^30179184-21-761-770

  • ⏱ 2021-10-13 09:14:04

• 📌 Segmentation: generalized base/limit. ^30179184-21-975-986

  • ⏱ 2021-10-13 09:14:08

• 📌 A pair of base and limit registers is given for each logical segment in the address space. A segment is simply a contiguous fixed-length area in the address space, and there are typically three logically distinct segments in a typical address space: code, stack, and heap. ^30179184-21-1106-1191

  • ⏱ 2021-10-14 12:41:07

Chapter 26 Concurrency: Introduction#

• 📌 Chapter 26 Concurrency: Introduction. ^30179184-33-350-360
  • ⏱ 2022-01-14 09:15:03

Chapter 46 Dialogue About Distribution#

• 📌 Messages can be lost, machines can fail, disks can corrupt data, as if the whole world is against you! ^30179184-51-1170-1203
  • ⏱ 2021-11-09 10:10:42

Chapter 47 Distributed Systems#

• 📌 Not just interacting with one machine. ^30179184-52-510-522

  • ⏱ 2021-11-09 10:11:50

• 📌 Our main focus is on failures. ^30179184-52-670-689

  • ⏱ 2021-11-09 10:12:02

• 📌 System performance is often critical. ^30179184-52-1471-1493

  • ⏱ 2021-11-09 10:12:41

• 📌 Failures will be the focus: how should the communication layer handle failures? ^30179184-52-1824-1842

  • ⏱ 2021-11-09 10:13:35

• 📌 Packet loss is a fundamental phenomenon of networks. So the question becomes: how should packet loss be handled? ^30179184-52-2489-2516

  • ⏱ 2021-11-09 10:14:16

• 📌 The core principle of modern networks is that communication is fundamentally unreliable. ^30179184-52-1933-1953

  • ⏱ 2021-11-09 10:14:30

• 📌 Packets are often lost, corrupted, or fail to reach their destination. ^30179184-52-1990-2012

  • ⏱ 2021-11-09 10:14:34

• 📌 A good example of this unreliable layer is the UDP/IP network stack, which is present in almost all modern systems. ^30179184-52-2727-2765

  • ⏱ 2021-11-09 10:15:39

• 📌 UDP is a good example of an unreliable communication layer. ^30179184-52-5420-5438

  • ⏱ 2021-11-09 10:19:55

• 📌 Use checksums to check integrity. ^30179184-52-5705-5715

  • ⏱ 2021-11-09 10:20:08

• 📌 We need an additional mechanism called timeout. ^30179184-52-6866-6890

  • ⏱ 2021-11-09 10:20:40

• 📌 The combination of timeout and retry has led some to refer to this method as timeout/retry. Very clever people, those who work with networks, aren't they? Figure 47.4 shows an example. ^30179184-52-7020-7122

  • ⏱ 2021-11-09 10:20:53

• 📌 In this example, what is lost is not the original message, but the acknowledgment message. ^30179184-52-7165-7187

  • ⏱ 2021-11-09 10:21:01

• 📌 Therefore, the sender receives acknowledgment, but the message was not received twice, ensuring the one-time semantics mentioned above. ^30179184-52-7992-8027

  • ⏱ 2021-11-09 10:21:41

• 📌 This mechanism is called a sequence counter. ^30179184-52-8195-8206

  • ⏱ 2021-11-09 10:21:59

• 📌 After sending a message, the sender increments this value (to N + 1). ^30179184-52-8334-8383

  • ⏱ 2021-11-09 10:22:42

• 📌 Therefore, the receiver knows it has received that message. Thus, it will acknowledge the message but will not pass it to the application. In this simple way, the sequence counter can avoid duplication. ^30179184-52-8754-8815

  • ⏱ 2021-11-09 10:23:06

• 📌 The commonly used reliable communication layer is called TCP/IP, or simply TCP. ^30179184-52-8845-8869

  • ⏱ 2021-11-09 10:23:09

• 📌 Distributed Shared Memory (DSM) systems allow processes on different machines to share a large virtual address space. ^30179184-52-9727-9796

  • ⏱ 2021-11-09 10:23:46

• 📌 For example, after the first timeout, the client might increase its timeout value to a higher amount, possibly double the original value. This exponential back-off scheme was implemented in early Aloha networks and adopted in early Ethernet, avoiding resource overload due to excessive retransmissions. Robust systems strive to avoid such overload. ^30179184-52-9504-9639

  • ⏱ 2021-11-09 10:24:08

• 📌 Another issue is performance. ^30179184-52-10271-10279

  • ⏱ 2021-11-09 10:25:11

• 📌 Remote Procedure Call (RPC). ^30179184-52-10496-10506

  • ⏱ 2021-11-09 10:25:57

• 📌 The main abstraction is based on Remote Procedure Calls (RPC). ^30179184-52-10591-10757

  • ⏱ 2021-11-09 10:26:11

• 📌 The rest is handled by the RPC system, which typically has two parts: a stub generator and a runtime library. These parts will be discussed in more detail next. ^30179184-52-10882-10994

  • ⏱ 2021-11-09 10:26:30

• 📌 The way servers are organized for concurrency. ^30179184-52-13277-13288

  • ⏱ 2021-11-09 10:29:00

• 📌 If an RPC call blocks (for example, on I/O), it wastes server resources. Therefore, most servers are constructed in some concurrent manner. A common organization is a thread pool. ^30179184-52-13342-13417

  • ⏱ 2021-11-09 10:29:16

• 📌 Threads continuously receive other requests and may send them to other worker threads. ^30179184-52-13488-13514

  • ⏱ 2021-11-09 10:29:27

• 📌 The hostnames and port numbers provided by the current internet protocol. ^30179184-52-13872-13889

  • ⏱ 2021-11-09 10:37:17

• 📌 Building RPC on top of a reliable communication layer may lead to inefficiencies in performance. ^30179184-52-14371-14395

  • ⏱ 2021-11-09 10:37:43

• 📌 How the reliable communication layer works: acknowledgment and timeout/retry. ^30179184-52-14404-14423

  • ⏱ 2021-11-09 10:37:48

• 📌 Many RPC packages are built on top of unreliable communication layers, such as UDP. ^30179184-52-14580-14608

  • ⏱ 2021-11-09 10:37:59

• 📌 47.6 Summary We introduced a new topic, distributed systems and their main issue: how to handle failures, which are now common events. As people say within Google, when you only have your desktop, failures are rare. When you have a data center with thousands of machines, failures are happening all the time. The key to all distributed systems is how to handle failures. We also saw that communication is at the core of all distributed systems. This common abstraction of communication can be seen in Remote Procedure Calls (RPC), which allow clients to make remote calls on servers. The RPC package handles all the details, including timeout/retry and acknowledgment, to provide a service very similar to local procedure calls. The best way to truly understand the RPC package is, of course, to use it yourself. Sun's RPC system uses the stub compiler rpcgen, which is quite common and available on many systems today, including Linux. Give it a try and see what all this fuss is about. References [A70] "The ALOHA System — Another Alternative for Computer Communications" Norman Abramson The 1970 Fall Joint Computer Conference The ALOHA network pioneered some fundamental concepts in networking, including exponential back-off and retransmission. Over the years, these have become the foundation of shared bus Ethernet network communication. [BN84] "Implementing Remote Procedure Calls" Andrew D. Birrell, Bruce Jay Nelson ACM TOCS, Volume 2:1, February 1984 The foundational RPC system, upon which all other theories are based. Yes, it is another pioneering effort from our friends at Xerox PARC. ^30179184-52-16865

  • ⏱ 2021-11-09 10:38:39

• 📌 To build reliable file transfers, end-to-end reliability checks must be included, such as reading the file on the receiver's disk after the entire transfer is complete, calculating checksums, and comparing that checksum with the checksum of the sender's file. The implication of this guideline is that sometimes, lower layers providing additional functionality can indeed improve system performance or optimize the system in other ways. Therefore, such mechanisms at lower layers of the system should not be excluded. In fact, you should carefully consider the practicality of such mechanisms and their ultimate role in the overall system or application. The RPC package often provides a clearly defined byte order in its message format, thus addressing this issue. In Sun's RPC package, the XDR (eXternal Data Representation) layer provides this functionality. If the computer sending or receiving the message matches the byte order of XDR, messages will be sent and received as expected. ^30179184-54-14643

  • ⏱ 2022-01-14 09:14:39

Chapter 48 Sun's Network File System (NFS)#

• 📌 One of the first uses of distributed client/server computing is in the field of distributed file systems. In this environment, there are many client machines and one (or several) servers. The server stores data on its disks, and clients request data through well-structured protocol messages. Figure 48.1 shows the basic setup. [illustration] Figure 48.1 General client/server system From the figure, it can be seen that the server has disks, and the clients sending messages access directories and files on the server's disks through the network. Why go through the trouble of adopting this arrangement? (That is, why not just let clients use their local disks?) Well, this setup allows for easy sharing of data between clients. Therefore, if you access a file on one computer (client 0) and then use another (client 2), you will have the same file system view. You can naturally share your data across these different machines. The second benefit is centralized administration. For example, backup files can be done through a few server machines without having to go through numerous clients. Another advantage might be security, keeping all servers in locked server rooms. The key question: how to build a distributed file system? What key aspects need to be considered? Where are the pitfalls? What can we learn from existing systems? ^30179184-53-359
  • ⏱ 2021-11-09 10:40:01

Reading Notes#

Chapter 4 Abstraction: Processes#

Highlighted Comments#

• 📌 It is still unclear whether this is a good decision. ^7265636-7uG78TXby
  • 💭 It can be allocated by priority, and immediate switching may cause frequent context switching, which is less efficient.
  • ⏱ 2021-11-09 09:28:51

Highlighted Comments#

• 📌 When a process initiates an I/O request to the disk, it will be blocked, allowing other processes to use the processor. ^7265636-7uG6WqgFc
  • 💭 Thread blocking.
  • ⏱ 2021-11-09 09:25:47

Chapter 16 Segmentation#

Highlighted Comments#

• 📌 The concept of segmentation has emerged. Segmentation is not a new concept; it can even be traced back to the early 1960s. The idea is simple: introduce more than one pair of base and limit registers in the MMU, giving each logical segment in the address space a pair. A segment is simply a contiguous fixed-length area in the address space, and there are typically three logically distinct segments in a typical address space: code, stack, and heap. The mechanism of segmentation allows the operating system to place different segments in different physical memory areas, thus avoiding unused portions of the virtual address space occupying physical memory. ^7265636-7u2LJkG3F
  • 💭 I see now.
  • ⏱ 2021-10-14 12:41:45

Chapter 46 Dialogue About Distribution#

Highlighted Comments#

• 📌 Replication, retries, and various other techniques. People have developed these techniques over time to detect failures and recover from them. ^7265636-7uG9Uidei
  • 💭 Distributed one.
  • ⏱ 2021-11-09 10:11:04

Chapter 47 Distributed Systems#

Highlighted Comments#

• 📌 In the best case, the page is already a local page on the machine, so data can be quickly accessed. In the second case, the page is currently on other machines. A page fault occurs, and the page fault handler sends a message to other computers to fetch the page, loads it into the requesting process's page table, and then continues execution. ^7265636-7uGaS0mwP
  • 💭 Not practical. Stability and performance issues.
  • ⏱ 2021-11-09 10:25:46

Highlighted Comments#

• 📌 To allow the receiver to detect duplicate message transmissions, the sender must uniquely identify each message in some way, and the receiver needs some way to track whether it has seen each message. ^7265636-7uGaER0eS
  • 💭 So that's how the sequence number comes about; acknowledgments may be lost.
  • ⏱ 2021-11-09 10:22:32

Highlighted Comments#

• 📌 To allow the receiver to detect duplicate message transmissions, the sender must uniquely identify each message in some way, and the receiver needs some way to track whether it has seen each message. ^7265636-7uGaC1Akt
  • 💭 Sequence number.
  • ⏱ 2021-11-09 10:21:50

Book Review#