Syllabus   Blank Homework  
Notes   Labs   Scores   Blank

Lecture Notes
Dr. Tong Lai Yu, March 2010
    0. Review and Overview
    1. An Introduction to Distributed Systems
    2. Deadlocks
    3. Distributed Systems Architecture
    4. Processes
    5. Communication
    6. Distributed OS Theories
        7. Distributed Mutual Exclusions
    8. Agreement Protocols
    9. Distributed Scheduling
    10. Distributed Resource Management
    11. Recovery and Fault Tolerance
    12. Security and Protection
    1. Threads Revisited

    2. We build virtual processors in software, on top of physical processors:

      Processor: Provides a set of instructions along with the capability of
        automatically executing a series of those instructions.

      Thread: A minimal software processor in whose context a series of
        instructions can be executed. Saving a thread context implies stopping
        the current execution and saving all the data needed to continue the
        execution at a later stage.

      Process: A software processor in whose context one or more threads may be
        executed. Executing a thread, means executing a series of instructions
        in the context of that thread.

    3. Context Switching:

      Processor context: The minimal collection of values stored in the
        registers of a processor used for the execution of a series of
        instructions (e.g., stack pointer, addressing registers, program

      Thread context
      Process context

    4. Notes:
      1. Threads share the same address space. Thread context switching
        can be done entirely independent of the operating system.
      2. Process switching is generally more expensive as it involves
        getting the OS in the loop, i.e., trapping to the kernel.
      3. Creating and destroying threads is much cheaper than doing so
        for processes.
    5. Threads and Operating Systems

      Main issue
      Should an OS kernel provide threads, or should they be implemented as
      user-level packages?

      User-space solution

    6. All operations can be completely handled within a single process =>
      implementations can be extremely efficient.
    7. All services provided by the kernel are done on behalf of the process in
      which a thread resides
      => if the kernel decides to block a thread, the
      entire process will be blocked.
    8. Threads are used when there are lots of external events: threads block
      on a per-event basis
      => if the kernel cannot distinguish threads, how can it
      support signaling events to them?

    9. Kernel solution
      The whole idea is to have the kernel contain the implementation of a thread
      package. This means that all operations return as system calls

    10. Operations that block a thread are no longer a problem: the kernel
      schedules another available thread
      within the same process.
    11. Handling external events is simple: the kernel (which catches all events)
      schedules the thread associated with the event.
      The big problem is the loss of efficiency due to the fact that each thread
      operation requires a trap to the kernel.
    12. Conclusion
      Try to mix user-level and kernel-level threads into a single concept.

    13. Threads and Distributed Systems

      Improve performance
    14. Starting a thread is much cheaper than starting a new process.
    15. Having a single-threaded server prohibits simple scale-up to a
      multiprocessor system.
    16. As with clients: hide network latency by reacting to next request while
      previous one is being replied.

    17. Better structure
    18. Most servers have high I/O demands. Using simple, well-understood
      blocking calls
      simplifies the overall structure.
    19. Multithreaded programs tend to be smaller and easier to understand due
      to simplified flow of control.
    20. Virtualization

      Virtualization is becoming increasingly important:

    21. Hardware changes faster than software
    22. Ease of portability and code migration
    23. Isolation of failing or attacked components
          (a) General organization between a program, interface and system.
          (b)General organization of virtulizing system A on top of system B.
    24. Architecture of VMs
      Virtualization can take place at very different levels, strongly depending
      on the interfaces as offered by various systems components:

        Figure: Various interfaces offered by computer systems.

      Process VMs versus VM Monitors

    25. Process VM: A program is compiled to intermediate (portable)
      code, which is then executed by a runtime system (Example: Java
    26. VM Monitor: A separate software layer mimics the instruction set
      of hardware => a complete operating system and its applications
      can be supported (Example: VMware, VirtualBox).
    27. VM Monitors on operating systems
      We are seeing VMMs run on top of existing operating systems.

    28. Perform binary translation: while executing an application or
      operating system, translate instructions to that of the underlying
    29. Distinguish sensitive instructions: traps to the orginal kernel (think
      of system calls, or privileged instructions).
    30. Sensitive instructions are replaced with calls to the VMM.
    31. Clients: User Interfaces

      A major part of client-side software is focused on (graphical) user

      X was specifically designed to be used over network connections.
      X kernel (display machine) is the server; remote application is the client.

      Compound documents

      User interface is application-aware => interapplication communication:

    32. drag-and-drop: move objects across the screen to invoke
      interaction with other applications
    33. in-place editing: integrate several applications at user-interface
      level ( word processing + drawing facilities)
    34. Client-Side Software

      Generally tailored for distribution transparency
      A client should not know that it is communicating via network or not
      Distribution is often less transparent to servers than to clients

    35. Access transparency: handled through client-side stubs for RPCs: provides
      same interface as at the server, hides different machine architectures
    36. Location/migration transparency: let client-side software keep track of
      actual location (if server changes location: client rebinds to the server if necessary).
      Hide server locations from user.
    37. Replication transparency: multiple invocations handled by client stub:
      Client-side software can collect all responses but passes only one to the client application.
    38. Failure transparency: mask server and communication failures: done through client middleware,
      e.g. connect to another machine.
      middleware may return cached data (e.g. some Web browsers)
    39. Servers: General organization

      Basic model
      A server is a process that waits for incoming service requests at a
      specific transport address. In practice, there is a one-to-one mapping
      between a port and a service.

      Type of servers:

    40. Iterative servers: handles a request itself; can handle only one client at a time,

    41. Concurrent servers: Does not handle a request itself; pass it to a separate thread or another process

    42. Superservers: Servers that listen to several ports, i.e., provide several
      independent services. In practice, when a service request comes
      in, they start a subprocess to handle the request (UNIX inetd)
    43. Out-of-band communication

      Is it possible to interrupt a server once it has accepted (or is in the process of
      accepting) a service request?
      One approach is that the user (client) exit the application; server will tear down the connection

      Solution 1
      Use a separate port for urgent data:
      • Server has a separate thread/process for urgent messages
      • Urgent message comes in => associated request is put on hold
      • Note: we require OS supports priority-based scheduling

      Solution 2
      Use out-of-band communication facilities of the transport layer:

      • Example: TCP allows for urgent messages in same connection
      • Urgent messages can be caught using OS signaling techniques

    44. Servers and state

      Stateless servers
      Never keep accurate information about the status of a client after having
      handled a request:

    45. Don't record whether a file has been opened (simply close it again after
    46. Don't promise to invalidate a client's cache
    47. Don't keep track of your clients
    48. Consequences

    49. Clients and servers are completely independent
    50. State inconsistencies due to client or server crashes are reduced
    51. Possible loss of performance because, e.g., a server cannot anticipate
      client behavior (think of prefetching file blocks)
    52. Stateful servers
      Keeps track of the status of its clients:

    53. Record that a file has been opened, so that prefetching can be
    54. Knows which data a client has cached, and allows clients to keep
      local copies of shared data
    55. Observation
      The performance of stateful servers can be extremely high, provided
      clients are allowed to keep local copies. As it turns out, reliability is not
      a major problem.

    56. Server clusters: three different tiers

      Crucial element
      The first tier is generally responsible for passing requests to an
      appropriate server.

      Request Handling
      Having the first tier handle all communication from/to the cluster may
      lead to a bottleneck.

      Various, but one popular one is TCP-handoff

    57. Example: PlanetLab

      Different organizations contribute machines, which they subsequently
      share for various experiments.

      We need to ensure that different distributed applications do not get into
      each other's way => virtualization

      Independent and protected environment with its own libraries,
      server versions, and so on. Distributed applications are assigned a
      collection of vservers distributed across multiple machines (slice).

      Virtual Machine Monitor ( VMM ):
      An enhanced Linux operating system,
      Ensures that vservers are separated in different vservers: processes are
      executed concurrently and independently.

    58. Code Migration

    59. Approaches to code migration
    60. Migration and local resources
    61. Migration in heterogeneous systems
    62. Some Context

      CS: Client-Server   REV: Remote evaluation
      e.g.perform database operations involving
      large amount of data
      CoD: Code-on-demand
      Migrate part of server to client
       MA: Mobile agents
      moves from site to site

      Strong and weak mobility
      Object components

    63. Code segment: contains the actual code
    64. Resource segment: contains references to resources needed by the process
    65. Execution segment: stores the executation state of a process

    66. Weak mobility
      Move only code segment and perhaps some initialization data (and reboot execution):
    67. Relatively simple, especially if code is portable (e.g. Java applet)
    68. Distinguish code shipping (push) from code fetching (pull)

    69. Strong mobility
      Move components, including execution segment
    70. Migration: move entire object from one machine to the other
      A process could be stopped, moved to another machine and resumed execution where it left off.
    71. Cloning: start a clone, and set it in the same execution state.
    72. Migration can be sender-initiated (e.g. uploading program), receiver-initiated (e.g. java applets) or hybrid.

      Managing local resources
      Process-to-resource binding

    73. By identifier: the object ( process ) requires a specific instance of a resource
      ( strongest binding ) (e.g. a specific database, a URL)
    74. By value: the object requires the value of a resource (e.g.
      a library ). The execution of a process would not be affected if another resource
      would provide the same value
    75. By type: the object requires that only a type of resource is
      available (e.g. local devices such as a color monitor)
    76. Also need to consider resource-to-machine binding:

    77. Unattached resources : can be easily moved between machines ( e.g. files associated with programs )
    78. Fastened resources: costly to move or copy them from one machine to another (e.g. local databases, web sites )
    79. Fixed resources: bound to one machine, cannot be moved (e.g. local devices such as printers )

      Migration in heterogenous systems
      Main problem

    80. The target machine may not be suitable to execute the migrated
    81. The definition of process/thread/processor context is highly
      dependent on local hardware, operating system and runtime
    82. Only solution
      Make use of an abstract machine that is implemented on different

    83. Interpreted languages, effectively having their own VM
    84. Using VM (as discussed previously)
    85. Software Agents

      What is an Agent?
      Definition: An autonomous process capable of reacting
      to, and initiating changes in its environment, possibly
      in collaboration with users and other agents

    86. collaborative agent: collaborate with others in a multiagent system
    87. mobile agent: can move between machines
    88. interface agent: assist users at user-interface level
    89. information agent: manage information from physically different sources
    90. Property Common? Description
      Autonomous Yes Can act on its own
      Reactive Yes Responds timely to changes in its environment
      Proactive Yes   Initiates actions that affect its environment
      Communicative   Yes Can exchange information with users and other agents
      Continuous No Has a relatively long lifespan
      Mobile No Can migrate from one site to another
      Adaptive No Capable of learning

      Agent Technology

      Management: Keeps track of where the agents on
      this platform are (mapping agent ID to port)
      Directory: Mapping of agent names & attributes to
      agent IDs
      ACC: Agent Communication Channel, used to communicate
      with other platforms