Syllabus   Blank Homework  
Notes   Labs   Scores   Blank

Lecture Notes
Dr. Tong Lai Yu, March 2010
    0. Review and Overview
    1. An Introduction to Distributed Systems
    2. Deadlocks
    3. Distributed Systems Architecture
    4. Processes
    5. Communication
    6. Distributed OS Theories
        7. Distributed Mutual Exclusions
    8. Agreement Protocols
    9. Distributed Scheduling
    10. Distributed Resource Management
    11. Recovery and Fault Tolerance
    12. Security and Protection
     
    
    The top software developers are more productive than
    average software developers not by a factor of 10X or
    100X or even 1000X but by 10,000X.
    
                                            Nathan Myhrvold
    
    
    Review and Overview
    1. Process and Threads
    2. Process -- programs in execution, including stacks, memory, resources consumed
      has states : New, Ready, Running, Blocked, Terminated
      Process Control block ( PCB ) saves complete information about a process.
    3. Concurrent processes -- execution overlaps in time, i.e. second process starts before first completes
      interact with each other through
      1. shared variables
      2. message passing -- two concurrent processes exchange information with each other by sending and receiving messages
    4. Serial processes -- execution of one must be complete before the other can start
    5. Threads -- light weight process ( LWP ), includes PC and stack, shares with peer threads resources
    6. Critical Section code section that accesses resources shared by several processes;
      if only one process can enter at a time => mutual exclusion
    7. Mutual Exclusion Mechanisms
    8. Busy waiting -- wasting CPU cycles
    9. Disabling interrupt -- has trouble with multiprocessor systems
    10. Test-and-set lock ( TSL ) -- atomic instruction
    11. Semaphores
    12. generalization of mutual exclusion
    13. integer variables
    14. counting semaphore; binary semaphore (mutex)
    15. atomic operations : UP ( V ), DOWN ( P )
      wait:   down ( mutex );
        access_CS();
      signal:   up ( mutex );

      C-code Guarded Commands
              void wait ( int S ) {   //atomic
                while ( S <= 0 ) 
                      ;               //wait
                S = S - 1;
              }
              
      when ( S > 0 ) [
          S = S - 1;
      ]

      when ~ guard
      only when the guarded statement is true,
      will the statements inside the square brackets,
      known as command sequence, be executed!

              void signal ( int S ) { //atomic     
                S = S + 1;
              }
      	
      [ S = S + 1; ]

    16. Weakness

      1. difficult to verify program correctness
      2. difficult to implement, omission of an operation may lead to deadlock
      3. need to know which other processes are using semaphore

    17. Java Synchronization
    18. Use monitor (see below) to achieve synchronization.
    19. Every object is associated with a lock; normally lock is ignored,
    20. lock activated by keyword synchronized; block-based
    21. JVM ensures that only one synchronized method or block of statements can be locked
        Example:
          public class Score
          {
            private double score;
            private double total;
        
            public synchronized void setScore ( double s )
            {
              score = s;
            }
        
            public synchronized void addScore ( double s )
            {
              total += s;
            }
        
            public double getScore ()
            {
              return score;
            }
            .....
          }
          
      • Only one thread can access either setScore or addScore at one time.
      • While a thread is accessing setScore, no thread can access addScore but a thread can access getScore,
        which is not synchronized.
      • Multiple threads can access getScore.
    22. Condition Variables
    23. For some problems, using semaphores could be complex.
    24. A condition variable is a queue of threads (or processes) waiting for some sort of notifications.
    25. Supported by POSIX and SDL; Win-32 events
    26. A condition variable queue can only be accessed with two methods associated with its queue.
      These methods are typically called wait and signal. The signal method is called notify in Java.
    27. Threads waiting for a guard to become true enter the queue.
    28. Threads that change the guard from false to true could wake up the waiting threads.
    29. General approach:

      Guarded CommandJava Implementation
       when ( guard ) [
            statement 1
            ..... 
            statement n
      ]
      
      class SharedResource   
      {   
           final Lock mutex = new ReentrantLock();   
           final Condition condVar = mutex.newCondition();   
           boolean condition = false;   
           .....   
           public  void methodA() throws InterruptedException   
           {   
              mutex.lock();   
              while ( !condition );   
                   condVar.await();   
              statement 1   
               .....   
               statement n   
      
               mutex.unlock();   
            }   
            .....   
      }

      //code modifying guard
            ..... 
       
      class SharedResource   
      {   
            .....   
            public  void methodB() throws InterruptedException   
            {   
                mutex.lock();   
                condition = true;  //or code modifying guard   
                .....   
                condVar.signal();   
                mutex.unlock();   
            }   
      } 

    30. To evaluate a guard safely, a thread must mutually exclude the access of all other threads.
    31. While the thread is waiting for the guard to become true, it does not lock the mutex,
      otherwise other threads cannot change the value of the guard.
    32. Example: Readers-writers problem
      1. Mutual Exclusion required only when is being modified.
      2. Multiple readers can read at the same time.
    33. No simple solution using semaphores, but can be solved easily using condition variables.
    34. Solution presented in guarded commands:
        void reader()
        {
          when ( writers == 0 ) [
            readers++;
          ]
        
          //read
        
          [readers--;]
        }               
           
        void writer()
        {
          when ( (readers == 0) && (writers == 0) )[
            writers++;
          ]
        
          //write
        
          [writers--;]
        }               
    35. Java Implementation
        import java.util.concurrent.locks.Condition;
        import java.util.concurrent.locks.Lock;
        import java.util.concurrent.locks.ReentrantLock;
        
        class ReaderWriter
        {
          final Lock mutex = new ReentrantLock();
          final Condition readerQueue=mutex.newCondition();  //cond variable
          final Condition writerQueue=mutex.newCondition();  //cond variable
          int   nReaders = 0; //number of reader threads
          int   nWriters = 0; //number of writer threads (0 or 1)
        
          void reader() throws InterruptedException
          {
            mutex.lock();         //mutual exclusion
            while ( !(nWriters == 0) )
              readerQueue.await();//wait in readerQueue till no more writers
            nReaders++;           //one more reader
            mutex.unlock();
            //read
            //........
            //finished reading
            mutex.lock();         //need mutual exclusion
            if ( --nReaders == 0 )
              writerQueue.signal();//wake up a waiting writer
            mutex.unlock();
          }
        
          void writer() throws InterruptedException
          {
            mutex.lock();
            while ( !((nReaders == 0) && (nWriters == 0)) )
              writerQueue.await();//wait in writerQueue
                                  // until no more writer & readers
            nWriters++; //one writer
            mutex.unlock();
            //write
            //........
            //finished writing
            mutex.lock(); //need mutual exclusion
            nWriters--; //only one writer at a time
            writerQueue.signal(); //wake up a waiting writer
            readerQueue.signalAll(); //wake up all waiting readers
            mutex.unlock();
          }
        }
        	
    36. If too many readers, writers may have starvation. Can be solved with writers-priority.
        void reader()
        {
          when ( writers == 0 ) [
            readers++;
          ]
        
          //read
        
          [readers--;]
        }               
           
        void writer()
        {
          [writers++;]
          when ( (readers == 0) && (active_writers == 0) )[
            active_writers++;
          ]
        
          //write
        
          [writers--;  active_writers--;]
        }               
      Here writers represents the number of threads that are either writing or waiting to write.
      The variable active_writers represents the number of threads ( 0 or 1 ) that are currently writing.
      Java Implementation:
        class ReaderWriterPriority
        {
          final Lock mutex = new ReentrantLock();
          final Condition readerQueue = mutex.newCondition(); //cond variable
          final Condition writerQueue = mutex.newCondition(); //cond variable
          int nReaders = 0;        //number of reader threads
          int nWriters = 0;        //number of writer threads (0 or 1)
          int nActiveWriters = 0;  //number of threads currently writing
        
          void reader() throws InterruptedException
          {
            mutex.lock();          //mutual exclusion
            while ( !(nWriters == 0) )
              readerQueue.await(); //wait in readerQueue until no more writers
            nReaders++;            //one more reader
            mutex.unlock();
            //read
            //........
            //finished reading
            mutex.lock();          //need mutual exclusion
            if ( --nReaders == 0 )
              writerQueue.signal();//wake up a waiting writer
            mutex.unlock();
          }
        
          void writer() throws InterruptedException
          {
            mutex.lock();
            nWriters++;            //a writer has arrived
            while ( !((nReaders == 0) && (nActiveWriters == 0)) )
              writerQueue.await(); //wait in writerQueue
                                   // until no more writer & readers
            nActiveWriters++;      //one active writer
            mutex.unlock();
            //write
            //........
            //finished writing
            mutex.lock();          //need mutual exclusion
            nActiveWriters--;      //only one active writer at a time
            if ( --nWriters == 0 ) //no more waiting writers, so wake
              readerQueue.signalAll();// up all waiting readers
            else //has waiting writer
              writerQueue.signal();//wake up one waiting writer
            mutex.unlock();
          }
        }
                       
    37. Monitors
    38. high-level synchronization,
      consists of procedures, shared resources, and administrative data
    39. only one process can be active inside a monitor
    40. procedures of monitor can only access data inside monitor
    41. local data of monitor cannot be accessed from outside
    42. When a task attempts to access a monitor method, it is put in the monitor's entry queue.
    43. A task that holds the monitor lock may give it up and enter a condition variable queue by executing the corresponding wait method.
    44. A task that holds the monitor lock may revive a task waiting in a condition variable queue with the notify method of that queue.
    45. The notify method removes one task from the condition variable queue if the queue is not empty.
    46. Processes in the monitor queues are waiting to acquire the monitor lock.
    47. When a task is removed from one of the condition variable queues, it is put in the waiting queue.

      Weakness:

      1. one process active inside monitor => defeats concurrency purpose
      2. nested monitor calls --> deadlock
      Java Synchronization ~ monitor
    48. Serializers

      Procedures



      Hollow Region


      ( multiple processes can be active here )
       


      Hollow Region


      ( multiple processes can be active here )
       

    49. Processes can be concurrently active in a hollow region
    50. when a process enters a hollow region, it releases the possession of the serializer to let other processes gain possession join-crowd ( <crowd> ) then <body> ) end
    51. Regain possession of serializer enqueue ( <priority>, <queue-name> ) until ( <condition> )
    52. Weakness
      1. more complex
      2. less efficient
    53. Path Expression
    54. Indicates the operations order on a shared resource
    55. proposed by Campbell and Habermann in 1970s

    56. Syntax:
        path S end

      S -- execution history: an expression operating on resource with operators:
      • ; sequencing -- statement executions are in sequence e.g. path open; read; close; end
      • + selection -- only one statement can be executed at a time e.g. path read + write end
        either read or write, but order doesn't matter
      • {} concurrency -- concurrent execution e.g. path { read } end
        several read can be executed at the same time

        e.g. path write + { read } end
        either write or several read

        e.g. path { write ; read } end
        at any time, there can be any number of instantiations of path write; read.

      • path-end -- repetition

      Example: readers-writers problem with weak reader's priority ( several readers can read file at the same time but only one writer can write to the file at a time; an arriving reader has higher priority than a waiting writer if reading has occurred, but when reading or writing is done, reader and writer have same priority, i.e. chosen randomly )

      path { read } + write end

      Example: Writer's priority: (when there is more than one path expression, the order of operations indicated by all the path expressions must be satisfied.


      path start_read + {start_write; write} end
      path {start_read; read} + write end

          1st expression: a writer cannot start a write when a writer is writing or when a reader is reading)
          2nd expression: no reader can execute start_read when a writer is writing
    57. Communicating Sequential Processes ( CSP )

      Use I/O commands for synchronization Input command = <source process id> ? <target variable>

      Output command = <destination process id> ! <expression>

      operators: ! (send)   ? (receive)

      Usage:

      • B!x   (message x is sent to process B)
      • A?y   (receive message from process A and save in buffer y)

      e.g. mouse?xy,   screen!xy
      ( read input from mouse, save value in xy; the value of xy is output to the screen )

      Concurrent processes :
      [process P1's code||process P2's code|| ..... ||process Pn's code]

      Guarded Command G -- > CL
      G -- guard, a boolean expression
      CL -- list of commands

      basically :
      if ( G )
          CL
      evaluate G, if false, ( i.e. guard fails ), CL won't be evaluated

      Alternative command

      G1 -->CL1 G2 -->CL2 ..... Gn -->CLn if all guarded commands fail, then the alternative command fails
      otherwise, one successfully guarded command is selected randomly and executed

      Repetitive command

      *[G1 -->CL1 G2 -->CL2 ..... Gn -->CLn ] repeatedly execute the alternate command until all guarded commands fail --> terminate

      CSP parallelism, on its own, does not introduce non-determinism.

      Example

      i := 0; *[ i < size; content(i) ≠ n → i := i + 1]

      start with i = 0, then repetitively scan until either isize or some content( i ) equals n

      Example

      *[c: character; P1 ? c → P2 ! c ]

      Repeatedly send process P2 a character received from process P1 until P1 terminates.