Please enable JavaScript.

Coggle requires JavaScript to display documents.

PARALLEL PROCESSING - Coggle Diagram

- - - - Multiport memory
      - Central control unit
      - Time shared or common bus
        
        Advantages
        
        Flexibility
        
        Reliability
        
        Simplicity
        
        Disadvantages
        
        Performance limited by bus cycle time
        
        Each processor should have local cache
        
        Leads to problems with cache coherence
        
        Features
        
        Arbitration
        
        Time-sharing
        
        Addressing
  - - - Synchronization
      - Memory management
      - Scheduling
      - Reliability and fault tolerance
      - Simultaneous concurrent processes
  - - - L2 cache
      - System control element (SCE)
      - Dual-core processor chip
      - Main store control (MSC)
      - Memory card
      - Memory bus adapter (MBA)
- - - - Directory protocols
        
        Centralized system collect and maintain information about copies of data in cache
        
        Creates central bottleneck
      - Snoopy protocols
        
        Increase bus traffic
        
        Two basic approach
        
        Write invalidate
        
        Write update
        
        Distribute cache coherence responsibility among cache controllers
  - - - Effect
        
        Exclusive
        
        Modified
        
        Shared
    - - Exclusive
      - Modified
      - Shared
      - Invalid
  - - - Can result in an inconsistent view of memory
- - - - Parametric computing
      - Parallelized application
      - Parallelizing compiler
- - - - Led to the development of cluster system
        
        Each node has its own private memory
        
        Software must be tailored to this environment to achieve maximum performance
    - - Silicon graphics origin
      - Sequent NUMA-Q systems
  - - - There is a single addressable memory
      - When a processor initiates a memory access
        
        Requested memory location is not in the cache
        
        L2 Cache initiates a fetch operation
        
        Disiredmline is in a remote portion of the main memory
        
        Automatic request is send to fetch that line
    - - Containing multiple processors
      - Each of which is an SMP organization
      - Nodes are interconnected by a communication facility
    - - Each node must maintain some sort of directory that gives it an indication of the location of various portions of memory and cache status information
      - Cache coherence is achieved through a cooperative protocol to take care of modifications
  - - - Deliver effective performance at higher levels of parallelism than SMP, without requiring major software changes
    - - Software changes will be required to move an operating system and applications from an SMP to a CC-NUMA system
      - Availability is a complex issue that depends on the exact implementations of the CC-NUMA system
- - - - Characteristics
        
        Resource ownership
        
        Scheduling/execution
  - - - Interleaved multithreaded scalar
      - Single-threaded scalar
      - Single-threaded scalar
    - - Simultaneous multithreading
      - Chip multiprocessor (multicore)
    - - Simultaneous multithreading (SMT)
      - Chip multiprocessing
      - Blocked multithreading
      - Interleaved multithreading
- - - - Single instruction, multiple data (SIMD) stream
        
        Array processor
        
        Vector processor
      - Multiple instruction, single data (MISD) stream
      - Single instruction, single data (SISD) stream
        
        Uniprocessor
      - Multiple instruction, multiple data (MMD) stream
        
        Shared memory (tightly coupled)
        
        Symmetric multiprocessor
        
        Nonuniform memory access
        
        Distributed memory (loosely coupled)
        
        Clusters
  - - - Shared memory
      - Distributed memory
    - - Each processor executes different instruction sequence
    - - Access times to different regions of memory many differ
    - - Each PU may have its own dedicated memory
    - - PU = processing unit
      - MU = memory unit
      - CU = control unit
      - IS = instruction
      - DS = data
- - - - The control data machines
        
        Operands are obtained directly from memory
      - Vector register
        
        Advantages
        
        The operation is decoupled from slower main memory
        
        Disadvantages
        
        Programmer or compiler must consider them for good performance
      - IBM and cray organizations
        
        A register-to-register approach, where vector operands are staged in vector registers
      - IBM 3090 vector facility
      - Three additional registers
        
        The vector-status register
        
        The vector-activity
        
        The vector-mask register
    - - Three compound instructions are available
        
        MULTIPLY-AND-SUBTRACT
        
        MULTIPLY-AND-SUM
        
        MULTIPLY-AND-ADD
      - do not require the use of additional registers
      - IBM vector facility does not support instruction chaining
      - Require one less register access
    - - IBM vector offers increased performance
        
        Allowing concurrent and parallel data-access and arithmetic operations
        
        Using vector registers
        
        Fixed and predetermined structure of vector data
      - System/370
        
        Vector instructions are interruptible and have the same arithmetic exceptions as scalar instructions
        
        Vector data resides in virtual storage
        
        Used for scalar operations
    - - Two summation instructions
        
        Accumulate
        
        Multiply-and-accumulate
      - Elements of the vector are added in such a way as to produce four partial sums
      - Target is always a vector register, except for comparison
      - A separate vector register is used to hold the partial sums
      - Many of the instructions use a three-operand format
      - The four partial sums are added together to produce the final result
  - - - Vector processing
        
        Way to reach
        
        Parallelism and pipelining
        
        The use of multiple ALU in a single processor
        
        Multiple parallel processors
        
        Chaining
      - Parallel processing
        
        Performing vector operations using multiple independent processors in parallel
    - - Parallel ALUs
      - Parallel processors
      - Pipelined ALU
    - - Recognize that the main task is to perform arithmetic operations on arrays or vectors of floating-point numbers