Please enable JavaScript.
Coggle requires JavaScript to display documents.
Parallel Processing - Coggle Diagram
Parallel Processing
Symmetric Multiprocessors
A standalone computer
Characteristics
Two or more similar processors of comparable capacity
Processors share same memory and I/O
Processors are connected by a bus or other internal connection
Memory access time is approximately the same for each processor
All processors share access to I/O
All processors can perform the same functions (hence symmetric)
System controlled by integrated operating system
Advantages
Performance
Availability
Incremental growth
Scaling
Organization Classification
Time shared or common bus
Good at simplicity, flexibility, and reliability
Limited by bus cycle time, cache coherence probleem
Multiport memory
Central control unit
OS issues
Simultaneous concurrent processes
Scheduling
Synchronization
Memory management
Reliability and fault tolerance
Taxonomy of Parallel Processor Architectures
Single instruction, single data stream (SISD)
Uniprocessor
Single instruction, multiple data stream (SIMD)
Vector processor
Array processor
Multiple instruction, single data stream (MISD)
Multiple instruction, multiple data stream (MIMD)
Shared memory (tightly coupled)
Symmetric multiprocessor (SMP)
Non uniform memory access (NUMA)
Distributed memory (loosely coupled)
Clusters
Multi-threading
Increase performance
MIPS rate = F * IPC
Limited by complexity and power consumption
Threads and processes
Thread
Multithreaded processors may or may not
be same as software threads
Dispatchable unit of work within
process
Process
An instance of program running on computer
Process switch
Switches the processor from one
process to another
Thread switch
Switchies processor between threads within same
process
Multithreading and Chip Multiprocessors
Instruction stream divided into smaller streams (threads)
Executed in parallel
No increase in complexity or power consumption
Implicit and Explicit Multithreading
Implicit
Concurrent execution of multiple threads extracted from single sequential program
Explicit
A separate program counter for each thread of execution to be executed concurrently
Approaches
Interleaved
Blocked
Simultaneous
Chip
Scalar processor
Single-threaded scalar
Interleaved multithreaded scalar
Blocked multithreaded scalar
Multiple Instruction Issue Processors
Superscalar
Interleaved multithreading superscalar
Blocked multithreaded superscalar
Very long instruction word (VLIW)
Interleaved multithreading VLIW
Blocked multithreaded VLIW
Parallel, Simultaneous
Execution of Multiple Threads
Simultaneous multithreading
Chip multiprocessor
Cache Coherence and MESI Protocol
Background
Multiple copies of same data in
different caches can result in an inconsistent view of memory
Write back policy and write through
Software solutions
Compiler and operating system deal with problem
Overhead transferred to compile time
Design complexity transferred from hardware to software
Analyze code to determine safe periods for caching shared variables
Hardware solutions
Cache coherence protocols
Dynamic recognition of potential problems
Run time
More efficient use of cache
Transparent to programmer
Examples
Directory protocols
Centralized system collect
Requests are checked against directory
Effective in large scale systems, but creates central bottleneck
Snoopy protocols
Distribute cache coherence responsibility
Suited to bus-based multiprocessor, but increases bus traffic
Two basic approaches from
directory and snoopy protocol
Write invalidate (MESI)
Write update