Please enable JavaScript.
Coggle requires JavaScript to display documents.
Parallel Processing - Coggle Diagram
Parallel Processing
Taxonomy of parallel processor
architectures
SISD
Parallel Organizations
control unit
processing unit
instruction
memory unit
data
Single processor
Single instruction stream
Data stored in single memory
Uni-processor
SIMD
Single machine instruction
Controls simultaneous execution
Number of processing elements
Lockstep basis
Each processing element has associated
data memory
Each instruction executed on different set
of data by different processors
Vector and array processors
Parallel Organizations
MIMD
MIMD Shared
Memory
Set of general-purpose processors
Simultaneously execute different instruction
sequences
Each can process all instructions necessary
Different sets of data
Further classified by method of processor
communication
SMPs, clusters and NUMA systems
Symmetric multiprocessors
Non-uniform memory access
MIMD
Distributed Memory
SMP
Processors share memory
Communicate via that shared memory
Symmetric Multiprocessor (SMP)
Share single memory or pool
Shared bus to access memory
Memory access time to given area of memory
is approximately the same for each processor
NUMA
Nonuniform memory access
Access times to different regions of
memory may differ
Clusters
Collection of independent uniprocessors or
SMPs
Interconnected to form a cluster
Communication via fixed path or network
connections
Symmetric multiprocessors
standalone computer
similar processors of comparable capacity
Processors are connected by a bus or other internal connection
Memory access time is approximately the same for each processor
All processors share access to I/O
Either through same channels or different channels giving paths to same devices
All processors can perform the same functions
System controlled by integrated operating system
providing interaction between processors
Interaction at job, task, file and data element
levels
Time Shared Bus
Simplest form
Structure and interface similar to single processor system
features provided
Addressing
Arbitration
Time sharing
Now have multiple processors as well as multiple I/O
modules
Advantages
Simplicity
Flexibility
Reliability
Disadvantage
Performance limited by bus cycle time
Each processor should have local cache
Leads to problems with cache coherence
Cache Coherence and MESI Protocol
Write through
can also give problems unless caches monitor memory traffic
Write back policy
can lead to inconsistency
Can result in an inconsistent view of memory
Problem - multiple copies of same data in different caches
Multi-threading
Multithreading and Chip Multiprocessors
Instruction stream divided into smaller streams
Executed in parallel
No increase in complexity or power consumption
Wide variety of multithreading designs
Definitions of Threads and Processes
Threads
Thread switch
Switching processor between threads within same process
Thread: dispatchable unit of work within
process
Interruptible: processor can turn to another thread
Processes
Process
Scheduling/execution
Process switch
An operation that switches the processor from one process to another
Implicit and Explicit Multithreading
Implicit
concurrent execution of multiple threads extracted from
single sequential program
Explicit
“Concurrently execute instructions
Approaches to Explicit Multithreading
Scalar variants
Interleaved
Blocked
Superscalar variants
Simultaneous
Chip multiprocessing
Examples
Some Pentium 4
IBM Power5
