How many processing elements cooperate on parts of one problem.
What is parallel computing?
Parallel computing divides a problem into tasks or data pieces that can be processed at the same time.
Parallelism can use multiple CPU cores, GPUs, clusters, vector units, FPGAs or specialized accelerators.
Why is it powerful?
Some computations contain thousands or millions of independent operations. Parallel execution can reduce elapsed time, enable larger simulations and process streams such as images or scientific measurements.
What makes it difficult?
Tasks may depend on one another. Communication, synchronization, load imbalance, memory bandwidth and failures can erase the expected speedup.
Amdahl's law reminds us that a serial fraction limits total acceleration.
Models and future
Shared-memory threads, distributed message passing, vectorization and data-parallel kernels are different ways to express parallel work.
Future systems will combine heterogeneous processors and high-speed interconnects. Good algorithms must consider energy, data movement, reproducibility and the cost of coordinating workers.
Ai disclosure: written with the help of AI (ChatGPT). You are encouraged to point out errors and omissions.
