Parallel programming models

There are two ways to think about parallel programming:

• you may think about how processes interact; or
• you may think about how problems decompose.

The first has to do with how parallel threads of a computation are allowed to communicate and/or share data.

The second has to do with how a programming task is divvied up to different processes, irrespective of how these communicate.

Process interaction

Suppose we have a computational situation where many processes and/or threads are running in parallel. There appear to be broadly four ways in which these threads can collaborate:

1. Through passing messages.
2. Through sharing memory in a global address space.
3. Through sharing memory in a partitioned global address space.
4. By not interacting at all. Instead, each process operates on a distinct set of data, which are then combined at the end (aka data parallelism).

Each language may support one or more of the above models.

Problem decomposition

Please expand

References

• There is a Wikipedia page.