The Parallel For Loop
A parallel for runs the iterations of a loop across multiple threads at once. It is the simplest doorway into parallelism, but it is only safe when the iterations are truly independent.
When it is safe
Iterations may run in any order or at the same time only if they do not interfere:
- No iteration may write a location that another iteration reads or writes.
- A loop carried dependency, where one iteration needs a value produced by an earlier one, breaks the rule.
- Shared accumulators must use a reduction rather than a plain shared variable.
Scheduling chunks
The runtime divides the index range into chunks and hands them to threads. Static scheduling assigns equal chunks up front, which is cheap but uneven if iterations vary in cost. Dynamic scheduling hands out small chunks on demand, balancing load at the price of more coordination.
Key idea
A parallel for is safe only when iterations are independent, and the scheduler splits the range into chunks that may be static or dynamic.