Parallel Programming is when you use more than one processor (or even computer) working simultaneously on the same problem.

One might think that if you have ten computers working on the same problem, that the problem will run in 10% of the time.  This never happens.  You never get perfect speedup (Amdahl's Law).  In fact, you have to know what you're doing to prevent the various processors from getting in each other's way.

Because we're really not using a cluster in this class, we're using the pthread library.  Our parallel stuff is really only "simulated" so you're really going to see the runtime improvement in this class.

There are several well-known examples of parallel computer.  Online gaming.  SETI.

There are two major forms of parallel programming.  Shared memory and distributed memory.  Shared memory is when all of your processors have access to the same memory.  If one processor sets a variable, then all the processors have access to that variable.  This is the most studied form of parallel processing.  However, distributed processing is what caught the world by storm.  Distributed is when the processors have their own memory, and they communicate by sending explicit messages to each other.

When you're programming a regular non-parallel computer using a non-parallel language.  What are problems programmers face?

1.  Compiler errors.
2.  Infinite loops.
3.  Crashes.
4.  Program appears to work, but answers are wrong.

Parallel programmers suffers the same problems but also has

5.  Deadlock
6.  Livelock
7.  Program appears to work, but not actually running in parallel.

Time sharing:  If you have multiple programs sharing a processor, each program can only run a small amount of time on the processor before another program gets to use it.  ("Time slicing.")  This is not truly parallel.  It simulates parallel, but you don't get the speedup of parallel.

if (x=0) { ... }     if (x==0) {...}

We won't be using anything fancy for our programs.  We will just be using g++.  GNU C++ Compiler (for Linux).


What is forking?  Why is it good?  Why is it bad?

forking creates a duplicate process.  Where there was one, there is now two.  Each copy has the same memory and variables.  And each copy is at the same point in the program as each other.  There are subtle differences.  The original and the copy have different PIDs.  (Process ID #'s.)  And, very importantly, in the original fork() returns true.  In the copy, fork() returns false.  (In the original, fork() returns the PID of the copy.  In the copy, fork() return 0.)

The copy is considered a child process of the original.

The processes on the system are organized in a hierarchical system, or a tree.