JimDempseyAtTheCove:

Before you duplicate the work of others you should go to www.gpgpu.org and look at the programming models such as Brook+ and CUDA. This site addresses the issues of using graphics cards for processing. The memory model and process interaction with GPU is very similar to what you have stated.

In this context I also have to mention RapidMind platform:
http://www.rapidmind.net/
The main distinguishing feature of RapidMind is that it can work transparently on multicore CPU or on GPU. User don't have to worry about this.

Also there is OpenCL:
http://en.wikipedia.org/wiki/OpenCL
Which also works on CPU or GPU.

And Stream Computing:
http://ati.amd.com/technology/streamcomputing/