I definitely agree with Dan on this one. Here at Pervasive we are adding our egg to the dozen, DataRush. A flow based or actor oriented java solution which works really well with problems that focus on large amounts of data manipulation.
JamesF
PervasiveDataRush

Here are a few of those lessons:
1. parallel programming is inherently hard, and tools and techniques claiming to avoid the problems never work as advertised
2. heterogeneous and asymmetric architectures are much harder to program effectively than homogeneous symmetric architectures
3. Programmer-managed memory is much harder to use than system-managed (whether OS or hardware)
4. Specialist instruction sets are much harder to use effectively than general-purpose ones

I nearly fell out of my chair laughing when the Cell was launched. It contained all 4 of these errors in one design. Despite the commercial advantages in bringing out a game that fully exploited its features, as I recall, only 1 game available when the Playstation 3 was launched came close.

I hereby announce Machanick’s corollory to Moore’s Law: any rate of improvement in the number of transistors you can buy for your money will be matched by erroneous expectations that programmers will become smarter.

Unfortunately there is no Moore’s Law for IQ.

The only real practical advantage of multicore over discrete chip multiprocessors (aside from the packaging and cost advantages) is a significant reduction of interprocess communication costs — provided IPC is core-to-core, i.e., if you communicate through shared memory, you’d better make sure that the data is cached before the communication occurs. That makes the programming problem harder, not easier (see Lesson 3 above).

Good luck with transactional memories and all the other cool new ideas. Ask yourself one question: do they make parallel programming easier, or do they add one more wrinkle for programmers to take care of — that may be different in the next generation or on a rival design?

Putting huge numbers of cores on-chip is a losing game. The more you add, the smaller the fraction of the problem space you are addressing, and the harder you make programming. I would much rather up the size of on-chip caches to the extent that they effectively become the main memory, with off-chip accesses becoming page faults. Whether you go multicore or aggressive uniprocessor, off-chip memory is a major bottleneck.

As Seymour Cray taught us, the thing to aim for is not peak throughput, but average throughput. 100 cores each running at 1% of its full speed because or programming inefficiencies, inherent nonparallelism of the workload and bottlenecks in the memory system is hardly an advance on 2 to 4 cores each running at at least 50% of its full speed.

In any case all of this misses the real excitement in the computing world: turn Moore’s Law on its head, and contemplate when something that cost $1,000,000 will cost $1. That’s the point where you can do something really exciting on a small, almost free device.

Now we’re at a point where sequential processors can’t easily be made faster – really! – so parallelism actually, honestly matters. Many, many commercially interesting problems can be solved by data parallelism, and we have a growing set of data parallel tools available commercially. I’m still toiling away in the fields of commercial parallel processing, and I don’t see the need for any massive government investment here.

I’m going to say that I cringe every time
I hear an academic arguing for a new
Manhattan project for his particular area.
Suppose DARPA or some other DOD agency did do a Manhattan Project. First lets make one thing clear. It was a TOP SECRET project. No clearances no work. Working on the Manhattan project was an interruption of one’s academic career to support the war. I believe it was Wheeler who received a letter by his brother serving in the European theater of operations saying “Hurry UP”. His brother would be killed a few days latter. Had the War Dept not viewed the effort as supporting combat operations they wouldn’t have gotten the resources to do it. Today, a huge effort directing a large part of the scientific community to say detecting IEDs would be a better analogy and it too would be top secret. So folks, lets put that one to rest for good and pick a better analogy.