Nov
17
In previous posts on this blog, Berkeley’s David Patterson, Intel’s Andrew Chien, and Microsoft’s Dan Reed presented their views on why research advances are needed to overcome the problems posed by multicore processors. In this piece — the fourth (and possibly final) entry in the series -– Marc Snir from UIUC argues that there are major challenges facing us but yet, the sky is not falling.
–
The CCC blog has published a couple of articles on the multi-core challenge, all emphasizing the difficulty of making parallel programming prevalent and, hence, the difficulty of leveraging multi-core systems in mass markets. The challenge is, indeed, significant and requires important investments in research and development; but, at UPCRC Illinois, we do not believe that the sky is falling.
Parallel programming, as currently practiced, is hard: Programs, especially shared memory programs, are prone to subtle, hard-to-find synchronization bugs and parallel performance is elusive. One can reach two possible conclusions from this situation: It is possible that parallel programming is inherently hard, in which case, indeed the sky is falling. An alternative view is that, intrinsically, parallel programming is not significantly harder than sequential programming; rather, it is hampered by the lack of adequate languages, tools and architectures. In this alternative view, different practices, supported by the right infrastructure, can make parallel programming prevalent.
This alternative, optimistic view is based on many years of experience with parallel programming. While some concurrent code, e.g., OS code, is often hard to write and debug, there are many forms of parallelism that are relatively easy to master: Many parallel scientific codes are written by scientists with limited CS education; the time spent handling parallelism is a small fraction of the time spent developing a large parallel scientific code. Parallelism can be hidden behind an SQL interface and exploited by programmers with little difficulty. Many programmers develop GUI’s that are, in effect, parallel programs, using specialized frameworks. Parallelism can be exposed using scripted object systems such as Squeak Etoys in ways that enable young children to write parallel programs. These examples seem to indicate that it is not parallelism per se that is hard to handle; rather it is the unstructured, unconstrained interaction between concurrent threads that result in code that is hard to understand both from a correctness and performance view, hence hard to debug and tune.
The state-of-the-art in parallel programming is what sequential computing was several decades ago. A major reason for this situation is that parallel programming has been an art exercised by a group of experts whose small population did not justify major investments in programming environments aimed at making their life easier. This reason disappears as parallelism becomes available on all platforms. Furthermore, we can make faster progress now because we understand well the principles it takes to make programming easier — principles such as safety, encapsulation, modularity, or separation of concerns; we also have more experience in developing sophisticated IDE’s.
What will it take to bring these principles of computer science to parallel programming? It will require a broad based attack across the system stack. As has been said in these blogs, we need research in languages, compilers, runtime, libraries, tools, hardware … What has not been said explicitly is that none of these areas are likely to produce the silver bullet on their own. The solution that will work eventually will be one that brings together technologies from all these areas to bear on each other. However, we do not have the luxury of doing this via incremental and reactive changes over decades. The research truly needs to be interdisciplinary and the idea of co-design needs to be internalized. Unfortunately, the mainstream systems community has all but abandoned this mode of research in the last several years. Language researchers are locked into mechanisms that will only be supported by commodity hardware and hardware researchers are locked into a mode that requires supporting the lowest common denominator software. It is imperative that we break out of these shells and get the research community into a mindset that we are truly looking to define a new age of computing — a mindset that nurtures research where a clean system slate is an acceptable starting point.
The sky is not falling, but the ground is shifting rapidly. The multi-core challenge requires a concerted effort of academia and industry to generate new capabilities. We are confident that in the future, as in the past, new capabilities will breed new applications. Multi-core parallelism can be leveraged to develop human-centered consumer products that provide more intelligent and more intuitive interfaces through better graphics and vision, better speech and text processing and better modeling of the user and the environment.
The task of providing better performance is shifting from the hardware to the software. This is an exciting time for Computer Science.
Marc Snir
4323 Siebel Center, 201 N Goodwin, IL 61801
Tel (217) 244 6568
Web http://www.cs.uiuc.edu/homes/snir
Sep
22
The Multicore Challenge, Part 2
Filed Under research horizons | 3 Comments
The problem of parallel computing is occupying the minds of a growing number of researchers. Why is this age-old concept so “hot” today? In this article — the second in a series of opinion pieces –Andrew Chien, Vice President and Corporate Technology Group Director for Intel Research, gives us his perspective on the issue, with a particular focus on the challenges facing us in education and funding.
–
Multicore (parallelism) represents a fundamental challenge and change for all of computing and computer science. It represents the fundamental constraints of physics — nature loves parallelism — surfacing and interacting with some fundamental tenets of computing. We have formulated our theory of computation and complexity primarily on sequence — in control and state. Fundamental physics (and consequently circuits and architecture) which makes parallelism fundamentally cheaper is now challenging us to broaden the foundation of computing with parallelism as a first class element. I believe that as a research community, this is a first-order challenge to respond — in nearly all disciplines of computer science. First and foremost, this is a major intellectual challenge to the computer science community to “reinvent” or at least broaden computer science in this way. Second, this is a major educational challenge, where students we are training today to think about “Computer Science founded on sequence” are being launched into a world of parallelism. For their benefit, we must mount a rapid response in pedagogy and curriculum to ensure these students emerge armed to deal with the future of computing in their careers.
Now, let me turn to research funding in parallelism — which is a critical need in all areas from architecture, runtimes, compilers, programming languages, algorithms, and theory. We need major increases in funding and research activity in all of these areas. Governments must take the primary role in funding research in information technology for the long term economic development and societal well-being. We would like to see aggressive large-scale funding of long-range research in parallelism, and that the fruits of that research be made broadly available for commercialization. In the U.S., DARPA has a long track record of funding such research in IT, but such investment has decreased in recent years. We would like to see it increase both in DARPA, as well as other parts of the US government, and yes around the world. History has proven that only governments are able to invest in this type long-term general economic development, and it is critical that the research outputs be generally available for society at large to benefit — not just a small population of gatekeepers. It is great to see this vision being pursued in many regions around the world.
At Intel, we depend heavily on a broad range of science and engineering research pursued by the global academic community. Many of the innovations we commercialize were first conceived in universities — often many years before their practicality — and we have contributed additional innovations and refinements to bring them to the broadest swath of society possible. We strongly support (and contribute our time, money, and leadership to) the health of the research and innovation community globally. We have made significant investments in education (multicore curriculum and training) and research (research grants, Universal Parallel Computing Research Center with Microsoft) for parallelism, and continue to encourage others to join us in doing so.
– Andrew Chien
–
To see the first article in this series, click here.
Aug
26
The Multicore Challenge
Filed Under research horizons | 13 Comments
Researchers working in areas spanning computer architecture, programming languages, operating systems, algorithms, and more have been thinking harder about the problem of parallel computing. Why has the age-old concept of parallelism become so “hot” today? To provide the first of an upcoming series of opinion pieces, we asked David Patterson, Professor in Computer Science at UC Berkeley, to give us his thoughts, and the rationale for increased government funding to solve the multicore challenge.
–
Since the first commercial computer in 1950, the information technology industry has improved cost-performance of computing by about 100 billion overall. For most of the last 20 years, architects used the rapidly increasing transistor speed and budget made possible by silicon technology advances to double performance every 18 months. The implicit hardware/software contract was that increases in transistor count and power dissipation were OK as long as architects maintained the existing programming model. This contract led to innovations that were inefficient in transistors and power but which increased performance. This contract worked fine until we hit the power limit that a chip could dissipate.
Computer architects were forced to find a new paradigm to sustain ever-increasing performance. The industry decided that only viable option was to replace the single power-inefficient processor by several more efficient processors on the same chip. The whole microprocessor industry thus declared that its future was in parallel computing, with a doubling of the number of processors or cores each technology generation, which occur every two years. This style of chip was labeled a multicore microprocessor. Hence, the leap to multicore is not based on a breakthrough in programming or architecture; it’s actually a retreat from the even harder task of building power-efficient, high-clock-rate, single-core chips.
Many startups tried commercializing multiple core hardware over the years. They all failed, as programmers accustomed to continuous improvements in sequential performance saw little need to explore parallelism. Convex, Encore, Floating Point Systems, INMOS, Kendall Square Research, MasPar, nCUBE, Sequent, and Thinking Machines are just the best-known members of the Dead Parallel Computer Society, whose ranks are legion. Given this sad history, there is plenty of reason for pessimism about the future of multicore. Quoting computing pioneer and Stanford President John Hennessy:
“…when we start talking about parallelism and ease of use of truly parallel computers, we’re talking about a problem that’s as hard as any that computer science has faced. … I would be panicked if I were in industry.”
Jeopardy for the IT industry means opportunity for the research community. If researchers meet the parallel challenge, the future of IT is rosy. If they don’t, it’s not. Failure could jeopardize both the IT field and the portions of the economy that depend upon rapidly improving information technology. It is also an opportunity for the leadership in IT to move from the US to wherever in the world someone invents the solution to make it easy to write efficient parallel software.
Given this current crisis, its ironic that since 2001 DARPA chose to decrease funding of academic research in computer systems research. Knowing what we know today, if we could go back in time we would have launched a Manhattan Project to bring together the best minds in applications, software architecture, programming languages and compilers, libraries, testing and correctness, operating systems, hardware architecture, and chip design to tackle this parallel challenge.
Since we don’t have time travel, there is an even greater sense of urgency to get such an effort underway. Indeed, industry has recently stepped in to fund three universities to get underway–Berkeley, Illinois, and Stanford–but its unrealistic to expect industry to fund many more. Its also clear given the urgency and importance to the industry and the nation, we can’t depend on just three academic projects to preserve the future of the US IT industry. We need the US Government to return to its historic role to bring the many more minds on these important problem. To make real progress, we would need a long-term, multi-hundred million dollar per year program.
The consequences of not funding aren’t a drop in Nobel prizes or research breakthroughs; its a decline in the US-led IT industry, a slowdown in portions of the US economy, and possibly ceding the leadership in IT to another part of the world were governments understand the potential economic impact of funding academic IT research on parallelism.
– David Patterson