Subject: FYI: another note on research Enclosed is an article that was written for the journal Research*Technology Management for their Jan/Feb '96 issue. They asked for an opinion piece rather than one of just analysis, so the presentation is different from that in "The decline of unfettered research." In particular, there is more discussion (in the last section) of what unfettered research can do. (It is also much shorter!) Andrew Odlyzko Footnote: This article is based on a more extensive essay, "The decline of unfettered research," which is to be published separately. We still need unfettered research Andrew Odlyzko AT&T Bell Laboratories 600 Mountain Avenue Murray Hill, NJ 07974 amo@research.att.com October 18, 1995 1. Introduction Technology is at the center of the profound changes transforming our society, and is widely regarded as crucial to national and corporate success. The importance that even Wall Street attaches to good technology is shown vividly by the initial public offering of Netscape Communications in August 1995. A company formed less than two years earlier, with total sales of under $20 million, an initial investment of a few million, and no profits, was suddenly judged to be worth $2 billion. The reason was that this company's main product, the Netscape World Wide Web browser, was marginally superior to competing products, and the market dominance it had achieved was thought to offer the potential to dominate Internet commerce. At the same time that Wall Street embraces the superior technology of companies like Netscape, and economists attribute at least half of economic growth to technological advances, there is great pressure on researchers to concentrate on short-term projects. "Blue-sky" projects are out, and "mission-oriented" work with "focus on customers' needs" is in. What is perhaps most remarkable is that unfettered research is almost totally gone from industrial research labs. This type of research, in which little justification was required of individual researchers in the selection of their work, was the centerpiece of science supported by the U.S. government after World War II. The justification for it was that "scientific progress on a broad front results from the free play of free intellects, working on subjects of their own choice, in the manner dictated by their curiosity" [Bush]. This type of research spread also to industrial labs in the 1950s and 1960s, but then was gradually reduced. It was never a large part of the total research effort (which itself was only a fraction of the total R&D work), but its decline is an indicator of changing attitudes and expectations towards research. Directed research does not have to be short-term, or lack basic scientific value, as the discovery of the transistor at Bell Labs demonstrates. Also, customer focus can involve long-term work, as the development of Lotus Notes by Ray Ozzie and his crew shows. However, the presence of unfettered research does imply an optimistic, long-term outlook and its absence is often taken as a sign of a lack of vision for the future. The story seems to be more complicated than that. I feel that the decline in unfettered research, and the more general tendency towards more directed and short-term work, is the outcome of a natural evolution of research. In particular, the accumulation of technical knowledge has made it easy to build new products and services, and so the most promising areas for research have moved away from the traditional ones that focus on basic components, and towards assembling these components into complete system. 2. The changing environment for research (a) Growth and increasing competitiveness of research The last half-century has seen an unprecedented increase in research. While science and engineering have been growing for centuries, after War World II the rate of growth jumped. In most fields, there has been about a ten-fold increase in the number of publications since 1950. This tremendous growth has led to changes in the nature of research. Increasingly, even in universities, large teams are required to achieve significant advances. Further, the competition is much fiercer, and it is hard for any team to stay in the lead for long. As an example, within weeks of the announcement of high-temperature superconductivity by the IBM Zurich research lab, several other groups had made major advances in the area. Thus even if high-temperature superconductivity had developed into a successful industry, IBM would not have been able to monopolize the benefits from its invention. (b) Fewer big "hits" While the research establishment grew, the number of big "hits" whose impact the public can appreciate, such as the Salk and Sabin vaccines that conquered polio, did not. Further, with increasing competition, the leaders do not stand out as much, and the economic benefits of technical superiority are often smaller. When Seymour Cray designed the Cray-1 two decades ago, it was about 10 times faster than any other computer on the market, and therefore it sold well even though hardly any software was available for it. In the 1990s, Cray designed the Cray-4, but could not find customers for it, since even with a variety of technical advances, he could offer at most a marginal improvement over competing machines from other companies. (c) Better technology does not always win Increasing competition in a well-defined area (whether it is in sports, or the market for PCs) almost always reduces the differentiation among leaders, and this means that to an increasing extent, better technology does not win. Microsoft Windows operating systems are still not up to the level of the Macintosh ones from 5 years ago, and the Intel x86 chips have for a long time been inferior to competing RISC microprocessors in performance. However, the differences were not large enough to convince users to give up on the compatibility advantages of staying with the market leaders. (d) Standards and interoperability As technologies mature, the systems that we build grow more complicated, and so interoperability becomes increasingly important. A new data-compression scheme has to be adopted as an industry standard to have a large impact, and this limits the profit potential for an inventor, who has to make the scheme widely available to have it accepted. Further, in more and more cases, there are several different technical solutions available. A decade ago, Narendra Karmarkar invented a method for solving resource allocation problems that was dramatically better than previous ones. This discovery spurred research on traditional simplex methods, so that today these techniques are competitive with the Karmarkar approach. (e) Incrementalism wins Incremental work has become much more important than the occasional breakthroughs. Although extensive work has gone into exotic technologies such as gallium arsenide, silicon still reigns supreme in integrated circuits. Extensive and ingenious work has to be carried out to advance the state of the art in silicon technologies, but it is incremental. (f) Predictability of science A major justification for "blue-sky" research is that results of scientific research cannot be predicted. However, science has to some extent become predictable. This is shown most vividly by "Moore's Law," which says that the number of transistors that can be placed on a chip doubles every 18 months. This "law" has applied over the last 20 years, and is widely expected to apply for at least another 10. Increasing research efforts are required to maintain the rate of progress consistent with "Moore's Law," but it is understood just how much effort is required, all the leading semiconductor companies have been able to keep up, and not one has been able to get ahead of the others for long. The concept of a predictable rate of advance in technology is becoming common. Although "Moore's Law" applies only to integrated chips, similar growth rates are observed in other areas, and are often relied on in business planning. Several leaders of advanced development projects have told me that they did not see any technical barriers to completing their projects. They were not able to complete them right then and there, but they were convinced that either microprocessors, or transmission facilities, or algorithms would provide them with the two- or five-fold improvement in performance they needed, and that they would be able to obtain such improvements on the open market, without any effort on their part. This creates a barrier for a researcher, since a 25% improvement in some process might only equal a 6-month delay in waiting for ""Moore's Law" to provide an equivalent advance. The idea of constant technological advance provides a way to explain business behavior such as the IBM purchase of Lotus for $3.5 billion. The main thing that IBM bought is Lotus Notes, an innovative and important product that has a dominant presence in its market segment, and has features that have not yet been duplicated by competitors. In today's competitive environment, it will not take long for other companies to develop equivalent or better products. The problem is that this does take some time. What IBM has bought for $3.5 billion is a place a bit ahead of everybody else on the moving escalator. (g) Unprecedented opportunities with existing knowledge The final factor affecting research is that of a threshold effect. The Internet has been growing at roughly a constant pace for about two decades, doubling every year, but in the last year or so it has become so large, that its promise (and threat) have finally become widely acknowledged. More generally, there has been a huge accumulation of science and basic technology as a result of the research over the preceding decades, and we are now going through the digital revolution, which uses all those results. The two months that Andreessen and Bina spent writing the basic code for the Mosaic World Wide Web browser is an extreme example. Mosaic did more than any other recent development to spur the growth of the Internet. Its development and impact were made possible by decades of work on data networking. However, those basic building blocks were in place, and available to everybody, and the actual work of developing Mosaic went fast. Given how easy it is to put together revolutionary new products and services out of available components, there is increased uncertainty, which decreases willingness to invest for the long term. The payoffs from building systems out of what is already available are so high, that work on fundamentally novel approaches is not seen as necessary. Researchers are being asked to "just do it," to build the systems that are feasible with existing knowledge, and not to worry about the distant future. 3. The future of research (a) Do we need research? Do we need long-range research? There is a tremendous amount of research that has not yet been fully exploited, there are experts willing to jump on any problem that is encountered, and anything not discovered today will simply be discovered later. Thus one could argue that the world could continue to advance technologically at the current pace without any effort in basic research. However, accepting such an argument might be dangerous. There are too many technologies that required two decades or more from basic idea to widespread use. Such was the case with combinatorial chemistry and optical amplifiers. Another good example is public key cryptography. It was invented in the mid-1970s, was widely recognized then as an important idea, but is only now beginning to play a significant role. Even though it was not used widely during those two decades, the knowledge of its existence was of great help, since it showed that electronic commerce and other building blocks of the information society could be constructed. Giving up on the advantages of having such knowledge in advance appears foolish. The main question is where such research should be performed and how much effort should be devoted to it. (b) Public funding of research The current trend is towards corporations that are narrowly focused on their market segments, and away from vertical integration. (This trend is made possible by the wide availability of research and the other themes described above.) Therefore it is hard to argue for a return to the days of extensive unfettered research in industry. If anybody manages to demonstrate the commercial advantages of unfettered research, it probably will be the Japanese, with their keiretsu industrial groupings and frequent collaborative ventures. So far, though, there is little sign of their moving away from the model they perfected and everybody else is imitating, namely of doing only directed research. This leaves taxpayers as the only likely source of support for unfettered research. There is indeed wide recognition of the need for such support, even by the industrial leaders who are cutting back their own companies' R&D efforts. The argument for taxpayer funding is strong, since the benefits of research are dispersed throughout the economy, and the available evidence points to a high rate of return on research spending on a national scale. Netscape, for example, achieved great prominence and high stock market valuation, but the basic technology of the Internet and of the World Wide Web that made its products useful had been generated with sustained public funding. However, there are questions even about public support [Armstrong], and the argument for it needs to be continually re-examined. Further, at all levels, choices have to be made [NAS] that the scientific community has not been comfortable making in the past, such as about increasing support for some areas over others. Universities will also have to respond to the pressures they face. Some of these pressures arise from the factors outlines in the preceding section, which make short-term research more attractive. Other pressures come from academic culture that does not accommodate change gracefully. (c) Unfettered research in industry The factors listed earlier argue that the decline in unfettered research in industry was a rational response to a changing environment. However, they do not argue for complete elimination of such research. While those factors do show that the probability of revolutionizing an industry ("creating a new Xerox") by investing in "blue-sky" research has decreased substantially, there are other benefits of unfettered research, some of which have increased. As research areas have expanded, and become increasingly esoteric, it is increasingly useful to have a staff that is knowledgeable about outside scientific and engineering developments. Usually the best way to acquire such expertise is to have researchers who do leading work in the given area or a related one. (Outside consultants and joint ventures with universities can provide some of the same capabilities, but have their own deficiencies.) Such experts can be helpful in recruiting, in internal consulting, and especially in identifying new opportunities. While researchers are seldom great marketers, they usually have a much better vision of what the technological possibilities are in their area than business planners. Few researchers involved in data networking were surprised by the emergence of the Internet as a major new factor in the economy, and many of them had devoted extensive (but largely fruitless) efforts to alerting their management to what was happening, The problem with "customer focus," the current trendy buzzword, is that such a focus can blind an organization to basic changes. (A beautiful set of examples where this has happened repeatedly is presented by the hard disk industry, cf. [BowerC].) A relatively unfettered research organization just might provide the understanding of science and technology that is needed for the successful evolution of a company over the next decade. References: [Armstrong] J. A. Armstrong, Is basic research a luxury our society can no longer afford?, The Bridge (quarterly published by Nat. Acad. Eng.), vol. 24, no. 2, 1994. [BowerC] J. L. Bower and C. M. Christensen, Disruptive technologies: Catching the wave, Harvard Bus. Rev. Jan.-Feb. 1995, pp. 43-53. [Bush] V. Bush, "Science: The Endless Frontier," Government Printing Office, Washington, D.C., 1945. [NAS] Science, technology, and the federal government: National goals for a new era, National Academy Press, 1993.