The unsolvable privacy problem and its implications for security technologies Andrew Odlyzko Digital Technology Center University of Minnesota odlyzko@umn.edu http://www.dtc.umn.edu/~odlyzko Version of April 30, 2003. Abstract: Privacy presents many puzzles. In particular, why is it eroding, given the high value people assign to their privacy? This extended abstract argues that there are strong incentives for decreasing privacy, rooted in the economic benefits of price discrimination. As a result, the privacy problem is unsolvable. The conflict between incentives to price discriminate and the public dislike of this practice will influence what security technologies are likely to succeed. 1. Introduction This is an extended abstract of the invited lecture by this title at the 2003 Australasian Conference on Information Security and Privacy. What causes the steady erosion of privacy we experience? Some of the push comes from government agencies, and their incentives are easy to understand. Moreover, in democratic societies it is understood how to control government privacy intrusions. However, most of the developments that reduce privacy are coming from corporations, which care about money, not politics. And on the private enterprise side, there are many puzzles. First of all, although huge amounts of data are being collected, overall not much use is being made of it right now. Second, given most people's concern about their privacy, why is there this insistence on very intrusive data collection? The consumer side of the privacy issue also presents puzzles. People are extremely concerned about their privacy, often citing this as a reason they are reluctant to engage in ecommerce. On the other hand, these same people appear unwilling to do much to protect their privacy, and rather freely share detailed information about their activities. Are they just paranoid, or is there something yet unseen that they have reason to fear? The thesis that is reported on very briefly here (with full details being written up in a series of papers) resolves these puzzles. The erosion of privacy is caused by the incentives to price discriminate. Privacy intrusions enable more effective price discrimination. The public's fears about decreased privacy are likely to prove justified from their personal perspective, since evidence shows that people strongly dislike price discrimination. This thesis is supported be extensive data and analysis, with some of the most illustrative examples coming from comparisons of 19th century railroad pricing practices with those in the modern economy. In particular, the railroad example then helps explain why the spread of price discrimination is taking place in a stealthy manner. 2. Benefits of price discrimination Price discrimination is the charging of varying prices when there are no cost justifications for the differences. The benefits of this practice have been understood in economics ever since the work of the French "econoengineers" on railroad pricing in the middle of the 19th century. It is part of the universally accepted economic theory that price discrimination increases the efficiency of the economy by enabling transaction that would not take place otherwise. 3. Increasing incentives for price discrimination The increasing upfront costs of providing goods and services, and declining marginal costs, combine to increase incentives to price discriminate, as is well established in the economics literature. Railroads were the major industry of the 19th century, and they had huge fixed and relatively low marginal costs, which again produced huge incentives to price discriminate. (These incentives were also present, sometimes to an even greater extent, in canals and some other industries. While the development of price discrimination did take place in those fields, and is part of the fascinating historical record, those industries were nowhere near as large and noticeable a part of the economy as railroads.) 4. Increasing ability to price discriminate One of the most important results of the development of improved information and communication technologies, especially the Internet, is the great improvement in the ability to price discriminate. It is possible to obtain much better information about customers' willingness to pay, and it is possible to control their usage much better than before. The comparison of 19th century railroads with airlines at the end of the 20th century is especially instructive. Railroad passenger tickets were transferable, which limited the extent of price discrimination that was feasible. On the other hand, airline tickets are contracts for service between a particular passenger named on the ticket and the airline. Therefore, with the help of "positive passenger identification" systems, airlines can engage in their "yield management" schemes. The benefits of these schemes are derived primarily from price discrimination, as in charging business travelers unwilling to stay over a Saturday night much higher fares than discretionary vacation travelers. 5. Competition and price discrimination The general thinking, often deeply embedded in legal doctrines, has long been that price discrimination comes from monopoly power. However, historically there have been many instances of very competitive markets where price discrimination was widely practiced, and such examples are growing increasingly numerous. (The development of airline "yield management" after deregulation is just one such development.) One of the most notable features of the evolving "New Economy" (as opposed to the discredited "New Economy" that had been widely forecasted a few years ago) will likely be the growth of price discrimination in competitive markets. This will require revisions in legal and economic doctrines. Among other upsets to "conventional wisdom," first degree price discrimination will come to be seen as feasible. 6. Public hate of price discrimination There is extensive evidence of the spread of price discrimination. This spread would undoubtedly be even more rapid if it were not for the public's hate and fear of rice discrimination. Again, the comparison with 19th century railroads is instructive. It was not profiteering by railroads that brought about intrusive government regulation, but price discrimination. (In the United States, the Interstate Commerce Act of 1887, the first serious federal government intrusion into commercial conduct, was aimed almost exclusively at railroad price discrimination.) Similar negative reaction to overt price discrimination have been seen in more recent times. The public dislike of price discrimination provides incentive to hide it. As it turns out, practices such as bundling and site licensing provide a way to attain the same end as explicit price discrimination, by reducing consumer surplus while appearing to offer great bargains to buyers. 7. Conclusions There is no easy resolution to the conflict between sellers' incentives to price discriminate and buyers' resistance to such measures. The continuing tension between these two factors will have important consequences for the nature of the economy. It will also determine which technologies will be adopted widely. Governments will likely play a role in controlling pricing, although their roles will probably be ambiguous. Sellers are likely to rely on bundling, since that allows them to extract more consumer surplus and also to conceal the extent of price discrimination. Micropayments and auctions are likely to play a smaller role than is often expected. In general, because of the strong conflicting influences, privacy is likely to prove an intractable problem that will be prominent on the the public agenda for the foreseeable future. More details will be available in a series of papers that are in preparation and which will be available on my home page, at . Some of the main theses of this lecture were already outlined in the earlier paper [Odlyzko]. There are also several sets of presentation slides (especially "The New Economy and old economics: What 19th century railroads can tell us about the future of ecommerce") available at that cover some of the material in this extended abstract. References [Odlyzko] A. M. Odlyzko, The bumpy road of electronic commerce, pp. 378-389 in "WebNet 96 - World Conf. Web Soc. Proc.," H. Maurer, ed., AACE, 1996. Available at .