Lessons Learned from Claude Shannon

• Is the style of research practiced by Shannon an endangered species today?
• If Claude were starting his career today, could he get a tenure at a major university or a good job at a premier research lab?
• Support for this type of research has always been fragile.
• In 1960, many higher ups told me that information theory was past its peak and I should start moving to an emerging field like vacuum tubes.
• In 1970, researchers themselves thought that IT was dead.

Lessons learned:

• Go to the source (do original research - not explanations by others).
• Follow curiosity in problem selection (avoid fads, avoid excessive perusal of journals).
• Abstract a problem into essential features.

Organizations that support research feel that they should choose important problems. The US government is increasingly tying research to perceived specific national goals. This leads to commercial / academic / military consortia working toward specific goals. Committees of commercial / academic / military leader plan these consortia. They are clueless about Shannon style research. They also have vested interests.

In the past, this type of research has been called "basic." It was justified by a serial model of basic research, then applied research, then test bed, then product. This makes no sense today, since product cycles are too small. This never really made sense, but manager previously accepted it, and thus supported basic research.

Claude Shannon had more than curiosity and ability to abstract. Other lessons:

• Be curious about real things.
• Be curious about the conceptual puzzles in things.
• After abstracting a problem, look at the simplest nontrivial version.
• Knowing all theorems or all engineering practice is not necessary (and may be harmful).

Shannon's development of information theory conceptualized communication as follows:

• mathematical models and theorems
• architectural principles
• collection of simple toy examples
• collection of successful real systems
• "back of the envelop" calculations

Today, all of these have been fleshed out and form the basis of both communication theory and technology.

To summarize these lessons, Shannon's curiosity was directed by the simplest coherent way to look at things. His papers are filled with results where all of us look at them and say "I could have done that (if only I had thought to ask the right question)." What chance is there for students today to learn how to do this kind of research?

Unfortunately, education system today is diametrically opposed to this approach. There are too many projects, too many facts, too many problem sets to allow for curiosity. We are very adept at programming students to solve standard type of problems (those problems that computers can be programmed to solve better).

When students enter graduate school, they start doing research. Occasionally, in the communication field it is the Shannon style research we have been discussing. This style of research has existed in the communication field for many years. It is characterized by an easy interaction between mathematical models and real systems. There are a number of other models, particularly in fields related to communication:

• Crystal ball model: preliminary work on future technology. This impresses businessmen and managers, but does not have a strong record, except coupled with conceptual research. The Shannon model of research is more bottom up, building conceptual structure necessary for applications over a broad range. The crystal ball model is "top down", and is only effective for shorter term research after the conceptual structures are developed.
• System model: given a vision, build a large system; see if it floats. This model has been very popular at ARPA, starting with ARPANET. This model at universities leads to large research stuffs, big empires. This is the dominant model for computer system & network research. This model is successful if enough of the conceptual structure is already in place. It often generates very useful side products.
• Complexity model: build a homogeneous model and see if a vision emerges. Many big thinks think that this is the way to understand complexity.
• Interdisciplinary model: build a homogeneous system by committee; see if the committee floats.
• Big physics model: build homogeneously expensive apparatus and do critical experiments.
• Edison model: invent something useful.
• Scholarly model: learn more and more about less and less.
• Performance analysis model: this is focused more on analysis and somewhat less on aiding designs.

These four latter models are all important and all have their place, but are not interchangeable.

None of the models above seem to meet the current needs of technology. The Shannon model is not an ideal preparation for designing large new systems. The system model is good for team work in building systems, but perhaps not good in developing insights about systems. Perhaps the problem is our systems are becoming too complex, or perhaps we are becoming too impatient. Powerful system tools make it easy to make complex systems work, but make it harder to understand them.

Conceptual complexity cannot be easily quantified. A system of a few simple components can be hard to understand. A parallel processing machine with $10^6$ processing elements can be as simple as one with four processing elements. What is complex to one person is easy to another (because of background and mode of thinking). None-the-less, reduction of complexity is the dominant problem of the information age.

The real legacy of Shannon's research, beyond all the neat results, is the existence proof that systems can be made understandable if we take the time to understand them. This takes genius, but might be possible if we let students and young researchers develop these talents.