Lessons Learned from Claude Shannon

Robert G. Gallager
May 18, 1998 (Shannon Day)
Lucent Bell Labs

We all learned information theory (directly or indirectly) from Claude Shannon. We all learned a bunch of other neat results from his papers. The focus here is on what we have learned about how to do research.

Questions to ask yourself as we proceed:

Lessons learned:

Organizations that support research feel that they should choose important problems. The US government is increasingly tying research to perceived specific national goals. This leads to commercial / academic / military consortia working toward specific goals. Committees of commercial / academic / military leader plan these consortia. They are clueless about Shannon style research. They also have vested interests.

In the past, this type of research has been called "basic." It was justified by a serial model of basic research, then applied research, then test bed, then product. This makes no sense today, since product cycles are too small. This never really made sense, but manager previously accepted it, and thus supported basic research.

Claude Shannon had more than curiosity and ability to abstract. Other lessons:

Shannon's development of information theory conceptualized communication as follows:

Today, all of these have been fleshed out and form the basis of both communication theory and technology.

To summarize these lessons, Shannon's curiosity was directed by the simplest coherent way to look at things. His papers are filled with results where all of us look at them and say "I could have done that (if only I had thought to ask the right question)." What chance is there for students today to learn how to do this kind of research?

Unfortunately, education system today is diametrically opposed to this approach. There are too many projects, too many facts, too many problem sets to allow for curiosity. We are very adept at programming students to solve standard type of problems (those problems that computers can be programmed to solve better).

When students enter graduate school, they start doing research. Occasionally, in the communication field it is the Shannon style research we have been discussing. This style of research has existed in the communication field for many years. It is characterized by an easy interaction between mathematical models and real systems. There are a number of other models, particularly in fields related to communication:

These four latter models are all important and all have their place, but are not interchangeable.

None of the models above seem to meet the current needs of technology. The Shannon model is not an ideal preparation for designing large new systems. The system model is good for team work in building systems, but perhaps not good in developing insights about systems. Perhaps the problem is our systems are becoming too complex, or perhaps we are becoming too impatient. Powerful system tools make it easy to make complex systems work, but make it harder to understand them.

Conceptual complexity cannot be easily quantified. A system of a few simple components can be hard to understand. A parallel processing machine with $10^6$ processing elements can be as simple as one with four processing elements. What is complex to one person is easy to another (because of background and mode of thinking). None-the-less, reduction of complexity is the dominant problem of the information age.

The real legacy of Shannon's research, beyond all the neat results, is the existence proof that systems can be made understandable if we take the time to understand them. This takes genius, but might be possible if we let students and young researchers develop these talents.