Shannon's Theory Explained
GreenTouch ™ looks to Claude Shannon’s theories to provide guidance for a new generation of high capacity, super low-energy networks
Claude Shannon's extraordinary idea – that basic principles of binary or digital information can be related to fundamental physical laws – was instrumental in shaping our digital era. Today, Shannon’s theory remains the guiding foundation for communication scientists and engineers in their ongoing quest for faster, more energy efficient, and more robust communication systems.
Our ability to transform information – phone calls, music, video, virtually everything – into digital bits of data and to transmit billions of them per second is founded upon the innovative work of Bell Labs and MITmathematician, Claude Shannon (1916-2001). Shannon’s seminal 1948 paper, “A Mathematical Theory of Communication,” established a whole new discipline, Information Theory, by showing how Boolean algebra and basic thermodynamic principles could be applied to communications.
The result was a revolutionary new way to conceive of information and communications systems, based on theoretical and experimental aspects of information transmission and processing. The applications which make contemporary digital information services possible, from the Internet to the iPod to HDTV, applications such as data and image compression, detection, estimation, prediction, cryptography, error-correction coding, modulation, and networking all depend on Information Theory. GreenTouch ™ will use Shannon’s work on theoretical power consumption as a touchstone as it will inform and inspire member scientists to develop the framework and architecture for a new generation of networks that consume only a tiny fraction of the energy required today.
Power and Capacity Limits
Shannon provided a mathematical theory for encoding information by applying a value to it – either 0 or 1. This formulation is commonly known as the basis for digital communications. Furthermore, he demonstrated that mathematics could be used to calculate the theoretical maximum amount of information carried by a communications system based upon the physical laws of thermodynamics.
The most valuable aspect of Shannon’s work may be his definition of “information” for communication networks. This made it possible to identify the critical relationships between various network elements. For example, Information Theory helps explain the interactions among the power of a particular signal, the bandwidth or frequency range of the channel through which the signal travels, and the channel noise, which alters the signal on its way to its destination.
In telecommunications, a channel is typically a path over an electrical wire, an optical fiber or air. In wireless systems, the channel is a tiny slice of radio spectrum used to transmit the message. Often many channels share the same wired or wireless links. Shannon’s equations told engineers how much information could be transmitted over the channels of an ideal system. He also spelled out mathematically the principles of “data compression,” which explains what the end of this sentence demonstrates, that “only infrmatn esentil to understandn mst b tranmitd.” And he showed how we can use controlled error rates to ensure integrity as information is transmitted over noisy channels.
Theoretical tools make it easier to compute the capacity of any given system, which is critical to the design of sophisticated, energy-efficient communication systems and networks. Nevertheless, design remains a formidable challenge for modern communications channels with multiple-antennas or high-speed fiber-optic devices. Building on contributions by Shannon and others, new mathematical methods must be developed to understand the capabilities of these channels. Based on this insight, new codes, signaling methods and equipment can be developed to greatly increase channel capacities while reducing their energy consumption.
We already know that existing multi-access communication methods can be improved through innovative signaling methods. For instance, wireless base stations are often equipped with multiple antennas, where signals are pre-coded to ensure that the right signals reach the right destination.
In the same way that postal codes help ensure rapid delivery of mail, electronic coding is used to improve the reliability and efficiency of information transfer. Codes are used for data compression, cryptography, and error-correction, and more recently also for network-layer coding. One of the major tasks facing scientists and engineers today is to design and implement codes that take system constraints into account, such as time variations in multi-user environments.
Shannon’s theory has been likened to a lighthouse. Its beacon tells communications scientists and engineers where they are, where they’re going, how far they can go, and significantly, where the theoretical limits lie. However, his theory does not explain how to get there. That’s the challenge we continue to face today as we aim to invent new communications systems and technologies – for networks that will move closer to the theoretical limits mapped out by Claude Shannon. With the GreenTouch ™ Initiative, Shannon’s theory is employed in a novel manner – to provide guidance and focus on discovering and developing new ways to minimize energy consumption while approaching the Shannon Limit of maximized network capacity.
Using Information Theory to Combat Climate Change
When Shannon published his theory in 1948, the largest communications cable in operation could carry up to 1,800 voice conversations. Twenty-five years later, the highest capacity cable was carrying 230,000 simultaneous conversations. Today a single strand of optical fiber as thin as a human hair can carry more than 6.4 million conversations. As communication channels today rapidly approach the theoretical limits set by Shannon, it becomes ever more important that we find innovative new methods of communication and methods of organizing communication systems and networks within these limits.
Scientists and engineers worldwide continue to expand on Claude Shannon’s ideas by conducting research into communications systems from a fundamental mathematical perspective and to guide the design of new telecommunications technologies. Some of the areas of focus in this discipline include physical layer signal processing for multi-access, modulation and coding for wireless and optical systems, and network-layer coding for reliable communication.
Theoretical limits built upon Shannon’s theory indicate that the network data traffic generated by network users today could be transported using as little as 1 milliwatt of power. That’s 25,000 times less than the 25 watts of energy estimated to be consumed by a network user today using state of the art equipment. Keeping pace with Internet growth requires high capacity solutions and thus the energy per data bit is the key quantity that needs to be reduced. Increasing network efficiency in this context can be expressed as increasing the traffic supported in bits per second for a given total network power. Guided by Shannon’s beacon, scientists and engineers within GreenTouch™ are striving to develop new approaches to network design that will dramatically increase network efficiency and, in so doing, contribute to the fight against climate change.
Gee Rittenhouse, Bell Labs, on Shannon's Theory
and the GreenTouch™ Initiative