information theory or communication theory, mathematical theory formulated principally by the American scientist Claude E. Shannon to explain aspects and problems of information and communication. While the theory is not specific in all respects, it proves the existence of optimum coding schemes without showing how to find them. For example, it succeeds remarkably in outlining the engineering requirements of communication systems and the limitations of such systems.

In information theory, the term information is used in a special sense; it is a measure of the freedom of choice with which a message is selected from the set of all possible messages. Information is thus distinct from meaning, since it is entirely possible for a string of nonsense words and a meaningful sentence to be equivalent with respect to information content.

Measurement of Information Content

Numerically, information is measured in bits (short for binary digit; see binary system). One bit is equivalent to the choice between two equally likely choices. For example, if we know that a coin is to be tossed but are unable to see it as it falls, a message telling whether the coin came up heads or tails gives us one bit of information. When there are several equally likely choices, the number of bits is equal to the logarithm of the number of choices taken to the base two. For example, if a message specifies one of sixteen equally likely choices, it is said to contain four bits of information. When the various choices are not equally probable, the situation is more complex.

Interestingly, the mathematical expression for information content closely resembles the expression for entropy in thermodynamics. The greater the information in a message, the lower its randomness, or "noisiness," and hence the smaller its entropy. Since the information content is, in general, associated with a source that generates messages, it is often called the entropy of the source. Often, because of constraints such as grammar, a source does not use its full range of choice. A source that uses just 70% of its freedom of choice would be said to have a relative entropy of 0.7. The redundancy of such a source is defined as 100% minus the relative entropy, or, in this case, 30%. The redundancy of English is estimated to be about 50%; i.e., about half of the elements used in writing or speaking are freely chosen, and the rest are required by the structure of the language.

Analysis of the Transfer of Messages through Channels

A message proceeds along a channel from the source to the receiver; information theory defines for any given channel a limiting capacity or rate at which it can carry information, expressed in bits per second. In general, it is necessary to process, or encode, information from a source before transmitting it through a given channel. For example, a human voice must be encoded before it can be transmitted by telephone. An important theorem of information theory states that if a source with a given entropy feeds information to a channel with a given capacity, and if the source entropy is less than the channel capacity, a code exists for which the frequency of errors may be reduced as low as desired. If the channel capacity is less than the source entropy, no such code exists.

The theory further shows that noise, or random disturbance of the channel, creates uncertainty as to the correspondence between the received signal and the transmitted signal. The average uncertainty in the message when the signal is known is called the equivocation. It is shown that the net effect of noise is to reduce the information capacity of the channel. However, redundancy in a message, as distinguished from redundancy in a source, makes it more likely that the message can be reconstructed at the receiver without error. For example, if something is already known as a certainty, then all messages about it give no information and are 100% redundant, and the information is thus immune to any disturbances of the channel. Using various mathematical means, Shannon was able to define channel capacity for continuous signals, such as music and speech.

information storage and retrieval, the systematic process of collecting and cataloging data so that they can be located and displayed on request. Computers and data processing techniques have made possible the high-speed, selective retrieval of large amounts of information for government, commercial, and academic purposes. There are several basic types of information-storage-and-retrieval systems. Document-retrieval systems store entire documents, which are usually retrieved by title or by key words associated with the document. In some systems, the text of documents is stored as data. This permits full text searching, enabling retrieval on the basis of any words in the document. In others, a digitized image of the document is stored, usually on a write-once optical disc. Database systems store the information as a series of discrete records that are, in turn, divided into discrete fields (e.g., name, address, and phone number); records can be searched and retrieved on the basis of the content of the fields (e.g., all people who have a particular telephone area code). The data are stored within the computer, either in main storage or auxiliary storage, for ready access. Reference-retrieval systems store references to documents rather than the documents themselves. Such systems, in response to a search request, provide the titles of relevant documents and frequently their physical locations. Such systems are efficient when large amounts of different types of printed data must be stored. They have proven extremely effective in libraries, where material is constantly changing.

information processing: see data processing.

information, in law: see indictment.

Office of War Information (OWI), U.S. agency created (1942) during World War II to consolidate government information services. The OWI absorbed the functions of the Office of Facts and Figures, the Office of Government Reports, the division of information of the Office for Emergency Management, and the foreign information service of the Coordinator of Information. Elmer Davis was named director. Besides coordinating the release of war news for domestic use, the office established an overseas branch, under Robert E. Sherwood, which launched a huge information and propaganda campaign abroad. Congressional opposition to the domestic operations of the OWI resulted in increasingly curtailed funds, and by 1944 the OWI operated mostly in the foreign field, contributing to undermining enemy morale. The agency was abolished in 1945, and its foreign functions were transferred to the Dept. of State.

Freedom of Information Act (1966), law requiring that U.S. government agencies release their records to the public on request, unless the information sought falls into a category specifically exempted, such as national security, an individual's right to privacy, or internal agency management. The act provides for court review of agency refusals to furnish identifiable records. The states also have similar laws. The federal government and some states have also adopted so-called sunshine laws that require governmental bodies, as a matter of general policy, to hold open meetings, announced in advance. Presidential papers remained under the control of individual American presidents until 1981, when the Presidential Records Act—enacted by Congress in 1978—took effect. Under it, presidential papers were to be released to the public 12 years after an administration ended. In 2001, however, President George W. Bush signed an executive order that gave a former president or a sitting president the right to prevent the release of a former president's papers to the public. The G. W. Bush administration has also has generally been more reluctant to release documents under the Freedom of Information Act.

American Standard Code for Information Interchange: see ASCII.

Field of mathematics that studies the problems of signal transmission, reception, and processing. It stems from Claude E. Shannon's mathematical methods for measuring the degree of order (nonrandomness) in a signal, which drew largely on probability theory and stochastic processes and led to techniques for determining a source's rate of information production, a channel's capacity to handle information, and the average amount of information in a given type of message. Crucial to the design of communications systems, these techniques have important applications in linguistics, psychology, and even literary theory.

Learn more about information theory with a free trial on Britannica.com.

Acquisition, recording, organization, retrieval, display, and dissemination of information. Today the term usually refers to computer-based operations. Information processing consists of locating and capturing information, using software to manipulate it into a desired form, and outputting the data. An Internet search engine is an example of an information-processing tool, as is any sophisticated information-retrieval system. Seealso data processing.

Learn more about information processing with a free trial on Britannica.com.

Discipline that deals with the processes of storing and transferring information. It attempts to bring together concepts and methods from such varied disciplines as library science, computer science and engineering, linguistics, and psychology to develop techniques and devices to aid in the handling of information. In its early stages in the 1960s, information science was concerned primarily with applying the then-new computer technology to the processing and managing of documents. The applied computer technologies and theoretical studies of information science have since permeated many other disciplines. Computer science and engineering still tend to absorb its theory- and technology-oriented subjects, and management science tends to absorb information-systems subjects.

Learn more about information science with a free trial on Britannica.com.

Computerized system that relates and displays data collected from a geographic entity in the form of a map. The ability of GIS to overlay existing data with new information and display it in colour on a computer screen is used primarily to conduct analyses and make decisions related to geology, ecology, land use, demographics, transportation, and other domains, most of which relate to the human use of the physical environment. Through the process of geocoding, geographic data from a database is converted into images in the form of maps.

Learn more about geographic information system (GIS) with a free trial on Britannica.com.

in full American Standard Code for Information Interchange.

Data-transmission code used to represent both text (letters, numbers, punctuation marks) and noninput device commands (control characters) for electronic exchange and storage. Standard ASCII uses a string of 7 bits (binary digits) for each symbol and can thus represent 2⁷ = 128 characters. Extended ASCII uses an 8-bit encoding system and can thus represent 2⁸ = 256 characters. While ASCII is still found in legacy data, Unicode, with 8-, 16-, and 32-bit versions, has become standard for modern operating systems and browsers. In particular, the 32-bit version now supports all of the characters in every major language.

Learn more about ASCII with a free trial on Britannica.com.