Shlomo Shamai

If information theory and estimation theory are thought of as two scientific languages, then their key vocabularies are information measures and estimation measures, respectively. The basic information measures are entropy, mutual information and relative entropy. Among the most important estimation measures are mean square error (MSE) and Fisher information. Playing a paramount role in information theory and estimation theory, those measures are akin to mass, force and velocity in classical mechanics, or energy, entropy and temperature in thermodynamics.The Interplay Between Information and Estimation Measures is intended as handbook of known formulas which directly relate to information measures and estimation measures. It provides intuition and draws connections between these formulas, highlights some important applications, and motivates further explorations. The main focus is on such formulas in the context of the additive Gaussian noise model, with lesser treatment of others such as the Poisson point process channel.Also included are a number of new results which are published here for the first time. Proofs of some basic results are provided, whereas many more technical proofs already available in the literature are omitted. In 2004, the authors of this monograph found a general differential relationship commonly referred to as the I-MMSE formula.In this book a new, complete proof for the I-MMSE formula is developed, which includes some technical details omitted in the original papers relating to this. It concludes by highlighting the impact of the information-estimation relationships on a variety of information-theoretic problems of current interest, and provide some further perspective on their applications.

Security is one of the most important issues in communications. Security issues arising in communication networks include confidentiality, integrity, authentication and non-repudiation. Attacks on the security of communication networks can be divided into two basic types: passive attacks and active attacks. An active attack corresponds to the situation in which a malicious actor intentionally disrupts the system. A passive attack corresponds to the situation in which a malicious actor attempts to interpret source information without injecting any information or trying to modify the information; i.e., passive attackers listen to the transmission without modifying it.Information Theoretic Security focuses on confidentiality issues, in which passive attacks are of primary concern. The information theoretic approach to achieving secure communication opens a promising new direction toward solving wireless networking security problems. Compared to contemporary cryptosystems, information theoretic approaches offer advantages such as eliminating the key management issue; are less vulnerable to the man-in-the-middle and achieve provable security that is robust to powerful eavesdroppers possessing unlimited computational resources, knowledge of the communication strategy employed including coding and decoding algorithms, and access to communication systems either through perfect or noisy channels.Information Theoretic Security surveys the research dating back to the 1970s which forms the basis of applying this technique in modern systems. It proceeds to provide an overview of how information theoretic approaches are developed to achieve secrecy for a basic wire-tap channel model as well as for its extensions to multiuser networks. It is an invaluable resource for students and researchers working in network security, information theory and communications.

This book focuses on the performance evaluation of linear codes under optimal maximum-likelihood (ML) decoding. Though the ML decoding algorithm is prohibitively complex for most practical codes, their performance analysis under ML decoding allows to predict their performance without resorting to computer simulations. It also provides a benchmark for testing the sub-optimality of iterative (or other practical) decoding algorithms.This analysis also establishes the goodness of linear codes (or ensembles), determined by the gap between their achievable rates under optimal ML decoding and information theoretical limits. In this book, upper and lower bounds on the error probability of linear codes under ML decoding are surveyed and applied to codes and ensembles of codes on graphs. For upper bounds, the authors discuss various bounds where focus is put on Gallager bounding techniques and their relation to a variety of other reported bounds. Within the class of lower bounds, they address de Caen's based bounds and their improvements, and also consider sphere-packing bounds with their recent improvements targeting codes of moderate block lengths.Performance Analysis of Linear Codes under Maximum-Likelihood Decoding is a comprehensive introduction to this important topic for students, practitioners and researchers working in communications and information theory.