The relevance of information theory to statistical theory and its applications to stochastic processes is a unifying influence in these TOPICS. The integral representation of discrimination information is presented in these TOPICS reviewing various approaches used in the literature, and is also developed herein using intrinsically information-theoretic methods. Log likelihood ratios associated with various stochastic processes are computed by an application of minimum discrimination information estimates. Linear discriminant functionals are used in the information-theoretic analysis of a variety of stochastic processes. Sections are numbered serially within each chapter, with a decimal notation for subsections. Equations, examples, theorems and lemmas, are numbered serially within each section with a decimal notation. The digits to the left of the decimal point represent the section and the digits to the right of the decimal point the serial number within the section. When reference is made to a section, equation, example, theorem or lemma within the same chapter only the section number or equation number, etc., is given. When the reference is to a section ,equation, etc., in a different chapter, then in addition to the section or equation etc., number, the chapter number is also given. References to the bibliography are by the author's name followed by the year of publication in parentheses. The transpose of a matrix is denoted by a prime; thus one-row matrices are denoted by primes as the transposes of one-column matrices (vectors).

This book presents new and original research in Statistical Information Theory, based on minimum divergence estimators and test statistics, from a theoretical and applied point of view, for different statistical problems with special emphasis on efficiency and robustness. Divergence statistics, based on maximum likelihood estimators, as well as Wald’s statistics, likelihood ratio statistics and Rao’s score statistics, share several optimum asymptotic properties, but are highly non-robust in cases of model misspecification under the presence of outlying observations. It is well-known that a small deviation from the underlying assumptions on the model can have drastic effect on the performance of these classical tests. Specifically, this book presents a robust version of the classical Wald statistical test, for testing simple and composite null hypotheses for general parametric models, based on minimum divergence estimators.

This book provides an up-to-date introduction to information theory. In addition to the classical topics discussed, it provides the first comprehensive treatment of the theory of I-Measure, network coding theory, Shannon and non-Shannon type information inequalities, and a relation between entropy and group theory. ITIP, a software package for proving information inequalities, is also included. With a large number of examples, illustrations, and original problems, this book is excellent as a textbook or reference book for a senior or graduate level course on the subject, as well as a reference for researchers in related fields.

Information Theory and Statistics: A Tutorial is concerned with applications of information theory concepts in statistics, in the finite alphabet setting. The topics covered include large deviations, hypothesis testing, maximum likelihood estimation in exponential families, analysis of contingency tables, and iterative algorithms with an "information geometry" background. Also, an introduction is provided to the theory of universal coding, and to statistical inference via the minimum description length principle motivated by that theory. The tutorial does not assume the reader has an in-depth knowledge of Information Theory or statistics. As such, Information Theory and Statistics: A Tutorial, is an excellent introductory text to this highly-important topic in mathematics, computer science and electrical engineering. It provides both students and researchers with an invaluable resource to quickly get up to speed in the field.

This book is an evolution from my book A First Course in Information Theory published in 2002 when network coding was still at its infancy. The last few years have witnessed the rapid development of network coding into a research ?eld of its own in information science. With its root in infor- tion theory, network coding has not only brought about a paradigm shift in network communications at large, but also had signi?cant in?uence on such speci?c research ?elds as coding theory, networking, switching, wireless c- munications,distributeddatastorage,cryptography,andoptimizationtheory. While new applications of network coding keep emerging, the fundamental - sults that lay the foundation of the subject are more or less mature. One of the main goals of this book therefore is to present these results in a unifying and coherent manner. While the previous book focused only on information theory for discrete random variables, the current book contains two new chapters on information theory for continuous random variables, namely the chapter on di?erential entropy and the chapter on continuous-valued channels. With these topics included, the book becomes more comprehensive and is more suitable to be used as a textbook for a course in an electrical engineering department.

Information theory always has the dual appeal of bringing important concepts to the study of communication in society, and of providing a calculus for information flows within systems. This book introduces readers to basic concepts of information theory, extending its original linear conception of communication to many variables, networks, and higher-order interactions (including loops) and developing it into a method for analyzing qualitative data. It elaborates on the algebra of entropy and information, shows how complex models of data are constructed and tested, describes algorithms for exploring multivariate structures using such models, and gives illustrative applications of these techniques. The book is designed as a text but it can also serve as a handbook for social researchers and systems theorists with an interest in communication.

This monograph presents an account of the asymptotic behaviour of the weighted bootstrap - a new and powerful statistical technique. Researchers and advanced graduate students studying bootstrap methods will find this a valuable technical survey which is thorough and rigorous. The main aim of this book is to answer two questions: How well does the generalized bootstrap work? What are the differences between all the different weighted schemes? Readers are assumed to have already some familiarity with the bootstrap, but otherwise the account is as self-contained as possible. Proofs are presented in detail, though some lengthy calculations are deferred to appendices.

The datafication of our world offers huge challenges and opportunities for social science. The ‘data-drivenness’ of computational research can occur at the expense of theoretical reflection and interpretation. Additionally, it can be difficult to reconcile the ‘quantitative’ dimensions of big data with the ‘qualitative’ sensibilities needed for its understanding. At the same time, this opens up possibilities for reimagining key principles of social inquiry. In this experimental and provocative book, Simon Lindgren argues that a hybrid approach to data and theory must be developed in order to make sense of today's ambivalent, turbulent, and media-saturated political landscape. He pushes for the development of a critical science of data, joining the interpretive theoretical and ethical sensibilities of social science with the predictive and prognostic powers of data science and computational methods. In order for theories and research methods to be more useful and relevant, they must be dismantled and put together in new, alternative, and unexpected ways. Data Theory is essential reading for social scientists and data scientists, as well as students taking courses in social theory and data, digital methods, big data, and data and society.

Big Data and Information Theory are a binding force between various areas of knowledge that allow for societal advancement. Rapid development of data analytic and information theory allows companies to store vast amounts of information about production, inventory, service, and consumer activities. More powerful CPUs and cloud computing make it possible to do complex optimization instead of using heuristic algorithms, as well as instant rather than offline decision-making. The era of "big data" challenges includes analysis, capture, curation, search, sharing, storage, transfer, visualization, and privacy violations. Big data calls for better integration of optimization, statistics, and data mining. In response to these challenges this book brings together leading researchers and engineers to exchange and share their experiences and research results about big data and information theory applications in various areas. This book covers a broad range of topics including statistics, data mining, data warehouse implementation, engineering management in large-scale infrastructure systems, data-driven sustainable supply chain network, information technology service offshoring project issues, online rumors governance, preliminary cost estimation, and information system project selection. The chapters in this book were originally published in the journal, International Journal of Management Science and Engineering Management.

The present monograph is a comprehensive summary of the research on visibility in random fields, which I have conducted with the late Professor Micha Yadin for over ten years. This research, which resulted in several published papers and technical reports (see bibliography), was motivated by some military problems, which were brought to our attention by Mr. Pete Shugart of the US Army TRADOC Systems Analysis Activity, presently called US Army TRADOC Analysis Command. The Director ofTRASANA at the time, the late Dr. Wilbur Payne, identified the problems and encouraged the support and funding of this research by the US Army. Research contracts were first administered through the Office of Naval Research, and subsequently by the Army Research Office. We are most grateful to all involved for this support and encouragement. In 1986 I administered a three-day workshop on problem solving in the area of sto chastic visibility. This workshop was held at the White Sands Missile Range facility. A set of notes with some software were written for this workshop. This workshop led to the incorporation of some of the methods discussed in the present book into the Army simulation package CASTFOREM. Several people encouraged me to extend those notes and write the present monograph on the level of those notes, so that the material will be more widely available for applications.