Machine learning and artificial intelligence increasingly use methodological tools rooted in statistical physics. Conversely, limitations and pitfalls encountered in AI question the very foundations of statistical physics. This interplay between AI and statistical physics has been attested since the birth of AI, and principles underpinning statistical physics can shed new light on the conceptual basis of AI. During the last fifty years, statistical physics has been investigated through new geometric structures allowing covariant formalization of the thermodynamics. Inference methods in machine learning have begun to adapt these new geometric structures to process data in more abstract representation spaces. This volume collects selected contributions on the interplay of statistical physics and artificial intelligence. The aim is to provide a constructive dialogue around a common foundation to allow the establishment of new principles and laws governing these two disciplines in a unified manner. The contributions were presented at the workshop on the Joint Structures and Common Foundation of Statistical Physics, Information Geometry and Inference for Learning which was held in Les Houches in July 2020. The various theoretical approaches are discussed in the context of potential applications in cognitive systems, machine learning, signal processing.

This book constitutes the proceedings of the 4th International Conference on Geometric Science of Information, GSI 2019, held in Toulouse, France, in August 2019. The 79 full papers presented in this volume were carefully reviewed and selected from 105 submissions. They cover all the main topics and highlights in the domain of geometric science of information, including information geometry manifolds of structured data/information and their advanced applications.

This book focuses on information-geometric manifolds of structured data and models and related applied mathematics. It features new and fruitful interactions between several branches of science: Advanced Signal/Image/Video Processing, Complex Data Modeling and Analysis, Statistics on Manifolds, Topology/Machine/Deep Learning and Artificial Intelligence. The selection of applications makes the book a substantial information source, not only for academic scientist but it is also highly relevant for industry. The book project was initiated following discussions at the international conference GSI’2019 – Geometric Science of Information that was held at ENAC, Toulouse (France).

Geometry and Statistics, Volume 46 in the Handbook of Statistics series, highlights new advances in the field, with this new volume presenting interesting chapters written by an international board of authors. - Provides the authority and expertise of leading contributors from an international board of authors - Presents the latest release in the Handbook of Statistics series - Updated release includes the latest information on Geometry and Statistics

This book publishes select papers presented at the 4th International Conference on Frontiers in Industrial and Applied Mathematics (FIAM-2021), held at the Sant Longowal Institute of Engineering and Technology, Longowal, Punjab, India, from 21–22 December 2021. Most of the papers deal with mathematical theory embedded with its applications to engineering and sciences. This book illustrates numerical simulation of scientific problems and the state-of-the-art research in industrial and applied mathematics, including various computational and modeling techniques with case studies and concrete examples. Graduate students and researchers, who are interested in real applications of mathematics in the areas of computational and theoretical fluid dynamics, solid mechanics, optimization and operations research, numerical analysis, bio-mathematics, fuzzy, control and systems theory, dynamical systems and nonlinear analysis, algebra and approximation theory, will find the book useful.

Frontiers in Entropy Across the Disciplines presents a panorama of entropy emphasizing mathematical theory, physical and scientific significance, computational methods, and applications in mathematics, physics, statistics, engineering, biomedical signals, and signal processing.In the last century classical concepts of entropy were introduced in the areas of thermodynamics, information theory, probability theory, statistics, dynamical systems, and ergodic theory. During the past 50 years, dozens of new concepts of entropy have been introduced and studied in many disciplines. This volume captures significant developments in this arena. It features expository, review, and research papers by distinguished mathematicians and scientists from many disciplines. The level of mathematics ranges from intermediate level to research level. Each chapter contains a comprehensive list of references. Topics include entropy and society, entropy and time, Souriau entropy on symplectic model of statistical physics, new definitions of entropy, geometric theory of heat and information, maximum entropy in Bayesian networks, maximum entropy methods, entropy analysis of biomedical signals (review and comparison of methods), spectral entropy and its application to video coding and speech coding, a comprehensive review of 50 years of entropy in dynamics, a comprehensive review on entropy, entropy-like quantities and applications, topological entropy of multimodal maps, entropy production in complex systems, entropy production and convergence to equilibrium, reversibility and irreversibility in entropy, nonequilibrium entropy, index of various entropy, entropy and the greatest blunder ever.

This book is a printed edition of the Special Issue "New Trends in Statistical Physics of Complex Systems" that was published in Entropy

This book explores minimum divergence methods of statistical machine learning for estimation, regression, prediction, and so forth, in which we engage in information geometry to elucidate their intrinsic properties of the corresponding loss functions, learning algorithms, and statistical models. One of the most elementary examples is Gauss's least squares estimator in a linear regression model, in which the estimator is given by minimization of the sum of squares between a response vector and a vector of the linear subspace hulled by explanatory vectors. This is extended to Fisher's maximum likelihood estimator (MLE) for an exponential model, in which the estimator is provided by minimization of the Kullback-Leibler (KL) divergence between a data distribution and a parametric distribution of the exponential model in an empirical analogue. Thus, we envisage a geometric interpretation of such minimization procedures such that a right triangle is kept with Pythagorean identity in the sense of the KL divergence. This understanding sublimates a dualistic interplay between a statistical estimation and model, which requires dual geodesic paths, called m-geodesic and e-geodesic paths, in a framework of information geometry. We extend such a dualistic structure of the MLE and exponential model to that of the minimum divergence estimator and the maximum entropy model, which is applied to robust statistics, maximum entropy, density estimation, principal component analysis, independent component analysis, regression analysis, manifold learning, boosting algorithm, clustering, dynamic treatment regimes, and so forth. We consider a variety of information divergence measures typically including KL divergence to express departure from one probability distribution to another. An information divergence is decomposed into the cross-entropy and the (diagonal) entropy in which the entropy associates with a generative model as a family of maximum entropy distributions; the cross entropy associates with a statistical estimation method via minimization of the empirical analogue based on given data. Thus any statistical divergence includes an intrinsic object between the generative model and the estimation method. Typically, KL divergence leads to the exponential model and the maximum likelihood estimation. It is shown that any information divergence leads to a Riemannian metric and a pair of the linear connections in the framework of information geometry. We focus on a class of information divergence generated by an increasing and convex function U, called U-divergence. It is shown that any generator function U generates the U-entropy and U-divergence, in which there is a dualistic structure between the U-divergence method and the maximum U-entropy model. We observe that a specific choice of U leads to a robust statistical procedure via the minimum U-divergence method. If U is selected as an exponential function, then the corresponding U-entropy and U-divergence are reduced to the Boltzmann-Shanon entropy and the KL divergence; the minimum U-divergence estimator is equivalent to the MLE. For robust supervised learning to predict a class label we observe that the U-boosting algorithm performs well for contamination of mislabel examples if U is appropriately selected. We present such maximal U-entropy and minimum U-divergence methods, in particular, selecting a power function as U to provide flexible performance in statistical machine learning.

This book focuses on information geometry manifolds of structured data/information and their advanced applications featuring new and fruitful interactions between several branches of science: information science, mathematics and physics. It addresses interrelations between different mathematical domains like shape spaces, probability/optimization & algorithms on manifolds, relational and discrete metric spaces, computational and Hessian information geometry, algebraic/infinite dimensional/Banach information manifolds, divergence geometry, tensor-valued morphology, optimal transport theory, manifold & topology learning, and applications like geometries of audio-processing, inverse problems and signal processing. The book collects the most important contributions to the conference GSI’2017 – Geometric Science of Information.

This book brings together geometric tools and their applications for Information analysis. It collects current and many uses of in the interdisciplinary fields of Information Geometry Manifolds in Advanced Signal, Image & Video Processing, Complex Data Modeling and Analysis, Information Ranking and Retrieval, Coding, Cognitive Systems, Optimal Control, Statistics on Manifolds, Machine Learning, Speech/sound recognition and natural language treatment which are also substantially relevant for the industry.