This book introduces readers to the minimum description length (MDL) principle and its applications in learning. The MDL is a fundamental principle for inductive inference, which is used in many applications including statistical modeling, pattern recognition and machine learning. At its core, the MDL is based on the premise that “the shortest code length leads to the best strategy for learning anything from data.” The MDL provides a broad and unifying view of statistical inferences such as estimation, prediction and testing and, of course, machine learning. The content covers the theoretical foundations of the MDL and broad practical areas such as detecting changes and anomalies, problems involving latent variable models, and high dimensional statistical inference, among others. The book offers an easy-to-follow guide to the MDL principle, together with other information criteria, explaining the differences between their standpoints. Written in a systematic, concise and comprehensive style, this book is suitable for researchers and graduate students of machine learning, statistics, information theory and computer science.

This introduction to the MDL Principle provides a reference accessible to graduate students and researchers in statistics, pattern classification, machine learning, and data mining, to philosophers interested in the foundations of statistics, and to researchers in other applied sciences that involve model selection.

Machine Learning Proceedings 1994

A source book for state-of-the-art MDL, including an extensive tutorial and recent theoretical advances and practical applications in fields ranging from bioinformatics to psychology.

This book is based on the workshop on New Approaches to Learning for Natural Language Processing, held in conjunction with the International Joint Conference on Artificial Intelligence, IJCAI'95, in Montreal, Canada in August 1995. Most of the 32 papers included in the book are revised selected workshop presentations; some papers were individually solicited from members of the workshop program committee to give the book an overall completeness. Also included, and written with the novice reader in mind, is a comprehensive introductory survey by the volume editors. The volume presents the state of the art in the most promising current approaches to learning for NLP and is thus compulsory reading for researchers in the field or for anyone applying the new techniques to challenging real-world NLP problems.

Rules – the clearest, most explored and best understood form of knowledge representation – are particularly important for data mining, as they offer the best tradeoff between human and machine understandability. This book presents the fundamentals of rule learning as investigated in classical machine learning and modern data mining. It introduces a feature-based view, as a unifying framework for propositional and relational rule learning, thus bridging the gap between attribute-value learning and inductive logic programming, and providing complete coverage of most important elements of rule learning. The book can be used as a textbook for teaching machine learning, as well as a comprehensive reference to research in the field of inductive rule learning. As such, it targets students, researchers and developers of rule learning algorithms, presenting the fundamental rule learning concepts in sufficient breadth and depth to enable the reader to understand, develop and apply rule learning techniques to real-world data.

Machine Learning: A Constraint-Based Approach, Second Edition provides readers with a refreshing look at the basic models and algorithms of machine learning, with an emphasis on current topics of interest that include neural networks and kernel machines. The book presents the information in a truly unified manner that is based on the notion of learning from environmental constraints. It draws a path towards deep integration with machine learning that relies on the idea of adopting multivalued logic formalisms, such as in fuzzy systems. Special attention is given to deep learning, which nicely fits the constrained-based approach followed in this book.The book presents a simpler unified notion of regularization, which is strictly connected with the parsimony principle, including many solved exercises that are classified according to the Donald Knuth ranking of difficulty, which essentially consists of a mix of warm-up exercises that lead to deeper research problems. A software simulator is also included. - Presents, in a unified manner, fundamental machine learning concepts, such as neural networks and kernel machines - Provides in-depth coverage of unsupervised and semi-supervised learning, with new content in hot growth areas such as deep learning - Includes a software simulator for kernel machines and learning from constraints that also covers exercises to facilitate learning - Contains hundreds of solved examples and exercises chosen particularly for their progression of difficulty from simple to complex - Supported by a free, downloadable companion book designed to facilitate students' acquisition of experimental skills

This book covers VC dimension and PAC learning, dimensionality reduction, evaluation of classifiers, Bayesian classifier and ML estimation, regression, decision trees, neural networks, sample questions, Bayesian learning, and Instance based learning.

Machine Learning Proceedings 1995

Machine Learning