Mikhail Moshkov

In this book, the following three approaches to data analysis are presented: - Test Theory, founded by Sergei V. Yablonskii (1924-1998); the first publications appeared in 1955 and 1958,- Rough Sets, founded by Zdzislaw I. Pawlak (1926-2006); the first publications appeared in 1981 and 1982,- Logical Analysis of Data, founded by Peter L. Hammer (1936-2006); the first publications appeared in 1986 and 1988.These three approaches have much in common, but researchers active in one of these areas often have a limited knowledge about the results and methods developed in the other two. On the other hand, each of the approaches shows some originality and we believe that the exchange of knowledge can stimulate further development of each of them. This can lead to new theoretical results and real-life applications and, in particular, new results based on combination of these three data analysis approaches can be expected.- Logical Analysis of Data, founded by Peter L. Hammer (1936-2006); the firstpublications appeared in 1986 and 1988.These three approaches have much in common, but researchers active in one of these areas often have a limited knowledge about the results and methods developed in the other two. On the other hand, each of the approaches shows some originality and we believe that the exchange of knowledge can stimulate further development of each of them. This can lead to new theoretical results and real-life applications and, in particular, new results based on combination of these three data analysis approaches can be expected.These three approaches have much in common, but researchers active in one of these areas often have a limited knowledge about the results and methods developed in the other two. On the other hand, each of the approaches shows some originality and we believe that the exchange of knowledge can stimulate further development of each of them. This can lead to new theoretical results and real-life applications and, in particular, new results based on combination of these three data analysis approaches can be expected.

In this book, the following three approaches to data analysis are presented: - Test Theory, founded by Sergei V. Yablonskii (1924-1998); the first publications appeared in 1955 and 1958,- Rough Sets, founded by Zdzislaw I. Pawlak (1926-2006); the first publications appeared in 1981 and 1982,- Logical Analysis of Data, founded by Peter L. Hammer (1936-2006); the first publications appeared in 1986 and 1988.These three approaches have much in common, but researchers active in one of these areas often have a limited knowledge about the results and methods developed in the other two. On the other hand, each of the approaches shows some originality and we believe that the exchange of knowledge can stimulate further development of each of them. This can lead to new theoretical results and real-life applications and, in particular, new results based on combination of these three data analysis approaches can be expected.- Logical Analysis of Data, founded by Peter L. Hammer (1936-2006); the firstpublications appeared in 1986 and 1988.These three approaches have much in common, but researchers active in one of these areas often have a limited knowledge about the results and methods developed in the other two. On the other hand, each of the approaches shows some originality and we believe that the exchange of knowledge can stimulate further development of each of them. This can lead to new theoretical results and real-life applications and, in particular, new results based on combination of these three data analysis approaches can be expected.These three approaches have much in common, but researchers active in one of these areas often have a limited knowledge about the results and methods developed in the other two. On the other hand, each of the approaches shows some originality and we believe that the exchange of knowledge can stimulate further development of each of them. This can lead to new theoretical results and real-life applications and, in particular, new results based on combination of these three data analysis approaches can be expected.

This book is devoted to the study of deterministic and nondeterministic computation trees. Computation trees are a natural generalization of decision trees: in addition to the one-place predicate-type operations (attributes) used in decision trees, computation trees can use multi-place predicate and function operations. They arise both where we deal with algorithms for solving problems of combinatorial optimization, computational geometry, etc., and where we solve classification or prediction problems, especially if we use combinations of input variables as attributes. This book mainly studies the complexity of computation trees and also examines related optimization problems. The results discussed in this book may be useful to researchers studying algorithms and using algorithm models similar to computation trees. These results may also be useful to researchers working with decision trees and decision rule systems in data analysis, particularly, in rough set theory, logical analysis of data, and test theory. The book is also used to create graduate courses.

This book explores, within the framework of rough set theory, the complexity of decision trees and decision rule systems and the relationships between them for problems over information systems, for decision tables from closed classes, and for problems involving formal languages. Decision trees and systems of decision rules are widely used as means of representing knowledge, as classifiers that predict decisions for new objects, as well as algorithms for solving various problems of fault diagnosis, combinatorial optimization, etc. Decision trees and systems of decision rules are among the most interpretable models of knowledge representation and classification. Investigating the relationships between these two models is an important task in computer science. The possibilities of transforming decision rule systems into decision trees are being studied in detail. The results are useful for researchers using decision trees and decision rule systems in data analysis, especially in rough set theory, logical analysis of data, and test theory. This book is also used to create courses for graduate students.

In this book, we study decision trees for fault diagnosis in circuits and switching networks, which are among the most fundamental models for computing Boolean functions. We consider two main cases: when the scheme (circuit or switching network) has the same mode of operation for both calculation and diagnostics, and when the scheme has two modes of operation—normal for calculation and special for diagnostics. In the former case, we get mostly negative results, including superpolynomial lower bounds on the minimum depth of diagnostic decision trees depending on scheme complexity and the NP-hardness of construction diagnostic decision trees. In the latter case, we describe classes of schemes and types of faults for which decision trees can be effectively used to diagnose schemes, when they are transformed into so-called iteration-free schemes. The tools and results discussed in this book help to understand both the possibilities and challenges of using decision trees to diagnosefaults in various schemes. The book is useful to specialists both in the field of theoretical and technical diagnostics.It can also be used for the creation of courses for graduate students.

In this book, the concept of a hypothesis about the values of all attributes is added to the standard decision tree model, considered, in particular, in test theory and rough set theory. This extension allows us to use the analog of equivalence queries from exact learning and explore decision trees that are based on various combinations of attributes, hypotheses, and proper hypotheses (analog of proper equivalence queries). The two main goals of this book are (i) to provide tools for the experimental and theoretical study of decision trees with hypotheses and (ii) to compare these decision trees with conventional decision trees that use only queries, each based on a single attribute. Both experimental and theoretical results show that decision trees with hypotheses can have less complexity than conventional decision trees. These results open up some prospects for using decision trees with hypotheses as a means of knowledge representation and algorithms for computing Boolean functions. The obtained theoretical results and tools for studying decision trees with hypotheses are useful for researchers using decision trees and rules in data analysis. This book can also be used as the basis for graduate courses.

In this book, we study decision trees for fault diagnosis in circuits and switching networks, which are among the most fundamental models for computing Boolean functions. We consider two main cases: when the scheme (circuit or switching network) has the same mode of operation for both calculation and diagnostics, and when the scheme has two modes of operation—normal for calculation and special for diagnostics. In the former case, we get mostly negative results, including superpolynomial lower bounds on the minimum depth of diagnostic decision trees depending on scheme complexity and the NP-hardness of construction diagnostic decision trees. In the latter case, we describe classes of schemes and types of faults for which decision trees can be effectively used to diagnose schemes, when they are transformed into so-called iteration-free schemes. The tools and results discussed in this book help to understand both the possibilities and challenges of using decision trees to diagnosefaults in various schemes. The book is useful to specialists both in the field of theoretical and technical diagnostics.It can also be used for the creation of courses for graduate students.

In this book, the concept of a hypothesis about the values of all attributes is added to the standard decision tree model, considered, in particular, in test theory and rough set theory. This extension allows us to use the analog of equivalence queries from exact learning and explore decision trees that are based on various combinations of attributes, hypotheses, and proper hypotheses (analog of proper equivalence queries). The two main goals of this book are (i) to provide tools for the experimental and theoretical study of decision trees with hypotheses and (ii) to compare these decision trees with conventional decision trees that use only queries, each based on a single attribute. Both experimental and theoretical results show that decision trees with hypotheses can have less complexity than conventional decision trees. These results open up some prospects for using decision trees with hypotheses as a means of knowledge representation and algorithms for computing Boolean functions. The obtained theoretical results and tools for studying decision trees with hypotheses are useful for researchers using decision trees and rules in data analysis. This book can also be used as the basis for graduate courses.

This book introduces a fairly universal approach to the design and analysis of exact optimization algorithms for multi-objective combinatorial optimization problems. It proposes the circuits without repetitions representing the sets of feasible solutions along with the increasing and strictly increasing cost functions as a model for such problems. The book designs the algorithms for multi-stage and bi-criteria optimization and for counting the solutions in the framework of this model.As applications, this book studies eleven known combinatorial optimization problems: matrix chain multiplication, global sequence alignment, optimal paths in directed graphs, binary search trees, convex polygon triangulation, line breaking (text justification), one-dimensional clustering, optimal bitonic tour, segmented least squares, optimization of matchings in trees, and 0/1 knapsack problem.The results presented are useful for researchers in combinatorial optimization. This book is also useful as the basis for graduate courses.

This book compares four parameters of problems in arbitrary information systems: complexity of problem representation and complexity of deterministic, nondeterministic, and strongly nondeterministic decision trees for problem solving. Deterministic decision trees are widely used as classifiers, as a means of knowledge representation, and as algorithms. Nondeterministic (strongly nondeterministic) decision trees can be interpreted as systems of true decision rules that cover all objects (objects from one decision class). This book develops tools for the study of decision trees, including bounds on complexity and algorithms for construction of decision trees for decision tables with many-valued decisions. It considers two approaches to the investigation of decision trees for problems in information systems: local, when decision trees can use only attributes from the problem representation; and global, when decision trees can use arbitrary attributes from the information system. For both approaches, it describes all possible types of relationships among the four parameters considered and discusses the algorithmic problems related to decision tree optimization. The results presented are useful for researchers who apply decision trees and rules to algorithm design and to data analysis, especially those working in rough set theory, test theory and logical analysis of data. This book can also be used as the basis for graduate courses.

This book introduces a fairly universal approach to the design and analysis of exact optimization algorithms for multi-objective combinatorial optimization problems. It proposes the circuits without repetitions representing the sets of feasible solutions along with the increasing and strictly increasing cost functions as a model for such problems. The book designs the algorithms for multi-stage and bi-criteria optimization and for counting the solutions in the framework of this model.As applications, this book studies eleven known combinatorial optimization problems: matrix chain multiplication, global sequence alignment, optimal paths in directed graphs, binary search trees, convex polygon triangulation, line breaking (text justification), one-dimensional clustering, optimal bitonic tour, segmented least squares, optimization of matchings in trees, and 0/1 knapsack problem.The results presented are useful for researchers in combinatorial optimization. This book is also useful as the basis for graduate courses.

The results presented here (including the assessment of a new tool – inhibitory trees) offer valuable tools for researchers in the areas of data mining, knowledge discovery, and machine learning, especially those whose work involves decision tables with many-valued decisions. The authors consider various examples of problems and corresponding decision tables with many-valued decisions, discuss the difference between decision and inhibitory trees and rules, and develop tools for their analysis and design. Applications include the study of totally optimal (optimal in relation to a number of criteria simultaneously) decision and inhibitory trees and rules; the comparison of greedy heuristics for tree and rule construction as single-criterion and bi-criteria optimization algorithms; and the development of a restricted multi-pruning approach used in classification and knowledge representation.

This book compares four parameters of problems in arbitrary information systems: complexity of problem representation and complexity of deterministic, nondeterministic, and strongly nondeterministic decision trees for problem solving. Deterministic decision trees are widely used as classifiers, as a means of knowledge representation, and as algorithms. Nondeterministic (strongly nondeterministic) decision trees can be interpreted as systems of true decision rules that cover all objects (objects from one decision class). This book develops tools for the study of decision trees, including bounds on complexity and algorithms for construction of decision trees for decision tables with many-valued decisions. It considers two approaches to the investigation of decision trees for problems in information systems: local, when decision trees can use only attributes from the problem representation; and global, when decision trees can use arbitrary attributes from the information system. For both approaches, it describes all possible types of relationships among the four parameters considered and discusses the algorithmic problems related to decision tree optimization. The results presented are useful for researchers who apply decision trees and rules to algorithm design and to data analysis, especially those working in rough set theory, test theory and logical analysis of data. This book can also be used as the basis for graduate courses.

The results presented here (including the assessment of a new tool – inhibitory trees) offer valuable tools for researchers in the areas of data mining, knowledge discovery, and machine learning, especially those whose work involves decision tables with many-valued decisions. The authors consider various examples of problems and corresponding decision tables with many-valued decisions, discuss the difference between decision and inhibitory trees and rules, and develop tools for their analysis and design. Applications include the study of totally optimal (optimal in relation to a number of criteria simultaneously) decision and inhibitory trees and rules; the comparison of greedy heuristics for tree and rule construction as single-criterion and bi-criteria optimization algorithms; and the development of a restricted multi-pruning approach used in classification and knowledge representation.

Dynamic programming is an efficient technique for solving optimization problems. It is based on breaking the initial problem down into simpler ones and solving these sub-problems, beginning with the simplest ones. A conventional dynamic programming algorithm returns an optimal object from a given set of objects. This book develops extensions of dynamic programming, enabling us to (i) describe the set of objects under consideration; (ii) perform a multi-stage optimization of objects relative to different criteria; (iii) count the number of optimal objects; (iv) find the set of Pareto optimal points for bi-criteria optimization problems; and (v) to study relationships between two criteria. It considers various applications, including optimization of decision trees and decision rule systems as algorithms for problem solving, as ways for knowledge representation, and as classifiers; optimization of element partition trees for rectangular meshes, which are used in finite element methods for solving PDEs; and multi-stage optimization for such classic combinatorial optimization problems as matrix chain multiplication, binary search trees, global sequence alignment, and shortest paths. The results presented are useful for researchers in combinatorial optimization, data mining, knowledge discovery, machine learning, and finite element methods, especially those working in rough set theory, test theory, logical analysis of data, and PDE solvers. This book can be used as the basis for graduate courses.

Dynamic programming is an efficient technique for solving optimization problems. It is based on breaking the initial problem down into simpler ones and solving these sub-problems, beginning with the simplest ones. A conventional dynamic programming algorithm returns an optimal object from a given set of objects. This book develops extensions of dynamic programming, enabling us to (i) describe the set of objects under consideration; (ii) perform a multi-stage optimization of objects relative to different criteria; (iii) count the number of optimal objects; (iv) find the set of Pareto optimal points for bi-criteria optimization problems; and (v) to study relationships between two criteria. It considers various applications, including optimization of decision trees and decision rule systems as algorithms for problem solving, as ways for knowledge representation, and as classifiers; optimization of element partition trees for rectangular meshes, which are used in finite element methods for solving PDEs; and multi-stage optimization for such classic combinatorial optimization problems as matrix chain multiplication, binary search trees, global sequence alignment, and shortest paths. The results presented are useful for researchers in combinatorial optimization, data mining, knowledge discovery, machine learning, and finite element methods, especially those working in rough set theory, test theory, logical analysis of data, and PDE solvers. This book can be used as the basis for graduate courses.

Decision trees and decision rule systems are widely used in different applicationsas algorithms for problem solving, as predictors, and as a way forknowledge representation. Reducts play key role in the problem of attribute(feature) selection. The aims of this book are (i) the consideration of the setsof decision trees, rules and reducts; (ii) study of relationships among theseobjects; (iii) design of algorithms for construction of trees, rules and reducts;and (iv) obtaining bounds on their complexity. Applications for supervisedmachine learning, discrete optimization, analysis of acyclic programs, faultdiagnosis, and pattern recognition are considered also. This is a mixture ofresearch monograph and lecture notes. It contains many unpublished results.However, proofs are carefully selected to be understandable for students.The results considered in this book can be useful for researchers in machinelearning, data mining and knowledge discovery, especially for those who areworking in rough set theory, test theory and logical analysis of data. The bookcan be used in the creation of courses for graduate students.

Decision trees and decision rule systems are widely used in different applicationsas algorithms for problem solving, as predictors, and as a way forknowledge representation. Reducts play key role in the problem of attribute(feature) selection. The aims of this book are (i) the consideration of the setsof decision trees, rules and reducts; (ii) study of relationships among theseobjects; (iii) design of algorithms for construction of trees, rules and reducts;and (iv) obtaining bounds on their complexity. Applications for supervisedmachine learning, discrete optimization, analysis of acyclic programs, faultdiagnosis, and pattern recognition are considered also. This is a mixture ofresearch monograph and lecture notes. It contains many unpublished results.However, proofs are carefully selected to be understandable for students.The results considered in this book can be useful for researchers in machinelearning, data mining and knowledge discovery, especially for those who areworking in rough set theory, test theory and logical analysis of data. The bookcan be used in the creation of courses for graduate students.