Anh Huy Phan

This monograph builds on Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions by discussing tensor network models for super-compressed higher-order representation of data/parameters and cost functions, together with an outline of their applications in machine learning and data analytics. A particular emphasis is on elucidating, through graphical illustrations, that by virtue of the underlying low-rank tensor approximations and sophisticated contractions of core tensors, tensor networks have the ability to perform distributed computations on otherwise prohibitively large volume of data/parameters, thereby alleviating the curse of dimensionality. The usefulness of this concept is illustrated over a number of applied areas, including generalized regression and classification, generalized eigenvalue decomposition and in the optimization of deep neural networks. The monograph focuses on tensor train (TT) and Hierarchical Tucker (HT) decompositions and their extensions, and on demonstrating the ability of tensor networks to provide scalable solutions for a variety of otherwise intractable largescale optimization problems. Tensor Networks for Dimensionality Reduction and Large-scale Optimization Parts 1 and 2 can be used as stand-alone texts, or together as a comprehensive review of the exciting field of low-rank tensor networks and tensor decompositions. See also: Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions.

Modern applications in engineering and data science are increasingly based on multidimensional data of exceedingly high volume, variety, and structural richness. However, standard machine learning and data mining algorithms typically scale exponentially with data volume and complexity of cross-modal couplings - the so called curse of dimensionality - which is prohibitive to the analysis of such large-scale, multi-modal and multi-relational datasets. Given that such data are often conveniently represented as multiway arrays or tensors, it is therefore timely and valuable for the multidisciplinary machine learning and data analytic communities to review tensor decompositions and tensor networks as emerging tools for dimensionality reduction and large scale optimization.This monograph provides a systematic and example-rich guide to the basic properties and applications of tensor network methodologies, and demonstrates their promise as a tool for the analysis of extreme-scale multidimensional data. It demonstrates the ability of tensor networks to provide linearly or even super-linearly, scalable solutions.The low-rank tensor network framework of analysis presented in this monograph is intended to both help demystify tensor decompositions for educational purposes and further empower practitioners with enhanced intuition and freedom in algorithmic design for the manifold applications. In addition, the material may be useful in lecture courses on large-scale machine learning and big data analytics, or indeed, as interesting reading for the intellectually curious and generally knowledgeable reader.

This book provides a broad survey of models and efficient algorithms for Nonnegative Matrix Factorization (NMF). This includes NMF’s various extensions and modifications, especially Nonnegative Tensor Factorizations (NTF) and Nonnegative Tucker Decompositions (NTD). NMF/NTF and their extensions are increasingly used as tools in signal and image processing, and data analysis, having garnered interest due to their capability to provide new insights and relevant information about the complex latent relationships in experimental data sets. It is suggested that NMF can provide meaningful components with physical interpretations; for example, in bioinformatics, NMF and its extensions have been successfully applied to gene expression, sequence analysis, the functional characterization of genes, clustering and text mining. As such, the authors focus on the algorithms that are most useful in practice, looking at the fastest, most robust, and suitable for large-scale models. Key features: Acts as a single source reference guide to NMF, collating information that is widely dispersed in current literature, including the authors’ own recently developed techniques in the subject area.Uses generalized cost functions such as Bregman, Alpha and Beta divergences, to present practical implementations of several types of robust algorithms, in particular Multiplicative, Alternating Least Squares, Projected Gradient and Quasi Newton algorithms.Provides a comparative analysis of the different methods in order to identify approximation error and complexity.Includes pseudo codes and optimized MATLAB source codes for almost all algorithms presented in the book. The increasing interest in nonnegative matrix and tensor factorizations, as well as decompositions and sparse representation of data, will ensure that this book is essential reading for engineers, scientists, researchers, industry practitioners and graduate students across signal and image processing; neuroscience; data mining and data analysis; computer science; bioinformatics; speech processing; biomedical engineering; and multimedia.