Andrzej Cichocki

This monograph builds on Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions by discussing tensor network models for super-compressed higher-order representation of data/parameters and cost functions, together with an outline of their applications in machine learning and data analytics. A particular emphasis is on elucidating, through graphical illustrations, that by virtue of the underlying low-rank tensor approximations and sophisticated contractions of core tensors, tensor networks have the ability to perform distributed computations on otherwise prohibitively large volume of data/parameters, thereby alleviating the curse of dimensionality. The usefulness of this concept is illustrated over a number of applied areas, including generalized regression and classification, generalized eigenvalue decomposition and in the optimization of deep neural networks. The monograph focuses on tensor train (TT) and Hierarchical Tucker (HT) decompositions and their extensions, and on demonstrating the ability of tensor networks to provide scalable solutions for a variety of otherwise intractable largescale optimization problems. Tensor Networks for Dimensionality Reduction and Large-scale Optimization Parts 1 and 2 can be used as stand-alone texts, or together as a comprehensive review of the exciting field of low-rank tensor networks and tensor decompositions. See also: Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions.

Modern applications in engineering and data science are increasingly based on multidimensional data of exceedingly high volume, variety, and structural richness. However, standard machine learning and data mining algorithms typically scale exponentially with data volume and complexity of cross-modal couplings - the so called curse of dimensionality - which is prohibitive to the analysis of such large-scale, multi-modal and multi-relational datasets. Given that such data are often conveniently represented as multiway arrays or tensors, it is therefore timely and valuable for the multidisciplinary machine learning and data analytic communities to review tensor decompositions and tensor networks as emerging tools for dimensionality reduction and large scale optimization.This monograph provides a systematic and example-rich guide to the basic properties and applications of tensor network methodologies, and demonstrates their promise as a tool for the analysis of extreme-scale multidimensional data. It demonstrates the ability of tensor networks to provide linearly or even super-linearly, scalable solutions.The low-rank tensor network framework of analysis presented in this monograph is intended to both help demystify tensor decompositions for educational purposes and further empower practitioners with enhanced intuition and freedom in algorithmic design for the manifold applications. In addition, the material may be useful in lecture courses on large-scale machine learning and big data analytics, or indeed, as interesting reading for the intellectually curious and generally knowledgeable reader.

Based on the results of the study carried out in 1996 to investigate the state of the art of workflow and process technology, MCC initiated the Collaboration Management Infrastructure (CMI) research project to develop innovative agent-based process technology that can support the process requirements of dynamically changing organizations and the requirements of nomadic computing. With a research focus on the flow of interaction among people and software agents representing people, the project deliverables will include a scalable, heterogeneous, ubiquitous and nomadic infrastructure for business processes. The resulting technology is being tested in applications that stress an intensive mobile collaboration among people as part of large, evolving business processes. Workflow and Process Automation: Concepts and Technology provides an overview of the problems and issues related to process and workflow technology, and in particular to definition and analysis of processes and workflows, and execution of their instances. The need for a transactional workflow model is discussed and a spectrum of related transaction models is covered in detail. A plethora of influential projects in workflow and process automation is summarized. The projects are drawn from both academia and industry. The monograph also provides a short overview of the most popular workflow management products, and the state of the workflow industry in general. Workflow and Process Automation: Concepts and Technology offers a road map through the shortcomings of existing solutions of process improvement by people with daily first-hand experience, and is suitable as a secondary text for graduate-level courses on workflow and process automation, and as a reference for practitioners in industry.

The purpose of this book is to present analysis and design principles, procedures and techniques of analog integrated circuits which are to be implemented in MOS (metal oxide semiconductor) technology. MOS technology is becoming dominant in the realization of digital systems, and its use for analog circuits opens new pos­ sibilities for the design of complex mixed analog/digital VLSI (very large scale in­ tegration) chips. Although we are focusing attention in this book principally on circuits and systems which can be implemented in CMOS technology, many con­ siderations and structures are of a general nature and can be adapted to other promising and emerging technologies, namely GaAs (Gallium Arsenide) and BI­ MOS (bipolar MOS, i. e. circuits which combine both bipolar and CMOS devices) technology. Moreover, some of the structures and circuits described in this book can also be useful without integration. In this book we describe two large classes of analog integrated circuits: • switched capacitor (SC) networks, • continuous-time CMOS (unswitched) circuits. SC networks are sampled-data systems in which electric charges are transferred from one point to another at regular discrete intervals of time and thus the signal samples are stored and processed. Other circuits belonging to this class of sampled-data systems are charge transfer devices (CTD) and charge coupled dev­ ices (CCD). In contrast to SC circuits, continuous-time CMOS circuits operate continuously in time. They can be considered as subcircuits or building blocks (e. g.

This book provides a broad survey of models and efficient algorithms for Nonnegative Matrix Factorization (NMF). This includes NMF’s various extensions and modifications, especially Nonnegative Tensor Factorizations (NTF) and Nonnegative Tucker Decompositions (NTD). NMF/NTF and their extensions are increasingly used as tools in signal and image processing, and data analysis, having garnered interest due to their capability to provide new insights and relevant information about the complex latent relationships in experimental data sets. It is suggested that NMF can provide meaningful components with physical interpretations; for example, in bioinformatics, NMF and its extensions have been successfully applied to gene expression, sequence analysis, the functional characterization of genes, clustering and text mining. As such, the authors focus on the algorithms that are most useful in practice, looking at the fastest, most robust, and suitable for large-scale models. Key features: Acts as a single source reference guide to NMF, collating information that is widely dispersed in current literature, including the authors’ own recently developed techniques in the subject area.Uses generalized cost functions such as Bregman, Alpha and Beta divergences, to present practical implementations of several types of robust algorithms, in particular Multiplicative, Alternating Least Squares, Projected Gradient and Quasi Newton algorithms.Provides a comparative analysis of the different methods in order to identify approximation error and complexity.Includes pseudo codes and optimized MATLAB source codes for almost all algorithms presented in the book. The increasing interest in nonnegative matrix and tensor factorizations, as well as decompositions and sparse representation of data, will ensure that this book is essential reading for engineers, scientists, researchers, industry practitioners and graduate students across signal and image processing; neuroscience; data mining and data analysis; computer science; bioinformatics; speech processing; biomedical engineering; and multimedia.

With solid theoretical foundations and numerous potential applications, Blind Signal Processing (BSP) is one of the hottest emerging areas in Signal Processing. This volume unifies and extends the theories of adaptive blind signal and image processing and provides practical and efficient algorithms for blind source separation, Independent, Principal, Minor Component Analysis, and Multichannel Blind Deconvolution (MBD) and Equalization. Containing over 1400 references and mathematical expressions Adaptive Blind Signal and Image Processing delivers an unprecedented collection of useful techniques for adaptive blind signal/image separation, extraction, decomposition and filtering of multi-variable signals and data. * Offers a broad coverage of blind signal processing techniques and algorithms both from a theoretical and practical point of view * Presents more than 50 simple algorithms that can be easily modified to suit the reader's specific real world problems * Provides a guide to fundamental mathematics of multi-input, multi-output and multi-sensory systems * Includes illustrative worked examples, computer simulations, tables, detailed graphs and conceptual models within self contained chapters to assist self study * Accompanying CD-ROM features an electronic, interactive version of the book with fully coloured figures and text. C and MATLAB® user-friendly software packages are also provided MATLAB® is a registered trademark of The MathWorks, Inc. By providing a detailed introduction to BSP, as well as presenting new results and recent developments, this informative and inspiring work will appeal to researchers, postgraduate students, engineers and scientists working in biomedical engineering, communications, electronics, computer science, optimisations, finance, geophysics and neural networks.

Based on the results of the study carried out in 1996 to investigate the state of the art of workflow and process technology, MCC initiated the Collaboration Management Infrastructure (CMI) research project to develop innovative agent-based process technology that can support the process requirements of dynamically changing organizations and the requirements of nomadic computing. With a research focus on the flow of interaction among people and software agents representing people, the project deliverables will include a scalable, heterogeneous, ubiquitous and nomadic infrastructure for business processes. The resulting technology is being tested in applications that stress an intensive mobile collaboration among people as part of large, evolving business processes. Workflow and Process Automation: Concepts and Technology provides an overview of the problems and issues related to process and workflow technology, and in particular to definition and analysis of processes and workflows, and execution of their instances. The need for a transactional workflow model is discussed and a spectrum of related transaction models is covered in detail. A plethora of influential projects in workflow and process automation is summarized. The projects are drawn from both academia and industry. The monograph also provides a short overview of the most popular workflow management products, and the state of the workflow industry in general. Workflow and Process Automation: Concepts and Technology offers a road map through the shortcomings of existing solutions of process improvement by people with daily first-hand experience, and is suitable as a secondary text for graduate-level courses on workflow and process automation, and as a reference for practitioners in industry.

A topical introduction on the ability of artificial neural networks to not only solve on-line a wide range of optimization problems but also to create new techniques and architectures. Provides in-depth coverage of mathematical modeling along with illustrative computer simulation results.