Abstract:

Abstract: The aim of this tutorial is to bring together signal processing engineers, computer and information scientists, applied mathematicians and statisticians, as well as systems engineers to carve out the role that analytical and experimental engineering has to play in Big Data research and development. This proposal will emphasize on signal analytics, networking, computation, optimization, as well as systems engineering aspects of Big Data. There are four main objectives. The first objective is to provide an introduction to the big data paradigm, from the signal processing perspective. The second objective is to introduce the key techniques to enable signal processing for big data in a comprehensive way. The third objective is to provide numerical datasets, and illuminate how signal processing approaches can be addressed to wireless datasets. The fourth objective is to present the state-of-the-art big data applications. This will include classifications of the different schemes and the technical details in each scheme.

Abstract:

Abstract:

Part of the power of neural networks comes from the fact that it is a very generic, almost "blind", tool to extract useful information directly from data. Unlike more conventional data analysis approaches, it does not assume any knowledge of the statistical model, the structure, relation or constraints in the data, but tries to use a universal network structure to learn and represent all types of models. On the other hand, if we do have some of such knowledge of the model, such as in communication systems and almost all other engineering systems, in principle we should be able to make the inference algorithms more efficiently by taking advantage of the domain knowledge. It is however not clear how the domain knowledge should be used in deep learning in general. This talk tries to address this issue. Conceptually, we need to identify what happens inside a neural network during the learning process, to find out what statistical quantities are being calculated and how are they stored inside a network. To that end, we formulate a new problem called the "universal feature selection" problem, where we need to select from the high dimensional data a low dimensional feature that can be used to solve, not one, but a family of inference problems. We solve this problem and show that 1) the solution is closely related to a number of concepts in information theory and statistics such as the HGR correlation and common information, and 2) a number of learning algorithms, PCA, CCA, Matrix Factorization, and Neural Networks, implicitly solve the same problem. We then demonstrate how such theoretical understanding of neural networks can help us to establish a performance limit and design network parameters more systematically; and to include specific domain knowledge in the design of new network structures.

Abstract: