Kazu Ghalamkari
Purpose:
The drastic spread of machine learning techniques has made it common to obtain scientific knowledge by analyzing data in various fields. Yet, the non-convexity and instability of many machine learning models leads to important issues of reproducibility with results differing from changes to initializations. Within data science and the modeling of biological data sets non-negative tensor networks (NTN) have become a prominent tool providing part based and interpretable representations of large and complex datasets. However, NTN faces issues of non-convexity, leading to instability of applications. This project will introduce an enhanced framework for NTN that provides convexity, expressiveness, and robustness, aiming to mitigate the instability of downstream tasks across various domains reliant on tensors.
Method:
To eliminate the non-convexity of tensor decomposition, I recently developed a novel framework, many-body approximation (MBA) [1], that globally minimizes error. MBA regards tensor modes (axes) as particles and learns the interactions among them with appropriately designed energy functions. Although the convexity of MBA potentially eliminates instability in the variety of downstream tasks, so far, MBA is explored “vanilla” and is currently unsuited to solve specific tasks whereas it remains unclear what makes MBA a convex optimization problem. To make MBA practical, I will discover the essence of convexity, improve complexity, and introduce certainty through noise robustness and prior distributions to the model. Specifically, I will advance MBA to
• Accommodate symmetry, which plays a central role in link prediction in a graph [2] via properly re-designed energy function. Include new state of particles that enables data fusion, where multiple tensors are analyzed at once.
• Expand its complexity by exploring the analogy between MBA and Boltzmann machines [3] and further introducing hidden variables and interference effects [4].
• Exploit prior distributions in the representation and perform noise-robust modeling without loss of convexity.
Significance of the Project:
This project develops an advanced tensor factorization that is convex, expressive, and robust. This can potentially lead to a paradigm shift towards solid data analysis to eliminate initial value dependency from downstream tasks of tensor decomposition and provide stable computational tools for the data science communities reliant on tensors.
This project develops an advanced tensor factorization that is convex, expressive, and robust. This can potentially lead to a paradigm shift towards solid data analysis to eliminate initial value dependency from downstream tasks of tensor decomposition and provide stable computational tools for the data science communities reliant on tensors.