Investigator: Stanley Kok.
Deep learning, statistical models and structured representations are cornerstones of artificial intelligence (AI). They each have their unique strengths and weaknesses, and separately, they have found numerous diverse applications. However, to date, these three pillars have largely not been integrated into a coherent whole that brings their unified strengths to bear on challenging real-world problems. Take for example the problem of thoroughly understanding the health status of a medical patient from various multi-modal sources (e.g., clinical images, medical sensors, nurses’ notes, and medical literature). On this problem, deep learning has the ability to compress images and text into vector-based representations to facilitate similarity comparisons among patients; statistical models can model the interactions of variables that affect a patient’s health; and structured representations (e.g., logic) can capture a doctor’s medical knowledge, hypotheses and intuitions. However, these three components function disparately, and each does not share its information with another, thereby only giving a partial view of a patient’s health. To better leverage the combined strengths of these components, we propose to integrate deep learning, statistical models and structured representations into a unified model.