Written by: Lee Wee Sun, Oct 2019
Deep learning methods tend to use distributed representations where information is distributed throughout a network. In contrast, traditional engineering methods tend to decompose systems into modules, clearly separated by interfaces in a way where information can only pass from module to module through the interfaces. How do they compare? What is the role of modularity in deep learning?