Motivated by the abundance and ubiquity of functionally modular systems in the real world, we propose a novel way of quantifying functional modularity of a system to analyze the source of functional modules. Because of its abundance, understanding how modular structures emerge can help us build better and more useful systems. Also, the ubiquity of modular structure should not be a coincidence because it is observed all over the places ranging from biological systems like humans to conceptual systems like computer programs. We want to find under what conditions such functional modules emerge. The Knockout Analysis described in this dissertation is a model independent method of measuring each component's importance and specialization. It allows us to quantify functional modularity of a system in terms of functional specialization of individual components. We find that modular structures are observed in almost all multitasking systems, except in extreme cases where the tasks are almost similar to each other. With a deeper investigation on the source of modular structures, we find that task dissimilarity plays a crucial role in formation of modules. Furthermore, functional modules emerge naturally from multitasking systems because specialized components are extremely more probable to form than multitasking components.