Cognitive Psychology Meets Art
February 9, 2015, Electronic Imaging Conference, San Francisco—Monica Lopez-Garcia from La Petite Noisuse Productions is studying the interactions of creativity, language, and emotion in film and theater. The availability of computers and other supporting technologies is helping scientists by adding words to their vocabulary and improve methodologies.
The new science is giving insights into visualization and interdisciplinary efforts are exposing new links in looking at creativity. The increased volumes of data, although unstructured and mostly audio video, clearly show that computers cannot match humans for creativity. At the same time, the work shows that there is a high degree of interrelationship between science, media, aesthetics, and technology.
The cross-domain interaction of science and philosophy for feelings have been considered for a very long time. In 1495, Leonardo da Vinci designed a mechanical knight the showed some features of entanglement of mind and body. The problem for artificial intelligence is that the computers are missing pieces for the how-when-why for problem solving and emotions.
Creativity and innovation are necessary because programmers cannot all of the types and evolution of new challenges. In addition, the machines cannot get to the why for a problem due to issues with language, new concepts, and others. For example, an artist is the epitome of creativity, but no model exists for their internal processes. Even general problem solving requires a creative cognition approach. Humans reason, think, plan and communicate to solve problems.
Divergent production calls for various cognitive states: spontaneous for emotion and cognition, and intentional. A spontaneous emotion is an epiphany while an intentional emotion is an “ah ha” moment. Spontaneous cognitive functions recognize discovery while intentional enable the discovery through sustained focus.
By definition, improvisation is spontaneous and is most easily seen in music forms. Researchers are trying to map brain functions to see if there is commonality or domain specificity for language and music. Current models of brain function indicate that both functions are most active in the frontal lobes, but the available tools are too coarse and invasive. How creative are you going to be inside of an MRI machine?
A set of studies with wearable EEG sensors looked at professional musicians to evaluate three facets of improvisation; the creation of new material, the perception of new material, and the communications of new materials. The results showed an increase in sensations and brain cortical activity when the musicians were “in the zone”.
The results didn’t show how the musicians integrated the new concepts, nor did they improve the mapping to specific areas of the brain. The resulting model shows dynamic searches and exchanges and high levels of mental computation. In addition, the musicians were influenced by the audience as the artists were creating their art.
Comparing and contrasting art and science show some significant differences. Science is about reductionism and control, numbers and quantization, and statistics while art favors openness and spontaneity with the context and narrative being significant contributors. As a result, the goal for the scientist is to move from patterns of behavior to models of creation. Art-based cognitive research needs to merge disciplines, methods, and data to find how spontaneity and creativity lead to emotions.
When people look at photographs, they search for meaning. In general, they use facial expressions and framing as a guide for their emotional responses. The growth of A-V media can have an even greater effect on the viewers’ responses and add physiological and scene imagery plus sounds and music to influence the emotions.
One study with photographs addressed 6 universal human emotions; sadness, surprise, disgust, fear, happiness, and anger. The various pictures were assembled into a movie and some professional musicians improvised a sound track. For sadness, the saxophone played a lament while the piano played some high arpeggios and the bass played a slow, minor rhythm like a pulse or breathing. In comparison, the happiness set was rolling and up-tempo in a major key that evolved into a fast blues. The musicians translated the scene and emotion into music within an group communication on tempo, key, and rhythms.
In another example, they considered improvisational music in a theater setting, where dialog and lighting are additional parts of the presentation. Once again, the musicians changed tempo and genre to follow the action and dialog. The interaction of the acting and music both contributed to the overall intent to influence the audience’s emotions. The musicians changed tempo, level, and rhythm to map to the actors in the theater.
The visual and linguistic functions set the time, place, mood, and context for the music, while the improvised music acted as an integrated feedback loop to change key, mode, pitch, rhythm, and tempo. The musicians seemed to use incoming ideas and past experiences for their creation. The link between emotion and music has many implications related to how the music will effect listeners, and what they are doing.
The studies showed that emotion is one key to creativity. The intersection of computers and people enables more to use music to express emotion, and music seems to map to distinct physiological changes.
