MIT Professor’s Invention Might Help You Send the Right Signals
People who want to improve their communication skills may one day have an unusual helper: software programs that analyze the tone, turn-taking behavior and other qualities of a conversation.
The programs would then tell the speakers whether they tend to interrupt others, for example, or whether they dominate meetings with monologues, or appear inattentive when others are talking.
The inventor of this technology is Alex P. Pentland PhD ’82 of the Massachusetts Institute of Technology, who has developed cell phone-like gadgets to listen to people as they chat, and computer programs that sift through these conversational cadences, studying communication signals that lie beneath the words.
If commercialized, such tools could help users better handle many subtleties of face-to-face and group interactions — or at least stop hogging the show at committee meetings.
With the help of his students, Pentland, a professor of media arts and sciences at MIT, has been equipping people in banks, universities and other places with customized smartphones or thin badges packed with sensors that they wear for days or even months. As these people talk with one another, the sensors collect data on the timing, energy and variability of their speech.
Pentland, known as Sandy, calls his gleaning and processing of conversational and other data “reality mining — using data mining algorithms to parse the real-life, analog world of social interactions.”
The tools he has developed might help people change their communication tactics, including those that lead to unproductive workplace dynamics, said David Lazer, an associate professor of public policy at the Kennedy School of Government at Harvard.
Lazer praised “the richness of the data” captured by the process — the “minute-by-minute, fine-grained data on whether you are talking, whom you prefer to talk with, what your tone is, and if you interrupt, for instance.”
That kind of tool is rare, Laser said. “Our existing research tools for gathering this kind of data aren’t very good,” he said — for example, questionnaires in which people self-report on conversations. Reality mining may be more accurate, and has the potential to show “all sorts of interactive patterns that may not be obvious to individuals in an organization,” he said.
Many of Pentland’s research studies with smartphones and badges with embedded sensors are discussed in his new book, “Honest Signals,” recently published by MIT Press. The badges use tools including infrared sensors to tell when people are facing one another, accelerometers to record gestures, and microphones and audio signal-processing to capture the tone of voice.
With the array of sensors, the badges can detect what Dr. Pentland calls “honest signals, unconscious face-to-face signaling behavior” that suggest, for example, when people are active, energetic followers of what other people are saying, and when they are not. He argues that these underlying signals are often as important in communication as words and logic.
For example, the badges register when listeners respond with regular nods or short acknowledgments like, “Right.” Such responses, he argues, are a kind of mirroring behavior that may help build empathy between speaker and listener. He also examines patterns of turn-taking in conversations, as well as gestures and other, often unconscious signals.
Future smartphones that take advantage of his technology may act as friendly personal assistants, automatically putting through calls from friends and family, but sending all others straight through to voice mail.
“The phone can be like a butler who really gets to know you,” he said, by deciding to ring brightly for an urgent call when its owner has forgotten to turn on the ringer.
In the research, many steps are taken to make sure the identities of participants remain anonymous, said Anmol P.P. Madan G, a graduate student of Pentland. For instance, when microphone audio data is collected, the microphone picks up tone and the length of speaking time but does not record any of the actual words spoken.
So far, Madan has found that the data gathered by mobile phones is far more accurate than accounts of the same information reported by participants.
“Humans have a lot of bias when they recall their behavior,” he said.
Tanzeem K. Choudhury PhD ’04, a former student and collaborator of Pentland and now an assistant professor of computer science at Dartmouth, continues to do reality mining with smartphones.
“We spend a lot of time talking about how to improve communication skills,” she said. “This work lets us pin down what makes conversations effective by analyzing people’s actual conversation in their social networks.”