Bluefin Labs: Tracking TV and Social Media Connections in Real-Time. Report from TED

Originally published: 03/07/2011

The Bluefin Labs system tracks television content, marries it to what's being said in social media streams and then creates a link between the two.

Deb Roy's TED TALK last week focused on three-years of almost non-stop video recording in every room in his home to track and profile his newborn son's learning influences, patterns and behavior. But the ultimate outcome of Roy's work is one of the most promising new research tools for gaining insights into how audiences are responding in real-time to TV ads and the programs they run in. Developed by Roy and his team at the MIT Media Lab, sophisticated tracking models are now deployed to identify – in real time -- the intersection of social media with the second-by-second content streams of 30 television networks. Roy's Bluefin Labs (www.bluefinlabs.com) has developed machine learning technology that provides brands, agencies and media companies with insights on the direct, real-time correlation between the shows and ads audiences are watching on TV and what those audiences are saying about that content via social media streams.

"We sift through the world's social media streams in order to identify and analyze comments about TV shows and ads," says Deb Roy, Bluefin Labs co-founder and machine learning and artificial intelligence expert. The system tracks television content, marries it to what's being said in social media streams and then creates a link between the two.

Check out Jack Myers' Curated TED Talks Recommendations.

Deb Roy is a leading expert on the study of how children learn language. But his work has led him on a path to become one of the leading experts in how audience behavior, interests and emotions are impacted by the TV programs and commercials they view. In the next few weeks Roy's TED Talk will be posted for public viewing at www.TED.com. It's is must viewing for all those engaged in media, marketing and entertainment. Bluefin Labs' white paper, Mass Media, Social Media & the Semantic Barrier, can be downloaded free here. The thesis of the White Paper follows:

The explosive rise of social media threatens to fundamentally disrupt current mass media marketing practices. To successfully build and maintain brands in the face of these new social forces, organizations must formulate new communication strategies that tightly integrate real time insights mined from social media into their use of mass media – which still accounts for a disproportionate share of their marketing and communications spending. Companies that approach social media in a silo isolated from their use of mass media risk spending massive amounts of money inefficiently while losing control of their brands. The semantic barrier is identified as a key problem that prevents organizations from effectively and efficiently integrating social and mass media. Drawing from 15 years of MIT research into the problem, an approach to breaking through the semantic barrier is outlined.

Bluefin Labs is a developer of Artificial Intelligence, which the company argues is poised to be a significant contributor to "innovating the customer experience" thanks to "a flood of information on how people communicate and connect with one another that can be harnessed to create a new breed of AI that assimilates human interests and experience into hybrid forms of human-machine intelligence. Google's search algorithm, which is grounded in human preferences and judgments, is an early example of the transformative effect of human-machine intelligence."

Bluefin claims their research reveals "unprecedented new levels of insight into human behavioral and social patterns." Predictive models grounded in observational video will enable better understanding of not only marketing and advertising influences, but also support marketers' and programmers' need to anticipate the needs and desires of customers and audiences at the individual level. The research has the potential to inform decision-making for TV network and online content development, advertising creative, media placement and cross-platform integration.

Merged with set-top box data being packaged by TRA Global, TiVo, Simulmedia and other providers, the Bluefin Labs AI could radically alter the media and marketing research landscape.

Roy believes "we are in a position to reveal sweeping new views into the human condition that will leave few areas of business and government untouched. From genome sequences and brain images to longitudinal video and cell phone traces, new data about people are revealing insights into how we develop, think, behave, and interact. At the same time, billions of people have come online and are expressing themselves in forms that simply did not exist just ten years ago – from blogposts and podcasts to YouTube videos and twitterfeeds. By analyzing and cross-pollinating these rich streams of human data, our ability to understand, predict, and effect human behavior will be expanded dramatically with deep implications for health, finance, retail, government, and beyond."

A related TED Talk Worth Spreading is by Aaron Koblin www.aaronkoblin.com, a data artist who takes community generated data and uses it to reflect on cultural trends and the changing relationship between humans and technology.

NOTICE: You are receiving this report as a corporate subscriber to Jack Myers Media Business Report. Redistribution is restricted to authorized individuals within your company only.