Endemol Shine Group automates its TV production of Big Brother with machine learning solution
Endemol Shine Group, creators of Big Brother, wanted to refresh the TV showâs brand and streamline its production. The company implemented a cloud-based serverless workflow solution powered by Microsoft Azure Media Analytics. It is built on the Azure Media Services platform and applies machine learning in order to index video files by using speech-to-text, face recognition, and related technology. This avoids a tedious manual review of hours of raw footage, speeds the production process, and frees up time for more creative development.
"In seconds, weâre able to locate video captured at, say, 1:10 AM that reflects exactly what we want to show on the program, even if no one was watching at the time. Thatâs a big savings on resources."
- Lisa Perrin: Chief Executive Officer of Creative Networks Endemol Shine Group
Eight hundred productions in 78 territories, airing on more than 275 channels around the worldâthat is the reach of global production powerhouse Endemol Shine Group (ESG). From Big Brother and MasterChef to Fear Factor and The Biggest Loser, the companyâs wildly popular programs entertain more than 250 million viewers and generate 2 billion views a month online.
Keeping the media propertyâs brand fresh
ESG devotes a lot of time, effort, and, increasingly, technology to keep its programming fresh, relevant, and a hot topic of the viewing publicâs conversation. For example, Big Brother follows a group of strangers brought together in one house outfitted with dozens of cameras and microphones that record every move 24 hours a day, seven days a week.
Lisa Perrin, Chief Executive Officer of Creative Networks at Endemol Shine Group, says, âAfter 18 years, it was time to refresh the Big Brother brand, make it look better, and keep it thriving across the world.â
Embracing innovative production ideas
As part of the creative refresh, ESG came up with sweeping ideas to improve production of the Spain edition of Big Brother, which would serve as a model for other editions. First, the company sought to implement a cloud-based production workflow system. âBy using cloud computing, we can shoot in Madrid, for instance, and edit in London or LAâwherever we want to be based,â says Perrin. The cloud is also ideal for Internet of Things (IoT) data collectionâin this case, data from wearable sensors. âWe could track peopleâs heart rates to hypothesize that there might be a love interest in the room,â she adds. âThatâs an example of a real change-up to offer more-exciting content to our broadcasters.â Additionally, ESG wanted to automate much of the selection of scenes, from hours of multicamera footage, to use in the published program.
Deploying the right technology
To make innovations like these possible, ESG joined forces with Microsoft in April 2017 to develop a production workflow solution powered by Microsoft Azure Media Analyticsâa collection of speech and vision components that analyze video files to provide actionable information through machine learning. Azure Media Analytics is hosted on the Azure Media Services platform, which includes technology to encode, encrypt, and stream audio and video at scale, live or on demand.
The ESG and Microsoft teams developed the back-end system that processes the live streams. It takes advantage of live channels (content pipelines that work with signal encoders for different distribution methods) and subclipping (a feature for dividing long footage into smaller, more manageable segments). Subclips are generated with Azure Media Encoder (Standard and Premium versions) using multiple formats for both the internet and broadcast video editing systems. And it delivers metadata (derived from speech-to-text, face detection, and motion detection) to Azure Cosmos DB, the globally distributed multimodel database service. All these components are orchestrated by Azure Logic Apps and Azure Functions. These serverless workflows run 24/7 and can be easily monitored and modified from the Azure portal.
Based on the solutionâs success so far, ESG plans to deploy it on a wider scale. Perrin says, âBy the end of 2018, we hope to have it rolled out in many of our Big Brother territories.â
Automating scene selection
The new solution replaces an entirely manual selection process in which production assistants identify the most important scenes to use in a Big Brother episode. Through machine learning, the system recognizes patterns of language, keywords, and emotional reactions. It tracks, monitors, and indexes the activities of a houseâs residents and infers what relationships they have. âWhen youâre filming around the clock, there can be hours of nothing happening. Watching screens for so many feeds and arduously logging moments is very tedious,â says Perrin. âNow, with Azure Media Analytics, we zero in on the most interesting actions rather than wading through hours of footage.â Specifically, the software indexes probable actions of interest. When humans search for various types of events that might be valuable to the showâs storyline, such as a fight between two characters, the software returns time stamps that correspond to events that match the search criteria. âIn seconds, weâre able to locate video captured at, say, 1:10 AM that reflects exactly what we want to show on the program, even if no one was watching at the time. Thatâs a big savings on resources.â
Perrin concludes, âAdopting this ground-breaking technology will completely revolutionize the way we produce our global formats and opens up an unprecedented level of creative freedom. In terms of time and efficiency, weâve really upped our game with Azure Media Analytics.â
Find out more about Endemol Shine Group on Twitter and LinkedIn.
"By using cloud computing, we can shoot in Madrid, for instance, and edit in London or LAâwherever we want to be based."
- Lisa Perrin: Chief Executive Officer of Creative Networks Endemol Shine Group
https://customers.microsoft.com/en-us/story/esg-media-telecommunications-azure