The advent of smart TVs/devices and advances in computer vision and sensing technologies facilitate automatic capture and real-time reporting of viewership. Sensors built into these devices can detect and recognize viewers and report face count (see Exhibit 13.3), enabling service providers such as Eyesight, Alphonso and Nielsen to report real-time audience metrics.

Some service providers also monitor the viewers’ attention based on head pose direction. It is possible too to detect viewers’ age and gender.

As regards the incorporation of facial coding, while it sounds feasible in the future, this capability is not provided by current service providers, perhaps because, to accurately code faces, requires the use of superior webcams and facial coding technologies.

The possibility though is exciting. When devices become capable of measuring audiences as well as their emotions and level of engagement, it will lead towards the convergence of audience measurement with engagement measurement.

Current reporting capabilities are illustrated in the Super Bowl Ad report by Alphonso. In addition to cross-channel tracking of active users depicted in Exhibit 13.3, the report incorporates metrics like “branded memorability” (loosely defined on a 1 to 5 point non-memorable to very memorable scale) obtained separately from quantitative research studies. It also provides visual depiction of TV watching households across the US, over hours.

Nielsen has the advantage of combining real-time viewing data from smart TVs with their traditional ratings data. Launched in April 2018, Nielsen viewership analytics platform, Grabix, started combining Nielsen ratings in top 56 TV US markets with real-time viewing data gathered from a pool of about 3 million smart TVs. This became feasible through the integration of Gracenote, which Nielsen acquired in late 2016.

Real-time audience measurement relies on automatic content recognition (ACR), an identification technology that recognizes content played on a media device. ACR uses the content’s “fingerprint”, i.e., a condensed digital summary of the sound/video recording, to identify the precise location within the content, over its duration. This software is increasingly integrated into smart TVs, cable set-top boxes and mobile devices such as smart phones and tablets.

The use of ACR technology to identify viewing habits of smart TV users raises legal issues. It is important therefore for service providers to explicitly seek users’ consent before turning on ACR functionality.

Besides audience measurement, ACR is used to enhance the viewers’ experience. Social media services use ACR to present relevant information to viewers, and to enable interactive features such as polls, coupons, lottery or the purchase of goods based on timestamp.