Smart TVs use image recognition to identify what is on your screen, regardless of input source, in order to collect data and target ads across devices, according to The New York Times. Sony, Sharp, TCL and Philips use Samba TV for this purpose.
Your TV knows what you’re watching
Streaming providers know exactly what you are watching on their respective services. They use this data as a basis for business decisions, statistics, and sometimes to target ads on the service.
Systems built deep into modern Smart TVs, however, are far more sophisticated. They use so-called ACR (Automatic Content Recognition) to do real-time image recognition. This allows them to track everything you are watching, regardless of the input source being an app, a TV channel, a console game, or a vacation photo.
Samba Interactive TV is one of the key players on the market. The New York Times reports that Samba TV is currently integrated in TVs from Sony, Sharp, TCL, and Philips, amongst others. In the US alone, they can collect usage data from 13.5 million Smart TVs but the system has also been deployed in other regions, including Europe. The company itself confirms that its technology is also found in TVs from Toshiba, Grundig, Beko, Arcelik, AOC, Westinghouse, Element, Magnavox, Seiki, and Sanyo.
Besides the tracking practices, several users have demonstrated how Samba TV negatively affects performance on Sony Android TVs. It simply makes the user interface noticeably more sluggish to navigate.
Samba TV is opt-in during setup of the TV, the company pointed out. On for example a Sony Android TV the user is presented with a screen to explain that Samba TV collects data in order to offer personalized recommendations.
The New York Times presents a different version of the truth. It details how the system initially creates a “content ID” for the viewer or household that is used to create a “device map” for all associated digital devices. In this way Samba TV can track 1 billion devices with the Smart TV as the “master key”. The method can be used to map behavior, habits, preferences, and more. The data can also be used to target ads across devices and The New York Times claims that Samba TV has “even offered advertisers the ability to base their targeting on whether people watch conservative or liberal media outlets”.
Samba TV is not alone
Samba TV is not alone in using ACR technology. Analyst company Nielsen has acquired Gracenote, which supplies programming data used in the TV guide on many Smart TVs. Through Gracenote, Nielsen is implementing its so-called Grabix system that collects statistical data using ACR technology.
The New York Times says that Inscape and Alphonso have employed similar systems.
Inscape is best known as Vizio’s partner in the US case where Vizio paid $2.2 million to settle with FTC because it had been collecting data without users’ consent.
Alphonso has previously been named in a case where mainly Android smartphones would use the built-in microphone to record audio from your living room TV, and use audio recognition to identify what you are watching.
But what do the Smart TV manufacturers stand to gain? They receive payment to include this kind of software, according to NYT.