User privacy in a technological age

Arguably the most pervasive technology conspiracy is that your smartphone is listening to your private conversations.


Arguably the most pervasive technology conspiracy is that your smartphone is listening to your private conversations. We have all felt some discomfort with the types of advertisements that pop up on our phones. Have you ever had a conversation or searched for a certain product on your browser, only to start seeing ads that correspond exactly with that conversation or browser search? Do you ever get the distinct impression that your phone is tailoring pop-up and SMS ads for you and your specific interests, without you knowing how it got that information? And it is an “open secret” that your mobile device uses your personal information in order to tailor your user experience. But to what extent? And with whose permission?

Earlier this year, the Guardian published a report which revealed that Apple was secretly paying human contractors to listen to recordings of customers talking to its Siri digital assistant in order to improve the service. Siri is Apple’s voice-controlled personal assistant and it uses speech recognition and natural language processing technologies to process voice commands that operate the mobile device and its apps.

Although not explicitly disclosed in consumer-facing privacy documentation, it was discovered that a small proportion of Siri recordings were being passed onto Apple’s contractors around the world so that they could grade the responses based on various factors including whether the activation of Siri was deliberate or accidental, whether the query was within the range of what Siri could expect to assist with, and whether Siri’s response to the query was appropriate. As part of their job of providing quality control, Apple’s contractors were regularly privy to private material including confidential medical information, crimes being committed and couples having sexual intercourse.

In the aftermath of that report, Apple issued a formal apology for its privacy practices, noting its failure to “fully live up to its high ideals”. In its statement, Apple announced that it would suspend the grading program along with other changes including the discontinued retention of audio recordings of Siri interactions in favour of computer-generated transcripts; the option for users to opt into the use of audio samples of their requests, and if opted in, the deletion of any inadvertent triggers of the Siri assistant. However, Apple was just one of several major tech companies that were caught using paid human contractors to review recordings from its digital assistant.

Who else is listening in?

Google faced controversy earlier this year when a report by Belgian Public Broadcaster VRT NWS revealed that Google employees are systematically listening to audio files recorded by Google Home smart speakers and the Google Assistant smartphone app. While clearly stated in Google’s terms and conditions, not everyone is aware of the fact that everything you say to your Google smart speakers and Google Assistant is recorded and stored. What people don’t know —because it isn’t mentioned in its terms and conditions—is that Google employees can listen to excerpts from those recordings. In response to the report, Google admitted that it works with language experts worldwide to improve speech technology by making transcripts of a small number of audio files – about 0.2% of all audio fragments. The company added that these transcripts are not linked to any personal or identifiable information, but are “of crucial importance to develop technologies sustaining products such as the Google Assistant”.

Similarly, a report by Bloomberg in April this year revealed that Inc employs thousands of people around the world to help improve the Alexa digital assistant powering its line of Echo speakers.

The team listens to voice recordings captured in Echo owners’ homes which are then transcribed, annotated and fed the recordings back into the software. This work is done as part of an effort to eliminate gaps in Alexa’s understanding of human speech and to help it respond better to commands. According to Amazon’s website, no audio is stored unless Echo detects the wake word or is activated by pressing a button.

While Alexa does sometimes appear to become activated inadvertently, thus triggering transcription, Amazon maintains that it has strict technical and operational safeguards in place including that employees do not have direct access to information that can identify a person or account; the use of multi-factor authentication to restrict access; service encryption and audits of the control environment.

Facebook has been denying for years that it listens to user conversations in order to generate targeted ads. In early August 2019, Bloomberg News published a story showing how Facebook had contracted an external company to transcribe audio conversations conducted through the Facebook Messenger app.

This was done to test the accuracy of an automatic transcription algorithm that Facebook would be rolling out, and the company claimed that all of the users who opted in to the transcription service were aware of the potential human review system. However, Facebook maintains that it does not use your phone’s microphone to inform ads or to change what you see in your news feed, but instead shows ads based on people’s interests and other profile information.

Debunking the theory

Recently Wondera, a mobile cyber security company, conducted an experiment to test the “phone snooping” theory. The experiment was to place an iPhone and a Samsung Galaxy in a room, then play an audio loop of pet food ads for 30 minutes per day, over three days. The user permissions for a large number of apps were all enabled and the same experiment was performed in a silent room with the same phone to act as a control. The experiment was to see whether pet food ads would suddenly appear in any streams after the three days, and the phones were observed to track data consumption, battery use and background activity. The results were clear: no pet food ads showed up on any app following the test and there was no difference in data consumption, battery use and background activity between the audio room tests and the silent room tests. This was significant because had an app been accessing a microphone and sending the audio to a cloud server for analysis, there would have been notable traces of data consumption.

A disconcerting truth

In 2017 Jingjing Ren, a PhD student at Northeastern University, and Elleen Pan, an undergraduate student, designed a study to investigate whether smartphones were eavesdropping on conversations without users’ knowledge. The study quickly showed that there were no audio leaks as not a single app had activated the microphone.

However, the team started to note unexpected things. Apps were automatically taking screenshots of themselves and sending them to third parties. In one case, an app took a video of the screen activity and sent that information to a third party. Out of over 17 000 Android apps examined, over 9 000 had potential permissions to take screenshots. And a number of them were actively doing so.

Though your phone may not be spying on you by randomly taking useless pictures or recording pointless conversations, it has the capacity to track you in many other ways. And it is this trackable data that enables companies to serve you targeted ads that appear to be unnervingly accurate.

The reality is that everything that makes your phone useful, like knowing where you are in order for Waze or Google Maps to operate, taking photos, enabling online shopping or banking, is exactly where all of the vulnerabilities are.

The more useful your smartphone is to you, the more data you will provide that entices opportunistic third parties such as advertisers and hackers.

The reality is that companies do not need to listen to your conversations because they have tons of data on you, they can tell where you are at any given moment, who your friends are and what they are interested in, track you on all your devices, and even watch you type and delete something that you never actually send. All of this data can be processed by an algorithm that can then predict your current needs and tailor ads for you at the exact moment that you need them. 

comments powered by Disqus

This edition

Issue 414


Leadership_Mag JJ's View - The rise and fall of leadership in the 26 years of democracy. #JJsview #leadership #may2020 Read on pag… 13 days - reply - retweet - favorite

Leadership_Mag Leadership May 2020 Cover Story - The future of South Africa post the Global Pandemic. Read it in our new interacti… 13 days - reply - retweet - favorite

Leadership_Mag The interactive version of the May 2020 #Leadership magazine is available now. 17 days - reply - retweet - favorite