Categories
My Opinion

My Connected Device is Listening

My Android phones have been listening to me for years. I have no doubt about this. When I started talking about something absurd late last year – “can you use hand lotion to condition a leather chair” – and then decided to look it up on my phone, the first suggestion google makes upon entering “Can you use” was “Can you use hand lotion to condition leather products”. And that’s just one example.

People call me paranoid for saying this, but I’m not. I just understand (or at least pretend I understand) what’s going on inside our connected devices.

Speech Recognition in the Cloud

If you have an Android phone with the Google+ Launcher applied, try this: Turn it on and just say “OK, Google”. This automatically opens the voice to text search box where you can talk to the device and get it to do things like a search or send an email or whatever. I’m sure the Fruit Company phone can do the same thing. And the one from Macrohard. It’s amazingly unamazing in a world where rapid technological advancement has made us jaded.

What’s actually happening here is really quite amazing: The phone is constantly listening for specific voice queues, and when they are triggered it starts doing stuff. It gets even more impressive when you start dictating. You can actually see the phone guessing and correcting itself in real time as it does to make sure it gets everything exactly right. And watching this happen it’s clear there is a lot of contextual semantic processing going on out there in the cloud.

What we have here is the dream of speech recognition come true in the cloud. And now that we have it people are (and should be?) terrified.

Your Samsung TV is Eavesdropping on Your Private Conversations

Earlier this week the tech media and everyone else suddenly got very interested in connected devices and their listening capabilities. In the Terms of Service for Samsung’s Smart TVs a line was discovered that said:

Please be aware that if your spoken words include personal or other sensitive information, that information will be among the data captured and transmitted to a third party

The reporting was quickly followed by a statement from Samsung saying:

If a consumer consents and uses the voice recognition feature, voice data is provided to a third party during a requested voice command search. At that time, the voice data is sent to a server, which searches for the requested content then returns the desired content to the TV

George Orwell fans with better memories than me immediately caught on to the striking similarities from the book 1984:

And the world burned for days.

The Clash of Dreams and Reality

What we have here is a clash of dreams and reality. In our dreams we want to be able to talk to our devices and have them do our bidding. In Star Trek they had the Universal Translator. You can now get much the same feature by downloading the Google Translate app on your smartphone. Try it. It’s absolutely mind blowing. But for this technology to work we can’t just rely on our phones or computers or TVs. Language is complex and can’t be simplified to algorithms that can run on our local devices. For this technology to work we need the Cloud. And that means literally recording and sending your conversations over the web to a server that then parses the data, gleans the meaning of it, and acts according to your instructions.

In short, when you talk to your device your device needs to actually understand what you are saying. Which is why your Samsung Smart TV and your cell phone and probably your computer and any other connected device in your house is in fact listening to you all the time

The Question Isn’t If You are Being Recorded, But Who Listens

Most people will find this revelation rather unsettling, but the reality is this is not new. It’s been going on for years. And it’s a direct response to what we as consumers have been asking for.

The bigger question is who’s listening. The device and service companies are pretty much unanimous in saying they are not recording and not listening. The recordings are purely for the computers. And in a way it is probably believable (unless you are talking about that website that wants to sell your face on a book. They are totally listening). I don’t actually fear the companies (much), but I do question their encryption algorithms.

Post-Snowden we have confirmation of what many of us have known all along: If you put it on the Internet the US intelligence system will be listening in. So does this mean that someone is sitting in a bunker somewhere in the US (or elsewhere) listening to our conversations while we watch TV, or eat dinner, or chat with our phones within reach? Not unlikely.

Somewhere George Orwell is shaking his head in shame while dictating his next novel through his phone.

Categories
My Opinion

Analogue

Last November web design luminary Frank Chimero published a talk – turned – article about web design and user experience called “What Screens Want“. It’s a compelling and thought provoking piece on how we think of screens as design surfaces and how we need to break from the confines and rigid frames of traditional print design to reach the full potential of screen-targeted design and give screens what they want. If you work with screens and especially if you work under the broad umbrella called “web design” you’d be well served to read the article and use it as a starting point to reevaluate how you think of screens.

Sitting in my couch last night I felt compelled to take out my Moleskine and a pencil and sketch out what was directly in front of me. If you’re a regular follower of this site you may have noticed I’ve started hand illustrating my articles. This is a feeble effort on my part to rediscover the skill of drawing, something I like most others pretty much abandoned after secondary school. But the drawing you see above is is not nor was it ever meant to be an illustration for an article. It was more than anything the physical manifestation of a sudden realization I had about what screens actually want.

Screens want attention.

From my vantage point I had four live screens: A TV (out of frame), my old laptop (left), my smartphone (on top) and my new laptop (right). All on, all displaying different information, all screaming for my attention. Though this is not a normal scenario for me (I usually only have one laptop in front of me at a time) it made me realize my exposure to screens is bordering on permanence. When I wake up in the morning I check my phone for emails. When I eat breakfast I watch news on TV. When I work I stare at a screen. When I’m done work I read up on articles and interact with social connections through one of several screens. Then for some relaxation in the evening I watch TV or a movie, again on a screen.

The crazy thing is that it doesn’t end there: In my car there is a screen telling me if it’s currently using electric or gasoline power. Most bars and restaurants have screens showing some sort of sporting event. At the mall there are screens running ads. There are screens on the backs of every seat on most planes. The supermarket cashier has been replaced by a screen. I touch a screen to buy a ticket to use public transit, a screen tells me what classes are currently available at the gym, there is even a tiny screen inside the viewfinder of my camera.

The screens that surround us are attention vampires. When a screen is within your field of vision you can’t help but let your eyes drift towards it, even when what it displays is irrelevant or uninteresting to you. And once it has your attention it feeds off your focus, draining you. Their bold colors, quick movements, and hyper-realism trigger something in the primitive parts of our brains that make us pay attention.

As a web designer I am relying on this effect and I’ve learned to exploit it. I know how to make something appear on a screen in such a way that you just have to look at it. And I know that others are far better at it than I am. So good they can make you look at a screen and not notice what is happening around you in real life. So good they can make you believe in their reality and doubt your own experience. So good they can alter our perceptions of ourselves, of others, of our world.

When I was a kid my parents limited me to one hour of TV per day. “Watching too much TV makes you a fool” my mom would say. And she was right. TV does make you stupid unless you are very careful about what you watch and ask a lot of critical questions. But it’s not just the TV any more. All these other screens, there to give us information and enlighten us, have the same ability to make us dumb and disinterested and desensitized through information overload.

So maybe in addition to asking  what screens want we should also ask “what do we want from screens”.

Categories
Internet My Opinion

The Hyperbole and the Damage Done

There are many lessons to be learned from the Instagram TOS (Terms of Service) debacle that has been playing out on social media over the last two days. Chief among them is this:

The social web is not a good source of legal interpretation and factual information.

For all the greatness of the social web it has some very big flaws, one of which is that we are still wearing our newspaper goggles. What I mean by that is that we are still treating information provided to us from seemingly reliable sources as if that information is in fact reliable. This is a historical artefact from a time when news and information came to us from large news and publishing conglomerates with tight editorial guidelines and requirements for fact checking and source research. This is no longer the reality we live in. Most of the information you’ll find on the web is the exact opposite: Poorly researched, often incorrect, and largely based on non-expert opinion and wild speculation.

Such is the case of the interpretation of the new Instagram TOS. And the damage might be irreparable.

If you are not familiar with what happened, here is the gist of it:

The Hyperbole, a.k.a. “Instagram wants to steal your photos!!!!!!!!”

On Monday December 17 Instagram released their new Terms of Service agreement. Among the changes was a new sentence:

“To help us deliver interesting paid or sponsored content or promotions, you agree that a business or other entity may pay us to display your username, likeness, photos (along with any associated metadata), and/or actions you take, in connection with paid or sponsored content or promotions, without any compensation to you.”

This was widely interpreted as “Instagram reserves the right to take your photos, sell them for large piles of cash to a company you disapprove of, and have that company use them along with your name on billboard posters thereby robbing you of your copyright and earning money on your creativity.

Completely ridiculous. And incorrect.

The social web responded with hundreds of articles on how to bail from Instagram, what other services you can use instead of Instagram to post photos of your feet and food and friends, and how to delete your Instagram account forever so that they can never exploit you. And judging from reaction on the web, many people followed that advice.

The Reality, a.k.a. You Don’t Understand Legalese

Of course this interpretation was total rubbish. But it was also great fodder for the social web. Every gadget/tech/web blog wrote extensive articles on it, and everyone and their mother voiced their outrage over this vile injustice on Twitter, Facebook, Google+, and yes, on Instagram.

Then the people at The Verge took a step back and said “Hm. This doesn’t really make any sense. Why would Instagram commit social suicide like this? Maybe someone got something wrong.” (I’m assuming that’s the type of conversation that takes place at The Verge. I could be wrong.) They read the TOS again and found that not surprisingly the hyperbole was just that: Hyperbole. The reality was widely different. For that take read the excellent article aptly titled “No, Instagram can’t sell your photos: what the new terms of service really mean“. This was soon followed by “Instagram says ‘it’s not our intention to sell your photos’” which referred to this statement directly from Instagram.

For those of us who voiced caution about the hyperbole this comes as a vindication. For the many who instantly jumped on the band wagon and deleted their Instagram accounts, it is a sobering wake up call. For Instagram and all other online services with murky revenue models it is a rude awakening: Faced with complicated legalese, people trust anyone with a cool logo to be a legal expert and act on information obtained from said cool-logo-owning entity without checking the facts.

Be like a philosopher to avoid looking like an idiot …

One of the things you learn when you study philosophy is that before you make any judgement or take any action you should take a few steps back and look at the bigger picture. That means questioning whether your understanding is the correct one or even if you are equipped to understand what you are seeing. It means questioning the sources of your information. And most importantly it means stepping in the other party’s shoes and looking at it from their perspective. Few actions are committed without forethought, and before you make any final judgements or act on any apparent fact it is vital that you understand the reasoning behind what you see.

In the Instagram case the widespread interpretation of the TOS – the one that claimed Instagram would steal your photos and sell them to the highest bidder – only makes sense from the perspective of a paranoid person thinking everyone is out to get him. From a rational cool headed vantage point a few steps back there is obviously more to the story. But that isn’t what brings readers to the blogs and clicks on ads, so the hyperbole wins every time.

… or be like both Mulder and Scully

(Pardon the ridiculous and old pop culture reference here. I’m watching The X-Files on Netflix.) When it comes to information you read on the web you need to be both like Fox Mulder and like Dana Scully. Like Mulder you should trust no one, and like Scully you should assume there is always a logical explanation. That way you might avoid deleting your accounts only to realize you did it for no good reason and now you can’t get them back.

For an alternate take on the story, check out fellow Vancouverite Rob Cottingham’s piece
Terms of service changes deserve more than just a shrug and a click.