As featured in this ipXchange thought piece. Written by Dr. Eamon Standing.
Picture this: you’re sitting in your kitchen watching your smart television, but you’re not watching the regularly scheduled programming, you’re watching a gaming speedrun on YouTube while preparing your favourite meal – it’s nice to have something on in the background.
Your phone is sitting on the counter in front of you, and it starts to ring. “Alexa, pause”, you say – your hands are too dirty to use the remote control – and you tap your chin on your smartwatch to take the call from there.
It’s Grandma, and her washing machine is having troubles again. You tell her that you’ll look into getting her a new model when you have a moment – you’re happy with yours, it even texts you when your load is done. But you don’t recommend that one; it might be too complicated for Grandma, even though your family just got her the latest smart phone.
The conversation ends, and you resume your YouTube content with a quick, “Alexa, play”. You’re feeling good. Good meal ahead. Good deed for Grandma. Good video with some nice cuts to world record times. Then comes the ad break, and it’s not a skippable one.
You’re a little annoyed, but it’ll pass, then your blood turns cold. It’s an ad for washing machines. Not only that, these aren’t even the top models. They’re a bit too simple for your taste, but they’re within your budget, they’re perfect for Grandma, and you can get one from the shop in town.
You look around the room at all the smart machines that surround you and start to wonder: which one of these devices was listening to that phone call?
Hello, ipXchange community. It’s your usual host Dr. Eamon Standing, but unlike most of the time that I appear on camera – usually at shows, in the intros and outros to videos, or weekly roundups of our content – this time, I’m reading from a script. You may be wondering why…
Well, at the start of 2024, ipXchange decided to go full force into the world of video content – we believe it’s what sets us apart in the world of electronics news, aside from our focus on disruptive technology – but I love to write, so we thought it was time for a new type of ipXchange content.
In lieu of my usual off-the-top-of-the-head rants regarding technology, once a month, I’ll be doing one of these: a high-concept thought piece about a particular issue or phenomenon in the industry, and what better way to start than with security versus privacy in the IoT, or more accurately these days, the AIoT.
Have you ever felt that your tech was listening to you, showing you ads before you’ve even Googled the product? We’ve had many conversations about this in the ipXchange office, and we’ve even talked to leading innovators in the field of smart-home assistants, who have assured us that there’s no code in these devices that is enabling it to listen to you and use your private conversations to sell you things, or worse.
In the dinner preparation story, there were a few points of vulnerability where one might think information was being leaked to third parties, eventually resulting in the advertisement on YouTube. Let’s highlight all the pieces of technology mentioned in this miniature sci-fi horror story:
The smart TV – it’s connected to the internet and interfaced with Alexa, whether that’s in the device itself or in a separate hardware product. The YouTube app is likely linked to a Google account.
The smart phone and smart watch – these devices are linked via Bluetooth so that the smart watch can take calls, and the smart phone is connected to the internet, likely on the same local Wi-Fi network as the smart TV and any other device in the house – the smart washing machine, for example. The phone is also most likely connected to the same Google account as the TV.
But then there’s Grandma’s phone – this device was temporarily connected to the previously mentioned phone through a cellular service. Or was it connected to the cloud via a supposed encrypted app. Didn’t you get that phone for Grandma because you wanted to video call her?
Strangely it seems that all these devices were very well connected through the cloud – *cough* hence internet of things *cough* – so where did security and privacy play a part in this exchange, and where could it have gone wrong?
Let’s start by defining both:
Security, in this context, is the practice of making sure that an IoT network – and any cloud infrastructure it uses – cannot be accessed by unwanted third parties. Data on such a network needs to remain within the confines of that network, unless you say otherwise.
Security also extends to the devices within that network. If someone were to get a hold of one of the devices, a good hardware security solution would prevent third parties from gaining access to the data within all devices on that network, using the compromised device as an inroad.
There are many ways of doing this, from data encryption to physical countermeasures at the circuit level – everyone likes a good Mission: Impossible-style self-destruct. And then there’s the next level down: security that prevents third parties from altering the software or firmware of the device to introduce vulnerabilities to the system – this could be done over the air or directly via a wired interface.
Privacy on the other hand is a rather different concept to security in general. When we think of privacy, we’re not thinking about the typical exchange of data between devices – we want our devices to communicate, so long as it is done securely. But we don’t necessarily want our devices to exchange data about us. Privacy in this context might include our faces, our voices, our age, demographic, financial circumstances. Our search history…
This sort of information – or data – is not something many people are comfortable with being exchanged between devices, especially when there is no need, or if it ends up in the cloud where a good hack could ruin an otherwise convenient service. There’s a damn good reason why many modern laptops have a privacy shutter for the camera.
And it’s not just the raw data of an audio or video feed that might make us a little nervous to have always-listening or always-watching devices in our lives. With the emergence of natural language artificial intelligence, the subject matter of our conversations is up for grabs too – all that’s required is some good AI transcription.
When it comes to our smart home example, one might suppose that the security argument could come into play at the point where YouTube ads are being tailored to what is being said in the room. But without getting into the argument that you may have actually agreed to this when you didn’t read the terms and conditions, one would think this also comes under the umbrella of privacy.
As well as possible access to your budgeting spreadsheet, which might also be backed up in the cloud through Google docs, intimate details of your conversation with Grandma were being used to manipulate you with targeted ads. Targeted ads are fine if it’s based on your browsing history or whatnot, right? But this begs the question as to whether both your phone and Grandma’s phone know the relationship between the two of you.
It’s one thing for technology to listen to your conversation, but it’s something different when that technology knows who is talking. And what if your IoT isn’t secure, meaning that this information is up for grabs?
What if, at some point, you get a call from someone pretending to be Grandma. There’s no-one on the end of the line, but in the short time before you hang up, they’ve found their way in. And now they know what you’ve been watching on YouTube, when you go out for a run and where – through the smartwatch – and what time you’re out of the house, because you’ve always got your phone on you, and they now know when it’s disconnected from the Wi-Fi. They know your face too, because how many hours do you spend staring at that screen, just south of a small but unsecured camera. They’ll easily recognise when you’ve just left your house. Or worse. When you’re still in it…
Perhaps I’ve weaved enough of a tale to make it very clear why security in the internet of things is – for lack of a better phrase – really *bleep* essential. But there’s still the issue of privacy to contend with.
While good security protects your data, including all your private information, surely some of that information is not something that you would have ever wanted to be captured by the system. By this point we’ve established that no third parties are going to get at this data, but it’s still in the hands of the first parties – you – and second parties – in this case, Google or whoever is managing your cloud, and for some cases, your internet provider. Sure, they can’t just go look at your search history. Sure…
So for a truly secure and private IoT, ideally, you don’t want any of your personal data leaving a device. It won’t go to the cloud, and it won’t travel between devices. Your home assistant listens to what you’re saying, but it doesn’t tell anyone or anything. It’s your little secret with the machine.
This is the all’s-well-that-ends-well scenario. Your IoT devices don’t relay your personal data to the cloud, and you can feel safe knowing that your home isn’t spying on you. Roll credits, subscribe for more disruptive content, and as always,
Keep—
Wait a sec. Don’t we need a load of our data to go to the cloud or travel between devices, even if some of it may be linked to something personal like what meetings you’ve got today. Isn’t that what makes it an internet of things? And more so, isn’t that the key selling point of the technology in order to make our lives easier?
Well, bummer, I guess we’ll just have to live in a world where our tech sees us naked and listens to our conversations, and so long as the security in our IoT is good, at least it’ll only be cloud industry billionaires that can see that if they wish. I’m sure they’ve got good enough lawyers.
But in all seriousness, while there is inevitably going to be a secure exchange of data between the cloud and your devices, is there a way that we can ensure all this data is far less personal in nature?
This is where recent innovations in edge AI truly get their chance to shine. Essentially, if IoT end devices could take in sensor data – for example a face or conversation – and only transmit the meaning of that data to the cloud, rather than the data itself, we would have no issues of privacy, right?
Here’s an example: your home security system looks at your face through a door camera, so that your door will open for you – and only you – when you’ve got your hands full with two bags of shopping. Facial recognition is a complex task, and your system has a Time-of-Flight-based dual-verification method so that the door won’t open to just an image of your face – the depth must be there too. That’s a lot to process, so send that data up to the cloud, have an AI verify you, then send the command back to open the door. Who cares if the internet goes down – more secure anyway, right? – if you have to get your keys out, that’s life.
OR
All of that verification is done on the device with local AI trained specifically for the task. It knows that it’s you at the door, but it never takes your picture, it never stores reference image in the device – all that training was done long ago – and all that remains now is a neural network that can carry out the task. Now that’s a private AIoT.
So let’s expand this beyond the home. Privacy in smart city applications – a much bigger IoT – is, or maybe should be, a big concern for anyone that does not wish to feel like Big Brother is watching them. If you’re on the internet as much as I am, you’ll see conspiracy theory videos about city-wide facial recognition deployment, which can be scary when you consider how vast the network of CCTV cameras is, especially in the UK. But these are systems designed to protect us, so why do we get so nervous when AI-based facial recognition is introduced to it.
That’s a story for another video – or more accurately another channel – but the essential paranoia is that we’re moving towards an interconnected, AI-driven society where if the system – the IoT – were ever to become compromised through poor security, your privacy – your anonymity – would be gone. But engineers are smarter than that.
People tracking, for example, is an application which is rarely interested in the actual identities of the people being tracked, at least not within the smart city concept. Companies may want to understand building occupancy or where people gather in a store so that they can gauge what is interesting to customers – at the end of the day it’s about increasing sales. Sure, they may want to understand the demographic of the people gathering in those areas – and that is a privacy concern – but let’s not get carried away with ourselves at this stage; they can gather that information when the customer fills out the warranty form.
To measure where people are gathered, you don’t need a camera to perform facial recognition, you need a people counting-and-tracking algorithm that recognises an image as containing a set number of people and seeing how they move in the frame. Is that too complex to do without a video feed being uploaded to the cloud? Perhaps, so I’ll do you one better. How about a lidar image? That’s not really going to have people’s faces contained within the data, so we don’t mind that going to the cloud. It’s private, right? Not good enough?
How about we have Wi-Fi hotspots dotted around and we see which phones try to connect. That has no connection – all puns intended – to the identity of the person, right? Better still, a bunch of Wi-Fi connection requests doesn’t need to go to the cloud to contain actionable data. Those addresses could be queued with a number based on the day’s visitors, and that number can be transmitted to a cloud server to see the overall picture of the occupancy of people around the city. Even if that IoT isn’t secure, the private information about every person it tracked remains safe. And isn’t that what we really need in our IoT?
So let’s wrap this first thought piece up; I hope you’ve had as much fun listening to it as I had writing it. Security and privacy in the IoT are no doubt huge concerns for the future of this technology – maybe even as big a concern as the battery and sustainability argument against it – but when it comes down to it, these words mean two very different things, and it’s important to distinguish between them, though you might find that they overlap a lot of the time in their real-world use.
Security in the IoT is something that is as obviously required as security guards to buildings and properties. You don’t want unwanted data leaving or entering your IoT, and with the IoT constantly growing and using new wireless protocols to keep power consumption down and build better networks, keeping on top of weak points remains a top priority at the hardware and software level, whether on the device or in the cloud.
Privacy in the IoT is, for lack of a better word, a human problem. Even though many of us share much of our lives within the public domain, certain lines will always be drawn under information that we don’t want to leave our zone of control. The simplest way is to stop private information leaving an IoT end device, and developments in edge AI have made this much easier by converting real-world data – such as a picture or voice sample – into an inference that does not directly compromise the raw identity data of the user.
I may have started this thought piece with a mini horror story about a house that is spying on you – as well as all the devices within it – but in reality, with the correct security and privacy measures in place, this should not be your experience of the internet of things – so long as big tech is being honest about its conduct.
When operating correctly, having your devices well connected means that you can do exactly what the story illustrated in the beginning: you can tell your television to pause your favourite program when your hand’s aren’t free, you can have your washing machine text you when a load is finished, and so much more. My dad even once controlled our family home’s central heating from a holiday in Vietnam because I had set it too high. And so long as no-one else was able to do that, I’m pretty happy.
Thank you for watching this thought piece, and I hope you found it enlightening or entertaining depending on your level in the field – I’ll always try to keep these relatively general-interest within the tech sphere. If you’ve got a topic that you want to see covered, let us know in the comments, and I’ll do my best to research and script an episode.
Special thanks to my video editors Jake and Harry who make all this possible, and to Guy and Rhianna for letting me go ahead with this sort of content. It really gives me a chance to think about and digest the key concepts behind the technology that we write about every day at ipXchange.
Subscribe for more disruptive technology, leave a like and ring the bell to help us in the algorithm, and as always…
Keep designing!