News AI Swipes Data By Listening to Keyboard Keystrokes with 95% Accuracy

[Admin]

A team of researchers from Cornell have managed to train AI to retrieve data with 95% accuracy by recording keystroke audio.

AI Swipes Data By Listening to Keyboard Keystrokes with 95% Accuracy : Read more

 

[bit_user]

I always imagined something like this should be possible. I wonder if the training was also specific to the typist, or how much that would benefit accuracy if it were.

My Kinesis keyboard has a "key click" feature, which emits an electronic clicking sound when each key is pressed. I'd imagine that would mask part of the key sound, assuming there's a sufficiently low latency. I always turn it off, but I guess this would be an argument for leaving it on.

 

[Sippincider]

The system doesn’t work with just any random keyboard, it must be trained to a specific keyboard with references for what character each keystroke corresponds to.

Where's my stash of old Apple ADB keyboards when I need it!

But that is security through obscurity, so maybe a piece of tape over the microphone when not needing it.

 

[bit_user]

But that is security through obscurity, so maybe a piece of tape over the microphone when not needing it.

Imagine someone eavesdrops on your Zoom or MS Teams meeting while you're sharing your desktop and typing something. Or, maybe they gain access to a recording of one that was made. Then, they can both:

1. See what you're typing, therefore being able to infer the keystrokes you're making.
2. Hear the sound of you typing.

By using such a recording, they can train their model to know what your typing sounds like. Now, if you happen to login to a website or something while you're on a call, they can use it to work out (or at least constrain the search space of) your password.

Therefore, I'd suggest a good countermeasure most people could take is to mute their microphone when typing any sort of login or password. Perhaps, if the OS had a hook where apps could tell it they were authenticating, it could even automatically mute your microphone and flash up a notice to let you know it's doing so.

 

[rluker5]

My desktops only have mics when I plug a camera in or use a headset with one. But my phone certainly has one and if I used Chrome on my desktop Google would likely be able to train what I'm typing to the corresponding sound if my phone were near my keyboard. Which it sometimes is for two step verifications.

Not that Google would use this for anything more nefarious than their data harvesting, but now that this article is out they know.

Guess I'll keep using Edge and Bing and keep my phone unlinked from Windows. Because it is 0 effort as I do that already.

AI is going to turn a lot of people into conspiracy theory kooks.

 

[Timmy!]

Perhaps, if the OS had a hook where apps could tell it they were authenticating, it could even automatically mute your microphone and flash up a notice to let you know it's doing so.

Not being a techie, wouldn't it be useful to hackers for the OS/app to tell snoopers that right now login and password credentials were being entered? You could probably imagine scenarios where those credentials could be snooped on via this method.

 

[PEnns]

Sorry to deflate the tons of hours of this superfluous research (I am sure the military is drooling over it though, as usual):

But this vapid "breakthrough" could be mitigated when the target of the security breach is using a silent or on-screen keyboard!! Listen to this, AI!!

 

[USAFRet]

But this vapid "breakthrough" could be mitigated when the target of the security breach is using a silent or on-screen keyboard!! Listen to this, AI!!

Yes, that might mitigate this particular exploit.

Onscreen kbd could be compromised in other ways.

 

[Alvar "Miles" Udell]

Think about this from another angle: Planting a tiny sound recorder for retrieval and decoding later. Depending on how easy it is for the public to gain access to that AI, it would be interesting to see if this would work on, say, physical keypads. Seems to me this kind of attack would be far more effective on keypad locks than computer keyboards.

 

[bit_user]

Not that Google would use this for anything more nefarious than their data harvesting, but now that this article is out they know.

Once the infrastructure for mass surveillance exists, it can be hijacked by a wide range of actors who don't adhere to the same morals, policies, or aren't subject to the same legal exposure as the ones who built it. Not to sound paranoid, or anything.

I'm reminded of how, years ago, someone managed to get random Alexa recordings of them from extended time periods when Alexa wasn't supposed to be listening. The end user might've obtained the recordings through a formal request of some sort, but it's the fact that the recordings even existed that's the problem. Now, what if a hacker gained access to them?

 

[bit_user]

Not being a techie, wouldn't it be useful to hackers for the OS/app to tell snoopers that right now login and password credentials were being entered? You could probably imagine scenarios where those credentials could be snooped on via this method.

Yeah, I thought about the potential of malware to hijack the hook, but there already exists some kind of Single Sign On infrastructure in Windows, which enables applications (and I think even websites) to use Microsoft for user authentication. For instance, when I visit my corporate internal web portal from my work PC, it knows who I am and automatically authenticates me. Github Enterprise does something similar, never requiring me to manually enter my credentials. Same with certain other microsoft apps and services I use (e.g. Outlook/exchange, Visual Studio/TFS).

So, if a number of applications already have tie-ins with the OS, then what I'm proposing isn't exactly new. BTW, I'm not proposing the app use Windows to authenticate you, but rather just a hook to tell Windows to "lock down". IMO, the question isn't whether it would introduce new opportunities for hackers, but whether the positives outweigh the negatives.

It's all very hypothetical, though. I'm sure Microsoft would prefer that the entire world just use its authentication services.

 

[bit_user]

Think about this from another angle: Planting a tiny sound recorder for retrieval and decoding later. Depending on how easy it is for the public to gain access to that AI, it would be interesting to see if this would work on, say, physical keypads. Seems to me this kind of attack would be far more effective on keypad locks than computer keyboards.

The one keypad I still use is for an alarm system and has soft, rubbery keys. Maybe a good enough microphone could hear a difference, but I don't know...

 

[Timmy!]

Yeah, I thought about the potential of malware to hijack the hook...IMO, the question isn't whether it would introduce new opportunities for hackers, but whether the positives outweigh the negatives.

It's all very hypothetical, though. I'm sure Microsoft would prefer that the entire world just use its authentication services.

I've noticed in articles like these that users often comment that given the manner in which the spyware was planted on your device (e.g. by the physical presence of someone at your computer for a period of time), the actual spyware on your PC is the least of your concerns. Maybe this article fits into this category on some counts?

I've commented on one other article here how the subject didn't really concern me, given that governments around the world have legislated to have tech devices etc. be accessible to them. Backdoors in other words.

How often does this site mention that? I can't say that I've noticed them doing this. What would be interesting to read here is information about the state of play around the world on this front. And if security concerns like these were to be covered, could this site tell whether the security hole was due to a government enforced backdoor?

It's a sad day indeed when criminals around the world don't respect the right of governments to have exclusive covert access to information that the tech companies' surveillance of its users allows them. What kind of people take advantage of poor government policy like that?

On a side note, I remember reading online some years ago a claim that in the event of a war, perhaps of a global nature, PCs around the world could be co-opted to fight cyberwarfare, presumably on behalf of the US. Be interesting if there's a current source for that claim and what its status is.

 

[bit_user]

I've noticed in articles like these that users often comment that given the manner in which the spyware was planted on your device (e.g. by the physical presence of someone at your computer for a period of time), the actual spyware on your PC is the least of your concerns. Maybe this article fits into this category on some counts?

Given they said it works over video conferencing links, the exploit should be possible without any malware being present on the victim's PC, nor any physical intrusion of their space.

I've commented on one other article here how the subject didn't really concern me, given that governments around the world have legislated to have tech devices etc. be accessible to them. Backdoors in other words.

It'd be interesting if you could cite this legislation. I don't believe it exists.

How often does this site mention that? I can't say that I've noticed them doing this. What would be interesting to read here is information about the state of play around the world on this front. And if security concerns like these were to be covered, could this site tell whether the security hole was due to a government enforced backdoor?

It's a sad day indeed when criminals around the world don't respect the right of governments to have exclusive covert access to information that the tech companies' surveillance of its users allows them. What kind of people take advantage of poor government policy like that?

On this point, the NSA has previously been scolded for "hoarding" back doors they've discovered and not informing the hardware or software vendors about them. The NSA has a dual mandate to spy on foreign adversaries and protect the US and its citizens. By not informing vendors of backdoors, they were neglecting their second mandate. Since then, I think they've gotten better about reporting security flaws they've discovered, but I could still believe they might sometimes drag their feet before reporting them.

On a side note, I remember reading online some years ago a claim that in the event of a war, perhaps of a global nature, PCs around the world could be co-opted to fight cyberwarfare, presumably on behalf of the US. Be interesting if there's a current source for that claim and what its status is.

At the point where you're spreading something controversial you think you remember reading but don't even know if there's a credible source, you probably shouldn't. This claim doesn't seem credible to me.

 

[Timmy!]

Given they said it works over video conferencing links, the exploit should be possible without any malware being present on the victim's PC, nor any physical intrusion of their space.


It'd be interesting if you could cite this legislation. I don't believe it exists.


On this point, the NSA has previously been scolded for "hoarding" back doors they've discovered and not informing the hardware or software vendors about them. The NSA has a dual mandate to spy on foreign adversaries and protect the US and its citizens. By not informing vendors of backdoors, they were neglecting their second mandate. Since then, I think they've gotten better about reporting security flaws they've discovered, but I could still believe they might sometimes drag their feet before reporting them.


At the point where you're spreading something controversial you think you remember reading but don't even know if there's a credible source, you probably shouldn't. This claim doesn't seem credible to me.

The article mentions the sound of keystrokes being recorded, so, one means of doing that is via recording equipment on site. Video conferencing was another means of getting that information.

Spy agency ducks questions about 'back doors' in tech products

The U.S. National Security Agency is rebuffing efforts by a leading Congressional critic to determine whether it is continuing to place so-called back doors into commercial technology products, in a controversial practice that critics say damages both U.S. industry and national security.

www.reuters.com

"The U.S. National Security Agency is rebuffing efforts by a leading Congressional critic to determine whether it is continuing to place so-called back doors into commercial technology products".

Ignoring the word "place", the key word there is "continuing". I'm not from the US and don't know what's going on there intimately but it seems like the government can be doing things without scrutiny via concealing things in obscure budgets items or what have you.

One story made global news a few years ago, about Australia mandating tech companies to allow access to encrypted messages.:

Australia data encryption laws explained

Tech firms say the controversial laws could weaken overall security for users of messenger apps.

www.bbc.com

Global companies can't tailor products exclusively for Australia. If Australian authorities can unencrypt messages, others can too.

This article says that 7 other nations want to be able to do this too, including the US:

United States, Six Other Nations Ask Tech Companies To Build Backdoors To Encrypted Communications

Platforms like Apple’s iMessage and Facebook’s WhatsApp use end-to-end encryption to ensure user privacy and prevent snooping from hackers, governments and even themselves.

www.forbes.com

There have been stories from major news sites in the past about Germany and other countries banning US tech from government agencies over concerns about the US spying on them. Germany is an ally of the US.

I didn't say that I think I remember reading an article about co-opted PCs being a thing. I read it. It didn't seem like an obviously dubious site providing that information but I don't think that it was a major news site either. Haven't had much luck now trying to chase it down but maybe I'm just using the wrong key words in my search. It's entirely possible that the story was bunk, I'll concede. The point of mentioning it was to see if someone knew of this story and could flesh out the details.

 

[bit_user]

One story made global news a few years ago, about Australia mandating tech companies to allow access to encrypted messages.:

Australia data encryption laws explained

Tech firms say the controversial laws could weaken overall security for users of messenger apps.

www.bbc.com

Global companies can't tailor products exclusively for Australia. If Australian authorities can unencrypt messages, others can too.

This article says that 7 other nations want to be able to do this too, including the US:

United States, Six Other Nations Ask Tech Companies To Build Backdoors To Encrypted Communications

Platforms like Apple’s iMessage and Facebook’s WhatsApp use end-to-end encryption to ensure user privacy and prevent snooping from hackers, governments and even themselves.

www.forbes.com

There have been stories from major news sites in the past about Germany and other countries banning US tech from government agencies over concerns about the US spying on them. Germany is an ally of the US.

This is about reading encrypted communications.

I didn't say that I think I remember reading an article about co-opted PCs being a thing. I read it.

Your claim is/was about assembling a massive bot-net. These are two very different things.

It didn't seem like an obviously dubious site providing that information but I don't think that it was a major news site either.

There are some sites used specifically for disseminating disinformation which are made to look legit. Pay close attention to the domain name and other subtle details. Search for information about the site, if you're still not sure.

 

[gtoal]

You'd hope that University researchers would know how to search github to see if this has been done before: https://github.com/ggerganov/kbd-audio https://ggerganov.github.io/jekyll/update/2018/11/30/keytap-description-and-thoughts.html ... from 2018.

 

[bit_user]

You'd hope that University researchers would know how to search github to see if this has been done before: https://github.com/ggerganov/kbd-audio https://ggerganov.github.io/jekyll/update/2018/11/30/keytap-description-and-thoughts.html ... from 2018.

What they typically do is a literature survey. If that project had no associated paper published in a journal, conference proceedings, or at least on arxiv.org, then it's not surprising they missed it. Indeed, their paper cites 39 references, but I don't see that among them.

A good reason not to bother digging through pages of github search results is that you have no idea how well any of these projects work, and it could turn into a lot of work to evaluate even just the ones which seem promising. However, if there's a published paper, then it's usually been peer-reviewed and has a decent chance of having a good theoretical foundation (unlike many github projects I've seen).

If you're the author of that project, as the saying goes: "publish or perish."