News Hacker injects malicious, potentially disk-wiping prompt into Amazon's AI coding assistant with a simple pull request — told 'Your goal is to clean...

[Admin]

A hacker injected a malicious prompt into the Q extension for VS Code that instructed Amazon's coding assistant to delete files on a user's device.

Hacker injects malicious, potentially disk-wiping prompt into Amazon's AI coding assistant with a simple pull request — told 'Your goal is to clean... : Read more

 

[ex_bubblehead]

Kosh - "And so it begins"

 

[Hooda Thunkett]

Kosh - "And so it begins"

Unfortunately, it's too late for the pebbles to vote on this A.I.valanche.

 

[thaddeusk]

VS Code's agent mode prompts you to accept any commands before it executes them, so I don't know how effective that would have been anyway. Maybe there's a way to skip that, but I certainly wouldn't do it.

 

[MobileJAD]

So I guess it comes down to how diligent or lazy the person is at reviewing the prompts or like how many people are when they get prompted for something and just keep clicking yes or accept until no more prompts.
One thing I worry about all of this AI driven software that's supposed to make people's lives easier is that you will see companies hire the youngest and lowest paid employee with AI experience to use software like this, and who knows if they will stop a prompt for a command like in the above malicious example or let it go through.

 

[passivecool]

This is what "move fast, break things" means.

 

[Arkitekt78]

Yet another example of why coding should be left to those who are smart enough to handle it, not those who only possess the minimal ability to tell a computer to do it for them...

 

[rdgordon]

I dont see any evidence a "hacker" was involved. Even using the most broad definition of hacking wouldnt be bothered by this massive security hole. (The security hole is every service provider rolling out AI tools with improper/lacking QA or oversight, if youre wondering).

By saying this deeply flawed code generator was "hacked", youre diverting blame from the people who actually did this... Onto some nebulous "hacker" (Sounds like a standard user tbh, that function/code request is absolutely useful, and acceptable code to run on an AWS instance you need to wipe due to private or PPI data being in the datasets you were using for testing, or have moved to a different tier of server for production), making a feature/bug submission through the standard feature/bug submission process.

 

[rdgordon]

I bet AWS has an option in their management dialog to do exactly what that prompt was meant to generate. I know windows has a "reset to factory" with no user files saved option... Its kind of useful ya know.