Google's Gemini AI caught scanning Google Drive PDF files without permission

Just reiterates that you don't own your data hosted on cloud providers; this time there's a clear sign, but I can guarantee that google's systems read and aggregated data inside your private docs ages ago.

This concern was first raised when Gmail started, 20 years ago now; at the time people reeled at the idea of "google reads your emails to give you ads", but at the same time the 1 GB inbox and fresh UI was a compelling argument.

I think they learned from it, and google drive and co were less "scary" or less overt with scanning the stuff you have in it, also because they wanted to get that sweet corporate money.

All AI should be opt-in, which includes both training and scanning. You should have to check a box that says "I would like to use AI features", and the accompanying text should be crystal clear what that means.

This should be mandatory, enforced, and come with strict fines for companies that do not comply.

Apart from the obvious misleading way this article is written. I am listing all the links shared in the tweet thread that the article mentioned -

- Manage your activity on Gemini : https://myactivity.google.com/product/gemini

- This page has most answers related to Google Workspace and opting out of different Google apps : https://support.google.com/docs/answer/13447104#:~:text=Turn...

The headline is a little unclear on the issue here.

It is not surprising that Gemini will summarize a document if you ask it to. "Scanning" is doing heavy lifting here; The headline implies Google is training Gemini on private documents, when the real issue is Gemini was run with a private document as input to do a summary when the user thought they had explicitly switched that off.

That having been said, it's a meaningful bug in Google's infrastructure that the setting is not being respected and the kind of thing that should make a person check their exit strategy if they are completely against using The new generation of AI in general.

The title is misleading, isn't it? I was expecting this was scanning for training or testing or something, but this is summarization of articles the user is looking at, so "caught" is disingenous. You don't "catch" people doing things they tell you they are doing, while they're doing it.

Only a matter of time before someone extracts something valuable out of googe's models. Bank passwords or crypto keys or something

Glue pizza incident illustrated they're just yolo'ing this

this is similar to the scramble for health data during covid where a number of groups tried (and some succeeded) at using the crisis to squeeze the toothpaste out of the tube in a similar way, as there are low costs to being reprimanded and high value in grabbing the data. bureaucratic smash-and-grabs, essentially. disappointing, but predictable to anyone who has worked in privacy, and most people just make a show of acting surprised then moving on because their careers depend on their ability to sustain a gallopingly absurd best-intentions narrative.

your hacked SMS messages from AT&T are probably next, and everyone will be just as surprised when keystrokes from your phones get hit, or there is a collection agent for model training (privacy enhanced for your pleasure, surely) added as an OS update to commercial platforms.

Make an example of the product managers and engineers behind this, or see it done worse and at a larger scale next time.

The original Tweet and this article are mixing terms in a deliberately misleading way.

They’re trying to suggest that exposing an LLM to a document in any way is equivalent to including that document in the LLM’s training set. That’s the hook in the article and the original Tweet, but the Tweet thread eventually acknowledges the differences and pivots to being angry about the existence of the AI feature at all.

There isn’t anything of substance to this story other than a Twitter user writing a rage-bait thread about being angry about an AI popup, while trying to spin it as something much more sinister.

I'm shocked, especially this being HN, that how many people are being successfully misled on what is actually going on here. Do people still read articles before posting?

Shocker, Google not going quite far enough with privacy and data access? They talk about it but its never quite far enough to avoid their own services accessing data.

We really need to get to the point that all data remotely stored needs to be encrypted and unable to be decrypted by the servers, only our devices. Otherwise we just allow the companies to mine the data as much as they want and we have zero insight into what they are doing.

Yes this requires the trust that they in fact cannot decrypt it. I don't have a good solution to that.

Any AI access to personal data needs to be done on device, or if it requires server processing (which is hopefully only a short term issue) a clear prompt about data being sent out of your device.

It doesn't matter if this isnt specifically being used to train the model at this point in time, it is not unreasonable to think that any data sent through Gemini (or any remote server) could be logged and later used for additional training, sitting plaintext in a log, or just viewable by testers.

There is no cloud. It's just someone else's computer.

Not stored on your device or encrypted with keys only you control, not yours.

I assume anything stored in such a system will be data mined for many purposes. That includes all of Gmail and Google Docs.

This shouldn't come as a surprise to anyone, their entire business is our data. I always encrypt anything important I want to backup on the cloud.

I tried an equivalent of copilot once, in the code base I typed `images = [` and the AI autofilled me the array with http links to real images. I never tried to do the same thing with private keys or other sensitive informations but it sucks that it happens

It is urgent to educate people about how these systems work. Search requires indexing. Summarizing with a LM requires inference. Data used for inference usually is forgotten forever after used, as it is not used for training.

Yeah, that should be obvious for many here, but even software engineers believe that AI are sentient things that will remember everything that they see. And that is a problem, because public is afraid of the tech due to a wrong understanding of how it works. Eventually they will demand laws protecting them from stuff that have never existed.

Yes, there are social issues with AI. But the article just shows a big tech illiteracy gap.

I just want to add that gmail has a very sneaky 'add to drive' button that is way too easy to click when working with email attachments.

How long til gmail attachments get uploaded into drive by default through some obscure update that toggles everything to 'yes'?

Your first mistake was storing your data on someone else's computer.

In the push for AGI do companies feel a recursive learning future is soon achievable and therefore getting to the first cycle of that is worth the cost of any legal issues that may arise?

If this is true, google needs to be charged with the violation of various privacy laws.

I’m not sure how they can claim they have informed consent for this from their customers

There is a fundamentally interesting nuance to highlight. I don't know precisely what google is doing, but if they're just shuttling the content through a closed-loop deterministic LLM, then, much like a spellchecker, I see no issue. Sure, it _feels_ creepy, but it's just an algo.

Perhaps someone can articulate the precise threshold of 'access' they wish to deny apps that we overtly use? And how would that threshold be defined?

"Do not run my content through anything more complicated than some arbitrary [complexity metric]" ??

The inability to disable this feature adds to the frustration

My wake up moment with google was when they accused a parent of being a pedophile, permanently banned their accounts, reported them the police, and then doubled down when they were proven wrong.

Not only due those degenerates have the gal to creep on people, they refuse to admit wrongdoing or make their victems whole.

Sickos. That's what they are. Sickos.

we should just embrace digital copying galore instead of trying to digitalize the physical constraints of regular assets

we should just ignore physical constraints of assets which do not have them, like any and all digital data

which do you prefer? everybody can access all digital data of everybody (read only mode), or what we have now which is trending towards having so many microtransactions that every keystroke gets reflected in my bank account

The title misleads the point, and the article is, imoo, badly written. The post implies there is indeed a setting to turn it off. So the author deliberately asked Gemini AI to summarize (so, scan) its documents...

Related to this news: https://news.ycombinator.com/item?id=40934670

Every single week I have to refuse enabling back up for my pictures on my Google pixel. I refuse it today, next week I open the app and the UI shows the back up option enabled with a button "continue using the app with back up".

Somebody took the time to talk down my comment about this being a strategy to give their AI more training data. I continue believing that if they have your data they will use it.

Meta commentary but still relevant I think:

The author first refers to his source as Kevin Bankston in the article's subtitle. This is also the name shown in the embedded tweet. But the following two references call him Kevin _Bankster_ (which seems like an amusing portmanteau of banker and gangster I guess).

Is the author not proofreading his own copy? Are there no editors? If the author can't even keep the name of his source straight and represent that consistently in the article, is there reason to think other details are being relayed correctly?

I now feel obligated to cram as much AI-f'n-up crap into my Drive as possible. Come'n get it!

this is not openai doing shady things so everyone should be up in arms