It's made by Ben Zhao? You mean the "anti AI plagerism" UChicago professor who illegally stole GPLv3 code from an open source program called DiffusionBee for his proprietary Glaze software (reddit link), and when pressed, only released the code for the "front end" while still being in violation of GPL?

The Glaze tool that promised to be invisible to the naked eyes, but contained obvious AI generated artifacts? The same Glaze that reddit defeated in like a day after release?

Don't take anything this grifter says seriously, I'm surprised he hasn't been suspended for academic integrity violation yet.

Oh no, another complicated way to jpeg an image that an ai training program will be able to just detect and discard in a week's time.

Here's the paper: https://arxiv.org/pdf/2302.04222.pdf

I find it very interesting that someone went in this direction to try to find a way to mitigate plagiarism. This is very akin to adversarial attacks in neural networks (you can read more in this short review https://arxiv.org/pdf/2303.06032.pdf)

I saw some comments saying that you could just build an AI that detects poisoned images, but that wouldn't be feasible with a simple NN classifier or feature-based approaches. This technique changes the artist style itself to something the AI would see differently in the latent space, yet, visually perceived as the same image. So if you're changing to a different style the AI has learned, it's fair to assume it will be realistic and coherent. Although maaaaaaaybe you could detect poisoned images with some dark magic tho, get the targeted AI then analyze the latent space to see if the image has been tampered with

On the other hand, I think if you build more robust features and just scale the data this problems might go away with more regularization in the network. Plus, it assumes you have the target of one AI generation tool, there are a dozen of these, and if someone trains with a few more images in a cluster, that's it, you shifted the features and the poisoned images are invalid

Lol... I just read the paper, and Dr Zhao actually just wrote a research paper on why it's actually legally OK to use images to train AI. Hear me out...

He changes the 'style' of input images to corrupt the ability of image generators to mimic them, and even shows that the super majority of artists even can't tell when this happens with his program, Glaze... Style is explicitly not copywriteable in US case law, and so he just provided evidence that the data OpenAI and others use to generate images is transformative which would legally mean that it falls under fair use.

No idea if this would actually get argued in court, but it certainly doesn't support the idea that these image generators are stealing actual artwork.

I am sure we already got a budget version of this called the jpeg.

New CAPCHA just dropped.

This is already a concept in the AI world and is often used while a model is being trained specifically to make it better. I believe it's called adversarial training or something like that.

"I can tell this is toxic by the pixels."

I remember in the early 2010s reading an article like this one on openai.com talking about the dangers of using AI for image search engines to moderate against unwanted content. At the time the concern was CSAM salted to prevent its detection (along with other content salted with CSAM to generate false positives).

My guess is since we're still training AI with pools of data-entry people who tag pictures with what they appear to be, so that AI reads more into images than their human trainers (the proverbial man inside the Iron Turk).

This is going to be an interesting technology war.

Ooo, this is fascinating. It reminds me of that weird face paint that bugs out facial-recognition in CCTV cameras.

I am waiting for the day that some obsessed person starts finding ways to do like code injection in pictures.

I generally don't believe in intellectual property, I think it creates artificial scarcity and limits creativity. Of course the real tragedies in this field have to do with medicine and other serious business. But still, artists claiming ownership of their style of painting is fundamentally no different. Why can't I paint in your style? Do you really own it? Are you suggesting you didn't base your idea mostly on the work of others, and no one in turn can take your idea, be inspired by it and do with it as they please? Do my means have to be a pencil, why can't my means be a computer, why not an algorythm? Limitations, limitations, limitations. We need to reform our system and make the public domain the standard for ideas (in all their forms). Society doesn't treat artists properly, I am well aware of that. Generally creative minds are often troubled because they fall outside norms. There are many tragic examples. Also money-wise many artists don't get enough credit for their contributions to society, but making every idea a restricted area is not the solution. People should support the artists they like on a voluntary basis. Pirate the album but go to concerts, pirate the artwork but donate to the artist. And if that doesn't make you enough money, that's very unfortunate. But make no mistake: that's how almost all artists live. Only the top 0.something% actually make enough money by selling their work, and that's is usually the percentile that's best at marketing their arts, in other words: it's usually the industry. The others already depend upon donations or other sources of income. We can surely keep art alive, while still removing all these artificial limitations, copying is, was and will never be in any way similar to stealing. Let freedom rule. Join your local pirate party.

Invisible changes to pixels sound like pure BS to me. I'm sure others know more about it than i do but I thought pixels were very simple things.

Obviously this is using some bug and/or weakness in the existing training process, so couldn't they just patch the mechanism being exploited?

Or at the very least you could take a bunch of images, purposely poison them, and now you have a set of poisoned images and their non-poisoned counterparts allowing you to train another model to undo it.

Sure you've set up a speedbump but this is hardly a solution.