“CSAM generated by AI is still CSAM,” DOJ says after rare arrest

Hoooooly hell, good luck getting that study going. No ethical concerns there!

I'm willing to bet it'll differ from person to person, to complicate matters further

I think the general consensus is that availability of CSAM is bad, because it desensitizes and makes harming of actual children more likely. But I must admit that I only remember reading about that and don't have a scientific source.

Image-generating AI is capable of generating images that are not like anything that was in its training set.

Very, very good point. Depending on the answer, I retract the "victimless" narrative.

On the other hand - I feel like it could be a gateway to something m

You mean like marijuana and violent video games?

First off, this is obviously a sticky topic. Every conversation is controversial and speculative.

Second, I don't really see a lot of legitimacy to the "gateway" concept. The vast majority of people use some variety of drug (caffeine, alcohol, nicotine), and that doesn't really reliably predict "harder" drug use. Lots of people use marijuana and that doesn't reliably predict hard drug use. Obviously, the people who use heroin and meth have probably used cocaine and ketamine, and weed before that, and alcohol/caffeine/nicotine before that, but that's not really a "gateway" pipeline so much as paying through finer and finer filters. As far as I know, the concept has fallen pretty heavily out of favor with serious researchers.

In light of that perspective, I think you have to consider the goal. Is your goal to punish people, or to reduce the number and severity of victims? Mine is the latter. Personally, I think this sort of thing peels off many more low-level offenders to low-effort outlets than it emboldens to higher-severity outlets. I think this is ultimately a mental-health problem, and zero-tolerance mandatory reporting (while well-meaning) does more harm than good.

I'd rather that those with these kinds of mental issues have 1. the tools to take the edge off in victimless ways 2. safe spaces to discuss these inclinations without fear of incarceration. I think blockading those avenues yields a net increase the number and severity of victims.

This seems like a net benefit, reducing the overall number and severity of actual victims.

Thanks for being honest and well-meaning. Sorry you're getting downvoted, we both said pretty much exactly the same thing! A difficult subject, but important to get right...

What if someone draws their own CSAM and they're terrible at drawing but it's still recognizable as CSAM?

Makes sense.

So it’s all good as long as they have elf ears or that counts as realistic too?

https://commoncrawl.org/

https://laion.ai/

It's not what's used by all of them, but it's pretty popular.

I'm a professional artist and have no issue banning ai generated CSAM. People can call it self expression if they want, but that doesn't change the real world consequences of it.

Allowing ai generated CSAM basically creates camouflage for real CSAM. As ai gets more advanced it will become harder to tell the difference. The scum making real CSAM will be emboldened to make even more because they can hide it amongst the increasing amounts of ai generated versions, or simply tag it as AI generated. Now authorities will have to sift through all of it trying to decipher what's artifical and what isn't.

The liklihood of them being able to identify, trace, and convict child abusers will become even more difficult as more and more of that material is generated and uploaded to various sites with real CSAM mixed in.

Even with hyper realistic paintings you can still tell it's a painting. Anime loli stuff can never be mistaken for real CSAM. Do I find that sort of art distasteful? Yep. But it's not creating an environment where real abusers can distribute CSAM and have a higher possibility of getting away with it.

So long as the generation is without actual model examples that are actual minors there's nothing technically illegal about having sexual material of what appears to be a child. They would then have a mens rea question and a content question, what actual defines in a visual sense a child? Could those same things equally define a person of smaller stature? And finally could someone like tiny texie be charged for producing csam as she by all appearance or of context looks to be a child.

It's not a difficult test. If a person can't reasonably distinguish it from an actual child, then it's CSAM.

Not necessarily. There's been a lot of advances in watermarking AI outputs.

As well, there's the opposite argument.

Right now, pedophile rings have very high price points to access CSAM or require users to upload original CSAM content, adding a significant motivator to actually harm children.

The same way rule 34 artists were very upset with AI being able to create what they were getting commissions to create, AI generated CSAM would be a significant dilution of the market.

Is the average user really going to risk prison, pay a huge amount of money or harm a child with an even greater prison risk when effectively identical material is available for free?

Pretty much overnight the CSAM dark markets would lose the vast majority of their market value and the only remaining offerings would be ones that could demonstrate they weren't artificial to justify the higher price point, which would undermine the notion of plausible deniability.

Legalization of AI generated CSAM would decimate the existing CSAM markets.

That said, the real question that needs to be answered from a social responsibility perspective is what the net effect of CSAM access by pedophiles has on their proclivity to offend. If there's a negative effect then it's an open and shut case that it should be legalized. If it's a positive effect than we should probably keep it very much illegal, even if that continues to enable dark markets for the real thing.

I agree, just the linguistics are interesting.

Better to just blanket ban the entire concept and save us all the trouble, in my opinion.

That’s the issue though, blindly banning things that can be victimless crimes never ends, like prohibition.

placing the burden on the legal system to prove it to the contrary.

That's how it should be. Everyone is innocent until proven otherwise.

True that. We have the means to fix so many problems, we just have a very very very small few that reeeeally don't like to do anything good with their money, and instead choose to hoard it, at the expense of everyone else.

In the eyes of the law, intent does matter, as well as how it's responded to.
For csam material, you have to knowingly possess it or have sought to possess it.

The AI companies use a project that indexes everything on the Internet, like Google, but with publicly available free output.

https://commoncrawl.org/

They use this data via another project, https://laion.ai/ , which uses the data to find images with descriptions attached, do some tricks to validate that the descriptions make sense, and then publish a list of "location of the image, description of the image" pairs.

The AI companies use that list to grab the images train an AI on them in conjunction with the description.

So, people at Stanford were doing research on the laion dataset when they found the instances of csam. The laion project pulled their datasets from being available while things were checked and new safeguards put in place.
The AI companies also pulled their models (if public) while the images were removed from the data set and new safeguards implemented.
Most of the csam images in the dataset were already gone by the time the AI companies would have attempted to access them, but some were not.

A very obvious lack of intent to acquire the material, in fact a lack of awareness the material was possessed at all, transparency in response, taking steps to prevent further distribution, and taking action to prevent it from happening again both provides a defensive against accusations, and will make anyone interested less likely to want to make those accusations.

On the other hand, the people who generated the images were knowingly doing so, which is a nono.

You cannot force people to use Micro$oft. But I'm sure that it would only increase market share for them because it will be mediatized in a way that depicts non-privacy invading operating systems as morally evil because good guys don't have anything to hide. Kinda like they did with pleading the fifth and shifting the public image of doing so being a silent admission to having committed a crime.

I sure hope so. It is important because otherwise copyright will mean jackshit.

*Rant I truly hope politicians spend their time on more pressing issues than squabbling among themselves. Climate change, technological advancement that outpaces our legal framework, consumer protection. So much shit to do.

And I suppose it’s also saying that the form it’s in doesn’t matter. Any type of material is the same.

Okay for anyone who might be confused on how a model that's not been trained on something can come up with something it wasn't trained for, a rough example of this is antialiasing.

In the simplest of terms antialiasing looks at a vector over a particular grid, sees what percentage it is covering, and then applies that percentage to to shade the image and reduce the jaggies.

There's no information to do this in the vector itself, it's the math that is what is giving the extra information. We're creating information from a source that did not originally have it. Now, yeah this is really simple approach and it might have you go "well technically we didn't create any new information".

At the end of the day, a tensor is a bunch of numbers that give weights to how pixels should arrange themselves on the canvas. We have weights that show us how to fall pixels to an adult. We have weights that show us how to fall pixels to children. We have weights that show us how to fall pixels to a nude adult. There's ways to adapt the lower order ranking of weights to find new approximations. I mean, that's literally what LoRAs do. I mean that's literally their name, Low-Rank Adaptation. As you train on this new novel approach, you can wrap that into a textual inversion. That's what that does, it allows an ontological approach to particular weights within a model.

Another way to think of this. Six finger people in AI art. I assure you that no model was fed six fingered subjects, so where do they come from? The answer is that the six finger person is a complex "averaging" of the tensors that make up the model's weights. We're getting new information where there originally was none.

We have to remember that these models ARE NOT databases. They are just multidimensional weights that tell pixels from a random seed where to go to in the next step in the diffusion process. If you text2image "hand" then there's a set of weights that push pixels around to form the average value of a hand. What it settles into could be a four fingered hand, five fingers, or six fingers, depends on the seed and how hard the diffuser should follow the guidance scale for that particular prompt's weight. But it's distinctly not recalling pixel for pixel some image it has seen earlier. It just has a bunch of averages of where pixels should go if someone says hand.

You can generate something new from the average of complex tensors. You can put your thumb on the scale for some of those weights, give new maths to find new averages, and then when it's getting close to the target you're after use a textual inversion to give a label to this "new" average you've discovered in the weights.

Antialiasing doesn't feel like new information is being added, but it is. That's how we can take the actual pixels being pushed out by a program and turn it into a smooth line that the program did not distinctly produce. I get that it feels like a stretch to go from antialiasing to generating completely novel information. But it's just numbers driving where pixels get moved to, it's maths, there's not really a lot of magic in these things. And given enough energy, anyone can push numbers to do things they weren't supposed to do in the first place.

The way models that come from folks who need their models to be on the up and up is to ensure that particular averages don't happen. Like say we want to avoid outcome B', but you can average A and C to arrive at B'. Then what you need is to add a negative weight to the formula. This is basically training A and C to average to something like R' that's really far from the point that we want to avoid. But like any number, if we know the outcome is R' for an average of A and C, we can add low rank weights that don't require new layers within the model. We can just say, anything with R' needs -P' weight, now because of averages we could land on C' but we could also land on A' or B' our target. We don't need to recalculate the approximation of the weights that A and C give R' within the model.

No, you're quite right that the combination didn't need to exist in the input for an output to be generated - this shit is so interesting because you can throw stuff like "A medieval castle but with Iranian architecture with a samurai standing on the ramparts" at it and get something neat out. I've leveraged AI image generation for visual D&D references and it's excellent at combining comprehended concepts... but it can't innovate a new thing - it excels at mixing things but it isn't creative or novel. So I don't disagree with anything you've said - but I'd reaffirm that it currently can make CSAM because it's trained on CSAM and, in my opinion, it would be unable to generate CSAM (at least to the quality level that would decrease demand for CSAM among pedos) without having CSAM in the training set.

I think it's impossible to produce CSAM without training data of CSAM (though this is just an opinion). Young people don't look like adults when naked so I don't think there's anyway an AI would hallucinate CSAM without some examples to train on.

Dunno, probably because they didn't knowingly train it on CSAM - maybe because it's difficult to prove what actually goes into neural network configuration so it's unclear how strongly weighted it is... and lastly, maybe because this stuff is so cloaked in obscurity and proprietaryness that nobody is confident how such a case would go.

Camera makers and pencil makers (and the users of those devices) aren't making massive server farms that spy on every drop of information they can get ahold of.

If AI has the means to generate inappropriate material, then that means the developers have allowed it to train from inappropriate material.

Now when that's the case, well where did the devs get the training data?.. 🤔

That’d be like outlawing hammers because someone figured out they make a great murder weapon.

Just because you can use a tool for crime, doesn’t mean that tool was designed/intended for crime.

Not exactly. This would be more akin to a company that will 3D printer metal parts and assemble them for you. You use this service and have them create and assemble a gun for you. Then you use that weapon in a violent crime. Should the company have known better that you were having them create an illegal weapon on your behalf?

Sadly that's what most of the gun laws are designed about. Book banning and anti-abortion both are limiting tools because of what a small minority choose to do with the tool.

AI image generation shouldn't be considered in obscenity laws. His distribution or pornography to minor should be the issue, because not everyone stuck with that disease should be deprived tools that can be used to keep them away from hurting others.

Using AI images to increase charges should be wrong. A pedophile contacting and distributing pornography to children should be all that it takes to charge a person. This will just setup new precedent that is beyond the scope of the judiciary.

It would be more like outlawing ivory grand pianos because they require dead elephants to make - the AI models under question here were trained on abuse.

That's not the point. You don't train a hammer from millions of user inputs.

You gotta ask, if the AI can produce inappropriate material, then where did the developers get the training data, and what exactly did they train those AI models for?

Check my other comments. My thought was compared to a hammer.

Hammers aren't trained to act or respond on their own from millions of user inputs.

I'd rather not fart bullets, but thank you for inviting me to the party.

Totally dismissing inappropriate usage, AI can be funny and entertaining, but on the flip side it's also taking people's jobs.

It shouldn't take a book, let alone 3 seconds of common sense thought, to realize that.