@futurebird @GossiTheDog

Because thats how the math works.

It takes random noise, and checks if it looks like the target description.
Then modulates the noise and repeats until its satisfied that it looks like what the user described.

@rep_movsd @GossiTheDog

There are things that these generators do well, and things that they struggle with, and things they simply can't generate. These limitations are set by the training data.

It's easy to come up with a prompt for an engine that it just can't manage to do since it had nothing to reference.

@futurebird @GossiTheDog

The models are getting better by the hour.

AI gets details wrong, but in general they are almost as good as any artist who can do photorealism.

Also prompting techniques matter a lot

@rep_movsd @GossiTheDog

But you could only state that it could generate something not in the training data... if you knew what was in the training data. But that is secret. So you don't know. You don't know if there is a near identical image to the one produced in the training data.

@futurebird @GossiTheDog

Fair enough, but I am pretty sure that a model that is trained on both images of children and adults, will very easily be able to create images of children in adult like clothes and so forth.

Its possible to put some guardrails on what the AI can be asked to do, but only as much as you can put guardrails on any intelligent being who tends to want to do a task for a reward.

@futurebird @GossiTheDog

Fair enough, but I am pretty sure that a model that is trained on both images of children and adults, will very easily be able to create images of children in adult like clothes and so forth.

Its possible to put some guardrails on what the AI can be asked to do, but only as much as you can put guardrails on any intelligent being who tends to want to do a task for a reward.

@futurebird
The same way you can use words to describe something to someone who has never been exposed to that thing and they imagine it only using intuition from their own model of the world.

Look, these things are mammal-brain-like, but with very weird training/life-experience, and devoid of life.
@rep_movsd @GossiTheDog

@RustedComputing @futurebird @rep_movsd @GossiTheDog these this are absolutely not in any way brain like.

@rep_movsd @GossiTheDog

"LLM doesn't need to be trained on such content to be able to generate them."

People say this but how do you know it is true?

@futurebird @rep_movsd @GossiTheDog

One way to think of these models (note: this is useful but not entirely accurate and contains some important oversimplifications) is that they are modelling an n-dimensional space of possible images. The training defines a bunch of points in that space and they interpolate into the gaps. It’s possible the there are points in the space that come from the training data and contain adults in sexually explicit activities, and others that show children. Interpolating between them would give CSAM, assuming the latent space is set up that way.

@futurebird @rep_movsd @GossiTheDog

One way to think of these models (note: this is useful but not entirely accurate and contains some important oversimplifications) is that they are modelling an n-dimensional space of possible images. The training defines a bunch of points in that space and they interpolate into the gaps. It’s possible the there are points in the space that come from the training data and contain adults in sexually explicit activities, and others that show children. Interpolating between them would give CSAM, assuming the latent space is set up that way.

@david_chisnall @rep_movsd @GossiTheDog

This is what I've learned by working with the public libraries I could find, and reading about how these things work.

To really know if an image isn't in the training data (or something very close to it) we'd need to compare it to the training data and we *can't* do that.

The training data are secret.

All that (maybe stolen) information is a big "trade secret."

So, when we are told "this isn't like anything in the data" the source is "trust me bro"

@rep_movsd @GossiTheDog

OK you came at me with "Because thats how the math works." a moment ago, yet *you* may think these programs are doing things they can't.

'Intelligence working towards a reward' is a bad metaphor. (Why some see the apology, think it means something.)

They will say "exclude X from influencing your next response" Or "tell me how you arrived at that result" and think, because an LLM will give a coherent-sounding response, it is really doing what they ask.

It can't.