@neckspike @futurebird @AT1ST In all seriousness, don’t take this the wrong way but:
So what?
Why is that important?
What do you mean by “know what it is saying”?

Do you know what you are saying, or are you just repeating arguments that you have read before?

But maybe you do have a meaningful distinction here.
If so, what question can we ask it, and how should we interpret the response, to tell the difference?

The pizza thing is not particularly interesting because it’s just a cultural literacy test. It’s common for humans new to an unfamiliar culture to be similarly pranked. And that was a particularly cheap AI.

@nazokiyoubinbou @futurebird @CptSuperlative @emilymbender I've used it for that sort of roleplay somewhat recently. I fed an LLM some emails from and described the personality of and group dynamics surrounding my nemesis (condo association governance is important but fuuuucckk), and asked it to help me brainstorm ways of engaging on our community listserv that wouldn't leave them too much room to be ... the way that they tend to be. It was actually super helpful for that.

@neckspike @futurebird @AT1ST In all seriousness, don’t take this the wrong way but:
So what?
Why is that important?
What do you mean by “know what it is saying”?

Do you know what you are saying, or are you just repeating arguments that you have read before?

But maybe you do have a meaningful distinction here.
If so, what question can we ask it, and how should we interpret the response, to tell the difference?

The pizza thing is not particularly interesting because it’s just a cultural literacy test. It’s common for humans new to an unfamiliar culture to be similarly pranked. And that was a particularly cheap AI.

@marshray @futurebird @AT1ST
Are you one of those people who believe you're the only truly sapient being and everyone else is an unaware "NPC" just going through the motions of life to trick you or something?

To me it defeats the point of communication, which is to share our thoughts and impressions and ideas with one another. A thing that can't think can only show you a distorted reflection of yourself in the slurry coming out of the chute.

It's just an extremely fancy version of the markov chain bots we used to play with on IRC because sometimes it would spit out a funny line, but with massive data centers of power behind it instead of some spare cycles on a desktop.

Treating these things like they're anything approaching thinking is simply marketing bullshit.

"Chat GPT told me that it *can't* alter its data set but it did say it could simulate what it would be like if it altered it's data set"

NO. It has no idea if it's telling the truth or not and when it says "I can simulate what this would be like"

This guy is pretty sharp about philosophy but people really really really does not *get* how this works.

"Chat GPT told me this is what it did"

No! It told you what you thought it should say if you asked it what it did!

@neckspike @futurebird @AT1ST Absolutely not. I consistently speak out against the term “NPC” as I feel it is the essence of dehumanization.

I’m going around begging people to come up with a solid argument that our happiness does in fact bring something important and irreplaceable to the universe.

"Chat GPT told me that it *can't* alter its data set but it did say it could simulate what it would be like if it altered it's data set"

NO. It has no idea if it's telling the truth or not and when it says "I can simulate what this would be like"

This guy is pretty sharp about philosophy but people really really really does not *get* how this works.

"Chat GPT told me this is what it did"

No! It told you what you thought it should say if you asked it what it did!

@futurebird He’s not wrong about that question though.

@futurebird @marshray @trochee

I do wish philosophy would get off its dead ass and provide us with a cogent vocabulary for what's going on in the machines.

The closest to a competent philosopher is WVO Quine and his school. Google "gavagai" and the indeterminacy of translation.

There are at least a dozen ridiculous attempts to Explain AI and they're all worthless. Our understanding of reality is always relative to a background theory

@tuban_muzuru @futurebird @marshray @trochee

Or maybe it's AI that's worthless?

Mathematics has pretty sound understanding of how generative AI works, but 99% of people ignore it because it's telling them that even the men behind the curtain don't understand why the trick works.

@marshray @neckspike @AT1ST

"I’m going around begging people to come up with a solid argument that our happiness does in fact bring something important and irreplaceable to the universe."

Can you expand on what you mean by this? It feels like a totally new topic.

@joby @futurebird @CptSuperlative @emilymbender Yeah, there are a lot of little scenarios where it can actually be useful and that's one of them. The best thing about that is it merely stimulates you to create on your own and you can just keep starting over and retrying until you have a pretty good pre-defined case in your head to start from with the real person.

As long as one doesn't forget that things may go wildly differently IRL, it can help build up towards a better version of what might otherwise be a tough conversation.

@futurebird @CptSuperlative @emilymbender To be clear on this, I'm one of the people actually using it -- though I'll be the first to admit that my uses aren't particularly vital or great. And I've seen a few other truly viable uses. I think my favorite was one where someone set it up to roleplay as the super of their facility so they could come up with arguments against anything the super might try to use to avoid fixing something, lol.

I just feel like it's always important to add that reminder "by the way, you can't 100% trust what it says" for anything where accuracy actually matters (such as summaries) because they work in such a way that people do legitimately forget this.

@futurebird He’s not wrong about that question though.

@dalias

Maybe but that still implies some kind of organization of concepts beyond just through language or the shape of their output.

I don't see any reason why it should be impossible to design a program with concepts, that could do something like reasoning ... you might even use an LLM to make the output more human readable.

Though I guess this metaphor works in that to the extent there is a "goal" it's to "make it pass" rather than to convey any idea or express anything.

@futurebird @dalias We *have* programs with concepts which do reasoning. The code for your favourite spaceship computer game clearly has the concepts of “spaceship”, “laser”, “planet” etc encoded in it, and all the necessary machinery to reason about what happens when a laser hits a spaceship while it’s orbiting a planet. We have real-world examples as well, reasoning about questions like “what happens if the tide is coming in and we close the Thames Barrier?”

@CptSuperlative @emilymbender

I have found one use case. Although, I wonder if it's cost effective. Give an LLM a bunch of scientific papers and ask for a summary. It makes a kind of nice summary to help you decide what order to read the papers in.

It's also OK at low stakes language translation.

I also tried to ask it for a vocabulary list for the papers. Some of it was good but it had a lot of serious but subtile and hard to catch errors.

It's kind of like a gaussian blur for text.

@paulywill @futurebird As far as I understand it, chatgpt will take your request and use a diffusion model (Dall-e 3) to create an image. It will come up with a prompt and feed it to the other model and then just show the result. In no sense does chatgpt actually generates the image.
BTW, it seems other diffusion models have the same issue. Here's Stable Diffusion's and Flux's takes

"Chat GPT told me that it *can't* alter its data set but it did say it could simulate what it would be like if it altered it's data set"

NO. It has no idea if it's telling the truth or not and when it says "I can simulate what this would be like"

This guy is pretty sharp about philosophy but people really really really does not *get* how this works.

"Chat GPT told me this is what it did"

No! It told you what you thought it should say if you asked it what it did!

@futurebird @CptSuperlative @emilymbender Summarisation is the one I’d be most nervous about because creating a summary is hard: it requires understanding the content and knowing which parts are relevant in a given context, which is why LLMs tend to be awful at it. They don’t understand the content, which is how you get news summaries that get the subject and object the wrong way around in a murder. They don’t know what is important, which is how you get email summaries that contain a scam message and strip all of the markers that would make it obvious that the message is a scam.

If you’re going to read all of them and are just picking an order, that’s probably fine. The worst that a bad summary can do is make you read them in the wrong order and that’s not really a problem.