@funkula

I don't think any serious person really believes in the AGI fairy-tale anymore. "The machine will learn to improve itself and become god like!"

But there is a softer version of LLMs growing to dominate our world. If a significant portion of the population decides to treat these systems like their best friend and confidant all of those people become very easy to manipulate.

And this is exactly how the people creating these systems expect to cash in.

Forget traditional advertising.

@funkula

I have recently been feeling very self-conscious and dissatisfied with the interior design of my apartment. I've posted about it a lot recently and by interacting with people on this website I've decided that I don't need to throw everything away and buy new items, I can elevate my space through organization and repairs.

Where might I be if I talked to an LLM instead? I suspect I'd be thinking more about spending money. And couldn't a company pay to make that more likely to happen?

@funkula

Our political opinions, our purchasing decision, even our emotional well being are shaped by interactions with other people.

Everyone rubs off on you a little. You are who you choose to talk to.

Who do you become if you talk to an LLM most of all? Whoever they want you to be I suppose.

And it seems likely that the LLMs are *already* designed to keep people using the product. So, the first brainwashing is "I'm the best one to talk to" then "Only I can really understand you."

@funkula

I *could* be convinced that I "need" to spend 10k on a bathroom renovation. But the people in my life: my family, my friends, their needs and values but also the people I choose to interact with online can make that rather large decision go the other way.

If I wanted to sell things to people and didn't have morals I'd want them ONLY talking to my LLMs and those LLMs would work on the assumption that buying my product was normal and essential.

Why doesn't this scare people more?

@funkula

I *could* be convinced that I "need" to spend 10k on a bathroom renovation. But the people in my life: my family, my friends, their needs and values but also the people I choose to interact with online can make that rather large decision go the other way.

If I wanted to sell things to people and didn't have morals I'd want them ONLY talking to my LLMs and those LLMs would work on the assumption that buying my product was normal and essential.

Why doesn't this scare people more?

@futurebird I believe, people think they’re being manipulated “anyway”, and it “won’t get much worse” - but it can get a 100 times worse. We westerners just aren’t used to those considerations…

@funkula

It gives me the same sad feeling I get when I hear people talking about how much they "depend" on an LLM.

But... it's just sanding all of the difference out of everything. Doesn't that make you sad too? Doesn't it feel gross to get any sense of validation from such a source?

One woman said she didn't want to bother real people with all her insecure questions. It's just sad all around.

@promovicz

I mean. "we are already being manipulated" is already true. One of the reasons I started thinking about renovation is due to all of the interior design videos and magazines enjoy. And these all have an undercurrent of "renovate and spend" because ... well, consider who makes them.

But that was easier for me to compartmentalize than a conversation. If everyone on here was posting about their renovation it WOULD make me wonder "why can't I do that too?" and "should I do that?"

@futurebird @funkula Yeah... That's a thing Eliza also brought out: people want to talk to some/one/thing they know won't judge them or remember what was said.

@funkula

I have recently been feeling very self-conscious and dissatisfied with the interior design of my apartment. I've posted about it a lot recently and by interacting with people on this website I've decided that I don't need to throw everything away and buy new items, I can elevate my space through organization and repairs.

Where might I be if I talked to an LLM instead? I suspect I'd be thinking more about spending money. And couldn't a company pay to make that more likely to happen?

@futurebird @funkula

Ok the topic of organization:

My apartment comes with basement storage (5 apartment brownstone so the basement is divided in 5 zones and each unit gets one)

Something I did recently which has undeniably made my life better was measure the shelves in my apartment and basement, find a box that would fit well on those shelves and buy 40 of that box.

Now moving things around is super easy and a bunch of previously unusable shelf space opened up

I search: garnet ant ring
Search "corrects" it: garnet and ring

I try: garnet "ant" ring
Search "corrects" it: garnet "and" ring

I try: garnet "ant" ant ring
Search "corrects" it: garnet "aunt" and ring

I try: garnet "ant" ant insect ant ring
Search "corrects" it: garnet "aunt" and insect ant ring

This is a wonderful technology that makes search so much easier. This was on etsy.

@promovicz

This copycat, monkey see, monkey do, instinct is very real and strong in all people. It's part of how our minds work. I also think there is an aspect of modern (American?) individualism that says that if you can be influenced in that way you are weak willed and terrible.

This is very silly. Of course what you see other people doing has a direct impact on what you do. No man is a rock.

@futurebird This reminds me of gender - because a lot of it is learned by mimicry. Cis gender is - more or less - the culture of rejecting that idea.