Seems like the more people pride themselves on spotting nonsense, the more they seem to be advocating this shit these days. People have entered into this weird phase of mass AI hysteria and only those that don't use it at all are sitting here like... Am I crazy or are the hoards of AI enthusiasts crazy? It's gotta be one or the other.

@Smoljaguar

Wouldn't you need to ask it about each article individually and track the results?

Not just give it a stack of articles and ask "how many of the articles mentioned X" ?

@futurebird yeah, that's what the correct thing to do would be, but it is still plausible that it could do the second, it's just more likely to make a mistake (though I think a task of this difficulty is pretty doable for current models with huge contexts (1M tokens), unlike older/cheaper models which had severe quality drop offs after maybe 10k tokens)

That's not even the point of what I said at all, but nm.

@Smoljaguar

If it says there are 67 articles that mention topic X, but you don't know if that number is correct, it's just a guess based on context and the bulk of text (and LLMs are also bad at following commands such as "consider only these sources" ... ) what is the point of saying the number.

Maybe could you ask if a topic is mentioned "frequently" or "infrequently" but beyond that I think it's deceptive and useless.

@futurebird @Smoljaguar I’d do it with a loop. For each document, does it contain X, Y or Z? I’d end up with a table of document names and booleans.

@Moss

Damn thing will sit there and tell you that's what it's doing.

But it can't count! It still can't count. I feel like I'm going crazy. Am I the only person who cares that the machine can't even count?

I mean LLMs are based on statistics, and they will produce results that look like frequency charts. But these charts only attempt to approximate the expected content. They aren't based on counting articles that meet any set of criteria.

It's... nonsense, and not even people who pride themselves on spotting nonsense seem to understand this.

@futurebird regrettably being that guy:

In context of how LLM deep research workflows are built, I do think you might need to show your work on this claim more than OP does

The model is not the only operative mechanism in such an investigation

In that approach, the model would be invoking (deterministic) tools that, among other things, could log instances of topic areas encountered within a corpus. OP says they are capturing abstracts and authors and grouping them by year. Objectively this category of work is something these tools can be built to do really well, including citations (to real, verifiable URLs). Statistical modeling tasks, including text analysis, can be offloaded to one-off scripts written and executed specifically for a requested job. Perhaps the model can’t tell you the “R” count in strawberry, but it can write Python which does quite well

Moreover, it is possible to objectively evaluate the performance of these tools for such tasks (and Anthropic, vendor of OP’s research tool, does this)

I mention all of this because I find this particular flavor of strawman quite pernicious: the limitations of the raw model architecture are entirely possible to mitigate through larger agent and tool scaffolding, and this work is constant, ongoing, and often quite effective. Critique of the technology and its vendors (essential) is meanwhile less effective when claims like this are so easily disproved by experience, usage, and public information

Here’s a bit more detail on the architecture point.

How we built our multi-agent research system

On the the engineering challenges and lessons learned from building Claude's Research system

(www.anthropic.com)

@danilo

Is that what the guy in the video is doing?

@futurebird according to what he describes in the methods section of the video, he is doing an entirely plausible research task with a tool well suited to it, yes

The post I linked describes how it works

@futurebird @Moss
“ But it can't count! It still can't count. I feel like I'm going crazy. Am I the only person who cares that the machine can't even count?” -
I also feel deep incredulity towards this corporate-grade “confabulation”.

@dahukanna @futurebird @Moss

It’s a shame that it lists summarisation as something LLMs are good at, when all of the studies that measure this show the opposite. LLMs are good at turning text into less text, but summarisation is the process of extracting the key points from text. LLMs will extract things that are shaped in the same way as a statistically large number of key points in the training set but they don’t understand either the text of the document or your context for requesting a summary and so are very likely to discard the thing that you think is most important. They also have a habit of inverting the meaning of sentences when shrinking them.

@dahukanna @futurebird @Moss

It’s a shame that it lists summarisation as something LLMs are good at, when all of the studies that measure this show the opposite. LLMs are good at turning text into less text, but summarisation is the process of extracting the key points from text. LLMs will extract things that are shaped in the same way as a statistically large number of key points in the training set but they don’t understand either the text of the document or your context for requesting a summary and so are very likely to discard the thing that you think is most important. They also have a habit of inverting the meaning of sentences when shrinking them.

@dahukanna @futurebird @Moss

It’s a shame that it lists summarisation as something LLMs are good at, when all of the studies that measure this show the opposite. LLMs are good at turning text into less text, but summarisation is the process of extracting the key points from text. LLMs will extract things that are shaped in the same way as a statistically large number of key points in the training set but they don’t understand either the text of the document or your context for requesting a summary and so are very likely to discard the thing that you think is most important. They also have a habit of inverting the meaning of sentences when shrinking them.

@david_chisnall @dahukanna @Moss

I'm not an AI prohibitionist or "hater" however I keep finding the effective use case is much much much more narrow than the UI we have been shown to use these tools would suggest.

And a lot of people really seem to find it "easier" than searching the web, which, given the current state of the web isn't saying very much.

Has web search been broken to push everyone to the chatbots? (adjusting my tin foil cap here)