Chebucto Regional Softball Club

@lucydev @KatS Curl is closing their bug bounty program because it's far too easy to use LLMs to produce slop. It doesn't mean you can't use LLMs to produce non-slop, just that it is a technique some people have found to get money with not too much effort, and we haven't yet sufficiently adapted to it. This is a genuine problem.

@lucydev @KatS I have very specifically said "unseen questions."

If memorizing answers was a viable strategy to pass that test, humans would have done so.

If you still believe that there's no possible use for a tool that can get gold on a never-before-used set of math olympiad question given a few hours of access to a reasonably powerful computer, and that the existence of that tool will have no interesting impact on the world... I don't know what to tell you.

@lucydev @KatS Autocorrect on steroids is basically GPT-3 tech. There's a lot more that goes into modern LLMs. A lot of the improvements are due to reinforcement learning, where LLMs learn to predict tokens that actually achieve some outcome, E.G. code that passes tests, answer that is judged "good" by a domain expert. There's still token prediction involved of course, but it somehow turns out that token prediction can get better scores than any human at (unseen) math olympiad questions. And people still say it's not in any way intelligent...

@lucydev @KatS Nothing is ever gonna work right, not even humans. Different technologies are at different points on the price-to-mistakes curve, our job is to find a combination that minimizes price while also minimizing mistakes and harm caused.

E.G. it is definitely true that humans are much, much better psychologists than LLMs, but LLLMs are free, much more widely available in abusive environments, speak your language, even if you are in a foreign country, and work at 4AM on a Saturday when you get dumped by your partner. Human psychologists do not. Very often, the choice isn't between an LLM and a human, the real choice is between an LLM and nothing (and the richer you are, the less true this is, hence the "class divide" in opinions about tech). And I'm genuinely unsure which option wins here, but considering the rate of change over the last 3 years, I woulndn't bet towards "nothing" winning for long.

@lucydev @KatS The more you know about LLMs, the more "calibrated" you are about where they work (and don't work) right now. People who don't know much about them are either hypesters (mmaking a company of a thousand LLMs and firing all their employees), or LLM deniers. Both are just as crazy.

I also see not just where LLMs are right now, but where they are going. We went from coding agents being basically a joke a year ago, to them semi-autonomously solving (some) complex mathematical problems and being used for boring gruntwork by world-class, fields-medal-winning mathematicians. They can now also solve an extremely complex GPU performance engineering task that Anthropic used as an interview question for the most brilliant engineers in that discipline, *better than any human given the same amount of time*.

They're still much better at small, well-scoped and bounded tasks than at large open-ended problems, but "small and well-scoped" went from "write me a linked list implementation unconnected to anything in my code" to "write me a small feature and follow the style of my codebase." In a year. What will happen in another year? 5 years? 10 years? God only knows, and he certainly isn't telling.

@KatS @lucydev Yes. At least the kind of art that people want to consume (and not want to make).

@lucydev @KatS Because I use it every day, and I can see how much it helps. And to be fair, it primarily helps people who get X done, not the doers of X. Just as automated telephones primarily help those who want to make phone calls (by making them cheaper, faster and much more convenient), not the phone operators who helped to make them in the past.

@lucydev @KatS Currently a PHD student in the field. Have papers published. Presented at conferences.

@KatS @lucydev It democratizes in the public transit way (by making transport available to non-drivers), not in the car way (by making it easy).

And btw: all art is uncredited and a lot of it is unconsensual. Outside of academia, it's extremely rare to credit every single influence that an artist used, down to Da Vinci or the Gregorian chants, as long as snippets significant snippets aren't extracted directly from that work, something that AI only does when prompted.

@lucydev Humans are fundamentally unreliable tech. Unreliable in different ways, but still unreliable.

@lucydev sometimes that's the right call, particularly when you want to find out what users like and you don't yet know what you're building. It very often isn't.

@lucydev I think we're painfully re-learning the lessons we learned in programming over the last 70 or so years with AI, just like crypto had to painfully re-learn the lessons that trad fi got to learn in the last five hundred years.

Yes, you can 20x your productivity with AI if you stop worrying at all about architecture and coding practices, just like you can 5x your productivity without AI if you do the same thing. Up to a point. Eventually, tech dept will rear its ugly head, and the initial gains in productivity will be lost due to the bad architectural decisions. Sometimes that

@lucydev I see putting a prompt into AI and hoping that the generated code is correct as a bad idea, especially in complex apps that have long-term maintainability considerations, or when security / money / lives are at stake.

For throwaway projects (think "secret santa style gift exchange for a local community with a few extra constraints, organized by somebody with 0 CS experience", vibe coding is probably fine.

For professional developers, LLMs can still be pretty useful. Even if you have to review the code manually, push back on stupidity, and give it direction on how to do things, not just what to do (which is honestly what I do for production codebases), it's still a force multiplier.

@lucydev It democratizes it by making it available for the people who can't / don't want to / don't have the time for learning it.

We're already seeing non-programmers successfully create quite substantial coding projects with AI, to an extend which surprises even me, who was a huge proponent for AI in coding from the start.

Same applies to art, there are many people who need or want art (small business owners, hobbyist game creators, wedding organizers, school teachers), but don't have the budget for the real thing.

Of course, many artists and programmers don't want this to happen and try to invent reasons why this is a bad idea, just as phone operators didn't want the phone company to "force" customers to make their own calls, and just as elevator drivers tried to come up with reasons why driverless elevators were unsafe.