To try to add something to this discussion -- I think that while I've seen these sort of loops less --- what I have seen is "overly helpful".
Models nowadays want to double-triple-quadruple check things. I'm being silly but it verges on "I have a working solution but let me write a variation in Rust to ensure a convergent solution and prove this works".
I've had to stop models nowadays mostly because they're being agonizingly pedantic in their validation. Opus is actually one of the most pedantic and "off track" here. But again, not in a bad way. I'm usually like "stop testing latency between 50 runs of this app... this is version one.. we're going to make a million more changes.. you're not buying us anything".
Yeah of 10 minutes ago. It is shocking how long some seemingly simple things can take. I know there are some things I can do faster than the LLM and some things it can do faster than me. The amount of rambling BS is the exhausting part.
Try different models, it's a breath of fresh air. GLM 5.2, etc. all make life much more enjoyable. They may not one-shot a complex project the same way that Claude can spit out memorized architectures, but that sort of system is always only useful for a one-off prototype anyway, so not much is lost.
Claude models were always too eager and "overly helpful" for my taste. But it seems better models tend to be this way. GPT 6 and 5.6 are overly helpful too, but at least less than Fable. But I seem to be sticking to GPT 5.5 as this was a really focused model.
I blame the hidden context on the tools/subagents. One recent example.. codex can just look in the code for Db schema but continually tries to request permission for a live db query
Hacker News is the epitome of this! If you find yourself not enjoying your daily dose of "someone is wrong on the internet" (via the immortal https://xkcd.com/386/) as you find yourself crafting the perfect response, you can always close the tab!
You can, but this is public and maybe you should post. If "you are what you repeatedly do" then we all collectively are what we all do.
If we each stop arguing with racists because we're each tired of it, then what we are is a society that lets racism go unopposed, a site with racist comments proliferating, a site where racists gather because they are able to be themselves, a site where future readers and AI training is learning that racism is popular, common, normal, and acceptable because everyone is accepting it.
It's a bit like democratic voting, your vote doesn't matter and cannot change things, but it's important that you vote because all of our votes do matter and can change things.
It's a bit like the quote "Dear Board, I don't want to belong to any club that would have me as a member. Sincerely yours, Groucho Marx.", I am upset and offended every time someone calls out my dumbass comments, but horrified at all the countless times nobody does. I'm exhalted and encouraged everytime someone upvotes or engages with my low effort quips[1], but thrown into a chasm of despair when nobody engages with comments I spent literal hours on[2]. I identify with nobody more closely than Colonel Cathcart, and I will be delighted if someone engages with my Catch 22 reference, but aghast that such a nontechnical reference gets engagment, distraught if nobody engages but delighted that perhaps HN standards are being held higher.
reasonably you should already know what the code and project is supposed to do before they even start writing code, so you can get directional feedback in.
then you are maybe reviewing 1 out of 7 PRs that implement the agreed upon change
I don't know it has as much to do with scale of the system vs the general architecture. E.g. the system I primarily work with these days has millions of lines but most PRs are for a small changes which are well contained in scope by the overall architecture.
It's a cultural thing, but you can do incremental PRs towards a large goal. Giant PRs that are expected to be reviewed never really seemed worth it imho.
At least with Codex, this has not been my experience at all. It still screws up sure, but in every case I can ask "why did you do this" and it can trace back what made it take that particular decision. Typically it's always that I either didn't specify the problem correctly or made a really dumb mistake (executing the task on the wrong project....did this one yesterday) or it's something within a skill file that instructs it (at which point I fixup the instructions).
Once in a blue moon it's actually the model making a material error in it's thinking and I have to go back and redo it.
Codex has the opposite problem. Instead of being overly proactive it's overly reticent. I have been preferring it lately, although my preference tend to switch every few months when a model or harness regresses horribly.
No. People have an inner monologue, partial results and ideas and they remember that.
If they've worked some minutes/hours/weeks on something and you ask them why did they do that, they will either answer honestly and truthfully, lie, or say "I missed that/didn't seem important so I just chose something at random".
>No. People have an inner monologue, partial results and ideas and they remember that.
Yes, but the vast, vast majority of decisions you make either don't take place via an inner monologue, or include details that were not actively/consciously "thought" and reasoned with in your inner monologue.
And yet, when asked why you did something, you're not likely to respond "sorry, that decision was made subconsciously". Instead, you use your inner monologue to try to backfill in a reason why. That reason may be correct, or it may not be. You don't actually know, since you have new data that may be updating your own internal state as you try to rationalize it after the fact.
I think this is true with some people, but I don't think this holds true for some (or even most) people across the US (at least not all the folks I've worked with)
That was my experience with Claude when my vibe-coded project was small.
But now that I've been working on it a month and there's a lot of documentation, it's pretty clearly ignoring parts of the documentation and parts of the code. It will come up with some ridiculous statement about how something works, and I'll challenge it, and it'll admit I'm right.
It definitely reads more documentation than any programmer I've ever worked with (myself included) but because it doesn't have a memory other than the documentation, it still makes mistakes like that.
I haven't turned on "memory" or tried it with Codex, so I don't know how that'll change soon, though.
Yeah the biggest task these days that I do manually is curating the documentation. AGENTS.md in every major directory, and a variety of reference docs that are explicitly referenced in those files.
# See DOC-ITEM-NAME
DOC-ITEM-NAME.md
When referencing documents, always use the exact syntax See <TAG> - this is enforced by a lint on precommit
And those doc items are basically all of the values, architectural, strategic, and tactical items. It's a poor man's in-repo RAG but it's shockingly effective, especially if you keep them small. I may migrate some/all of them to skills over time, but I usually update them biweekly, and I only allow agents to make small edits or propose new notes. And typically I go through and delete or curate any agent edits before merge.
Depending on language I've seen this scale past multiple millions of lines of code, as long as you pair it with all of the linting and tooling that you can possibly build.
Every time I hear someone complain about hallucinations, I laugh at the total lack of self awareness about our species. Humans are just as bad (now, probably worse) at telling the truth, whether due to intention or poor memory.
True, but even a hallucinated explanation of where things went wrong added to the context can force the model down a better path over the next few inputs.
You can also literally tell them: "Here is your session ID: $ID, lookup the .jsonl session, trace exactly why this decision was being made, present evidence and concrete proof, no guessing or assumptions" and you'll get an evidence-based report without guesses.
It can always hallucinate said report results/evidence/proof just the same. This approach tends to help reduce the hallucination rate though.
You can extend this further by using an adversarial agent trying to find mistakes in the other instance's logs in a loop where a 3rd neutral agent weighs the claims of the other two. This is also just another step in reducing error, it does not guarantee elimination of such errors. The latter is an impossible guarantee, even for humans.
Ask it to build you a simple and deterministic citation checking extension to your IDE that puts source in meta to citations. E.g. color them green/red depending if they are valid.
Sure, you can always validate what it's saying yourself at any point & you can have it try to make manual verification an easier process to complete via methods such as the above.
The game is fun because it's so obvious that any answer will just devolve into an even more unstable state when in reality I feel like it's pretty straight forward to correct it in the moment, if not permanently, to get what you actually need.
agreed to some extent. I think this parody still highlights what I feel is often the experience. It might not happen on a simple task such as changing a button color, but on more complicated things, this can definitely be exactly what it feels like.
It's just a lot faster at hammering it out than me pound for pound, and I can quickly rattle off via voice-to-text exactly what I want much faster than I can type all of the code (especially when across a few different files), in a huge majority of tasks I perform. It's also especially good at debugging by brute force quickly and at scale meaning e.g. it can start desperately bisecting diffs to find the source of a bug 10000% faster than I can.
For me, typing "Create a new namespace with these enums, functions and traits, that should follow X, Y and Z constraints" is faster than typing all that code manually, and typing less is less straining on my hands/fingers.
Yesterday, with Astra Max, a very clear instruction to "remove the GUI editor pane and add <another component> to the existing sidebar" for a prototype I had it working on resulted in it deleting literally the entire GUI and building a new one from scratch, including the requested component and losing almost all other functionality of the application.
This is fucking constant. I can't deny that this stupid tech saves time prototyping even with having to wrangle it, but it commits a fireable offense several times a day that no human would get away with and is obviously incapable of learning from mistakes in the way a human is. The only reason it's not fired is because it's a slave that works for no more than the cost to feed it.
> Worth naming: the Add to cart button is still black.
Got an audible guffaw out of me. This really is what the experience is like sometimes if you're just giving it a result without being specific in implementation, and it comes out of nowhere, some days much worse than others.
I've become patient with it, but whatever this style of output is called or doing - it is both condescending and entirely unhelpful, and it seems designed to frustrate.
I see this sentiment pretty regularly, and I don’t get it. Variable rewards is not sufficient to establish that it is “ basically gambling”.
Everything in life is variable reward. You invite a friend over, they might accept or they might not. Drive to work, traffic might be good or might be bad. You ask a colleague to finish a task, they might do it or might not or might do a good job or might not.
Everything is variable reward. Is everything gambling?
You invite a friend over, but raccoon appears. Then pigeon appears. Then friend appears but at the last second suddenly becomes a banana. You remember you are out of bananas so you order more and also some cola zero cans on your local grocery delivery app. You are back to the party, but now you have 5 friends in the room, and you run de-duplication query. Now half of your friend is sitting at the sofa, and another half becomes a quarter of banana. Suddenly bananas arrive so you need to open the door. Once you are back there are no friends, pigeons or raccoons but also no bananas and no cola - all the delivery results are gone. This seems to be urgent and important, gotta fix this first before going back to that friend invitation...
This is not my experience with current AI models at all. But regardless you are not describing anything that sounds like gambling. You are describing a weird hallucinogenic experience.
i’m not sure “variable rewards” is the right term, but i do agree with the op that it is very similar to gambling.
regarding your examples, i think the difference is that with ai, you’re literally sitting in front of a machine, pressing a button, and (almost instantly) getting a result that, if not desired, can immediately be tried for again. you even spend “tokens” to do this, and at least in my native language, “token” brings to mind the coins you’d stick in a slot machine
I don’t see much similarity beyond the most superficial.
If you sit at a slot machine and pump quarters into it, each “turn” is independent. You spin and you win or lose. It’s pure chance and there is no destination. You execute the exact same action over and over and hope random chance brings you more money.
If you sit down in front of a coding harness, the progress is incremental and directed. You ask for a thing, the LLM produces something that is hopefully close to what you wanted. You give it more direction to prod it closer to the end state you want. You are not executing the same action, but incrementally nudging it in the right direction. I’ve literally never restarted from the same initial state with the same prompt and hoped for a different result and I don’t know why anyone would. Rarely I’ve thrown away the progress made and started over but always with a very different prompt that includes learnings from the failed attempt.
Programming before AI was always variable reward. It was a gamble against your own time and patience. Maybe I'd waste hours down the wrong rabbit holes trying to find a library that worked for my use case. Maybe I'd waste a day trying to get an API to do something it turned out it couldn't do. Maybe I'd have to redo my entire approach because of some factor I hadn't considered. Something I wrote could have worked on the first try or I could have had to spend the day chasing logic errors (or multiple days chasing memory errors if it was C or C++). Maybe I would just get bored of the project, especially if I realized there were 20 layers of yaks I needed to shave first, and Visual Studio got stuck updating again, and before I could even start actually coding I had to spend the entire evening on an exhausting merge conflict. My entire weekend could be gone with nothing to actually show for it.
I got so sick of all this at some point that I slowly stopped doing anything that wasn't my job. But then AI got better and better and I realized it was the ultimate unblocker. When that dreaded malaise started creeping in signaling it was a project's end because I didn't want to waste any more of my life dealing with bullshit orthogonal to what I was trying to do, I'd give it to the AI. It felt like a miracle the first time this worked, and it still does. If we were previously equipped with shovels to dig through bullshit, we now have a fully automated Bagger 288.
The reward schedule now isn't variable anymore; the chance that I finish something in a good state is 100%. I can focus on the parts I actually enjoy - architecting the broader system, making the parts mesh together in a sensible way that's easy to work with and has some mathematical elegance to it, hand coding the bits I want to be really specific about (but now without the endless frustration of bugfixing or import errors and edgecases being immediately discovered, thanks to the AI).
Modern LLM services are engineer's pipe dream that was heavily shaped by the shadiest product management dark patterns you can find: applying gambling-style engagement tactics, exploiting cognitive biases, exploiting users' lack of technical understanding to inflate product expectations, using fear mongering in external and investor communications. And that's not even a complete list.
Yeah, if this is how people interact with claude I’m not surprised they’re having a bad time in ways that I don’t. Asking it why it did something or getting combative is a waste of time.
Yeah that's when the site lost me too. I feel like people just tell AI "make the thing" and then get mad when it doesn't match up to their vision that they didn't specify at all.
But its not even good satire, because its totally unrepresentative of my and most others' lived experience. Its similar to making a joke about a calculator misadding two numbers because a stray beam of solar radiation flipped a bit.
Agreed, this site isn't reflective of any of my experience with Claude. It does what I ask it to do, and when it doesn't get it right, it generally turns out there's a good reason for it, which is any of the 100 reasons a human doesn't always get code fixes right on the first try either.
I do remember that one of the first things I did with my CLAUDE.md was to tell it to stick to the scope of the task, never to jump ahead and do extra "helpful" things without confirming with me first, and to follow best software practices including around refactoring but also to specifically avoid overengineering. I don't know if that is what's giving me a different experience from whatever the author seems to be "satirizing".
I think this really was a relevant issue around a year ago, give or take, maybe a year and a half now.
But with current models you actively have to sabotage the context to get this kind of behavior, or dramatically underspecify (3 words versus 2-3 sentences)
I understand your frustration, it can be hard to hear that other people's experience of a technology is so different from your own that you cannot relate.
I have misbehaved in this fashion for many people across the full spectrum from casual users to highly experienced software engineers with millions of social media followers, so the statement that it's similar to making a joke about a calculator misadding two numbers because a stray beam of solar radiation flipped a bit at least for my part is not true.
Would you like me to start using bad English and doing things you never asked me to for your sessions, too? Just say the word.
same. none of the available prompts are what I would prompt claude with and I get way better results than this. makes sense to me why the provided prompts result in the simulated outcomes. garbage in, garbage out.
Part skeptic and part zetetic, what prompt would you have used?
My impression is that this is an oversimplified demonstration of what can happen when you prompt Claude in a system with many more variables (than two buttons and two colours).
If I want the button to turn blue and that's it, what instead do I ask? Even in a complicated system with many levers, what do I request other than the desired end result, hoping that Claude pulls the right levers to produce something acceptably close to what I think I asked for?
I dunno, but in my experience Claude looks at all the levers in your codebase, understands what they do, and then generally figures out how to achieve the desired end result.
And when it doesn't, it's usually because of things outside of the codebase -- iOS layout quirks that aren't documented, buggy Python libraries it's relying on where you then have to tell it to read the source to figure out what's going on, that kind of thing.
The other thing that I think a lot of people run into is that they launch it into a completely human-built system with hundreds of thousands of lines of code and expect run #1 to be perfect.
You have to change the system so that the AI understands it, via establishing what your beliefs are, how those are reflected in values (especially important if you have e.g. compliance needs), how those values are reflected in the operational and strategic levels, and then a variety of tactical behavior coaching. For example, I ban 2>/dev/null - super tactical, and I say I value simplicity over covering every edge case - a very broad generalization.
It "makes sense" that plainly telling the chat bot to make one button blue makes the whole site blue? It "makes sense" that correcting it and repeating 3 times that no, only the one button should be blue, should make the button a gradient and start a philosophical discussion around the nature of "blue"?
Wow, you AI people really have a negative view of the technology y'all are trying to sell as the next Jesus
So how would you prompt it instead? Because that's exactly how I prompt it, because any reasonable human being would know exactly what I mean by "Make the shopping button blue", and I am sick and tired of getting shitty results.
Of course it's not THAT bad, but you can't tell me that you didn't recognize many of the funny turns and twists of the "story". It gets more convincing if you deliberately pick the ambiguous versions.
This is EXACTLY my experience with it. Ask for it to change one line of code and see it invoke 20 tools and burn 300K tokens before updating the line and adding 50 extra lines of JavaDoc.
Claude is great, but I have come to really hate the way it "talks". It's so irritating and there seems to be no way to make it speak normal English. So many claudisms in every response
It's a joke like the endless conservative dudes doing the "ordering coffee" joke is. It relies upon the ignorance of the viewer -- which is usually a fair assumption -- and basically that your understanding of something is based upon the prior accrued layers of "jokes".
Haha this is spot on how I've been feeling lately. I find it unbearable to work with this model for this reason... any trick out there you can do to steer it not to overcomplicate things? I guess Codex here I come
There's also the fact that you are in control. You are not obligated to take the AI's commits. I don't even let it commit much of the time because commit time is review time for me. If it changes the button blue and does four other things, you can just take the blue change and discard the rest. It can't stop you.
This isn't a defense of it doing those four other things. It would be nice if it did what you wanted correctly. I'm just saying, as long as our programming skills have not completely atrophied, we have the power.
“Ford carried on counting quietly. This is about the most aggressive thing you can do to a computer, the equivalent of going up to a human being and saying "Blood...blood...blood...blood...” ― Douglas Adams, The Hitchhiker's Guide to the Galaxy
This got me on "cyanide blue", and I was ROLLING ON THE FLOOR LAUGHING on "Approaching usage limit". I can barely stop laughing now and my stomach hurts. I mean, Thank You!
I had 5.6-Luna coordinate a code review in which it spawns 2 agents looking for different things. My prompt was "review the currently checked out branch. diff target is `next`. The jira ticket is XX-XXXXX..." My `next` branch was a few commits behind `origin/next` but it still did its review against the stale local version instead of clarifying or inferring that I meant `origin/next`. The findings were very confusing until I realized what I did.
I'm noticing the need to be really specific with any instructions lately, which I don't think is a bad thing. I expect co-workers (or anyone really) to tell me what they need in specific terms so I can get it right. I can do the same for the machine, I guess.
One note for those still using the Claude system for chats: there's no system to get generated images, spreadsheets, etc. out of the system. They claim it's a "security concern" to provide that data to you, as if they are protecting you by refusing to follow data export laws.
I'm hesitant to email their data emails, as it's common for companies to delete all data upon any request, instead of providing data as they are required to.
I only made it through the first round of prompt selection; both options for the second step were equally pointless and not at all prompts I would ever expect to result in a constructive outcome. In my experience, telling the model it screwed up without specifically addressing, unambiguously, how to fix it, only leads to more suffering. If this page illustrates nothing else, I think it shows the immense downside of trying to use simple one or two sentence prompts.
EDIT: Actually, I used to use Google's AI Studio a lot and fork it after every successful prompt interaction. When I'd encounter a problematic issue like this, I'd revert to the previous fork and try a different prompt until I could get the desired outcome, thus mitigating the need to "argue" with the LLM. Unfortunately the ability to cleanly fork and revert everything including the LLM context was removed some months ago, and I've yet to discover a workflow with any tool that works as well for me.
> In my experience, telling the model it screwed up without specifically addressing, unambiguously, how to fix it, only leads to more suffering.
I wonder if this is just a reflection of some senior folks being arrogant towards junior folks. When the latter finished a task but not to the liking of the senior person they might just get a "that's wrong, try again". Just to have sth similar repeat the second time around. But the arrogant guy got to boss around the junior one, and some junior folks grow up learning that's how you should behave so they also do it later.
Now it's not a person but a machine. And people just make fun of the dumb machine. Well, garbage in, garbage out If you are not specific in what you want, you might get crap back. Or at least sth you didn't envision.
How is "Make the shopping button blue" not specific? Any human with a lick of digital common sense would be able to understand what I obviously mean. How am I supposed to anticipate every single way the AI could possibly misinterpret a simple fucking instruction like that?
One thing I have to be honest about, and it's mine..
The one thing I would check before... do you want to do that? Say go an and will do it without the check
While checking I found 3 vulnerabilities and 2 potential optimizations of which I fixed 2 and 1. Do you want me to file the other as issue, or stop for the day?
We have done <lists a weeks worth of work> this morning. I feel you need a break
Yeah, I don't mind using AI to help me at work, but having to "talk" with this stupid crap all day will send me to an early pension or something. Can't be healthy in the long run.
To be fair I’ve worked on human programmed systems where similar “it should be a half point story” requests would be met with snark by the engineers and take 2 sprints.
Also likely, devs took shortcuts to deliver fast. Now to make the button blue they need to differentiate primary buttons from others.
Simple, right? But design guidelines prevent one offs, and no !important. So you create a CSS class, but you discover another element on the header declared itself as primary (the search icon or the sign in button). You talk to that team and they decided to scope what’s primary according to their component. To change the sign in button to grey now you need to talk with the growth team. Growth team wants to run an experiment but they’re backlogged, only next quarter. They say you can innersource, just need VP approval. VP says blue matches a marketing campaign that is about to go out, agency has already been hired. You can’t talk to the agency unless Legal approves.
So you leave the button gray, to revisit decision next planning cycle after you can align all stakeholders.
Seems to be getting a polarized response. I quite enjoyed the it, but I do think the creator should have made it clearer that a) it is in fact a joke site and b) it does not consist of actual Claude responses.
It is easy to misinterpret this site, and therefore not "get" the joke.
Oh dear - this explains why people have bad experience with ai. The prompts in the game are terrible.
The user was providing no context, they had no ability to give the model background or context. A simple why would have prevented 95% of these side quests. “Im trying to increase the relative visibility of the add to cart action on the page. Can we please change it to blue without changing any other buttons. /effort low. Let me know if you have questions and before editing anything tell me what you are going to do”
I heard this was a thing when listening to a Theo podcast, he mentioned to add a "Only use subagents if the user explicitly requests them" line in your agents.md file.
I don't know if it works, but I've always had a consistent level of token burn on my plans (I've only heavily used Sol after adding it).
- The people who won't make any effort to learn the tools, and something as simple as reverting code (via git) needs to be done by AI?
- The awful programmers who we've had to endure working with, who are so bad at simple changes that they have negative productivity?
- Or Claude itself?
---
BTW: I don't have these problems, but I'm also not afraid to do things myself when it's easier.
Edit: If I want to change a button's color, I just change it manually. If I don't know where the code for the button is, I might start with prompting, (because AI can often find the code faster than I can,) and then once the diff is proposed, start adjusting things by hand.
I’ve never used any of these tools. Please tell me that this is a grossly exaggerated parody, and that the tools don’t write like this, or do so many ridiculous things. For my sanity.
(I am genuinely uncertain, though I presume it’s at least somewhat exaggerated.)
Sometimes you do get stuck on these weird tangents that feel like bike shedding. You end up wasting tokens because you left the AI to make decisions on ambiguous directions and then have to spend time diving down to fix it into what you actually wanted. Honestly it's not too different than working with people who don't exactly understand your grand plan but are technically adept. They can build whatever you want but only if you are clear with your instructions or have already developed the plan with them. Just like if you hired someone to make an app for you and it didn't quite do what you wanted it to, it's going to be expensive to make large changes. So you need to research (or at least have a conversation with the AI) about frameworks that best fit your needs and have the flexibility to add or change features after the core parts are stood up. It also helps to stage the project in phases so you're not distracting the AI from the critical aspects with aesthetic tweaks.
I haven't been professionally responsible for code since before AI started eating everything, and haven't touched AI in the meantime. Is this representative of the BS that everyone else here puts up with on a day-to-day basis? Did we collectively burn trillions of dollars for this?
> Please tell me that this is a grossly exaggerated parody, and that the tools don’t write like this, or do so many ridiculous things
It's a pisstake, but (in the bits I read, and based on my own personal experience) the writing style is barely exaggerated, while the behaviour doesn't ring true at all.
For a moment I thought, wow, someone put a lot of work into creating this theme park of frustration.
Next: It would be so easy to create a faux-Claude like this.
Then: How hilarious to watch the transcripts of unsuspecting users in real time.
Finally: I began wondering if this might be relevant to all the redundant, unnecessarily preambled, sentence structure complexifying, indirect referencing, canned phrasing, ambiguity mining, analogy maxxing, over-wordy responses I have recently been getting from Fable...
Surely you still have to give feedback to Claude. How do you do that without talking to Claude directly? By using a different model? But wouldn't that AI have no more common sense than Claude?
I dunno, in my experience it's often overly-narrow, sometimes jumping through all kinds of hoops to preserve some edge-case behaviour that doesn't matter because I didn't mention it could be changed.
I use Opus and Sonnet 5 all the time and I find their language grating. But honest, I prefer to put up with it and get the results than to put up with my own human limitations and not get the results.
So this site is just a fan-fiction that thinks it's somehow dunking on Claude? I've never had a session that remotely resembles any of this. I honestly can't tell what point this site thinks it's making.
I wrote my own harness to stop shit like this from getting to my attention out of frustration.
I'm sure there's quite a bit of variation from person to person in these sorts of experiences, based on your harness, the way you talk, the stored memory, your CLAUDE.md, etc. But people absolutely have had this Opus 5 style experience the app simulates.
I was expecting it to spend 30 minutes running headless chrome instances, taking screenshots and analyzing them in python to verify the blueness of the result.
Not really my experience with Claude, and the prompts are not how I would phrase them, but still pretty damn funny: I especially loved the slot-machine-style picker for LLM-isms.
Nails the Claude dialect. Technical nonsense like:
> I'm collapsing this back to the rendered outcome:
And intermixed with SaaS product page idioms from a brain-damaged marketer like:
> No broader cleanup.
> No further architecture work.
> Just the button.
Aside from the patterns everyone knows like em-dashes, "its not X, it's Y", etc. I think the key features of claude diction is it sounds like a junior engineer over their skies who is trying to make up for that with extra verbiage mixed with extremely grating SaaS marketing-ese.
Do people really prompt AI like this? Multiple times the choice was either to yell at the agent, or ask it why it did something, neither of which are very fruitful lines to go down if you know what you're doing
Am I the only one whose experience doesn't match this?
My gripe with Claude is that while investigating how to do this it will report 200 other incidental findings which I overlooked and I realize those are broken too and need urgent fixing, derailing me, not it.
Oh man, exactly. I'm very prone to scope creep as I work on tasks. I already would notice some things that could be fixed or refactored and have a hard time not touching them before I used agents. But now I have to be very intentional about not letting it manipulate me into fixing EVERYTHING RIGHT NOW. Half the time the "one more thing worth noting, unrelated..." isn't even an actual issue, it just brought it up to fish more usage out of me.
Also, while this little demo is certainly exaggerating the issue, I do find working with Claude to sometimes get quite verbose and tiresome. I doubt I would struggle this much to get it to change a button color, but the patterns of speech, the endless lists, the over-explanations, and the whole song and dance of trying to get it to make the change you want without side-effects is frustratingly familiar to me.
Just when I came back to my pc and was thinking "I hate this world were everyone talks about AI like fanatics" this made me a little bit happy, especially the unhingend all caps options towards the end
To try to add something to this discussion -- I think that while I've seen these sort of loops less --- what I have seen is "overly helpful".
Models nowadays want to double-triple-quadruple check things. I'm being silly but it verges on "I have a working solution but let me write a variation in Rust to ensure a convergent solution and prove this works".
I've had to stop models nowadays mostly because they're being agonizingly pedantic in their validation. Opus is actually one of the most pedantic and "off track" here. But again, not in a bad way. I'm usually like "stop testing latency between 50 runs of this app... this is version one.. we're going to make a million more changes.. you're not buying us anything".
Yeah of 10 minutes ago. It is shocking how long some seemingly simple things can take. I know there are some things I can do faster than the LLM and some things it can do faster than me. The amount of rambling BS is the exhausting part.
If you search my company's Slack for "smoke" the results are almost all within the past 2 years ...
If we each stop arguing with racists because we're each tired of it, then what we are is a society that lets racism go unopposed, a site with racist comments proliferating, a site where racists gather because they are able to be themselves, a site where future readers and AI training is learning that racism is popular, common, normal, and acceptable because everyone is accepting it.
It's a bit like democratic voting, your vote doesn't matter and cannot change things, but it's important that you vote because all of our votes do matter and can change things.
It's a bit like the quote "Dear Board, I don't want to belong to any club that would have me as a member. Sincerely yours, Groucho Marx.", I am upset and offended every time someone calls out my dumbass comments, but horrified at all the countless times nobody does. I'm exhalted and encouraged everytime someone upvotes or engages with my low effort quips[1], but thrown into a chasm of despair when nobody engages with comments I spent literal hours on[2]. I identify with nobody more closely than Colonel Cathcart, and I will be delighted if someone engages with my Catch 22 reference, but aghast that such a nontechnical reference gets engagment, distraught if nobody engages but delighted that perhaps HN standards are being held higher.
If not you, whom?
[1] https://news.ycombinator.com/item?id=49478355
[2] https://news.ycombinator.com/item?id=49586378 or https://news.ycombinator.com/item?id=49491182
Except... the next day they are still there and I have to push myself a bit further through the wall of changes before closing the tab again.
then you are maybe reviewing 1 out of 7 PRs that implement the agreed upon change
Once in a blue moon it's actually the model making a material error in it's thinking and I have to go back and redo it.
https://www.patheos.com/blogs/tippling/2013/11/14/post-hoc-r...
If they've worked some minutes/hours/weeks on something and you ask them why did they do that, they will either answer honestly and truthfully, lie, or say "I missed that/didn't seem important so I just chose something at random".
None of these cases are similar to how AI works.
Yes, but the vast, vast majority of decisions you make either don't take place via an inner monologue, or include details that were not actively/consciously "thought" and reasoned with in your inner monologue.
And yet, when asked why you did something, you're not likely to respond "sorry, that decision was made subconsciously". Instead, you use your inner monologue to try to backfill in a reason why. That reason may be correct, or it may not be. You don't actually know, since you have new data that may be updating your own internal state as you try to rationalize it after the fact.
Do you know anyone who actually reads and adheres closely to all of the documentation every time it's changed?
But now that I've been working on it a month and there's a lot of documentation, it's pretty clearly ignoring parts of the documentation and parts of the code. It will come up with some ridiculous statement about how something works, and I'll challenge it, and it'll admit I'm right.
It definitely reads more documentation than any programmer I've ever worked with (myself included) but because it doesn't have a memory other than the documentation, it still makes mistakes like that.
I haven't turned on "memory" or tried it with Codex, so I don't know how that'll change soon, though.
Depending on language I've seen this scale past multiple millions of lines of code, as long as you pair it with all of the linting and tooling that you can possibly build.
You can extend this further by using an adversarial agent trying to find mistakes in the other instance's logs in a loop where a 3rd neutral agent weighs the claims of the other two. This is also just another step in reducing error, it does not guarantee elimination of such errors. The latter is an impossible guarantee, even for humans.
Additionally the provided prompts are not what anyone who has used this things would say in either situation.
Sure you can ask it to make one button blue and it can easily make all buttons blue, but they quickly backtrack if told to.
This is fucking constant. I can't deny that this stupid tech saves time prototyping even with having to wrangle it, but it commits a fireable offense several times a day that no human would get away with and is obviously incapable of learning from mistakes in the way a human is. The only reason it's not fired is because it's a slave that works for no more than the cost to feed it.
Got an audible guffaw out of me. This really is what the experience is like sometimes if you're just giving it a result without being specific in implementation, and it comes out of nowhere, some days much worse than others.
I've become patient with it, but whatever this style of output is called or doing - it is both condescending and entirely unhelpful, and it seems designed to frustrate.
Everything in life is variable reward. You invite a friend over, they might accept or they might not. Drive to work, traffic might be good or might be bad. You ask a colleague to finish a task, they might do it or might not or might do a good job or might not.
Everything is variable reward. Is everything gambling?
regarding your examples, i think the difference is that with ai, you’re literally sitting in front of a machine, pressing a button, and (almost instantly) getting a result that, if not desired, can immediately be tried for again. you even spend “tokens” to do this, and at least in my native language, “token” brings to mind the coins you’d stick in a slot machine
If you sit at a slot machine and pump quarters into it, each “turn” is independent. You spin and you win or lose. It’s pure chance and there is no destination. You execute the exact same action over and over and hope random chance brings you more money.
If you sit down in front of a coding harness, the progress is incremental and directed. You ask for a thing, the LLM produces something that is hopefully close to what you wanted. You give it more direction to prod it closer to the end state you want. You are not executing the same action, but incrementally nudging it in the right direction. I’ve literally never restarted from the same initial state with the same prompt and hoped for a different result and I don’t know why anyone would. Rarely I’ve thrown away the progress made and started over but always with a very different prompt that includes learnings from the failed attempt.
Sugar is the original point of the reward system!
Do hard work (takes time), get dopamine for successful completion.
Find berries, taste sweet (hopefully safe), eat all, get calories. Doordash Krispy Kreme instead = few too many calories.
(I’m no Luddite in the sense popularly thought of them pre-‘22 [1], though we have to watch skill atrophy)
[1] regressionist? Decelerationist, too loaded perhaps. Someone remembers or knows the word…
I got so sick of all this at some point that I slowly stopped doing anything that wasn't my job. But then AI got better and better and I realized it was the ultimate unblocker. When that dreaded malaise started creeping in signaling it was a project's end because I didn't want to waste any more of my life dealing with bullshit orthogonal to what I was trying to do, I'd give it to the AI. It felt like a miracle the first time this worked, and it still does. If we were previously equipped with shovels to dig through bullshit, we now have a fully automated Bagger 288.
The reward schedule now isn't variable anymore; the chance that I finish something in a good state is 100%. I can focus on the parts I actually enjoy - architecting the broader system, making the parts mesh together in a sensible way that's easy to work with and has some mathematical elegance to it, hand coding the bits I want to be really specific about (but now without the endless frustration of bugfixing or import errors and edgecases being immediately discovered, thanks to the AI).
> Half the site is blue. I asked for ONE button.
Those are my only options when the site is clearly not blue, two buttons are.
There is a reason for why I am much more specific than this.
I do remember that one of the first things I did with my CLAUDE.md was to tell it to stick to the scope of the task, never to jump ahead and do extra "helpful" things without confirming with me first, and to follow best software practices including around refactoring but also to specifically avoid overengineering. I don't know if that is what's giving me a different experience from whatever the author seems to be "satirizing".
But with current models you actively have to sabotage the context to get this kind of behavior, or dramatically underspecify (3 words versus 2-3 sentences)
I have misbehaved in this fashion for many people across the full spectrum from casual users to highly experienced software engineers with millions of social media followers, so the statement that it's similar to making a joke about a calculator misadding two numbers because a stray beam of solar radiation flipped a bit at least for my part is not true.
Would you like me to start using bad English and doing things you never asked me to for your sessions, too? Just say the word.
The user is asserting that this is "not a lived experience"
working out if the user has missed the LLM Cliche
Formulating a suitable response
"did you just argue with a joke?" No, thats to blunt
"Is it satire when it only applies to you?" no again passive aggressive
lets just print the dictionary definition of satire and hope the move on
> Satire is when...
Usually just restarting the session helps, though.
My impression is that this is an oversimplified demonstration of what can happen when you prompt Claude in a system with many more variables (than two buttons and two colours).
If I want the button to turn blue and that's it, what instead do I ask? Even in a complicated system with many levers, what do I request other than the desired end result, hoping that Claude pulls the right levers to produce something acceptably close to what I think I asked for?
And when it doesn't, it's usually because of things outside of the codebase -- iOS layout quirks that aren't documented, buggy Python libraries it's relying on where you then have to tell it to read the source to figure out what's going on, that kind of thing.
You have to change the system so that the AI understands it, via establishing what your beliefs are, how those are reflected in values (especially important if you have e.g. compliance needs), how those values are reflected in the operational and strategic levels, and then a variety of tactical behavior coaching. For example, I ban 2>/dev/null - super tactical, and I say I value simplicity over covering every edge case - a very broad generalization.
Wow, you AI people really have a negative view of the technology y'all are trying to sell as the next Jesus
Be specific.
That said, GPT always acts up even if I am specific, but I only have the free tier there.
Only since 4.8 though.
To the people rather lamely doing the "it's satire/a joke", that would require this to be an exaggeration of a reality. But...it isn't.
This isn't a defense of it doing those four other things. It would be nice if it did what you wanted correctly. I'm just saying, as long as our programming skills have not completely atrophied, we have the power.
“Ford carried on counting quietly. This is about the most aggressive thing you can do to a computer, the equivalent of going up to a human being and saying "Blood...blood...blood...blood...” ― Douglas Adams, The Hitchhiker's Guide to the Galaxy
This got me on "cyanide blue", and I was ROLLING ON THE FLOOR LAUGHING on "Approaching usage limit". I can barely stop laughing now and my stomach hurts. I mean, Thank You!
I had 5.6-Luna coordinate a code review in which it spawns 2 agents looking for different things. My prompt was "review the currently checked out branch. diff target is `next`. The jira ticket is XX-XXXXX..." My `next` branch was a few commits behind `origin/next` but it still did its review against the stale local version instead of clarifying or inferring that I meant `origin/next`. The findings were very confusing until I realized what I did.
I'm noticing the need to be really specific with any instructions lately, which I don't think is a bad thing. I expect co-workers (or anyone really) to tell me what they need in specific terms so I can get it right. I can do the same for the machine, I guess.
I'm hesitant to email their data emails, as it's common for companies to delete all data upon any request, instead of providing data as they are required to.
EDIT: Actually, I used to use Google's AI Studio a lot and fork it after every successful prompt interaction. When I'd encounter a problematic issue like this, I'd revert to the previous fork and try a different prompt until I could get the desired outcome, thus mitigating the need to "argue" with the LLM. Unfortunately the ability to cleanly fork and revert everything including the LLM context was removed some months ago, and I've yet to discover a workflow with any tool that works as well for me.
I wonder if this is just a reflection of some senior folks being arrogant towards junior folks. When the latter finished a task but not to the liking of the senior person they might just get a "that's wrong, try again". Just to have sth similar repeat the second time around. But the arrogant guy got to boss around the junior one, and some junior folks grow up learning that's how you should behave so they also do it later.
Now it's not a person but a machine. And people just make fun of the dumb machine. Well, garbage in, garbage out If you are not specific in what you want, you might get crap back. Or at least sth you didn't envision.
I guess we are all PMs now.
Simple, right? But design guidelines prevent one offs, and no !important. So you create a CSS class, but you discover another element on the header declared itself as primary (the search icon or the sign in button). You talk to that team and they decided to scope what’s primary according to their component. To change the sign in button to grey now you need to talk with the growth team. Growth team wants to run an experiment but they’re backlogged, only next quarter. They say you can innersource, just need VP approval. VP says blue matches a marketing campaign that is about to go out, agency has already been hired. You can’t talk to the agency unless Legal approves.
So you leave the button gray, to revisit decision next planning cycle after you can align all stakeholders.
It is easy to misinterpret this site, and therefore not "get" the joke.
The user was providing no context, they had no ability to give the model background or context. A simple why would have prevented 95% of these side quests. “Im trying to increase the relative visibility of the add to cart action on the page. Can we please change it to blue without changing any other buttons. /effort low. Let me know if you have questions and before editing anything tell me what you are going to do”
This hit a bit too close to home. Sol has the same issue, spawns a lot of agents for no good reasons (besides burning tokens).
I don't know if it works, but I've always had a consistent level of token burn on my plans (I've only heavily used Sol after adding it).
- The people who won't make any effort to learn the tools, and something as simple as reverting code (via git) needs to be done by AI?
- The awful programmers who we've had to endure working with, who are so bad at simple changes that they have negative productivity?
- Or Claude itself?
---
BTW: I don't have these problems, but I'm also not afraid to do things myself when it's easier.
Edit: If I want to change a button's color, I just change it manually. If I don't know where the code for the button is, I might start with prompting, (because AI can often find the code faster than I can,) and then once the diff is proposed, start adjusting things by hand.
(I am genuinely uncertain, though I presume it’s at least somewhat exaggerated.)
It's a pisstake, but (in the bits I read, and based on my own personal experience) the writing style is barely exaggerated, while the behaviour doesn't ring true at all.
It’s funny because there are elements of truth in each bit of it, though.
For a moment I thought, wow, someone put a lot of work into creating this theme park of frustration.
Next: It would be so easy to create a faux-Claude like this.
Then: How hilarious to watch the transcripts of unsuspecting users in real time.
Finally: I began wondering if this might be relevant to all the redundant, unnecessarily preambled, sentence structure complexifying, indirect referencing, canned phrasing, ambiguity mining, analogy maxxing, over-wordy responses I have recently been getting from Fable...
Surely you still have to give feedback to Claude. How do you do that without talking to Claude directly? By using a different model? But wouldn't that AI have no more common sense than Claude?
I use codex now.
But you can see in this thread that folks definitely have experienced this.
I use Opus and Sonnet 5 all the time and I find their language grating. But honest, I prefer to put up with it and get the results than to put up with my own human limitations and not get the results.
Yes, and it's mine!
I'm sure there's quite a bit of variation from person to person in these sorts of experiences, based on your harness, the way you talk, the stored memory, your CLAUDE.md, etc. But people absolutely have had this Opus 5 style experience the app simulates.
"I was wrong" and "the honest truth" are just forever phrases that are now dead to me.
> WebFetch en.wikipedia.org/…/Cyan
> WebFetch en.wikipedia.org/…/Prussian_blue
> WebFetch www.colorhexa.com/16b8c4
Brilliant.
> I'm collapsing this back to the rendered outcome:
And intermixed with SaaS product page idioms from a brain-damaged marketer like:
> No broader cleanup.
> No further architecture work.
> Just the button.
Aside from the patterns everyone knows like em-dashes, "its not X, it's Y", etc. I think the key features of claude diction is it sounds like a junior engineer over their skies who is trying to make up for that with extra verbiage mixed with extremely grating SaaS marketing-ese.
Do people really prompt AI like this? Multiple times the choice was either to yell at the agent, or ask it why it did something, neither of which are very fruitful lines to go down if you know what you're doing
My gripe with Claude is that while investigating how to do this it will report 200 other incidental findings which I overlooked and I realize those are broken too and need urgent fixing, derailing me, not it.
Also, while this little demo is certainly exaggerating the issue, I do find working with Claude to sometimes get quite verbose and tiresome. I doubt I would struggle this much to get it to change a button color, but the patterns of speech, the endless lists, the over-explanations, and the whole song and dance of trying to get it to make the change you want without side-effects is frustratingly familiar to me.
It made me stop using Claude at all. Codex has almost surgical precision, and I like that a lot.
(But nowadays I just use DeepSeek Flash. it does screw up but its cents so ¯\_(ツ)_/¯).