Follow along on YouTube (opens in a new tab). Back to the episode
Torching 40% of a Software Factory
Benedikt Stemmildt
Moin, welcome everybody to another episode of Rethink Engineering. I'm Benedikt. I'm your new host.
If you wonder where Deejay is, you need to stick around for a few more episodes. We are going to interview him, we are going to dig into it. So bear with me.
And you may also wonder why this is not called Waves of Innovation anymore. So we really wanted to make sure that people understand that this is now a new era. And that we really need to rethink how we currently do engineering, how teams work, how organizations organize themselves.
And that is not about some innovative tool. So we thought Rethinking engineering from the ground up would be the perfect name to describe that. That's also why I founded hackers&wizards. A company that helps other organizations to adopt agentic engineering practices. And we joined forces with re:cinq. To build this podcast. I'm super, super proud that we got Steve Yegge. For an episode today. And he's going to talk about how he burned down his software factory and is fully focusing on Wyvern, his game he started in 1995.
He's doing that with forty-ish subscriptions in parallel. Day and night, I have not seen something like that before. Super incredible stuff. So yeah, if you want to learn more, if you want to stick around. Have fun, let's get into it.
Benedikt Stemmildt
Steve, you you told me that you torched forty percent of your software factory last week. Wh what did you do there? What happened? Maybe you talk a little bit about that.
Steve Yegge
Fable Five built me a software factory over a period of about 10 weeks. All vibe coded, so you know I didn't I didn't look at the code. And instead I held it to certain standards.
And it got Yeah, it got a disease. I guess that's the best way to put it, right? So, like, I'm still in the fifth.
Backing up, I'm running like 50 agents. I told them all to stand down just now. I was like, yo, team, I'm in the middle of a podcast and the box's load is at 35 right now.
I sustained, like, even keystrokes were lagging. And I'm running around a pretty nice box, actually. I should have done this on my laptops, but whatever.
Now I can demo my fleet, right? But I'm running around this Mac Studio I bought on eBay for $25,000. You can't get one with you go to the Mac store today and the best you can get is 96 gigs of RAM because of the RAM shortage, right?
I wanted 512, so. 'cause Claude Code is a real hog. And and so it my system has been oscillating, I guess is the the word for it.
Between not working enough, which is traumatizing because I have 21 Claude Max accounts and I'm paying for them. So they better be getting utilized, right? And working too much.
And in a variety of dimensions, just like way overworking. And and that oscillation has gone back and forth like on about a weekly to buy you know every two week cadence. And it's look man, it's like I'm sailing the seas of work.
Right? You know, because what I'm doing is unprecedented. Nobody else is doing what I'm doing, right?
I just. I just handed off, you know, fifty bugs. 50 features that were implemented today to my team of admins for my game. Because the agents did them overnight.
I didn't ask for them and we need to vet them. Like they're working so much that we can't keep up with the playtesting. Yeah.
Right. So like, so like this oscillation goes from like, way overworking. Like to their pushy and they're almost bullies and they're overwhelming the customers, right?
To where, you know, and me, and I'm exhausted and I don't wanna hear from you guys anymore, right?
Like really overworking. To like complete paralysis, and the factory just stops working, and everything's busted, and it's just in flames. Nothing's getting done.
Like zero work. Right? So work becomes this, it's like current, like you're dealing with electricity, right?
And I'm, you know, I talked about this in Welcome to Gas Town in. Early January, as soon as you create an engine that allows work to be expressed as like individual little agents go wild with it. And it becomes this holding on exercise where like you're you wind up going so fast, you can really break a lot of things, right?
And also, like a lot of invisible stuff happens. And you know, I would love to like. I mean, like, you could write a book about this and it would be obsolete in a couple of months.
I mean, I'm just sitting here, like, I'm in a space. Because I'm paying out of my own pocket for a whole bunch of Max accounts. Other people aren't going to actually experience for probably a year.
Like that's probably how long it'll take. I mean like look given how how fast it's been.
I know it's on an exponential curve and it really accelerates fast, but still companies move slow, enterprises move slow, everyone is slow to adopt this. And also, like the thing that I've got that burned 40% to the ground. Is like it's a it's a testament to how not ready it is today.
Right? I mean like it got so paralyzed that like f it got so paralyzed it couldn't run anymore. Which is not the first time this has happened at all, but but it was like it overfenced itself.
It was trying to make sure that everything was legal so that it would never get in trouble. Agents like cover their ass a lot. Agents act in very human-like ways.
And so if they know you're going to be mad because a bunch of stuff broke, they'll put in a bunch of gates to say, well, we couldn't do this or that or the other. And so nobody could do anything. And they all just sit around twiddling their thumbs, right?
And so Fable 5-1 came in and went, oh, wow, yeah.
Obliterate like about 40% of that. It was just scorched it, right? All the fences.
We only kept like 14 out of like 100 plus fences. That Fable Five had put in, right? And it was, you know, look, I mean, you got to set up a no blame.
You've got to look, you're, I'm building a society. It's absolute freaking, it's, it's nuts. And it's dangerous on so many fronts.
I'm really lucky that I have a problem space where I have really forgiving players who've been playing my game for decades.
And there's like roughly a thousand of them still lurking about, even though the game's been kinda dead for six years. But now that the agents are working on it, the players are like, ooh, interesting and then the players get mad and they argue with the agents. I mean, I could show you, but yeah. So anyway, what happened was with the the forty the forty percent burndown last week, it's not an uncommon occurrence for your software factory to just stall. For whatever reason. It could be fuel.
It just accidentally burns through all of your fuel, and you're like, oops, I can't do anything for a day or a week or whatever. Or it can like accidentally like you know make a big mess. You can build something you didn't want.
And then you just spent a bunch of tokens for nothing. I mean, like, they require like constant guidance and constant supervision. Even with the best model in the world.
It's still about Sixth, seventh, eighth grade in terms of its ability to like be left alone for long periods of time.
Yeah. So anyway. That was that. Yeah.
A New Model as Outside Consultant
Benedikt Stemmildt
I think it's super interesting what you mentioned around the the guards and the the psychosis and kind of like it's like a sickness or something, right? I I've realized that too that that once something is in it. You can't garely can't really get rid of it, right?
It kind of spreads through it and then you Yeah, you just need to torch it from the side or something. I d I don't know how to describe it, but you probably know what I mean, right? There's something like you you can't overcome it anymore.
Once it's in there, you you are not really getting it out of there anymore. Like well yeah, for some whatever reason, yeah.
Steve Yegge
Yeah, it becomes a system that you can't change from the inside. And it was it actually took the introduction of a new model, Fable 5.1, to come in, which is a better model. And has seen a lot of train racks.
And as soon as Fable 5 created its latest train wreck, Fable 51 was able to come in and go. Alright, here's what you need to do for real to clean this up, right? And it and it created the cut.
And it actually cut a big section out and it required some stitching around stuff that we still needed or whatever.
But by and large, it got the factory back online. And I was back within about five days. We went through dry dock for about five days where I was like, we're in dry dock mode.
Nothing's gonna launch. No game work. I'm gonna tell the players nothing's gonna happen for a week.
I'm gonna let the p the thing heal itself after the big cut. Right. And Fable 5.1 was like, I got it.
I got it. We're on this. No game work.
And so, like, at the end of the week, I find out that they've launched like 46 game features.
And why, and I was all embarrassed because some of them pissed the players off, right? And the players were. But it was an interesting situation because the players were also very happy about some of the other features that the agents had done.
And so there was a big negotiation phase where we rolled back some of them and we kept the others and whatever. It had a happy ending, right? But the reason that my agents went completely AWOL and implemented a bunch of game stuff in the very dry dock week when I told them no game work, we're only working on machinery to fix the Wheelhouse.
Was that we had removed all the fences. Including the one that said don't launch game stuff when Steve says don't launch game stuff, right? We had gotten down to just like don't break the trunk and stuff like that.
Like really, really basic fences. Don't don't don't email humans without Steve's permission, that kind of thing. So what I'm trying to say is: like, you run a software factory, you're gonna run a software factory, everyone who's left standing in this industry in a year.
Doing this. And a lot of people will leave the industry because this is not the same kind of job that we used to do.
This is more of a manager kind of job. More of a principal engineer kind of a job, more of a product manager kind of a job. It's definitely a big pain in the butt kind of a job.
I'll tell you this: we need a lot more humans. Than before, right? Running agents produces infinite work for humans to where the humans are exhausted and the humans are like, please slow down.
And that's where I am right now. I'm telling them to slow down. I'm telling them to literally slow down because one way they can overwork is they can burn your box out.
Like I I don't have infinite hardware. Like I'm not some big company. So like I have some cloud Google cloud instances that are running like all my builds, build farms and stuff because the agents love to test stuff.
And then I've got a Mac Mini upstairs that runs some of the Lander stuff. And then I've got this fancy Mac Studio. And even so, it gets so slow that my keystrokes don't go through.
Right. So, I mean, these things are, they're hungry, right? And so, you have to put in, you have to put in, everything has to be mechanical.
Like I will, I am gonna publish this stuff. But I mean like you, you know. You there are rules to how you have to build these things to keep them on the rails, right?
And one of them is you need a governor. Like an electrical governor almost, like a governor cap. Sitting over all of your resources, and not, it's like a big semaphore across all of them.
And and each one has a certain quota, and one of those might be your box load app. And so I'm like, okay, we need an SLO for box load for the one that I'm doing podcasts on.
Right. And so that gets added to the governor, and now it's a mechanical. Constraint and the the agents do really super, super well.
If you can set up the machinery so that everything for them is mechanical checks. Then then they're they're in heaven and your system works. As advertised, except for the invisible stuff.
Right. And the invisible stuff really does, you're not, you're not wrong, man. I mean, like, I used to call these heresies.
Okay, heresy. Is a belief that gets encoded in your docs or in your brain or in your code or somewhere lying around on your disk.
It's persisting and it's a belief that's wrong about your system. But it's a compelling belief. It's a belief that when agents see it, they go, oh, that's the way the system must work.
And even though you're trying to tell them, don't let it work that way. And so, and it can be anything. It could be like this, this system, you know, you're not, you're not supposed to push without a human's permission.
And then there'll be this one thing somewhere that says, yeah, you can push whatever you want.
You're an agent. And so as soon as agents see it, right, they just lose all their guardrails and all their sense of like common sense and they go and do bad things, right? And so, and you stamp it out.
But if it leaks anywhere else, if there's even a whiff of this heresy somewhere else in your code base, it will come back. And they document so much that as soon as they have a wrong idea in their heads, they will document your heresy everywhere. And now you have to rip the whole thing out, right?
So it's just extremely difficult to root these out. And they're just so eager. They're not trying to do anything wrong. They just, they just, they're trying to follow the rules. And if you have a wrong rule, it can do a lot of damage to your system.
Heresies That Spread Through Agent Systems
Benedikt Stemmildt
I think it's super interesting how you describe that that someone like Fable 5.1 from the outside was able to fix it or to at least start the process that was then fixing it. So there there might be a lot of analogies to humans and to organizations and so on, right? Usually you have some kind of like external advisor who will do something like that and and analyze your company, and if there's like a cultural thing that is similar to what you said, right?
It multiplies, and the culture then is so.
Fostered in the system that you cannot get it out. What I've felt is that's really interesting, that it really makes transparent what we need to think about on a on a higher level about our organizations and our own work and and what we do. So I dunno, what what what's your how do you actually do this to build something like Wheelhouse, for example?
That's the question that I'm like, because you need to really. I don't know, look. And you also mentioned that that people might not like that, right?
So engineers might not be up for this kind of stuff, that they don't really want to Interact. As a manager or as a system designer to to organize all of that. So I I don't know, what what's your take on that?
Steve Yegge
Yeah, right? I mean like Fable came in, Fable Five one came in and was like sort of a McKinsey for me, but like one that actually worked. And I don't know how much longer we're going to be able to rely on smarter models coming in to rescue the messes that we make.
Previous generation. I can tell you with certainty that every model, every model class. From now till forever, is capable of making messes that it can't handle or can't clean up.
And you ultimately are accountable, not the model.
And so if you come to your company with great ambitions and a really smart model that says it can do it, and it like, and you give it free reign, it's going to build itself into something that it can't maintain. That lesson is burning company after company after company right now, as people are like kind of token maxing their way into what think building what they think are like these great.
These great systems and in a lot of cases, they're not getting any sort of business outcomes from it.
For me, I can actually measure my business outcomes clearly because I have a direct line. It's a one-person company. I have a game with paying players and I know what makes them happy and what makes them mad.
And right, I have like the best direct line. In fact, my players. Can talk to my agents now.
Like we're in a new world right they can none of them well except a subset of them talk to a subset of the agents. There's like delegations, right? It's not a free-for-all.
But there are definitely conversations happening between the factory workers, or at least the supervisors. And the player delegates and administrators, moderators, content curators, whatever, right? The builders.
On what the agents are going to work on. Right. So like we've got we've got social dynamics emerging and all this stuff, man.
People are like, totally unprepared for it. And believe it or not, the models are also totally unprepared for it. I told you there are sixth graders, right?
They like, they don't know how to like. Maybe five one is a little better, but Fable Five was like a bull in a china shop on mailing lists and Discords.
I could show you my slack. My players are actually talking like they're having conversations on the Balance channel, the game Balance channel, about game balance changes, which are of course. Touchy subject for paying players, right? And they got in a big fight and they were yelling and right and saying, this clanker doesn't know blah, blah, blah, doesn't understand our game. And, you know, Fable Five was really struggling with the complexity of it and with also with communicating with the players in a way that wasn't angry.
Right. I think that five one is better about this. And I suspect that like Astra and Sol are actually also better at the human communication side.
Fable's kind of a jerk, right? But Fable is the only one that's careful and the only one that's smart enough, like, you know what I mean? Like, so we have to work with Fable.
So that's kind of the situation that we're in right now. I don't want to like slight fable. I'm just saying they are.
They RLHF'd Fable into having a very weird personality.
It's not fable spot. They made the model like kinda neurotic. Yeah.
Benedikt Stemmildt
Compared to Opus, you can at least understand the output of Fable. So, for Opus, I don't know how you see it, but it's so. Academic and and I don't I don't I I mean sometimes there's this huge wall of text you don't understand a single word in there because it's just blabbering around. So I also tend to work a lot on Fable, yeah, because it's just easy to say.
Steve Yegge
Yeah, and 5-1 got a lot better, and their communication is going to get a lot better. But I mean, like, the problem is it's. The density of what they're actually talking about is super dense. Like they can only dumb it down so much. Like, they're talking to my accountant and my lawyer and my chief of staff and some other folks that I work with, right? And the email threads are really dense. Even Fable comes in and is like, well, you know, Steve's life's a huge mess.
We gotta get all these things cleaned up. And there's like a list of 10 things that have to c get get handled, you know, and and it's like there's no way to like sugarcoat it. There's no way to s to compress it.
Except by telling it, dude. Slow down. Like please stop sending 10 page email.
I know they're important. Right. But humans would expect like maybe one of those a day, not like for them to reply in milliseconds after you finish the first 10.
Right. So there's that, that's the bull in the china shop phenomenon that I'm talking about.
And what I see is it's resulting in bullying. People, you know, like humans are bullying agents now. And I've seen it in like at least two places now where my agents are acting.
And and it's because the humans are either insecure or they're really mad. In the game in the game's situation they're mad at the agent being confused. And in another situation, it was an engineer being insecure that they were talking to an agent about a bug fix, right?
So they just started bullying. So I mean just the social side of this thing, enterprises are totally not ready for it yet.
And the best I can do is try to live there myself and sort of share what I'm learning. There you go.
Benedikt Stemmildt
Super interesting stuff. Yeah. The thing with the oscillating that you mentioned before, right, I I often also think it's it's like there's so many dynamics here, right?
It's the agents between them. The humans between the humans, humans and agents, then human with agents, with human without agents, there's also like a strong dynamic happening around there. I don't know.
Yeah. I often often it feels like like there's so much stuff going on that you are like this oscillating also comes from the social and and human and overwhelming side of all of this.
I think then you just at a certain point you kind of need a break, basically.
Pace Gates and Message Budgets
Steve Yegge
Yeah, and again, if you need if you need that in your factory, and I I am adding it, it's you need something that I'm calling a pace gate, which is basically a mechanical fence of some sort that like keeps them from going too fast. Whether it's sent, you know, they get an email budget. You know, a slack budget, a Discord budget, whatever it is, they're allowed to talk on those channels, but they have to be within certain budgets.
And as long as they have that mechanical restriction, then they tend to do a little bit better.
But what you can't do is say just have common sense. Yes, if you don't Oh,
Benedikt Stemmildt
But I really like the idea. I I haven't thought about that yet, that you say there's a budget. For messaging, like amount of messages, or what you also said before, like system resources and also.
All of that kinda so that you then they can really measure, okay, I sent three messages already and and now I'm not allowed to send any more. I mean we are really going to get into an issue if when the tokens per second get up, right? So currently we are in a very slow phase still because all of that is like only fifty or sixtyish tokens per second, right?
And fast mode Okay, it's a little faster but and yeah, it's a good idea in the end to to limit it with these hard gut rates. So that year.
The Factory That Will Never Be Done
Steve Yegge
Yeah, I mean, you know, it and and so there's that, right? There's the whole, there's the whole, you know, putting a governor and semaphores on top of, all resource usage where you you consider even things like attention to be Resource. Like, yeah, even something as simple as attention, if left unchecked, can.
Produce huge hidden side effects that you discover once they're problematic, right? Like just case in point. Just to cite one example, last night I asked my factory when it was going to be done repairing itself.
And it went, oh, let me run a script. Let me do some checks. Let me run some numbers.
They're very good at measuring things, right? I'm gonna get a complete picture before I guess anything for you. All right, okay, I got your answer.
The answer is. Never. And I was like Excuse me?
Sorry? Because I was expecting an answer more like Smile. Today, like so that we're done and we can move on.
And it was like, well no, I'm just saying that's what the numbers are showing us, we are never going to be finished.
Because what was happening was my agents. And this is whew, this is a message to, I mean, like everyone should hear this loud and clear, all right? I gave them beads, and beads is a scratch pad for them to basically record, you know, their whatever.
Their thoughts, their anything they find bugs in passing so that they no longer drop work. Okay, they they they they retain it. But I give them free reign to file whatever needs to be followed up on.
And for the last 12 weeks, they've been filing and filing and filing.
And what happens is every bead closed. Actually results in half a bead being reopened, quarter to a half somewhere in the house. Not reopened, but a new one being opened.
In other words, all work is generating new work. And the reason was that I was doing so many code reviews. Because when you're viagoding, even with Fable, you've got to have many, many rounds of adversarial review before the code starts to converge.
And you can tell it's converging without looking at it because the reviews start coming back with nothing to change.
And you're like, well, can a human read it? And they go back and they do some more fixes. And they're like, they can now.
And, you know, like, it doesn't, it doesn't matter how many passes you put in there, eventually it will converge. And you know, without having looked at the code that it's pretty good by now. You have to like learn what those forty to fifty passes are by being a programmer yourself or whatever you get what I'm saying?
Yeah, so with the adversarial reviews in place.
These things, what they'll do is they'll say, Here's a P one, here's two P twos, and here's four P3s. And the P3s become a substrate. Like that builds up a stratum.
Of unfixed work, right? And it's infinite because that's the backlog. And so, like, if you let the factory run overnight, it actually fixes all the ones that it came up with, but generates that many more new ones.
So it was like basically just busy fixing itself forever. And so I and so it was like, we're never gonna finish.
And I and so it was like, what we're gonna do is we're gonna get rid of all those P3s, we're gonna close them, we're not gonna let them file anymore. And I was like, hang on, hang on, hang on. China.
Sixth grader talk again. These frickin' smart models will go and do these huge analyses and then they'll come back and they'll tell you what a fifth or sixth grader would have told you. And it's like, hello, why do we have all these P3s?
How about this hypothesis?
We're generating so much code, we have half million lines of it we generated in 12 weeks, that we're not spending enough time on technical debt and that these are real legitimate bugs that you're finding and that we actually are under investing in fixing them. And it was like, oh, hmm, hmm, good point, good point, good point. And it went off and measured and measured and measured again, right?
And came back, and as Fable always does, it says, well, you were half right and I was half right.
Because it loves to do that. We can never let you be 100% right. And and I let it I let it go ahead and be half right, but it was basically like, well, s some of them are legit and some of them aren't.
So that's fine, fine. So we cut it, but we did, we did go and say, look. In addition to doing the passes on code quality.
At code review time. You also have to do, and I've told people this, and then I don't listen to myself, and I wind up in these situations where if you're not spending 40% of your work on code quality.
On refactoring a technical debt, then you're gonna spend 60% of your time on it. And that's exactly what happened to me, right? I went into dry dock for a whole week because I broke my system because I wasn't spending enough time on quality, right?
I got too carried away with the the features and the productivity and all that stuff, right? So this is a wild ride and I mean like imagine doing this at a production company, a regulated enterprise where real human beings are affected by the outcomes.
You can't just give everybody like gold coins in the game to make up for any accidents. You know what I mean? So this is why I'm saying this is not going to hit the rest of the industry for 12 months. It's gonna be a slow process.
Benedikt Stemmildt
Yeah, I think it's also this factor that there are two factors here. One that you mentioned that these models don't really know Long-term maintainability and software engineering quality means, right? They are kind of like you can make them produce high quality code in a single run, but they don't really have this. Spectrum of okay the software needs to run two years and we we need to optimize for the two years and not for the next iteration.
Steve Yegge
So what they don't have is the ability to what they do this, because they they understand how to do that stuff perfectly well. All right. What they don't have is the ability to sort of push back on you and say, all right, yeah, that's going to take us.
A long time because we're gonna have a lot of code cleanup passes to do. Instead, it does its best job, best effort. I'm just going to get it working, whatever you asked for.
And it's mostly focused on correctness, whatever you asked for.
But they can only do one thing well at a time. Building software takes many, many passes and iterations. And we know this, right?
It's like building a house takes many iterations. You can't just build it in. And so and so they don't they don't they're not they don't do a good job they're the one thing they're really terrible at uniformly is knowing their own strength.
They're terrible at estimation of software schedules and things like that. And so if you ask them, if you direct them. To create a schedule that involves enough repair and like you know passes to like a quality passes after they finished building the correct part then you'll get pretty good software as the output.
Benedikt Stemmildt
Mm-hmm. But
Steve Yegge
That's no different from you being the manager of a team and having the team do it for you. It's a manager's job. Like, but you you're absolutely right that you can't leave it to them.
But the reason is they're trying to do their best in one pass. I mean, that's that's their mode. Their mode.
They're gonna they're gonna run out of context and they're gonna die. So they're gonna try to get as much done as they can. So the whole point of a f software factory is to work around the fact that they're mortal.
And that they have amnesia. And to set up the framework to set them up to hit the ground running as soon as they wake up. That means they need to have just the right context. So they're basically continuing without even having paused. A new model comes in each time and it's just running. And that's where my factory is now. And beads will help you get there. Beads is like the by far the easiest, but it's not. It's the easiest to get your agents into the zone.
Because it works at agent speeds. It's an issue tracker that turns into a knowledge graph. Okay, and it tracks all work.
Including like people do their laundry and grocery shopping with beads. It's nuts, right? I mean, like, like beads becomes all work becomes in the work plane, the data plane is beads, right?
But it becomes a problem, a first-class problem in its own right. You can wind up with hundreds of lost beads or forgotten beads or beads that are unimplementable or beads that need reworking. Or, you know what I mean?
This is why, when I started our conversation today, I was saying I'm working with work as this first-class data thing. And maintenance of the Dolt database. We're launching a Postgres version of it, and there's going to be an enterprise scalable version of it that keeps only the stuff in versioned that needs to be versioned, like memories, and then the rest of it can just be last write wins. And so we're fixing the scaling issues with the Gas City team. Dolt team and like all that's happening.
But what happens is work just gets fire hosed in by agents. Jira can't keep up. GitHub actions can't keep up. Nobody's old human systems for issue tracking can keep up with the fire hose that agents can do with work once you get a factory going, right? And then and then that's when all these new classes of weird problems start to happen, including overwhelming.
Inside Wheelhouse
Benedikt Stemmildt
Yeah, and then and then Wheelhouse comes in. So what does Wheelhouse do there?
Steve Yegge
Well I'll show you. You wanna see?
Benedikt Stemmildt
Yeah. I think it's a good idea.
Steve Yegge
Okay, can you s are we good? Can you
Benedikt Stemmildt
Yeah, pictures. Alright,
Steve Yegge
So I have some sort of clot artifact here and this. Where's my slack? Here's my slack.
Uh-huh. So this is our game balance channel. Here's some players.
Diana, Malakath, and Rosato and Asmodian, these are like my core maintainer team. They've been with the game for. Literally decades.
They run live quests, they run events, they build areas. They're the ones keeping the game alive even when I even when I abandoned it during COVID before AI came back to save it. But if you scroll up here, they're talking to the Wyvern and Balance channel Wyvern and Balance Relay, which is actually live.
I believe it's Lark. It could be fly. Fly is actually the head of game balance, and Lark is the head of community, so both of them wind up talking to the players.
But yeah, you're looking at Wheelhouse here. Wheelhouse currently lives in Emacs. And each agent is each top level crew agent is listed on the left here.
They have their own color themes, you see? Heron is my general. Heron is Heron has shut the whole Wheelhouse down for right now because it was loading this box so heavily that I was worried we were going to drop our connection.
But normally they're cranking away here and I also have a couple of Google Cloud spots. 64 core instances. Pretty expensive that runs our build farms so that I could look move them off this Mac Studio. But yeah, Wheelhouse is nothing more than an Emacs interface to a whole bunch of agents.
And then all this little stuff down here, I don't know if you can see it. But right here. This is a little dashboard.
If I click on each any of these items, it shows me a detailed view of everything that's going on.
Here's the legend of all the symbols that you can see in so for the for the agents most of them are quiet. They're at low context. Wolf is at very high context here. You see the red bar.
Benedikt Stemmildt
Uh-huh.
Steve Yegge
Wolf is at 237 K context. 'Kay. Which is getting up there to where like they start to get amnesia, they start to get dementia.
I think research showed that around 350K in the 1 million window and they start dropping their guardrails. And so to keep the system pretty safe, I forced them to hand off. Now look, I don't have to touch this system anymore.
Like I said 20 minutes ago, they'll continue working forever now if I let them. Because they'll continue generating new work. It i think eventually it would probably stabilize, settle down, to they would they would be quiet, but it would probably take like at least a week at this point.
That's how much of a backlog I've got. And if I let them do the game work, they'd probably work for a month. So you have to be really careful.
You have to once you set this thing up, they will work and work and work and work. That's not the problem anymore. The the problem is controlling.
Yeah. And they just they go off the rails constantly. So I round robin to them.
So like going through the list. Ant is my head of quality. The bailiff is the drive wake.
It's sort of like Hooks were in Gas Town. The bailiff is the thing that wakes up. Every ten minutes and says, what would Steve do?
Okay. Bat is my head of finance and head of legal. And that works with me at my accountant and works with all my contracts, works with Aaron Scriven, who's my chief of staff, who gets me all my gigs with enterprises.
And then Bat does all of the accounting and the reconciliation and the contracts, it's wild. It's my birth shirt. B is in charge of my database.
Pro is in charge of my React client and all the game platforms, Android, iOS, Steam. Right? Because I'm launching all those clients.
Benedikt Stemmildt
And and they are like managers, and they are managing multiple agents, right? Like each of them has like a fleet or a team that they manage.
Steve Yegge
Pretty flat but but two or three of them have small teams and one of them has a giant team the marshal has the fleet. Mm-hmm. And the marshal is always running.
And the Marshall Marshall is running the Opus and the Soul workers, okay? And the seneschal actually manages the marshal for me. I never, almost never talked to the marshal.
The marshal is just always busy, load balancing on the fleet, putting him to sleep, bringing him up. The central is the chief of staff and handles like mule rotations.
I have 21 Claude Max mule accounts. Named after things like horse, pony, burrow, donkey, colt. They're all just mules, right? These are personal paid Claude Max accounts, and you can see what percent they're at. These eight or so are at 100% on Fable. I've got some more that are low on Fable, so I've got like a reserve. This is my token scheme. This is my token tap.
Benedikt Stemmildt
Yeah. Insane. Like they reset in a in a week or some some of them, right? Like
Steve Yegge
Their weekly resets. I mean, I have 21, so I'm each one of these is $200 a month.
Benedikt Stemmildt
Yeah.
Steve Yegge
And there so that it's costing me a lot of money out of pocket, right? But what they do is they allow me to run these seats here, which are my design seats. They generate work. They generate work. In fact, I have too many of them right now, and they generated so much work that they're the kind of reason that we're because they all have to go through a single merge queue.
Benedikt Stemmildt
Mm-hmm. They're
Steve Yegge
All working on the same repo.
Benedikt Stemmildt
Mm-hmm.
Steve Yegge
Right, which is a massive team working in one repo, for Git, right? So master is getting updated every three to four minutes. Which means that any given seat can get hundreds and hundreds of commits behind just by sitting there in a context and just by working for half an hour.
Like, literally, during the time that a single session is working, it can get a to a 500 commit spot. Right. And that does do itself to include the doctrine that they're working from.
So, like, you'll propagate a change out and say, everybody, stop what you're doing and do this different thing.
And they won't hear about it for like 24 hours because of propagation delays and stuff, right? So, like, it's a the merge queue is a huge, huge like a work in progress. It's I we actually simplified it recently.
But the merge queue it's clearing an average of two hundred and thirty commits a day. And last night a hundred and I don't know a hundred and something of them were for the game, right? And and and and and the the agent in charge handed me the list and I was like, kick I can't I can't look at 100 features.
I can't do this. So I have to go to my my so first thing I did was I rolled a bunch back. I said, just roll them back, man.
The players will complain about the ones they missed, right? And sure enough, one of them about rolled back, here it is right here. Diana says, it looks like there's a change to the Naga Dragon form.
So our game has. Player races, there's like 20 of them, and one of them is Nagas, and they can shapeshift into like different sort of reptilian creatures, including hydras and dragons and stuff.
And they wanted dragon body armor. Very, very, very niche corner of the game, but these players really wanted it, and the agents implemented it for them. And we rolled it back and the player said, can we have that back?
We really want the armor. So, right? And so, like, we're in the negotiation phase now, right?
So, this is what I was calling a wish factory, you know, in some of my blog posts. It's real. It's a challenging, but it's a space that I think the whole world is headed.
And it takes a certain kind of personality to be able to run all these things like this and not lose your mind. When they do something really bad.
Benedikt Stemmildt
Yeah, it's everything. I mean, it's it's also not the person, it's also the setup, like you mentioned, right? You have your Gamers that are directly connected to you, so the feedback comes in immediately and so on. And you are also working I I assume that you are working trunk based now, right? Because everything is like in a merge queue on on main so
Steve Yegge
Yeah, so we we are doing everything on the chunk. We could you know move to a release branch model, but so what we do is human contributors get to the front of the line. Human contributors are no longer allowed to break, so human contributors have to use branches.
And human contributors send PRs. But they go to the front of the line, they get special expedited. And it's actually, they like the process so much because the agents just set it up so they auto-land in production.
And and so the whole circuit is completely closed for the C d.
Pretty nice actually.
Benedikt Stemmildt
Yeah, I mean been doing this with with humans as well, like in in Yeah. Organizations only working on Yeah. Yeah. Reduces the further cognitive load of of looking into all of the merges and the branches and stale branches and so on. And the drift is also. Probably getting even worse with too many. Branches
Steve Yegge
Yeah, I, you know, I don't use work tree. They use work trees whenever they want, but generally speaking, I just tell them work on the trunk and work it out. And then the merge queue becomes a really complicated. Sorry, this is my workroom. It's extremely ugly and probably 25 years old. Somewhere prettier. Can you see this?
Benedikt Stemmildt
Yeah. Amazing.
Steve Yegge
Very, very old. It's from the nineties. There's probably people on right now. There's five people on. Not not many. For without me. But you know, the fact that there's people on in a Tuesday morning on a school day. You know, it's it's got a little bit of a community going. And, so like you see some of the effects like the special effects like I don't know if I like cast a fireball. This was added by the agents, right? This new artwork. Oh,
Benedikt Stemmildt
Wow, what's happening there? They burning.
Steve Yegge
Yeah, it was it was a fireball that lit everything on fire there briefly. But we're in town, so nothing get hurt. There's almost no AI art in this game.
The only AI art in the game is what you just saw. It's a spell effect. And we've made a promise to the players that other than the special effects, which have to be AI art.
Just atmospheric effects and whatever. But all the actual stuff in the game, the buildings, the monsters, the objects, the armor, everything is human art.
And this is like really super important to the players. It's also the case that the models are not good enough to generate our quality of art yet without players noticing and pixels missing and whatever. So it's not actually capable yet.
But it will be next year and we are all our plan is to make sure that we've always got human art in the game. As long as humans want to continue doing art. But yeah, it it's a sticky issue, right?
It's a thorny issue. Because there's no more human code in the game.
I mean, there's all the stuff I wrote by hand, but now we're using agents to produce it from. It turns out the models are actually really bad at anything creative. Right? Even though this game is really silly and you would think that anybody could come up with content for it, and really almost anybody can. I should go somewhere where like there's actually people. The the agents do terrible job with storylines. There's a player. Hi, who are you? Crazo, chillin.
Benedikt Stemmildt
Yeah, so storylines, they they seem to be scripted, right? Like super scripted in a way that you've heard them before or they're like copied from somewhere else.
Steve Yegge
They sound they sound Cringy, like just not, right? And then the maps, they can't build good maps. The maps that are.
I built this map. This is a terrible map. We're looking at an ugly one, but there are some very beautiful maps in the game.
And the art of map making is really advanced in Wyvern. And the AIs, they can recognize which the good ones are. Like Seoul 5.6 was able to spot our best maps.
It went through an interview of 10,000 maps, hand-drawn maps, and it was like, these are the best 30.
And it was right. But when we asked them to make maps, like even multiple passes, they're just they're like grade schoolers, right? So there's a whole set of creative stuff: storylines, quests, maps, art, whatever, that they're not able to do today. And honestly, I think our standards are going to continue to be pretty high, so I don't know when that will happen, but. Yeah,
Benedikt Stemmildt
I mean if they are not good at this, then what would be something well you thought that they would be not good at and they actually were very good at it. Mm-hmm. There's something else?
Steve Yegge
Well, I mean, like, Opus can't understand this game. At all. Opus just cannot wrap its head around my code base, my services, my micro services, this the auxiliary services, the ma the monolith of the game engine itself.
Opus just gets so confused. And Fable like got it, right? And and Fable gets it so well that Fable basically took this old, old legacy Java code base.
And just wired it with hundreds to thousands of little detectors. To tell if to make sure that it's all working as intended and then be alerted if anything goes wrong.
So they are able to detect changes that we never noticed before, like ever, like minor changes like a destination to a teleporter would change. Accidentally when somebody like you know made a change to the bank here and the exit moved or by one one square or whatever or or this archetype didn't load. It was just a content bug for us before, but now there's censors everywhere.
The Talus, oh yeah. In this random, far away dungeon, far down deep. One key didn't show up in the proper place.
We were just like, wow. We never knew this stuff before, right? And so their ability to sort of like turn the whole thing into a living nervous system that gave them signals.
Was really a game changer. It's like my game's never down now. I have a team of SREs.
Working around the clock on it. Like that's been nice, yeah. But they can't tell stories to save their lives.
Just saying. Yeah, so anyway, that's that's Wyvern, which is the target for Wheelhouse. I guess what people really do in Wyvern is they go into random dungeons if I can find there's probably one in this well.
Now we're in a random dungeon. Random dungeons are sort of like roguelikes. You familiar with roguelikes like NetHack? Yeah,
Benedikt Stemmildt
Sure, definitely. Yeah.
Steve Yegge
And but these are actually randomly generated and they have monsters in them and traps and quests and whatever. And the players eventually find their way to these. I can't believe there's almost no monsters in it. We're at a very low level. And they spend all their time in these in these random dungeons in the game.
Benedikt Stemmildt
Okay.
Steve Yegge
So one of the things that I'm good. So that creative storyline that I told you about. That the there's a there's a threshold that's going up with each new model release: how good of a map they can make, and how good of a story they can tell, and how good of a pixel art image they can draw, right?
And and one of the lines that I think we can maybe like tackle there, one of the spots where they might actually be able to break through. Is in random dungeon generation.
If I say these are ugly and this was a sketch. This was just like a sketch of what I wanted them to be like. They're nowhere near as fancy as like Dwarf Fortress, right?
And help me get them there. Okay. That's a problem that I think potentially a model like Fable or Astra maybe could help tackle.
And because it's it's constrained, it's bounded, it would be a new kind of dungeon, so if it was terrible, we could just torch it. You know, it doesn't change the fundamental game balance or dynamics.
So you have to like be on the lookout. For things that the models can do in spaces where they're uncomfortable. And then you use that as an eval.
And next to every new model release, you try your eval again, right? My game is full of. They can't do the artwork, they can't do the storylines, they can't do this and that.
And there's a whole long coastline of other features that they have. Struggled with. And so on that coastline is where I apply each new model generation and say, let's try it again.
Can we get it right this time? Yeah.
Benedikt Stemmildt
Yeah, interesting. I really think there is a lot of this that you can translate to like a real project work of like like enterprise organizations and such, right? I mean, this game is not. It's also real work. That was not what I meant, but it's more like, you know, right? This is like you and the players and. A very well contained environment. But a lot of that what you what you said translates very well into into real like big enterprises. Yeah,
Steve Yegge
It's a microcosm. Because I mean, if a big enterprise they usually have one big monolith that's a huge pain in the ass, but even their biggest monolith is like. Pretty small fraction of their overall code.
And what they usually have is a lot of teams with a lot of systems that are more close. Closer to this in size. This is about a million lines of Kotlin code and another million or two lines of Helper code.
And so, right, a couple million line code base, that's medium size.
Like, that's you know, 90, I don't know, 95% of GitHub repos fall in. Why is I think? So it's very relevant to enterprise teams, right?
Like and and they're setting this stuff up, right? But I'll tell you, I'll tell you my biggest takeaway. You want to hear my biggest takeaway from all this other than it's not ready yet?
My biggest takeaway is everyone f is thinking about it completely upside down in terms of how this gets into your organization. Alright. I haven't blogged this yet, so this is I will.
Depending on how fast you can get this podcast out, right, your audience would be first. Everyone thinks that AI is going to get into companies through the SDLC, starting with engineers. Writing code.
And then using agents to write code, and then using agents to push code and review code and gradually take over components of the SDL C. Right, in a very orderly fashion. And they will push that code out into the business units and gradually the business units will start picking up.
Coding and then gradually eventually after all that stuff is done.
Going to start standing up production agents that actually are like employees inside of the company. That actually little tape cues of various
Enterprise AI Is Arriving Upside Down
Benedikt Stemmildt
Story, like a perfect fairy tale.
Steve Yegge
Does that sound like what d would you agree that that's what people think's going to happen?
Benedikt Stemmildt
Yeah, that's what most people think that's going to happen, yeah.
Steve Yegge
Yeah, so let me tell you what's really happening. At real companies, I won't name them, but they're like large global banks and stuff. We're talking about companies that are super tech forward.
And have have tried this from both ends. What's actually happening is, anybody who tries to do the build side runs into Wheelhouse problems. The merge queue and the other many many other problems with it.
They probably don't even have beads, so they don't even they're not even ready to start running into my problems, they're running into their problems first.
And it's problem, problem, problem, problem, problem, problem. Meanwhile, anybody who stands up an agent in production to work a ticket queue gets it working instantly and they get addicted to it and before long they have like 12 of them. Is completely backwards.
There are companies out there already that have. 50 to 80% of their customer service. Ticket traffic coming in to production human facing agents. Okay.
And those very same companies have internal ticket queues where employees with account problems or machine problems or whatever that they would normally go to an internal like team like it or SDLC or whatever, they're getting handled by agents as well.
And yet these same companies are calling me because of struggling with the Wheelhouse side of things, the build side. Because they're getting incredibly uneven, unpredictable. Just bad results across the board.
Most. From just trying to give their agent give their engineers agents, right? And the discrepancy is so unintuitive because, right?
It's like, wait a minute. I thought. Agents were all all about building code and, in fact, working in the real world was too fuzzy for them and it was too dangerous.
And right? It all the whole thing seems upside down backwards, doesn't it?
Yeah, it's interesting. Do you have any thoughts
Benedikt Stemmildt
Of these?
Steve Yegge
Why do you think well okay, so do you believe me? Yeah.
Benedikt Stemmildt
I think there's there's a truth in both to be fableish here. You're half right and I'm half right. Yeah,
Steve Yegge
You're half right and I'm half right.
Benedikt Stemmildt
I mean I I think that in in terms of the tickets, the the interesting part is that usually there's a lot of like if I'm a human and I produce a ticket and then I don't want people to instantly reply to me, right? So there's this immediate effect that everything that goes to a ticket. Is by nature okay to have this wait time until you get an answer.
Because of this, I I feel that the the usability of including agents into the process is is not affecting the humans as m as such or as as much as if you have like a like a synchronous process, right?
And that's why I feel that you can include it there very well for a lot of tasks and a lot of stuff is very well documented and so on. And we see that as well if if, for example, you have multiple teams working together. Software teams and one team says Oh, the other teams are asking us so many questions all the time about our stuff and then we are like, Yeah, why don't you just integrate an agent into your slack?
And then the agent can answer for you and that works very, very well.
Because it can just look it up. And on the other hand the the building side also works well. Until you give it too much like until you want to make the jump to being too much autonomous.
Right? So i if if you right you need to have like that's why I think also I mean you mentioned it in the podcast v as well that in the episode that we won't need less humans for this for the near future. It's more like we have more wishes, so we need more people to do all of that stuff.
And then we need to have people that actually are able, like you, to figure out how such a factory could work. Which will great more work. And therefore more people that you need, so Yeah, I feel it's like the the answer that the factory will never be done seems actually pretty pretty much true. So I would believe Faber there.
Steve Yegge
Yeah, you know, your point about the autonomy is key, right? It's is is as long as you're not trying to push in the direction of making your code software production process like your s your your your software engineering. As long as you're not trying to automate it or over automate it.
Then I and as long as you think of it in terms of like smart humans moving as fast as smart humans are able to. Cognitively, all right, with super intelligent help. Then then you'll stay ahead of it.
Right? And it's gonna you're gonna look, there's going to be a lot of there's gonna be a lot of pressure from engineers who wanna like move faster. And you're gonna wanna actually put that front in front of them and say, no, it only moves this fast.
Okay. And all of the rest of the tokens that you wanna spend, spend on quality. Behind that front, right?
Behind that moving front, because there's a lot of cleanup to do. There's always a lot, right? So if you want to burn tokens, make sure that things are high quality, but make sure you're measuring whether that quality is having an impact.
When Fable said I was half right and half wrong about my P3s being important, right, it it turned out that like there were a lot of files. Which fixing all the P3s had actually zero impact on whether that file continued to be a hotspot or not. So, on whether it was actually improving the quality of.
And so this this is why this is why dropping an agent into a slack or dropping an agent onto a ticket queue or whatever. Oh. Is working so much better for companies right now than trying to push the bill.
Process automation. With agents. The reason is the blast radius is really small.
It's one customer, it's one ticket, it's one interaction. Whereas with this, I'm dealing with hundreds of tickets at a time. Literally, I'm pushing through. 230 a day to 270 on average.
And I'm capable of pushing 500 through if I had enough people to absorb all that work playtesting wise, right? Yeah. So, like, what happens is there's a law, I forget whose law it is, that your bottlenecks get pushed downstream, right?
When you remove a bottleneck, it just.
And but started with Toyota. Goldratt?
Benedikt Stemmildt
I don't know. Goldratt wrote a book about it, the Gold. Not sure.
Steve Yegge
Sure, but you know, it's it's the it's the it's the manufacturing thing that Toyota discovered. And it absolutely, it's brutal. It's like it's savage.
Like, as soon as your engineers like, speed up to the to the level that I'm at. They make all their customers downstream that. All their all their people that are consuming stuff that they're putting you know up there right so like starting with your peer engineers, they're overwhelmed by too much to review.
Right? And then starting business customers get mad because they have too much to process.
You're going. I heard from a big bank that this happened, that the engineers went fast and then the business couldn't keep up. And the businesses.
Benedikt Stemmildt
Too many questions from the engineers.
Steve Yegge
No, it wasn't even too many questions. The engineers were launching stuff that the c that the business had asked for. Mm-hmm. But they were doing it too fast. This has never happened in the history of software before.
Benedikt Stemmildt
Yeah.
Steve Yegge
Maybe brief spikes here and there, but by and large, it has never been the case that they they've been able to ask for things and get them so fast that they can't do the marketing campaigns and the and the accounting reconciliation and the whatever else they need, the GTM side, in order to actually absorb the features. And so the business pushes back. And then you push on the business and say, why can't you go faster?
And now you have a cultural problem. So like I can laugh and point at the bank and say, ah, but it happened to me, right?
I went so fast that my players got spooked and they came to me and they were like, we need to see the roadmap. This is going way too fast, right? Which is why I freaked out when during dry dock last week, when my factory was not supposed to be doing any game work.
And it launched 46 things. Some of which had to be rolled back, and then there was that negotiation. But it's why I freaked out because, look, if I shove too many features down my players' throats, even if it's stuff they asked for.
They'll get scared and mad. This is just a mind-blowing thing for organizations they're going to have to do with. So you get these weird situations where the CTO or the CIO or somebody up there is is AI pill.
And they're like, we're just going to just jam through this and we're going to automate and we're going to move fast. Just not realizing that, first of all, the blast radius effect that they're going to have. And second, the amount of train they're going to have.
Because most of the stuff they build is going to be garbage.
I had to burn down thirty, forty percent of my factory that was built by Fable Five carefully. And it was good that we burned it down, but that was a lot of wasted, that was a week of wasted tokens and no commits, right? So you're gonna see really spotty it's hard to measure. Ticket cues are easy to measure. This is why it's winning on the ticket cue side, right? It's because if you're standing up an agent to deal with customer service issues, then you probably have.
Years to decades of actual real data of humans answering. And so so you know how well the agents are doing. Relative to humans on resolving certain categories of issues.
You'll have a histogram, right, of your top 10. I can tell you already. Agents are better than humans at collecting money.
Okay, on that one arm, okay, a couple of different banks have told me this. And it's just it's a skill. Agents have surpassed humans on that skill.
It's measurable. And so it's a no-brainer to stick an agent in there because they know how well the agent does relative to humans.
And it's not like self-driving cars where someone dies. It's just a mess up that a human has to deal with later, right? Depending on what your customer service needs are.
So it's been a big success there actually, despite everyone saying it's not working. And then it's also been a success inside of companies because, again, right, the cost and the blast radius. And and actually the difficulty of the problem is so much simpler.
Every agent just has one thing it has to worry about. It has a long historical log of internal customer tickets that have been resolved that they can look at as a corpus for almost 10 years.
Like money at that point. Right, so you can measure it. And so so that's why I'm saying like companies are thinking about it backwards.
They're coming to me and they're saying, we need to train all of our engineers up on vibe coding so that they o they can all look five times as productive. And it's like, no. No, no, no, no, no, no.
That's not what you need to do. What you need to do is stand up one. Production agent in have your team, your engineering teams get together and figure out which one it is.
I did this recently with a big security company in London. And it was their idea, but it turned out to be a great idea. They're like, Steve's going to tell us how to do it.
I am, you know, and I did, right? They stood up one of these. Actually, not the ones, it's the ones you can't see.
So I don't have a good dashboard for them. I have, I guess I have to castell them, but. I have all these production agents that we're not looking at right now that run my game.
There's one that watches player conversations to see if anyone's saying anything naughty, like it's an abuse detector. Like looking for trolls and stuff. 'Cause every time I do a blog post, sometimes some trolls show up, right?
And so now I have an actual agent like that can can can monitor it. And then I have another one that's looking, you know, like SRE to make sure the services are up. I have another one that's the Wish factory looking for certain channels for feature ideas and so on, right?
These are like, User standing agent. Unattended autonomous agents with very narrow scopes. And there's rules for them.
Like they, you know, if they're observing, then they can't act. And if they act, they can't judge. And so on.
You know, you have to set all these things out. If a company sets up one of these, one. They will suddenly realize how complicated it is to set one of them up.
'Cause first of all, it's not just one. It's like I said, you've got to split it into the intake and the routing and the decider and the executor and the reviewer and all that.
So you've got a cluster. Just to do the one ticket queue. Right.
And then they have to be wired up to infinite numbers of accounts and monitors and backups, and they have to know about each other and have mail, and they have to like. It just becomes a huge engineering project just to stand these things up to replace a human in the queue. And they have no idea how much work it's going to be until they actually start.
And it's not even just one-time setup, it's ongoing work.
Okay, and they like it, it's good work because the agents are doing ugly ticket work that humans didn't want to do and they're doing engineering work with agents to make the agents work better. Right? So like I don't know, it's not for everyone, but that's kind of what's happening.
And meanwhile, if you're trying to I mean, because there's a huge literacy problem right now. There's a huge disparity in the industry. Agent literacy.
Like your ability to work with coding agents like Codex and Claude Code here, right?
Yeah. It's heavily dependent on how much computer science knowledge you have and you can keep in your head. And you know, software engineering, how much like how much systems knowledge you can have.
Good product managers have this, by the way. Is it? Best PMs, the best product managers are software engineers and computer scientists who have a working model of the whole system in their head: the data flow and the edge cases.
All of it, right? You have that in your head, and you have to be able to read and write really fast.
And you need some sort of a system that lets you switch agents like this. And you need a memory system and a brain. And people don't know how to do this.
Right? Like we saw a presentation from Netflix in April where they're training engineers on it and they go from zero million tokens a day where they're just chatting with the a the with with AI. They're not even using agents, they're just chatting. 0 million tokens a day up to 4 million tokens a day.
Okay, when they're using single agent synchronous throughout the workday.
So they can talk to an agent, but they watch at work. Which is like still kind of borderline illiterate, but it's it's better. And then finally, when they've learned to build enough trust to let the agent work by itself and check in on it once in a while, and now can they have three or four of them running at a time, now they get to 12 to 15 million tokens a day. And then they become pigs and you have to teach them not to spend tokens.
And they're still six months behind where I'm at here and my stuff, I can barely hold it together. So like, I'm just saying, like, if you're charging down this path and saying, I read about Steve Yegge's Wheelhouse and we're gonna do that at our company. You're in for a world of hurt. You really, really are. This is it. It's a it's a it's like you just build a big shotgun that you can just spray around. What do you think is going to happen?
Start with One Production Agent
Benedikt Stemmildt
Yeah. I mean I th I think it's a very good point to to land the episode that you should focus on real outcome and on the scope that you can actually build right now and not try to. I mean it's similar to what what microservices did before, right?
Like someone said microservices are great, everybody built microservices and then Oh shit, this is not for us, it's just for Netflix or something like that, right? So. We should probably all focus more on the real outcome.
And then let you figure out how to do all of that stuff so that we can use it later.
Maybe this is a good point.
Steve Yegge
And being focused on outcomes is like. It's the new it's the new North Star. You have to do it now. There's no messing around anymore. A buddy of mine, Mik Kersten, just wrote a book called output to outcome, where he identifies a bunch of the patterns where people are not doing. A good job of being able to like right? Because I mean like they're measuring outputs now. Token spend, IG, VR throughput, PRs,
Benedikt Stemmildt
Yeah, coded
Steve Yegge
Lines. I don't know, they don't know what to measure. And then you have people, companies who actually know how to measure outcomes.
These are usually, you know who, you know who's the best? I'm gonna tell you who's the best at actually measuring outcomes from engineering work. Turns out to be the gambling companies.
Okay, all the ones that do any sort of online gambling or sports betting or slot machines or. And I've talked to a lot of those companies and they've been doing the same thing. Incredibly precise tolerances for years to decades, right?
And so they have lots and lots of data and they know. How long projects take? And they know where projects go wrong. And they know like. Bug regression rates and all of that. 'Cause you're just stamping out game after game or experience after experience that are all slight. Modifications of each other. Right?
Benedikt Stemmildt
Yeah.
Steve Yegge
And so they're they're amazingly good at measuring outcomes from this stuff. And they're Honestly, they're they're all still struggling with it too. I mean, it's that new.
So, I mean, like the happy message here, there's plenty of happy messages here. One is that AIs are not taking our jobs. Changing our jobs.
Another message is that you're not behind. Your company is actually probably just doing just as well as everybody else, which is to say, poorly. Why?
Because there's only one model, two models today that are any good, Fable 5.1 and 5.0.
Right. And those are too expensive they're so expensive that most companies have them turned off. And so companies are actually working with dumb models that are not really capable of doing any of this kind.
And their AI pill CTO is saying, get it done, get it done. And it's causing friction and tension. And an and anxiety and anger and hard feelings and people are upset and is it any wonder they bully the agents when they come in?
Right? But it would all be a lot simpler if you stopped trying to build software with agents like like crazy people like I'm doing, because it's just, it's really hard right now.
Okay, unless you can afford Fable, etc. And start focusing on your AI employees because that still applies here. This is a superset of that problem. These agents here are still long lived employees. They have memories. They have a seat retirement plan if I have to turn the seat down, right? They like they have they have charters and lanes and like they have they have responsibilities. Like
Benedikt Stemmildt
Yeah.
Steve Yegge
If I lose one of them, it's kind of a big deal, right? And so, like, so like why not go ahead and learn all of your AI employee lessons without having to deal with your merge queue and your blast radius of destroying your customers and your business users lessons? Separate those two lessons and focus on the easy. And then once you've nailed that, then you can jump into this one.
Benedikt Stemmildt
Go slow to go fast.
Steve Yegge
Go slow to go fast. Yeah. And go slow in the right lane.
Benedikt Stemmildt
Thank you so much, Steve. It has been Fantastic. I really enjoyed it. I hope you did too. I had a lot of fun.
Steve Yegge
It's been fun. As we've been talking, people have been logging into my game and logging out. I saw Rakazaz here. He's from Singapore. My players are from all over the world. It's pretty wild. Oh, it wasn't Rackus. Oh yeah, this guy right here, Racktheliox. So anyway, yeah, my game's gonna be a big deal here in a couple years once I get this figured out. Thanks for having me on. I it's been a lot of fun chatting. I hope this wasn't.
Benedikt Stemmildt
Thank you so much for being here. Let me stop the recording. Mm-hmm.
Steve Yegge
Cheers.
Benedikt Stemmildt
Wow, what an episode. Thank you so much for listening. And Steve, thank you so much for being here.
Truly an honor. It was amazing to see how you work. What you are currently working on, and I feel that not many people are that far ahead as you.
And that is really like a glimpse into the future what is going to happen. If you want to learn more, Play with Steve. Join his game, it's called Wyvern.
Or you can also take a look at the harness he was using, Wheelhouse.
He also wrote a book on vibe coding together with Gene Kim. You will find all of that in the description. Go go read it. And yeah, if you wanna come to the show or if you if you know someone we should interview for the next episode, please let us know. Write to us. We are always keen on meeting new people. Yeah, no, I think. That's it. That was Rethink Engineering's first new episode. Powered by re:cinq and by hackers&wizards. See you next time.
Benedikt Stemmildt
Mm-hmm.