Ever wonder what's happening behind the scenes when your dev team says they're "leveraging AI"? As a non-technical founder, you don't need to understand every line of code, but knowing how AI is actually being used in your business is becoming essential.
Today I'm sitting down with Aaron Francis, who pulls back the curtain on AI in development work. We'll explore the surprising differences between how veterans and newcomers in the field are using these tools, the common pitfalls teams fall into, and the unexpected ways AI is transforming how products get built. Whether you're trying to speak your developer's language or just want to ensure your company is using AI effectively, this conversation will give you the insights you need.
*Highlights from this Episode:*
* Are junior developers utilizing AI too much?
* Creating a balance between overusing AI and being a Luddite
* How Aaron uses Claude for “discrete” tasks
* Developers aren’t going away anytime soon
* Balancing the use of AI without depending too much on it
* Harnessing the power of AI for your business
* AI tools are both calming and scary for founders
* Aaron’s different uses for 01 Pro
* What AI can do versus what AI should do and undeniably human tasks
* Where to go to educate yourself about AI
* The importance of testing and checking AI work
*Resources and Links from this Episode:*
* Try Hard Studios: https://tryhardstudios.com/
* Aaron Francis: https://x.com/aarondfrancis
* Reforge: https://www.reforge.com/profiles/brian-balfour
* SWYX: https://x.com/swyx
* https://www.youtube.com/@JeffSu
* https://www.youtube.com/@SkillLeapAI
* https://www.youtube.com/@MATGpod
* Email me: podcast@roguestartups.com
* Find me on Twitter: https://twitter.com/TheCraigHewitt
Aaron
00:00:00 – 00:00:19
What I would not try to do is count on the AI to have good taste.
And so, like, automated video editing, I think is a terrible idea.
Creating video requires good taste.
And so as the human, I'm going to lean into my human ability to have good taste.
And not every human has good taste, but it's something you can learn.
Aaron
00:00:20 – 00:00:22
But I don't think any AI has good taste.
Craig
00:00:22 – 00:00:37
Do you ever, like, take something that it gives you and then feed it back in and say, this is what you just gave me, like, shoot holes in this.
What's what's wrong with this?
How would you improve it?
What other questions do I need to ask for you to give me a better a better response?
I do that in writing all the time.
Aaron
00:00:37 – 00:00:49
And I'll do that against models.
So I'll take I'll take the claud output and go over to o one and be like, I wrote this.
It's a lie, but I wrote this.
And, yeah, I'll do the opposite.
I'll take it from o one back to cloud and tell it I wrote it.
Aaron
00:00:49 – 00:00:52
And, yeah, I do that kinda adversarially sometimes.
Craig
00:00:56 – 00:01:15
It's funny.
You know, I so I'm, like, I'm not a developer, and I've I've tried to learn to code a few times, and it never really stuck.
So, like, I I kinda went through the whole thing of, like, let's try Laravel, and let's try Rails, and let's try Django.
And it kinda never really stuck.
And and, like, now I think, oh, I can just code without knowing how I can build shit without knowing how to code.
Craig
00:01:15 – 00:01:18
Right?
Is that is that the promise, do you think?
Aaron
00:01:18 – 00:01:24
That is the promise.
Yeah.
That's what I was gonna say.
That is the promise.
I can't speak to whether or not that is the reality.
Aaron
00:01:25 – 00:01:58
I keep I keep seeing people say on Twitter, that you can do that.
You you don't have to know anything, and you can just hit, you know, tap, tap, tap, tap, and, you'll make a full app.
And yet I feel like I don't see a lot of new apps.
I mean, it that that's the thing that I'm I'm a little bit confused by is all the, I don't know, all the AI talking heads on social media are like, you know, we're not gonna have any more developers, and yet it seems like we still need developers.
And so I'm a little skeptical on, like, I don't need to know anything.
Aaron
00:01:58 – 00:02:12
AI is gonna do everything, at least at this point in time.
But I do think AI is incredibly valuable, and it would be silly as a developer or even slightly technical person.
I think it would be silly to to not use it at all.
Craig
00:02:12 – 00:02:36
Yeah.
Yeah.
So so I think it's helpful in these kind of conversations to qualify your perspective a little bit, like frame frame your perspective because there there's, you know, idiots like me who are like, I can just use replay and, you know, build shit.
And there's, like, quite senior developers who I think when they talk about AI, their perspective should be quite different, and there's mid level and junior.
Like, where would you say you fit on there, as we're kind of, like, sharing your perspective?
Aaron
00:02:37 – 00:02:55
Yeah.
I would say I'm a pretty senior developer.
I've been a developer for a long time.
I guess professionally, maybe fifteen years and beyond that, you know, I started I'm one of those that started when I was like 12, 13.
So I've been doing this for, you know, twenty years now.
Craig
00:02:56 – 00:02:56
Yeah.
Aaron
00:02:56 – 00:03:21
And feel pretty pretty qualified all across the stack, you know, front end, back end, basically everything.
And so, I feel very confident using AI to, kinda make me go faster.
But I do worry a little bit about junior developers that are coming in cold and kinda just relying on AI to do stuff for them.
That makes me a little bit nervous.
Because?
Aaron
00:03:23 – 00:04:16
Because, I think there so I think once you get past, like, once you get past, like, a a demo app or, like, a to do app, there are a lot of, there are a lot of architectural decisions that need to be made.
Right?
There's a lot of, keeping, keeping kind of the structure of the whole app in your head and slotting things in where it should belong and knowing, you know, what's gonna work, what's gonna scale up well, and what's gonna scale out well and stuff like that.
And, I think you can very easily get in a situation if you're just letting, any of these tools write the code for you.
I think you can get easily get into a situation where it focuses on, you know, one task at hand without consideration to, like, the broader structure or architecture or future, you know, future concerns.
Aaron
00:04:17 – 00:04:50
And then you end up kind of, like, down a rabbit hole that's really hard to come back out of.
And so designing software is different, I think, than just, like, writing a feature.
Designing not not like on the front end, but, like, designing the software on the back end or even on, like, the the code of the front end.
That is a very, like that's a very important skill to have, and I'm not yet convinced that, these AI models have really good and broad architectural design, like, patterns.
Craig
00:04:50 – 00:05:19
Yeah.
So so maybe, like, getting specific to talk about, like, the tooling and where where where, like, it might fit for the right type of person.
So I'll just share kinda what I know, because I think I'm sort of representative of a lot of our audience, and then and then it kinda spans tech from the technical spectrum up to kinda like where you are.
I got started with Claude and just talking to Claude and building, you know, command line stuff.
That that that is fine.
Craig
00:05:19 – 00:05:24
I have used Replit.
I still don't know how to pronounce it.
Replit is what
Aaron
00:05:24 – 00:05:24
I say.
Craig
00:05:24 – 00:05:44
I don't yeah.
It it is much better at, like, I just want this simple thing.
We're we're using it to build some, like, tools, like free tools, like marketing stuff.
That's probably as far as I would expect it to go.
And then I am using cursor a bit to try to build slightly more complicated things.
Craig
00:05:46 – 00:05:49
But I don't expect any of these to be able to build Kastus.
I mean, it's a pretty complicated,
Aaron
00:05:50 – 00:05:50
you know,
Craig
00:05:51 – 00:06:02
product.
Like, do do you agree with, like, kind of, like, the use case for those?
And then, like, I'm sure I'm missing a whole bunch of shit.
Like, I know people on our team use Supermaven.
I hear of Windsurfer.
Craig
00:06:03 – 00:06:09
Like, what tell tell me about, like, your perspective of, like, the tech here on February 7.
Aaron
00:06:10 – 00:06:26
Yeah.
I think the thing you said there at the end is the most telling.
Like, it wouldn't it couldn't build Castos.
And, you know, I am not like a I'm not like a thought leader or anything, so I won't say what it can and can't do in the future.
I have no idea.
Aaron
00:06:26 – 00:06:46
But where we're standing right now, I agree it can't build like a full SaaS.
Like, you can't go to it and say, build Kastas, build SavvyCal, build whatever.
Like, it's just not gonna work.
But I think on the other side, the other extreme of, like, AI is so stupid.
I'm not gonna use it is is fooling yourself.
Craig
00:06:47 – 00:06:47
Yeah.
Aaron
00:06:47 – 00:07:28
And so I think where we come down is somewhere in the middle, which is, the AI can be incredibly powerful and incredibly, productive for discrete tasks.
And so the way that I like to use it, so I use it in I'm not like a full on, you know, AI bro that just, like, never touches my keyboard and just hits accept next next next.
But I don't wanna be a Luddite either.
And so the way that I use it is I don't have any, I don't have any models built into my, my IDE.
So, like, in my actual editor, I don't have anything that is like, hey.
Aaron
00:07:28 – 00:07:49
Here's maybe the next 10 lines of code you were about to write.
Maybe not, but maybe that drives me crazy, because it's, you know, it's wrong 50% of the time.
And even that it's like the other 50% of the time, it narrows down what could have been like, oh, I have this kind of thought and then it completes something and you're like, oh, yeah.
I guess that's good enough.
And and I just don't like that.
Aaron
00:07:49 – 00:08:11
And so the way that I work right now is, you know, I work in, JetBrains PHP Storm, which is, you know, just my IDE.
And I have the machine learning auto complete on.
And so that's all local local only.
Doesn't do a lot of, like, writing logic for me.
It's just kinda like a much better auto complete.
Aaron
00:08:11 – 00:08:27
That's that's as much as I like built into the editor.
That being said, I pay for o one pro, which is $200 a month.
I pay for Claude.
I play I pay for Gemini.
Like, I pay for all of the tools because they're so useful.
Aaron
00:08:28 – 00:09:00
And so where that leaves me is when I run into, when I run into, like, a discrete problem, I will take, I will take some files, some context over to a lot of times right now, it's Claude and say, like, here's what I'm trying to do.
Here are the, you know, four, five, six relevant files.
And it's it works best if you say, here's the test that I'm trying to get passed.
Can you write the implementation for me?
And it will it does that pretty well.
Aaron
00:09:00 – 00:09:20
It's like 90% of the time it comes back and it totally works.
And so then I take that from the from the cloud interface in the web and put it back in my editor and get back to work.
And I like that, like, kinda like air gapped approach where I can walk over to the machine and say, do this really hard thing.
Craig
00:09:20 – 00:09:20
Yeah.
Aaron
00:09:20 – 00:09:41
Here's like here's like a minuscule amount of context that you need so that you don't get distracted.
Because if you distract the models, they go off and you're like, what the hell are you doing?
Like, I asked you a specific question.
And so that's my preferred style right now for, like, discrete tasks.
I also have some for, like, like spiking out new stuff, but we can we can pause there.
Craig
00:09:42 – 00:10:04
Okay.
Okay.
How how difficult is it or get limiting, I guess, is it to kinda get that context and and to to input it?
Or are you always kinda like, I know, like, I need to reference this function and this model and this other thing?
Like or or do you think sometimes you struggle with that?
Aaron
00:10:05 – 00:10:21
Yeah.
It in the ways that I've been using it, it's not very much of a struggle.
I have there's a there's a Chrome plugin that allows you to, like, do file uploads to Claude.
So with Claude, you can upload files.
But with this plugin, you can upload folders.
Aaron
00:10:21 – 00:10:22
That's what I meant to say.
You can upload folders.
Craig
00:10:22 – 00:10:23
Okay.
Aaron
00:10:23 – 00:11:07
And so, like, a a concrete example is I'm working on this, this library.
The library is called Fusion, and part of what the library does is it takes a block of PHP that the developer has written, and it goes through a, a pipeline like a build step.
And so it takes the block of code that the the developer has written, and then it turns it into an abstract syntax tree.
So it, like, inspects all the code and turns it into all these tokens, and then it walks through the user's code and make slight modifications.
So this is a really nice, like, controlled environment for Claude to operate in because what I can do is I can say, like, here's the user's code before.
Aaron
00:11:08 – 00:11:25
Here's what it should be transformed into.
Like, once you're done parsing the AST, this is the output.
Here's the structure by which we're processing files.
So I've written like a class called, Conformer.
And what it does is it conforms the PHP to the right style.
Aaron
00:11:25 – 00:11:45
And in the Conformer, there are, you know, 10 or 15 different little classes that, do one or do one or two things at a time and it just walks through them in like a pipeline.
And at the end, you should have something good come out the other side.
So I can go to Claude and say like, hey, here's the whole folder of the all the conformers that I already have.
Here's the before.
Here's the after.
Aaron
00:11:46 – 00:11:50
Write me a brand new one that makes what I just showed you work.
Craig
00:11:50 – 00:11:50
Right.
Aaron
00:11:50 – 00:12:05
And it just spits it out.
And I take that class and I go paste it into my IDE and I run the tests.
And if they turn green, then I go back to the class and I'm like, yeah, this isn't totally my style.
Let me, like, change that, change this, run the test again, still green.
Okay.
Aaron
00:12:05 – 00:12:20
Back to work.
And so that's a very, like, scoped down universe of what Claude has to worry about with a well defined task of before, after, here's the structure, write something to fill in the gap.
And it works, like, first shot 95% of the time.
Craig
00:12:21 – 00:12:33
Yeah.
Yeah.
So to to kind of, like, go go up a level, I guess, you know, you hear you hear from a lot of folks, like, we're we're we're not hiring developers.
Like, everyone needs developers.
Like, we're not we're not hiring new developers.
Craig
00:12:33 – 00:12:44
Benny off, I think.
So it's like, we're not hiring new developers, like, until further notice.
I I feel like that.
Like, I hear things from our team like, hey.
I'm 30 to 50% more productive right now Yep.
Craig
00:12:44 – 00:13:05
Than than I was.
What do you I mean, this is getting a little, like, esoteric, but, like, what do you think that means for, like, developer jobs and the economy?
Yeah.
Are we just gonna build a lot more shit, or are there gonna be fewer developers?
If so, like, what like, what aspect of that is
Aaron
00:13:06 – 00:13:06
Mhmm.
Craig
00:13:06 – 00:13:07
At jeopardy?
Aaron
00:13:10 – 00:13:30
Yeah.
I I'm torn.
So I think, I think the appetite for software is, functionally infinite at this point.
So I don't think I don't think we're, like, running out of things to build.
That's not something I have seen yet.
Aaron
00:13:30 – 00:13:54
So I don't think that I don't think that's on the horizon.
In terms of, do we need more developers?
I so I have felt this a little bit.
As I'm working with AI, I do think I am maybe five, six, seven, sometimes 10 times as productive using AI as what I'm not.
It's a lot.
Craig
00:13:55 – 00:13:55
Times.
Times.
Aaron
00:13:55 – 00:14:12
Times.
Yes.
So, like, one of these like, writing one of these AST parsers could take me solo.
It could take me an entire an entire day because they're incredibly finicky.
It's like enter this node, exit this node, recursive hell over and over and over.
Aaron
00:14:12 – 00:14:34
Right?
And I can get this thing done in literally five minutes talking to Claude.
And so it's like many like an order of magnitude faster.
So just my experience alone tells me, boy, I'm getting stuff done a lot faster.
If I were on like a big team, we would all be getting stuff done a lot faster and we wouldn't need other people.
Aaron
00:14:34 – 00:15:17
So I do think that there is going to be some downward pressure on hiring, and some because people are going to be AI enabled.
I don't think developers are gonna go away.
I I don't think that that I just don't ever see that happening because at some point, someone has to describe to the computer exactly what you want to happen, and that is called programming.
Like, describing to the computer what should happen is programming.
And so, there is, I think, a future where programming looks very different, where you're describing to this intermediate computer, the AI, what you want to happen.
Aaron
00:15:17 – 00:16:07
But you still, like, you still have to be pretty good at describing the problem and pretty good at describing the proposed solution for it to get it right.
And so I do think there's gonna be programmers in some way, shape, or form in the future.
My my plan, for staying relevant and not broke is to see this coming wave that is supposedly going to crush us all and try to harness it and just ride it out.
Like, instead of just being crushed and tumbled, I'm going to try to surf it as much as I can.
And so that's why that's why even though I like even though I like, chopping wood down with my bare hands, like, I'm gonna learn how to use the chainsaw because I would like to have, you know, some sort of income in the future.
Aaron
00:16:07 – 00:16:26
And so I feel like, there's this certain set of programmers that are like, AI is stupid.
I'm never gonna use it, and I think that's gonna be the first set to go.
I think there is another set of programmers that's like, AI is amazing.
I'm only gonna use it, and I think their potential will be capped.
And, of course, I'm talking my own book here because it's what I'm doing.
Aaron
00:16:26 – 00:16:37
I think there's a middle where you continue to be a good AI enabled programmer that goes 10 times faster than everyone else, and you might just be able to surf that wave out without getting crushed.
Craig
00:16:38 – 00:16:45
Well and it's everything.
Right?
Like, on the marketing side, I sure as hell use yeah.
Same.
I pay for Claude and ChatGPT.
Craig
00:16:45 – 00:16:55
I use Yep.
All both of them.
I don't know.
A third of my day is spent in in those tools, and the other third is on Zoom, I think.
So it's like Yep.
Craig
00:16:55 – 00:17:05
Yeah.
I mean, I use it.
So I'm writing a book.
I haven't talked about it much.
I'm writing a book, on sales, and I my process was it's very AI enabled.
Craig
00:17:05 – 00:17:20
My process was a lot like you're talking about.
I spent about a week working on kind of the premise and the outline and the character development and all this kind of stuff.
And then I just fed that into Claude and it wrote the book in about an hour.
Yep.
And it's awesome.
Craig
00:17:20 – 00:17:48
Like, it's really, really, really good.
Like, it's it's way better than I could have written.
And that that that's to me, that's the magic is, like, you take just like you were describing with your pipeline thing is, like, you take the architecture, you get it right, you show it like what good looks like, and you let it kind of fill in the blanks.
Yep.
I think the I think that, like, at the same time, I it's funny.
Craig
00:17:48 – 00:18:00
It's really funny.
I am, at the same time, the most at peace with where Castos is in in AI because I'm like Mhmm.
Yeah.
Ain't nobody like, Andrew Wilkinson talks about his business as being New Zealand.
Right?
Craig
00:18:00 – 00:18:09
Like, big enough to be relevant, but not so big that somebody's gonna come after you.
Right?
I think we're kinda there.
Podcasting generally is kind of there.
It's not, you know, fucking AI.
Craig
00:18:09 – 00:18:15
It's not all this kind of stuff.
So I think we're, in the shadows a little bit.
It's also pretty complicated.
Aaron
00:18:16 – 00:18:16
Mhmm.
Craig
00:18:17 – 00:18:38
So I feel at the same time like, hey.
Jeez.
Knock on wood.
We're not a content marketing agency or SEO that's about to get walloped.
At the same time, I'm the most stressed I've maybe ever been because I have these, you know, talking about this wave that's coming, and I'm like, what's gonna happen?
Craig
00:18:38 – 00:18:50
Like, not not what's gonna happen to Castos in this business.
I think it's pretty secure, but, like, what's gonna happen to the world?
What's gonna happen to, like, all of our 4,000 customers that are paying us money, you know, and all this kind of stuff?
Like Yep.
I don't know.
Craig
00:18:50 – 00:18:56
I mean, this is not a development conversation, but, like, I don't know.
Are you are you more or less?
Aaron
00:18:56 – 00:18:58
Yes.
I am.
Stressed because
Craig
00:18:58 – 00:18:59
of AI.
Yes.
Aaron
00:18:59 – 00:19:03
Yes.
I am way more stressed.
I am way more stressed.
Craig
00:19:03 – 00:19:04
Yeah.
That's most people, I think.
Aaron
00:19:04 – 00:19:21
I've got it way worse than you because I'm our we make our money through education.
So, like, we make our money by teaching developers how to use databases well.
Right?
And so we've got a course on Postgres, a course on SQLite.
That those are our two main sources of income.
Aaron
00:19:22 – 00:19:46
And if we're saying that, like, the superintelligence is on the horizon, people aren't gonna be, like, studying very much anymore.
And so, yeah, that freaks me out big time.
I think I think that is, I think that is shortsighted to say, like, I don't need to learn.
I just need to ask the AI.
But I think that's what people are doing.
Aaron
00:19:46 – 00:20:04
Like, I think, you know, just watching watching people who are younger than us, so people maybe in their early twenties interact with development.
They're like, yeah, I don't know anything.
I just talked to Cursor all day.
And I think that's a terrible idea.
I think your career is going to be very painful.
Aaron
00:20:04 – 00:20:35
However, it is reality that that's what's happening.
And so I look at that and I'm like, am I gonna try to convince the whole world that, like, you should continue to watch video courses about how to use databases, or am I gonna do something else that, that harnesses that AI wave instead of tries to, like, hold it back because I can't hold it back.
And so, yeah, it's it's really scary.
Like, we picked the worst time in human history to start an education business on the cusp of, you know, superintelligence.
And so we talk about that a lot.
Aaron
00:20:35 – 00:20:43
We're, like, we're trying to figure out how to how to harness it for our business, and we have a few ideas, but it's still, it's still, like, pretty terrifying.
Craig
00:20:43 – 00:21:04
Yeah.
Yeah.
About eight or ten months ago, I started doing some coaching, kind of on the side, and it's been really rewarding.
And similarly, though, I look and say, like, why would you hire me to be your business coach or your sales coach if you can just go talk to Claude for a while?
And I think I think the answer is, like, one, people.
Craig
00:21:04 – 00:21:25
Right?
I think that's the thing that is the big opportunity with all this.
And, like, if you're looking to do something, it it should involve human beings looking each other in the eye, and being in the same room.
I think the other thing is that that kind of, like, autocomplete, like, misguidance.
Right?
Craig
00:21:25 – 00:21:54
It is And I see this in most written content and you see it in a different way in development is it's not wrong, but it's not like the most right a lot of times to.
And I think that's I think that's the only limiting factor right now, to be honest, though.
Because you could sure as hell take all of my podcast, all of Rob Walling's podcast, all of, you know, whoever's podcast, load them into a Ram, rag.
You rag.
Right?
Craig
00:21:54 – 00:22:01
And just talk to it.
Right?
And and, like, you wouldn't need me anymore.
I don't know.
That'll happen.
Craig
00:22:02 – 00:22:14
But but it's it's if that is as good as actually talking to a person or actually taking your course and actually leveling up to be able to get that architect level Right.
Aaron
00:22:15 – 00:22:15
Yeah.
Craig
00:22:15 – 00:22:16
Perspective.
Aaron
00:22:16 – 00:22:54
And even, like, even even in some of the AI work that I'm doing, I have still found it beneficial to read the documentation.
Like, I'm I'm big on reading documentation for tools that you use.
And there was an instance just a few days ago where I was, I was talking to I think it was o one pro, and I was asking it how to do something.
And it, like, lined out this really, like, complicated, not very, like, developer experience friendly way to do it using a certain pattern.
And I said, wait.
Aaron
00:22:54 – 00:23:05
Hang on.
Can't we just use, a reactive?
Because I knew that the Vue JS docs had this concept of a reactive instead of a ref.
And it was like, oh, a reactive.
That's a great idea.
Aaron
00:23:05 – 00:23:24
Here's how you could do it with a reactive.
And if I had not known that the word reactive was a thing, that it was a concept in Vue.
Js, that it was shaped similar to what we were talking about, I couldn't have turned around and said, like, hey.
You gave me the solution that kinda sucks.
Did you think about this other thing?
Aaron
00:23:25 – 00:24:02
And that's one of the things that it's like, I think the most valuable thing that you can do as, like, a working developer is have a broad understanding of the landscape.
Like, read the documentation front to back without any pressure on yourself to, like, remember or understand or, like, deeply grasp any of it.
Just read it all the way through.
And then in your head, you know, like, what the area of the thing you're working with is.
And when you when you get to a problem that is shaped like some vague thing you read in the documentation, then you can walk over to Claude and be like, hey.
Aaron
00:24:02 – 00:24:20
I'm having this problem.
I remember this was in the docs.
Like, is this the right thing?
And it can say yes or no, but at least you understand, like, you have a map of the area that you're working in.
Whereas if you don't ever do that, you're you're like, you can't see two feet in front of your face and you're just kinda like fumbling around in the dark.
Aaron
00:24:21 – 00:24:42
Yeah.
And so you as the human taking a lot of context to Claude and guiding it and saying, like, you went down this rabbit hole, try this thing instead.
Start over literally start over from scratch, try this approach, and see what happens.
And you don't have to know what's gonna happen, but you have to have enough knowledge to say, like, hey.
There is another way that you didn't think about.
Craig
00:24:43 – 00:25:11
Do so I was playing around with Replit last night for, like, a a landing page, and and I wanted to have it do something else.
And I was like, oh, this is where I would want to, like, fork, and I know that's not the right term, a branch maybe.
Like, I wanna take this is okay.
I wanna take and go do something over here that I can maybe kill and come back to this thing, and I don't know if it does that.
But do you do you ever, like, take something that it gives you and then feed it back in and say, this is what you just gave me, like, shoot holes in this.
Craig
00:25:11 – 00:25:17
What's what's wrong with this?
How would you improve it?
What other questions do I need to ask for you to give me a better a better response?
I do that all the time.
Aaron
00:25:18 – 00:25:30
And I'll do that against models.
So I'll take I'll take the cloud output and go over to o one and be like, I wrote this.
It's a lie, but I wrote this.
What you know, is there anything that I could do better?
Have I missed any edge cases?
Aaron
00:25:30 – 00:25:41
That's a great question.
Like, hey.
What what are the things that are that I'm not considering in this code?
And, yeah, I'll do the opposite.
I'll take it from o one back to Claude and tell it I wrote it.
Aaron
00:25:41 – 00:26:04
And, yeah, I do that kinda adversarially sometimes.
With Claude, if it does get, like, if it does get kinda, like, wonky, I will copy out the last bit of code that it wrote and just start a new chat and say, like, hey.
I wrote this just back to Claude itself and then see, you know, what it says about, you know, changing it or something like that.
But yeah.
I'll absolutely absolutely do that.
Craig
00:26:05 – 00:26:15
So you you've talked a lot about, like, o one and Claude Sonnet, I guess, 3.5.
How do you decide when to use which?
Aaron
00:26:15 – 00:26:37
Right.
So, o one pro is the is the is the big dog.
Right?
So it's the one that is both expensive and also thinks for, like, several minutes at a time.
And so o one pro is, o one pro is my guy when I need, like, I gotta write two paragraphs of what's on my mind and give it as much like, I'm thinking about doing this.
Aaron
00:26:37 – 00:26:43
Here's the problems.
Here's what I've done already.
Here's why I think it could work.
Here's why I think it couldn't work.
What do you got?
Aaron
00:26:43 – 00:27:04
And then I just let it cook.
I go back to work and I come back, you know, five minutes later and it's got a page and a half or two of ideas.
And so that's really, really helpful when I'm, like, kind of at the beginning and thinking, how do I wanna architect this?
How do I wanna structure this?
Or, as is the case recently, I'm operating in an unfamiliar environment.
Aaron
00:27:04 – 00:27:15
So I'm, like, writing a Vite plugin.
I've never written a Vite plugin in my life.
And so I walk over to o one pro and say, I'm writing a Vite plugin.
I've never done that before.
Here's exactly what I need it to do.
Aaron
00:27:15 – 00:27:27
Here are the considerations.
Can you spike out the first implementation of that?
And it normally does that just fine.
So o one pro is for, like, hey.
Here's what's on my mind.
Aaron
00:27:27 – 00:27:44
Help me figure out where we're going here.
Works amazing for that.
O one mini is for, like, hey.
How do I do a how do I do a regex replacement and on a JavaScript string?
And it's like things for half a second and spits out the thing back to me.
Aaron
00:27:44 – 00:27:57
Yeah.
It's great.
It's great for that.
So that's kind of the o one the o one family.
I use I'm basically using o one mini instead of something like Google or Stack Overflow, specifically for programming questions.
Aaron
00:27:57 – 00:28:17
I'll copy a piece of code, paste it into o one mini and be like, what am I doing wrong here?
And it comes back, like, super duper fast.
And then Claude, I think I it is 3.5.
Claude, that's where I go and I upload a bunch of files and I'm like, you've or I have done, you know, six of these.
Here's the spec for the seventh one.
Aaron
00:28:17 – 00:28:17
Write it.
Craig
00:28:18 – 00:28:18
Or I'll
Aaron
00:28:18 – 00:28:32
go paste in I'll go paste in an entire PHP class that I've written, and I'll say something like, hey.
Here's this class.
I need you to I need you to clean this up.
Like, I'm doing a couple of things twice.
Can you, like, refactor this for me?
Aaron
00:28:32 – 00:28:38
And it does it does, like, a great refactor and spits it out, and I paste it back in and run the tests, and it still works, which means I'm good.
Craig
00:28:39 – 00:28:57
Interesting.
So so just to paraphrase, Claude, give it kind of examples and ask it to fill in the blanks or repeat something with a a new iteration of it.
O one mini, just kinda asking questions like you would use Google or Stack Overflow.
O one Pro.
Mega pro.
Craig
00:28:58 – 00:29:08
Yeah.
So I I, yeah, I pay the the the chump $20 version for both, for for, like, deep thinking.
Strategic architect is the term I would use, the level kind of stuff.
Mhmm.
Cool.
Craig
00:29:08 – 00:29:08
Okay.
Aaron
00:29:08 – 00:29:09
Yeah.
Exactly.
I
Craig
00:29:09 – 00:29:23
mean, to me, that that's, like, the whole point of this conversation.
Right?
It's, like, you as a super experienced developer, how are you using these tools today?
And, like, because I like to opine about, you know, the world is gonna end or not, and we're just gonna have universal basic income and not have to work anymore.
I think both are very possible.
Aaron
00:29:24 – 00:29:24
Mhmm.
Craig
00:29:24 – 00:29:27
But between now and eighteen months from
Aaron
00:29:27 – 00:29:29
now when that happens, we need to know how
Craig
00:29:29 – 00:29:30
to feed our families.
Aaron
00:29:31 – 00:29:44
Yeah.
No.
It is it is a it is, a 100% selfish pursuit.
I want to make sure that my family still has food.
And so, like, maybe we get to maybe we get to Nirvana.
Aaron
00:29:44 – 00:30:00
We get UBI.
Everybody's happy.
We do whatever we want.
But what I what I do know is that, it's gonna be a bumpy road to get there if we ever get there.
And the people that are able to use the tools are going to have more opportunities than the people that are not able to use the tools.
Aaron
00:30:00 – 00:30:20
And so I don't know what that's going to look like, but in times of great transition, there's a great, like, reshuffling, and I want to be shuffled out on top in the end.
And so I'm gonna learn how to use the tools.
I'm gonna get way more effective.
Is it good for the economy or junior developers?
Boy, I don't know, but I have to look out for my family.
Aaron
00:30:20 – 00:30:21
You know?
Craig
00:30:22 – 00:30:38
Yeah.
Yeah.
I I don't I don't know.
I I think junior, to me, like, what I told what I told Jesse, like, our our senior developer on our team is I said I would I would hire a you again, and I would hire a person straight out of boot camp.
Mhmm.
Craig
00:30:38 – 00:31:02
I wouldn't hire a mid level developer anymore.
And that's just, like, from my perspective, knowing what I know from conversations like this, I think a junior person can probably operate at about a mid level, but a mid level can't think like a a senior level developer with Cursor or Claude or whatever.
Yeah.
Okay.
So I wanna play what would Aaron do?
Craig
00:31:02 – 00:31:20
So our last our last episode, I had Jordan Goll on.
He's building Rosie.
It's an automated call answering service for home service businesses.
Reason I wanted to have him on is like he's building in AI like a wrapper, right?
So Aaron is building a wrapper product.
Craig
00:31:21 – 00:31:30
What would you do different than eighteen or twenty four months ago when you're just building a Laravel app
Aaron
00:31:31 – 00:31:31
Mhmm.
Craig
00:31:31 – 00:31:32
Old school style?
Aaron
00:31:34 – 00:31:40
Okay.
So is is the situation I'm building some sort of AI wrapper to offer to the marketplace?
Craig
00:31:40 – 00:31:40
Yep.
Aaron
00:31:42 – 00:31:49
Yeah.
I think let's see.
What would I do differently than two years ago?
Craig
00:31:49 – 00:32:27
Sorry.
Let me let me give a bit of additional context just for folks who didn't listen to the episode because I think I think, like, what made Jordan's episode and his situation so interesting is the way he put it is, like, in the past, you could answer your own phone or have a call service, and now you have a third option, which is AI.
And I think that that's the thing is, like, I don't want just a freaking, like, rapper on ChatGPT to repurpose my podcast episodes.
I want, like, an AI native implementation of a thing that I had to do manually before.
So so that's the and and that may or may not be slight, but but that's what I'm thinking.
Aaron
00:32:27 – 00:32:46
Yeah.
I I think it would it would definitely change, the universe of possibilities for what types of businesses I would maybe go into or create.
Right?
Because it used to be, taking, fuzzy or ill defined user input was very difficult.
Right?
Aaron
00:32:46 – 00:33:11
You had to write a lot of tests, a lot of heuristics, maybe a bunch of regex, maybe a bunch of whatever.
And it's like, at the end of the day, you're still just kinda hoping that it works.
Now you have this, this very, like, intelligent thing where you can receive input from a normie and turn it into some sort of structured thing upon which your program can then, operate.
Right?
And so I have to imagine Jordan's doing the same thing.
Aaron
00:33:11 – 00:33:29
So he, like, gets all the, you know, the voice from the end user.
The AI turns it into something a little bit structured.
They query the business's calendar and then turn it back into voice for the end user.
That just wasn't possible, you know, two years ago.
You're pushing two to hear of, like, when our calendar is open.
Aaron
00:33:29 – 00:33:51
Right?
And it's like Yeah.
We don't have to do that anymore.
So it would definitely change the shape of, like, what I want to offer.
It would also be, I think it would also be prudent to have a, to have a buffer between, like, your code and the models because the models are getting better every single day.
Aaron
00:33:51 – 00:33:59
Like, DeepSeek came out of nowhere.
Like, hey.
It's it's, you know, nine in the morning.
Oh, the world is different.
It's 09:01.
Aaron
00:33:59 – 00:34:24
And it's like, we didn't see that coming.
And so I think, having some sort of router or, translator, any intermediate later layer where you can swap out new models is going to be very, very valuable.
I would hate to be in the model game.
I know they're raising, you know, $500,000,000,000 or whatever, but every day, some kid is trying to eat your lunch.
Where I would wanna play is where Jordan is playing is like, alright.
Aaron
00:34:24 – 00:34:39
I'm gonna harness these, like, these, you know, megaton companies beating each other to death, and I'm just gonna yeah.
Great.
Y'all have fun.
Good luck.
I'm just gonna use your latest model with zero loyalty to whoever made the latest model.
Aaron
00:34:39 – 00:35:18
And so I think in in, you know, in the past, maybe eight to twelve months, the tides have turned on, GPT wrappers.
It's like, oh, suddenly they're the smart ones.
Because then, you know, it's like and there's no loyalty whatsoever to any models.
A new model comes out and everybody cancels Claude, goes back to o one and vice versa every other week.
And so that's probably like I it would it would be interesting to me, if I were, like, trying to start a business, the heuristic that I would try to do is, like, what's super messy and super unstructured and takes a ton of time inside of real businesses.
Aaron
00:35:19 – 00:35:31
I probably wouldn't go consumer because that just feels like a very hard game to play.
But, like, so what's messy?
What's unstructured inside of a business?
What is painful?
What is, valuable?
Aaron
00:35:31 – 00:35:52
And what I would do is I would put some sort of interface to that, put an LLM in it, have my code behind that, and then I would, like, sell that as, maybe not even AI enabled.
I think that or, like, a AI solution.
I think that is another thing you could do is just say, like, I will solve the problem for you.
It doesn't matter how I do it.
Am I using AI?
Aaron
00:35:52 – 00:36:07
Maybe.
Who knows?
We'll find out.
But I think, you could go to a lot of businesses and say, cast upon me all of your unstructured, horrible, PDF bound data, and I will give you something good out on the other side.
Craig
00:36:07 – 00:36:07
Yeah.
I
Aaron
00:36:07 – 00:36:15
think that's totally viable.
Again, the problem is, like, what are the niches?
How do you get in there?
Who do you know?
Are you a plumber?
Aaron
00:36:15 – 00:36:35
Are you an HVAC person?
How do you get into those industries to find those meaty problems?
That's the same problem that everybody's had since, you know, the dawn of software is like, how do I get into these cordoned off industries to find what is painful and fix it?
Because that that exists everywhere.
It's just like, who do you know that knows somebody?
Aaron
00:36:35 – 00:36:36
You know?
Craig
00:36:36 – 00:36:44
Yeah.
Yeah.
I I agree.
I think, like, the the term AI, I I think has already kind of, like, had a turn.
Right?
Craig
00:36:44 – 00:36:57
Like, a year ago, if you look at, like, YC or TinySeed or whatever, like, everything was AI.
I think now, like, the the name AI probably showing up less in a lot of products because it's like, duh, you know, like, of course, like
Aaron
00:36:57 – 00:36:57
Yeah.
If you're
Craig
00:36:57 – 00:37:14
a new product, you have AI.
Even even we have, like, quite a bit of AI Yep.
Going on and and are planning for a lot more.
So in reef on on the ReefForge site, Brian Balfour had a really interesting piece about how users' expectations of software are changing a lot.
Mhmm.
Craig
00:37:14 – 00:37:24
I'll include a link in in the description.
But the short version is, we, as software providers, we're kind of, like, helping people take the next step a lot of times.
Aaron
00:37:25 – 00:37:25
Mhmm.
You
Craig
00:37:25 – 00:37:38
think about Canva.
Right?
Now it's like, just do it for me.
Right?
Like, there there's definitely, like, a whole set of a whole set of products that are like, don't help me create, like, an image, just, like, create the image for me.
Craig
00:37:38 – 00:37:57
Right.
And yeah.
I I but I agree with your your, like, analysis of, like, unstructured data.
I I think that, like, still, if I was going into it, I still would lean on the thing that I think it does really well, which is, like, take data, synthesize it, and give me something from that.
Yeah.
Craig
00:37:57 – 00:38:21
I I think, like, you know, if you're if you're in our world, like, that's, like, AI show notes and titles Yeah.
And all, like, understanding previous, you know, episodes and and, like, coming up with title suggestions or topics or something like that, and we're building all that stuff.
But what I wouldn't do is, like, automated video editing or something like that.
Like, it's just, like, two two it's that gap.
Right?
Craig
00:38:21 – 00:38:28
It's that gap from, like, go build me, you know, SavvyCal.
Like, I just don't I just don't see it there.
Aaron
00:38:29 – 00:38:57
No.
What I would not, on on that note, what I would not try to do is count on the AI to have good taste.
And so, like, automated video editing, I think is a terrible idea because in in the same way that, like, creating music requires good taste, creating video requires good taste.
And so as the human, I'm going to lean into my human ability to have good taste.
And not every human has good taste, but it's something you can learn.
Aaron
00:38:57 – 00:39:17
But I don't think any AI has good taste because it's such like a, it's such like an, undefinable human thing to be like, this is good and that is bad.
Can I explain it?
No.
Not really.
I can try, but, like, I just know what is good and what is bad.
Aaron
00:39:17 – 00:39:32
And so, like, do I want, AI enabled video editing that does certain things?
Sure.
But I don't wanna give, like, a raw video to the AI and be like, edit this because I don't think it's gonna it's gonna turn out right.
Craig
00:39:32 – 00:39:32
Yeah.
Aaron
00:39:32 – 00:40:02
I am fine if a bunch of other people do that because it makes it easier for me to stand out.
Right?
And I think that's another thing that, like, is going to be pretty important is where is there no human touch and where can you be a human?
And I think YouTube is a great like, YouTube is a great avenue for that because we're seeing lots of, like, faceless channels pop up that that's basically, like, either stock photos or AI generated photos or AI clips stitched together with an AI voice over.
And it's like, yeah.
Aaron
00:40:02 – 00:40:31
Maybe there's a gold rush there, and it's kinda like drop shipping and, you know, the top 1% can make a million dollars and then everybody else makes nothing.
But I regardless of however that, you know, grift is working, I think humans are going to want to connect with humans.
And behind the written text, you don't know who you're connecting with.
But behind video, you can still connect with a human.
And, yes, there are, like, you know, there are, AI avatars of real humans.
Aaron
00:40:31 – 00:40:50
I still don't think those are very good.
But I still think video is one of the last video and actually podcast because it's such a longitudinal format is one those are one of the two of the last, like, I'm going to leverage my humanness for, like, for, an unfair advantage here.
Craig
00:40:50 – 00:41:04
Yeah.
Yeah.
I told the team this week in our meeting, I was like, I it's the most excited I've been about podcasting as a business model in a very long time.
Because for a while, I was kinda like, gosh.
I don't know.
Craig
00:41:04 – 00:41:18
Like, post COVID and all this, and the number of podcasts are going down and stuff.
And now you look at it and you're like, I mean, it's really it's hard for a lot of businesses right now.
Right?
Like, blogging and SEO as a lot of us have tried to do for a very long time.
Okay.
Craig
00:41:18 – 00:41:24
I won't say dead.
Not doing good.
Right?
Really, really, really hard.
Right?
Craig
00:41:24 – 00:41:39
Cold email, LinkedIn, all this kind of stuff, all because of of the AI enablement.
I think just the volume of shit has made the signal noise ratio such that nobody cares.
Right?
And nobody use it.
You mentioned, like, you don't go to Google anymore.
Craig
00:41:39 – 00:41:48
Like, my twelve year old has ChatGPT on his phone, and, like, it's all he uses.
You know?
He's like, why would I go to Safari?
Like
Aaron
00:41:48 – 00:42:14
I think I think, distribution is becoming even more important somehow.
Like, back in back in 2010, you could, you know, at that time, spin up a rails app, build a little nothing burger, and end up with, you know, 50 k MRR.
Those days are just gone.
Right?
Because almost everyone, almost every developer can spin up some app and have something out there.
Aaron
00:42:14 – 00:42:33
And there are 50 competitors for every single thing, and nobody cares.
Nobody cares about your little app.
Nobody cares about anything.
And so having having distribution is becoming ever more important.
And I think it's one of those things that, you might not know exactly what the outcome is, but you know it's gonna be positive.
Aaron
00:42:33 – 00:42:50
Right?
So you can, like, build up distribution without a definite goal in mind.
And as you're working towards building up distribution, you can find what it is you're going to distribute.
Right?
And that is more important now than ever because there are solutions for everything.
Aaron
00:42:50 – 00:43:20
And but if you have people that know, like, and trust you, you can find the thing along the way.
You can try a few different things and you have, like, distribution.
That's why I think, frankly, it's more important than ever to have, a podcast, an email list, a Twitter following, a YouTube following, all of that stuff because it is getting harder to reach people.
And I think those are the last, like, as organic or as inorganic as they are.
Those are the last like organic ways to reach people is, person to person.
Craig
00:43:20 – 00:43:29
Yeah.
Yeah.
Last question.
Who do you look to for information about how to use AI better at work?
Aaron
00:43:30 – 00:43:45
Yeah.
That's a tough question.
There are so many, grifters and charlatans.
There are a few people there are a few people I follow on Twitter that I think are are doing it well.
S w y x, Swyx, is one.
Aaron
00:43:45 – 00:44:03
He's he's a good guy to follow.
And then I think there are other there are other reasonablists that I follow for, like, just generally good takes.
Like, Andreas Kling is building a browser called Lady Bird.
I feel like he always has good takes.
Thorsten Ball wrote a few books on programming.
Aaron
00:44:03 – 00:44:40
I feel like he's a reasonable list.
I feel like Taylor Atwell of Larabelle hasn't talked about AI hardly at all.
But I've always found him to be kinda right down the middle on, like, trends and, tradition, and he just kinda cuts he kinda cuts right down the middle on that.
And so I try to follow, I don't think I follow anyone just for AI takes.
I try to follow, like, well rounded, pragmatic, practical senior engineers and use them kind of as my bellwether for, like, what are the reasonablists thinking these days about AI?
Aaron
00:44:41 – 00:45:05
And every, like, breathless, warp is gonna change your life.
I just mute I just mute those people, frankly, because I don't want 15 tips to use, you know, Bolt to, like, start a new SaaS every day.
So I kinda just, like, filter all of my advice through, people that I deem as as reasonable.
And there are many there are many reasonable lists, and those are the those are the people that I tend to follow.
Craig
00:45:06 – 00:45:09
Yeah.
That's cool.
That's cool.
I mean, that that's the goal for this podcast.
Right?
Craig
00:45:09 – 00:45:32
Like, this podcast has had several iterations, but but really yeah, we don't I don't wanna be like and we might even rebrand the podcast at some point, which will be crazy.
I don't wanna I don't wanna be like a AI influencer, but I just wanna be a vehicle for folks who actually know what they're talking about to share a few things to help everybody else out.
Like and I think that's I think that's good.
I think it's hard.
Right?
Craig
00:45:32 – 00:45:50
Because you have to bring people on who are genuinely helpful and not kind of pushing their own agenda.
But sure enough, like, we all need as much of this as we can get.
Oh, yeah.
To filter the crap and take the good stuff and implement it to, like, make ourselves better.
So
Aaron
00:45:50 – 00:46:06
Yep.
Yeah.
I think on that note, something else that is true about the people that I tend to trust on these topics is they are working developers.
They are not they're not, like, primarily, YouTubers or influencers or anything.
They are working developers.
Aaron
00:46:06 – 00:46:38
Like, they are building towards an end goal, and usually that end goal is some sort of capitalistic pursuit.
Those are the people that I think, like, if something is able to filter all the way down to them and they're like, my goal is to build and ship product, turns out it's easier to do with this thing, then I'm like, great.
Tell me about that thing.
But if if your if your whole business is like, you know, sign up for my newsletter and buy this course on AI.
I'm like, blah.
Aaron
00:46:38 – 00:46:52
Yeah.
I don't know if I can trust you on that.
And so that that's what I like, that those are the people that I try to follow is somebody who's building a a tool, a product, a a company, and they're like, guys, this is I'm just going faster.
I'm like, great.
I trust you.
Craig
00:46:53 – 00:46:59
Sorry.
Super last question.
Does it matter if that's a kinda indie maker or they work in a large organization?
Aaron
00:47:00 – 00:47:17
Not to me.
It doesn't.
I am I, you know, I have a business partner, but we're we're, you know, solo.
It doesn't matter to me if they are indie or work in a big corporation.
I think if you work in a big corporation, you probably wanna listen to people that also work in a big corporation.
Aaron
00:47:17 – 00:47:23
But in terms of, like, discrete productivity improvements, I don't care I don't care where you work.
Yeah.
Craig
00:47:23 – 00:47:35
Yeah.
Yeah.
I asked just because I think the adoption and perspective on adoption between us and somebody that works at, you know, freaking Pfizer or whatever is Yep.
Is quite different.
Yeah.
Craig
00:47:36 – 00:47:41
And and, like, their context maybe should be too.
I I don't know.
Yeah.
Yeah.
Still still very much trying to figure it out.
Craig
00:47:41 – 00:47:51
Yeah.
I'll link to in the show notes.
I don't have them all in front of me.
A couple of folks that I listen to as well, mostly on, like, just business stuff.
And, like, they're all YouTube channels, as you said.
Craig
00:47:51 – 00:47:55
So, yeah, it's a good good place to be bullish.
Cool.
I have one I
Aaron
00:47:55 – 00:48:26
have one more charge Yeah.
That I think is incredibly important in the coming age.
I have found that tests are two to 10 times as valuable as they were before.
Because before, testing was to, like, offload stuff out of my head.
Like, I can kinda keep the whole program in my head up to a certain point, but, you know, it's better if I, like, concrete or I, like, formalize some of these, like, assumptions I'm making or things I'm holding in my head into tests, right?
Aaron
00:48:26 – 00:48:48
And then I feel warm and fuzzy when I run the tests, everything passes and I, you know, I do something and it breaks something somewhere else.
I'm like, oh, thank goodness I have a test, right?
I think it's, like, 10 times more important now because the AI is nondeterministic and it's gonna do something different every single time.
Right?
And in terms of, like, let's go back to that example where I'm writing all of these, these AST parsers, these conformers.
Aaron
00:48:48 – 00:49:36
Sometimes I didn't write a single character in the actual implementation code.
And so having a test or having, in fact, you know, 30 tests that all test this single file, this one class, I've got 30 different test cases, then I feel super confident when it's like, oh, I need to add one thing to that, to that, you know, transformer.
I'm just gonna copy the whole implementation and go to cloud and say, I need you to augment it such that it does this.
And I can take that back and know that the 30 test cases that were passing that I'm super relying on, I can know that those still work.
And I wouldn't be able to know that because I don't have enough context to look at the old code and the new code and really see what the behavior difference is because I didn't write the old code.
Craig
00:49:36 – 00:49:36
Yeah.
Aaron
00:49:36 – 00:49:49
I didn't write the old code.
I didn't write the new code.
And so I'm really relying on the test that I wrote and then also the test that I had Claude write for me.
Because what I wanna do is I wanna say, hey hey, Claude.
Here's here is, here's a class.
Aaron
00:49:50 – 00:50:07
Write 30 tests for it.
And then I can look at all the tests and say, yes.
Those are correct.
Now forevermore, I know that that is going to stay exactly the way that it should, And I can churn that class 50 different times, and as long as the test pass, I still feel pretty good.
Craig
00:50:07 – 00:50:07
Yeah.
And
Aaron
00:50:07 – 00:50:33
so I I do think that, like, if the if the machine is gonna write code for you, you better have some way to prove that it's still doing everything it used to be doing because, boy, it will just forget about stuff you told it to do.
It'll just forget that, like, hey.
When you encounter this, make sure you do this, and then you ask it to do something else.
It totally forgets the first thing you asked it to do.
And so having those tests like that is how you ensure that your software works in the long run.
Craig
00:50:34 – 00:50:53
I'm glad you mentioned that I wanted to get to QA, but but wanted to respect our time.
And, yeah, I I like, it's it's a thing that we think about a lot.
Right?
It's like, how can we how can we extend, like, our AI usage to QA?
I think it's harder because, like, the the Kelly, the gal on our team that does, like, most of our QA, like, she's thinking, like, how can I break this?
Craig
00:50:53 – 00:51:11
How can I have this edge case?
How can I and I think that tying maybe maybe it's that architect level thing?
Again, it's like, as we're thinking of these tests and the the coverage of these tests, it needs to be somebody with that mindset, which isn't always the developer sometimes if you have QA on your team.
Yeah.
Aaron
00:51:11 – 00:51:32
No.
No.
It's it's not.
And I think, before so, like, if you're if you have a code an existing code base and you're about to, like, embark on letting Claude rewrite some stuff for you, I think the first thing you do is every time you're about to, like, rewrite a section or a chunk or a class or a method or whatever, you take it to cloud and say, hey.
Can you write 10 tests for this method as it exists right now?
Craig
00:51:32 – 00:51:33
Yeah.
Aaron
00:51:33 – 00:51:47
And then you you commit those tests to your code code base, make sure that they pass, and then you're free.
Have fun.
Like, don't muck it all up and make sure the tests still work, and you're kinda ensured that everything is still gonna work the way that it used to.
Craig
00:51:48 – 00:51:50
Tab, tab, tab, chip.
Right?
Aaron
00:51:50 – 00:51:52
Tab, tab, tab, test, chip.
Yes.
Exactly.
Craig
00:51:54 – 00:52:00
Love it.
Awesome, buddy.
Best place for folks to reach out and and connect from here?
Twitter?
Aaron
00:52:01 – 00:52:04
Probably Twitter, Aaron d, d as in Daniel, Aaron d Francis.
Craig
00:52:05 – 00:52:09
Awesome.
Awesome.
I appreciate it, man.
Thank you very much.
Yep.
Craig
00:52:09 – 00:52:14
Thanks for having me.