Aaron Francis

Aaron can barely move, Colleen is working with billions of records. Sean is on vacation! Lucky Sean.

Learn more →

Transcript

Aaron

00:00:00 – 00:00:07

So this is actually the first time I've been back at my desk since the last time we recorded. I've been Oh,

Colleen

00:00:07 – 00:00:08

my gosh. Are you serious?

Aaron

00:00:09 – 00:00:27

I'm serious. I've been so we recorded last Friday and then on Saturday and Sunday I was doing some stuff in the shed. And then Sunday afternoon is when my back just locked up. And I've been in bed I've been in bed the entire week. Literally, the

Colleen

00:00:27 – 00:00:28

entire week.

Aaron

00:00:28 – 00:00:43

This is the first time I've been back at my desk. I know. It sucks. So I got I got an MRI. So I've seen an orthopedic because I was the first an orthopedist because I was the first person I could get in to see on Tuesday morning.

Aaron

00:00:45 – 00:01:03

And he was like, yeah. You need to get an MRI and then go talk to a a spine doctor. So I got an MRI, a big old bulging disc right into right into my nerve. I saw 2 spine doctors. I I guess they're called, spine surgeons, but that makes me nervous.

Aaron

00:01:05 – 00:01:19

And they both recommended getting an epidural injection before doing any kind of surgery, which makes me feel good because that seems way more conservative than, like, taking out the disc or something. So Heather: And

Colleen

00:01:19 – 00:01:21

they shave the disc down. Isn't that what they do in

Aaron

00:01:21 – 00:01:21

the surgery?

Colleen

00:01:21 – 00:01:23

They shave it. Yeah. Derek Mhmm.

Aaron

00:01:25 – 00:01:46

And both of them said like, you know, this steroid injection may fix it altogether. It may not do a thing, though. So it's kinda I'm getting it at 3 o'clock today, so we'll find out tonight. Well, they said it could take a day or 2. So I'm really hoping that it just fixes it altogether, and I don't have to have surgery.

Aaron

00:01:47 – 00:01:50

I'm 32. I'm not old enough to have back surgery. I have

Colleen

00:01:50 – 00:01:56

to ask, like, is this a chronic problem? Because my first thought when you told me was you're kinda young to have this kind of a problem.

Aaron

00:01:57 – 00:02:14

No. It's not. I mean, I've I've been in pain for the I've been in a different pain for, like, 18 months. My neck and my shoulder have been in pain for a year and a half, but I've been in I've had this leg pain for, like, 2 months. And I've been, like, stretching and getting massages.

Aaron

00:02:15 – 00:02:23

And then finally on Sunday, it was like, nope. You can't even straighten your leg anymore. You can't walk. It's like, oh my gosh. Awful.

Aaron

00:02:23 – 00:02:45

So, yeah, the one the one doctor that I really like said, you know, this this can just happen. Like, you can just be doing something, and it just like, the disc spurts out, and this just happens. Like, I wouldn't I wouldn't be depressed about it. I was like, oh, that's good. Because I feel pretty bummed about being 32 and looking down at back surgery at this point.

Aaron

00:02:45 – 00:03:07

So Yeah. But I am just hoping that the steroid shot calms everything down. I got an inversion table off of Amazon for a $130, which is insane. So I can, like, flip over and, like, stretch my back out. So we'll see.

Aaron

00:03:07 – 00:03:09

But, yeah, been kind of a kind of a rough week.

Colleen

00:03:09 – 00:03:10

Oh, that's awful.

Aaron

00:03:11 – 00:03:32

And the, like, the kicker is I've been having to lay on my side because it's the only way that's really comfortable. And so I can't, like, sit in bed with my laptop and, like, do stuff. So I've just I've just been laying there watching TV. I've watched the entire season of, Alone on Netflix. Have you heard of that show?

Colleen

00:03:32 – 00:03:33

Mhmm. I haven't heard of it.

Aaron

00:03:33 – 00:03:45

It's crazy. They drop people into the wilderness just to live. And then Cocaine Cowboys, I watched all of that. But it's kinda it's been kinda depressing, honestly, because I, like, can't do anything. I wanna, like, be working on stuff.

Aaron

00:03:46 – 00:03:48

I can't do a freaking thing. So

Colleen

00:03:48 – 00:03:48

Wow.

Aaron

00:03:49 – 00:03:54

Yeah. So anyway, that's how I'm doing.

Colleen

00:03:55 – 00:03:56

Yeah. So mostly terrible,

Aaron

00:03:57 – 00:04:12

but Mostly terrible. Mostly terrible, but with a glimmer of hope today at 3 o'clock. Although I am fasting all day, so that makes things crappy too. No no food or drink past 8 AM. So I'm both thirsty and hungry.

Colleen

00:04:12 – 00:04:14

Hopefully tomorrow is better.

Aaron

00:04:14 – 00:04:25

Hopefully. Welcome to the show, people. This is a good one. Okay. So tell me, you talked to Andrew today.

Aaron

00:04:26 – 00:04:27

What are y'all doing?

Colleen

00:04:27 – 00:04:40

We yeah. We're doing great. Excited to kind of get ramped up with this client and, look at some of the issues they're having. I think this is such a great place to start because they're gonna have such a huge dataset.

Aaron

00:04:41 – 00:04:41

Mhmm.

Colleen

00:04:41 – 00:04:46

It's gonna be really fun and really challenging. I think to think

Aaron

00:04:46 – 00:04:48

We're talking like billions. Right?

Colleen

00:04:48 – 00:04:49

Billions of records. Right?

Aaron

00:04:49 – 00:04:50

Billions with a b. Yeah. Okay.

Colleen

00:04:50 – 00:05:17

Yeah. Billions with a b records. And I think what what it's gonna look like, what we talked about is that the data's gonna come in, you know, kind of like a JSON dump. And then we're gonna deserialize that and put it in this events table. And then the thing is when we query when we have these nested queries, the there's concern that the joint the way we have it set up right now to just automatically do all these nested queries is not gonna work.

Colleen

00:05:17 – 00:05:35

We're just gonna lock up our database. So we talked about kind of doing these nested queries in a way that they just return the IDs, which I don't think is gonna be an issue, but I've gotta look at it. So it's a little different than the way we have it set up now. It'll actually be breaking out individual filters. Right?

Colleen

00:05:35 – 00:05:41

So the different models will have different filters that all come from this singular query. But the different filters

Aaron

00:05:41 – 00:05:55

Wait. Flesh flesh that out for me. So different models, I e a customer and a product, those are those will have different filters. Right? Show me customers that 123 and products that a b c.

Aaron

00:05:55 – 00:05:56

Is that what you're saying?

Colleen

00:05:56 – 00:06:09

Right. So when you look at the way I think we're gonna be querying this data is you're saying send an email to a contact where the contact has purchased a product, and the product was green.

Aaron

00:06:10 – 00:06:11

K. Got

Colleen

00:06:11 – 00:06:15

it. But because of the nature of the size of these tables

Aaron

00:06:15 – 00:06:16

Mhmm.

Colleen

00:06:16 – 00:06:37

We're discussing and returning it in such a way that you get that we receive that query and you go down to your product. The first thing you're gonna do is query for products that are green, return the IDs, and then feed the IDs into the one level higher query in order to make it a little more performant. So that's what we're talking about, like, playing around with right now.

Aaron

00:06:37 – 00:06:50

Okay. This is this is I love this. This is I'm so glad you're here because this is the stuff I wanted to talk about. Okay. So tell me and I don't know how far y'all have gotten.

Aaron

00:06:50 – 00:06:50

So Like,

Colleen

00:06:50 – 00:06:54

literally, this was a 20 minute conversation. We haven't looked at it yet. So this is

Aaron

00:06:54 – 00:07:13

So is the thought to pull the IDs into memory, like, pull them into Ruby and then bind them into a different query in memory? Or is the thought to push that all down into Postgres somehow? Because they're using Postgres. Right? Or So time stream or whatever it's called.

Colleen

00:07:13 – 00:07:15

Yeah. So the thought right now is to do it in Ruby.

Aaron

00:07:16 – 00:07:18

Okay. So the

Colleen

00:07:18 – 00:07:34

the, I was gonna say we are talking now about Timescale DB, not being, like, our sole source of truth. I think we had talked about this a little bit. And so that data is gonna be dumped into we're gonna go with the model domain structure. Right? So that data is gonna be dumped into the model tables.

Colleen

00:07:35 – 00:07:44

And so, yeah, that would be so you would basically go into your product filter, pull out the IDs. That would be in Ruby that you're keeping track

Aaron

00:07:44 – 00:08:05

of that. Okay. That is interesting to me. That, to me, sounds less performant to pull it into memory versus doing so I know that there were some discussion. The way that we do it currently is we use independent subqueries.

Aaron

00:08:06 – 00:08:58

Right? So we use our our refine builds up subqueries and then executes everything all at once as a giant statement, and the statement contains multiple subqueries. And so what we do by default, just as background for other people, what we do by default is we push everything down to the database layer. So we don't pull anything into memory, but we do essentially the same thing that you're suggesting where you do basically, like, a where in, and then you pass in, you know, a bunch of IDs. What we're doing is we're doing a where in, and then instead of passing in IDs, we pass in well, we have another SQL statement in there with the thought being that the engine the database engine can optimize whichever route is best to do that.

Aaron

00:08:58 – 00:09:11

And so it can run all of those and then build them all up together. So tell talk to me about the other way. So that as background, that's what we're doing. You know this, but that's what we're doing. And so what's the thought with pulling it into Ruby instead?

Colleen

00:09:12 – 00:09:17

So the concern is that's gonna lock up the database. And I have no idea. I don't know yet. So this is all new. Right?

Colleen

00:09:17 – 00:09:19

We just we just talked about this. Mhmm.

Aaron

00:09:19 – 00:09:19

Colleen

00:09:19 – 00:09:44

spent a short period of time talking about it. The concern is that we're gonna have trouble with that query because there will be some joins in there based on the table relationships. And I haven't looked at this model in all of its relationships, but the concern is is trying to do those joins on those multiple tables where necessary is going to give us some trouble.

Aaron

00:09:45 – 00:10:16

Okay. I think this is gonna be interesting because this is like, obviously, this is the biggest dataset we'll have done this with. And this is actually the first Ruby implementation that we've done, so it's kind of a it's kind of a double whammy. So by default, refine as far as I remember, refine doesn't use any joins whatsoever. We use the sub selects.

Aaron

00:10:16 – 00:10:18

Is that Oh, no. We changed it for Ruby.

Colleen

00:10:18 – 00:10:20

We changed it for Ruby. Right.

Aaron

00:10:20 – 00:10:21

That's right.

Colleen

00:10:21 – 00:10:25

And I'm I'm just coming back into this after being off it for 2 months.

Aaron

00:10:25 – 00:10:27

I know. You gotta load it all into your

Colleen

00:10:27 – 00:10:36

I gotta, like, load it back in my brain, but I am relatively confident there was some relationship where we were like, it's better to you use the joints here.

Aaron

00:10:36 – 00:10:36

Mhmm.

Colleen

00:10:36 – 00:10:48

And so maybe that's part of it too, Aaron. Maybe we look at it. I mean, I think the first step here is I just have to see what do we provide out of the box for this particular filter. Right? Mhmm.

Colleen

00:10:48 – 00:11:06

Because we what we're doing now is we're examining the relationships, and we're building up the query, you know, based on the relationship. So I need to see what we're providing out of the box. Mhmm. And maybe that's something we can further optimize instead of returning the IDs as one thing and then putting those IDs into the next filter.

Aaron

00:11:07 – 00:11:07

Mhmm.

Colleen

00:11:07 – 00:11:10

So Hello, child in

Aaron

00:11:10 – 00:11:16

the background. That's fine. We're we're a very casual show. Hey, buddy.

Colleen

00:11:18 – 00:11:22

Sorry. Okay. Visit from my child.

Aaron

00:11:22 – 00:11:23

Well, that's good. Alright.

Colleen

00:11:23 – 00:11:25

What were we talking about? Right. So You

Aaron

00:11:25 – 00:11:27

were saying pulling the IDs into memory.

Colleen

00:11:27 – 00:11:40

Right. So I think I think the first step here is to look at exactly what we're providing out of the box. Mhmm. And why are you telling me in the Laravel side, you don't use the joins ever anywhere? You In the

Aaron

00:11:40 – 00:11:41

Laravel side, do you wear in?

Colleen

00:11:41 – 00:11:42

We don't

Aaron

00:11:42 – 00:12:30

mhmm. We don't use a join anywhere ever. So part of the part of the thesis there is or the hypothesis there is, on the Laravel side, we don't touch so the developer provides the initial query, right, where they say, you know, it's it's the base query, like employees or act or, users or customers, basically just a wide open query. We don't join anything in because then then that means we're adding, we're adding stuff to the returned dataset. So if we were to if the developer were to pass in a base query of, like, customers

Colleen

00:12:31 – 00:12:31

Mhmm.

Aaron

00:12:31 – 00:13:09

Right, and then we were to join in product purchases, etcetera, all of that, all the way down the line, then the dataset that we return is gonna have all of that data left joined into it. And the, developer hasn't explicitly selected that data, but we have added it in. And so we have, unknowingly changed what the developer's expectation is. Right? So if the developer is saying, I want my base query to be customers, and then we return a query with customers full of products, full of variants, full of colors, full of prices, they're gonna be like, what the hell?

Aaron

00:13:09 – 00:13:35

Why did you give me 8 tables worth of data when all I asked for was customers? Right? And we can't we can't modify the selection either because what if the developer has already modified the selection? Then we have to go in and look at, okay, which columns are they selecting, and how do we, like, how do we add or remove the ones that we're modifying. So we just leave the select alone completely.

Aaron

00:13:35 – 00:13:36

Does that make sense?

Colleen

00:13:37 – 00:13:38

Yep. That makes sense.

Aaron

00:13:38 – 00:13:48

So whatever the developer asks for, we will give them, and we will add in the necessary subqueries to get to the place we need to be.

Colleen

00:13:49 – 00:13:59

Right. Yep. That makes sense. Yeah. I have to go back and look at this implementation to remember, why it's a little bit different on the Ruby side.

Colleen

00:13:59 – 00:14:01

I don't recall off the top of my head.

Aaron

00:14:01 – 00:14:12

I don't either. But I do remember talking to you about why like, needing to change it to left joints for something, but I cannot remember why we did that.

Colleen

00:14:13 – 00:14:20

Well, this might be a good opportunity to have I mean, maybe we surfaced a good issue here. Right? Like, this might be a good opportunity to check that out and The the Yeah.

Aaron

00:14:21 – 00:14:27

Remember. The frustrating thing is, like, how do we, oh, gosh. I can barely move.

Colleen

00:14:27 – 00:14:27

Oh my gosh.

Aaron

00:14:27 – 00:14:34

How do we, how do we test anything against a 1000000000 records without having a 1000000000 records?

Colleen

00:14:34 – 00:14:39

I have no idea. That's gonna be fun. That'll be fun to figure out. I don't know. Like, how do you actually

Aaron

00:14:40 – 00:14:45

Do we have to run do we have to run a seed to get a 1000000000 records on your local machine?

Colleen

00:14:46 – 00:14:50

Oh, my gosh. My Mac won't make it. It'll just Oh. It'll explode.

Aaron

00:14:50 – 00:14:50

I bet.

Colleen

00:14:50 – 00:14:59

It'll just start smoking. There's got I mean, I don't really know enough about tables this big. I'm sure there's a way to do that. Right? There's got Yeah.

Aaron

00:14:59 – 00:15:13

And I don't know if there's a functional difference between a 100,000,000 and a 1000000000, honestly. The biggest tables we have at Resolute are, I think, 50,000,000, maybe 80,000,000, something like that.

Colleen

00:15:13 – 00:15:14

Okay.

Aaron

00:15:15 – 00:15:17

But, no, we don't touch we don't touch a 1,000,000,000.

Colleen

00:15:19 – 00:15:20

Yeah. And it's

Aaron

00:15:20 – 00:15:23

gonna be so dependent on their on their, like, hardware.

Colleen

00:15:25 – 00:15:30

This is, like, a lot of interesting challenges that this is gonna surface. I think this is gonna

Aaron

00:15:30 – 00:15:33

be super fun. Sucked you back sucked you back in.

Colleen

00:15:33 – 00:15:45

Totally is. I totally is. I was like, I could do this 1,000,000,000 records. This is so cool. I'm super pumped to see, like, to, to get into these problems and to see where this takes us.

Colleen

00:15:45 – 00:15:50

And, you know, we'll be rock solid once we figure out everything on this side because this is data.

Aaron

00:15:52 – 00:16:05

Yeah. So their main source of data is gonna be events table. Is that right? And that's gonna hold, like, that's gonna hold, like, clicks and purchases and email opens and all of that stuff.

Colleen

00:16:06 – 00:16:06

Yes.

Aaron

00:16:07 – 00:16:19

And then the side the little bitty stuff on the side is gonna be, like, customers who did something in the events table. Right? So customers is, like, millions, just a few million. And then

Colleen

00:16:19 – 00:16:20

Just a few million.

Aaron

00:16:20 – 00:16:28

The events is the 1,000,000,000, and they're gonna want customers, but customers who did, you know, x y z in the events table.

Colleen

00:16:29 – 00:16:53

Right. That is my understanding. Is so you're looking at something like, the the sample I was given was send an email to a contact where the contact has purchased a product that was read and has not yet been emailed as part of campaign 3. And his name is like, these are crazy queries. Right?

Colleen

00:16:53 – 00:16:57

And his name starts with c. You know what? It's gonna be exciting.

Aaron

00:16:58 – 00:17:09

You know what is so just kinda thrilling to hear about that sentence is if you hold constant the fact that we'll figure out the 1,000,000,000 rows, refine does that. Like we can, we can build that query.

Colleen

00:17:09 – 00:17:11

We can build that right now. It's pretty cool.

Aaron

00:17:11 – 00:17:13

Right? That's wild.

Colleen

00:17:13 – 00:17:17

Yeah. Yeah. It's gonna be pretty spectacular.

Aaron

00:17:18 – 00:17:19

That's really exciting.

Colleen

00:17:20 – 00:17:20

Yes.

Aaron

00:17:21 – 00:17:26

Okay. Well, that's cool. So you and Andrew were just, like, catching up to see where y'all are at right now?

Colleen

00:17:27 – 00:17:53

Yes. Because I'm just getting back into this after a couple it's been a couple months. So I just need to get the lay of the land in terms of what's the status of this database and what do we need. And so we were just reviewing some of that. And so we were brainstorming how what this is gonna look like when these events start coming in and how we're gonna save them because there had been discussions of different databases and using different, you know, engines.

Colleen

00:17:54 – 00:17:55

But I think we're gonna go

Aaron

00:17:55 – 00:17:56

esoteric stuff.

Colleen

00:17:56 – 00:18:13

Yeah. We're gonna go, like, model based, which is what you and I had talked about ages ago, I think. I think that's gonna be great for Refine because that's gonna really enable us use the power that we've built here instead of writing another, you know, DSL on top of it. So I think we're we're all good. It's gonna be fun problems.

Colleen

00:18:13 – 00:18:17

But, yeah, I'm just getting my head back in the space to remember what we're looking at.

Aaron

00:18:18 – 00:18:21

And you start for real for real on the 23rd?

Colleen

00:18:24 – 00:18:28

Yeah. So I'll I'll start working more with them, something like that. Yeah.

Aaron

00:18:28 – 00:18:29

Okay.

Colleen

00:18:29 – 00:18:30

Relatively soon.

Aaron

00:18:31 – 00:18:36

Hopefully, very soon. Okay. Do you wanna talk, contract stuff?

Colleen

00:18:37 – 00:18:38

What contract stuff?

Aaron

00:18:39 – 00:18:40

The stuff that we're waiting on.

Colleen

00:18:41 – 00:18:46

Right. I don't know that we need to talk about it. I think we're good.

Aaron

00:18:46 – 00:18:46

Okay.

Colleen

00:18:46 – 00:18:48

We're just waiting for it to be signed.

Aaron

00:18:48 – 00:18:50

Waiting on signatures, and then I'll

Colleen

00:18:50 – 00:18:57

get it. So, yeah, we're just waiting on that to be signed, and I don't think there's gonna be I haven't heard that there's gonna be any problem. So, hopefully, that'll happen today.

Aaron

00:18:58 – 00:18:58

Colleen

00:18:59 – 00:19:06

And then you saw the thing from the lawyer about the LLC being in Texas and Let's talk about that. My god.

Aaron

00:19:06 – 00:19:23

So in in my in my stupor, I I read a little bit of the attorney's email, which is not a good idea. But, basically, what I gathered was they're basically saying, like, if it's a Texas LLC, which it is, then we should get somebody else to do it. Yes.

Colleen

00:19:23 – 00:19:26

Is that the gist? That's the gist.

Aaron

00:19:26 – 00:19:30

So Okay. That's a bad gist in my opinion. That doesn't seem great.

Colleen

00:19:31 – 00:19:45

No. I mean, it sounded to me my interpretation of that email was we don't know the Texas laws. So if you wanna keep it at Texas LLC, we cannot help you and we cannot represent you. If you wanna disband that and start an LLC in Illinois, we can help you with that.

Aaron

00:19:47 – 00:19:50

Thoughts before I have my thoughts?

Colleen

00:19:51 – 00:19:53

I I have no thoughts.

Aaron

00:19:53 – 00:19:55

I have many thoughts. I have

Colleen

00:19:55 – 00:19:56

I have Tell me your

Aaron

00:19:56 – 00:20:02

thoughts, sir. In fact, I have one thought, which is many. I kinda think we should just start over.

Colleen

00:20:03 – 00:20:05

Yeah. I'm okay. I mean,

Aaron

00:20:07 – 00:20:46

We would need to figure out the accounting ramifications of that, but I I kinda think at this point so Sean and I have no operating agreement. We own this LLC 5050. It's made, you know, x amount of money of which we've paid, you know, point 9 x out to you. I guess less than that, because Sean has been billing hours, and that money has been staying in the company. So and the LLC was formed by me going on to the secretary of state website and paying $300 to file the LLC.

Aaron

00:20:46 – 00:21:23

So I'm not tied. I would like to keep the name. But if we could I think if we could just start over and have the attorney do it, because that would gain us, obviously, the attorney, which we'd need, and the ability to have you, like, written in from the start instead of, like, somehow added on to whatever Sean and I already have. I kinda think we should just disband or kill the Hammerstone Texas entity and start a Hammerstone Illinois entity.

Colleen

00:21:24 – 00:21:26

That certainly seems like the easiest solution. I don't know

Aaron

00:21:26 – 00:21:27

It does.

Colleen

00:21:27 – 00:21:36

To your point, there's nothing about it being incorporated in Texas that makes it extra special. And there's not it doesn't sound like there's any I mean, that seems reasonable to me.

Aaron

00:21:37 – 00:21:58

Yeah. Okay. Well, Sean's on vacation, and since we bothered him so much last week, we shouldn't bother him. But when he gets back, we should talk about that. And I'll ask my accountant, and he can ask his accountant named our accountant named Aaron, if there what the accounting ramifications of that are.

Aaron

00:21:58 – 00:22:07

It'd be great if we could just, like if Hammerstone Illinois could acquire Hammerstone Texas and just close it down.

Colleen

00:22:07 – 00:22:07

Yeah.

Aaron

00:22:09 – 00:22:14

Okay. Yeah. Fun stuff. I freaking hate that stuff, man. I

Colleen

00:22:14 – 00:22:15

hate Me too.

Aaron

00:22:15 – 00:22:25

I hate the attorney stuff. I hate the accounting stuff. I am actually an accountant, and I hate it. I hate I hate it so much. It's just so frustrating.

Colleen

00:22:26 – 00:22:36

Yes. And And so complicated. Doesn't it just feel so unnecessarily complicated? Why does it have to be so hard?

Aaron

00:22:36 – 00:22:39

I don't know. We're not dummies. We could Right.

Colleen

00:22:39 – 00:22:39

We could

Aaron

00:22:39 – 00:22:42

figure this out, but why does it seem so complicated?

Colleen

00:22:43 – 00:22:43

Right. Seriously.

Aaron

00:22:45 – 00:22:48

Oh, goodness. Okay. Anything else on your side?

Colleen

00:22:49 – 00:22:54

No. No. I'm just getting up and running, and I'm good.

Aaron

00:22:54 – 00:22:55

Colleen

00:22:56 – 00:22:59

Let's see. On my side, I've been in bed all week.

Aaron

00:23:02 – 00:23:26

I got an inversion table. These are the highlights of my week. Yeah. Not not super great stuff. We have so it's in it's interesting now because you're a part of like, you're a partner in Hammerstone, and we've I've had got all these other, like, Hammerstone side projects going on.

Aaron

00:23:26 – 00:23:43

So Hammerstone is a side project, and then I have side projects for the side project, obviously, as one does. So Torchlight is one of them, and Torchlight is the syntax highlighting, API. Are you have you seen all of this? Are you familiar with Torchlight?

Colleen

00:23:43 – 00:23:45

I'm not actually really familiar with it.

Aaron

00:23:46 – 00:23:49

Great. Well, you own part of it, so I will explain it to you now.

Colleen

00:23:49 – 00:23:49

Please do.

Aaron

00:23:49 – 00:24:44

So, when I was building the Hammerstone website, I was writing all the docs for refine and everything, and the syntax highlighting options out there were really frustrating and really crappy. Prism JS, Highlight JS, there's another one. And I tried them all, and each one missed something in a novel and frustrating way, and so everything ended up, like, looking slightly wrong. And so I started writing this, like I found I found this open source library called Sheeky that does, like, basically, Versus Code compatible, syntax highlighting, but it's really frustrating to use because it's it's not client side. It it's run it's a node process, so you have to install node and do the whole thing.

Aaron

00:24:46 – 00:24:53

And their renderer kinda sucks. Like, it doesn't have line numbers. It doesn't have git diffing. It doesn't have a bunch of stuff. Right?

Aaron

00:24:54 – 00:25:26

So I started writing this solution for the Hammerstone website, that includes line numbers and focusing so you can, like, blur out parts of the code. You can collapse parts of the code. You can add line numbers, do all sorts of stuff. And so as I was writing that, I spun that out into a stand alone service called Torchlight. So it's now an it's now an API with a couple of different clients, one being Laravel, and then let's see.

Aaron

00:25:26 – 00:25:49

There's a there's a markdown client for PHP. There's one for there are a couple for static site generators in PHP, so we have some specific static site generators. Jigsaw is one of them, and then statamic is a a CMS that's not quite the same, but a little bit. And so it's got all these clients. I've launched it.

Aaron

00:25:49 – 00:26:11

I've got I've got we've got, like, 200 people on the waiting list. We've got several several people using it in production. A few people have signed up for paid plans. The Fathom analytics team is using it for their website, for their docs website. Laravel News is using it for their website.

Aaron

00:26:12 – 00:26:35

So we've got a lot, like, we've got a lot of people using it, and it's got a lot of traction, at least, you know, from my perspective. And Andrew wants to use it in bullet train. So that was the thing I messaged I messaged y'all about last week is we were gonna have one of our chat friends, Barry. Have you ever talked

Colleen

00:26:35 – 00:26:37

to Barry in the chat? Yeah. He was at retreat to your friend.

Aaron

00:26:37 – 00:27:11

That's right. We're gonna have Barry do it, but Barry got, slammed with work. And so now we're gonna have somebody else, write us a Ruby client, and it's gonna be, Andrew's gonna pull it into the bullet train docks, so Torchlight will power those docks as well. So kind of the thought with all of that is it's like a it's basically like I don't think it's gonna make that much money. I mean, the business plan is $14 a month, so there's not there's not really a lot of room to get rich on it.

Aaron

00:27:13 – 00:27:36

But I think there's a lot of room to get, 1, to get backlinks, because if if you use the, you know, the open source version, you have to link back to us, which is gonna be great. And 2, to get, like, to get mind share. So I've gotten a lot of people who have messaged me and been like, this is fantastic. I would love to use this. Highlighting sucks.

Aaron

00:27:36 – 00:28:19

I wanna put it on my website. And people just start, like, tweeting about how much they, you know, they install it, they put it on our website, and they love it. And I think this has been 10 times, maybe a 100 times as valuable as trying to write blog posts about, you know, hey, we're Hammerstone. Please notice us. But, yeah, this week has been this week has been good because a couple more people have picked it up and started tweeting about it, particularly the Fathom Analytics team when they, like, when they bought when they signed up and started tweeting about it, he DM'd me and was like, dude, I didn't understand I didn't understand before, like, the value of this.

Aaron

00:28:19 – 00:28:27

And now this is, like, this is extremely valuable and it's worth every penny that I pay, and you've done a fantastic job. Was like, oh my gosh. Nice.

Colleen

00:28:27 – 00:28:28

That's a good year, Aaron.

Aaron

00:28:28 – 00:28:32

That makes me feel so good. Like, it's so encouraging to to hear that.

Colleen

00:28:33 – 00:28:33

Yeah.

Aaron

00:28:34 – 00:28:55

So we're getting close to just opening it up, opening you know, right now it's behind it's behind a form, so you go there and you sign up for the early access list. But I think Okay. We're getting close to just, like, opening it wide up and anybody can sign up and anybody can use it. So it's been a lot of fun. Hopefully, we can get a Ruby client written.

Aaron

00:28:55 – 00:29:18

Andrew can start using it. We can start getting some adoption in the Ruby, ecosystem, the Rails world. But this is a nice little, like, a nice little beachhead to get into people's, you know, to get into people's minds and kinda get our stuff out there. So that's what Beachlight is, and you Sweet. You own part of it.

Aaron

00:29:18 – 00:29:25

Sweet. So if you ever see it, you gotta be a hype person because now this is this is part of your this is part of your portfolio.

Colleen

00:29:26 – 00:29:28

Nice. Cool. That's great.

Aaron

00:29:28 – 00:29:39

Yeah. So that I've been working a little bit on that recently. Good lord. There's no comfortable position. Position.

Aaron

00:29:39 – 00:29:50

And I think that's about it. Oh, there's one other thing. Have you been do you know what Sidecar is? It's a Laravel package that I've written.

Colleen

00:29:50 – 00:29:51

I feel like I've heard of it, but I don't know

Aaron

00:29:52 – 00:29:52

So that

Colleen

00:29:52 – 00:29:53

it's either.

Aaron

00:29:53 – 00:30:26

This is and I'll be curious, like I think we've talked about this a little bit. I'll be curious how this works long term. Like, in terms of is this an is this a good strategy that we should pursue in the Ruby world as well? Like, should we should we write open source packages for Ruby or Rails as a way to gain mindshare, going forward? So that'll be something that I think will be interesting.

Aaron

00:30:26 – 00:31:03

But what Sidecar is is it allows you to deploy AWS Lambda functions straight from your Laravel application. So you can deploy and execute Lambda functions from your Laravel app without having to go through, like, the AWS UI or API or anything. And then you don't have to set up API gateway. You don't have to set up any of that stuff. You just directly execute it straight from your Laravel application.

Aaron

00:31:03 – 00:31:28

And so that has been another one. I actually got to go on an, a podcast recently and talk about that one. And people have been contributing, like, pull requests to that one, which is a first for me. I've never written anything where someone else has contributed to it, which is kinda cool. So that one this week, somebody added a PR to support container images or something.

Colleen

00:31:29 – 00:31:38

So when you say from your Laravel application, like, how much abstracting do you do from from do you, like do they have to go in and set up a SAM configuration?

Aaron

00:31:39 – 00:32:01

Yeah. So what they do so when you when you install it, there's a configure command. So you just run artisan configure. Artisan is our little is the name of our CLI in in Laravel. So you run artisan configure, and what it does is it directs you to the AWS IAM portal.

Aaron

00:32:01 – 00:32:24

So it gives you a link in in your console, it gives you a link. Here's you know, go to the IAM portal, make a, set of admin access keys, and it walks you through how to do that. So, like, full privileges to everything. Give me those keys. So you paste those keys into the terminal, and then it walks you through step by step what it's doing.

Aaron

00:32:24 – 00:32:55

And so it creates, an execution role, it creates a deployment bucket, it creates a policy, and it attaches it all to the right, I don't know what you would call the right objects or instances or whatever. And then at the end, it gives you the option to self destruct those admin keys. So you can use keys to delete keys, and so it gives you the option to say, okay. We don't need you know, side car doesn't need these admin keys anymore. Do you wanna destroy them?

Aaron

00:32:55 – 00:33:24

And if you say yes, those admin keys are gone, and you're just left with your your sidecar access keys. So in terms of, abstraction, then all you do is you say, alright. Here's my Lambda function. The package is, you know, these 10 files in this folder of node modules or whatever, and then the handler is, you know, image dot create or whatever. And then on you just call it from PHP.

Aaron

00:33:24 – 00:33:34

You just say, you know, image you know, create OG image, execute, and it just creates an like, it goes out to Lambda, creates the image, and comes back to PHP.

Colleen

00:33:36 – 00:33:44

Cool. Yeah. We have something like that in Ruby. It was actually made by, like, my very first mentor, which is kinda cool. But, yeah, we

Aaron

00:33:44 – 00:33:44

What's it called?

Colleen

00:33:44 – 00:33:46

Yeah. Lam Lambie?

Aaron

00:33:47 – 00:33:49

I I've heard of Lamby. Is that Yeah. It's custom ink.

Colleen

00:33:50 – 00:33:57

Really cool. It's custom ink. So Ken Collins, who is, like Yeah. Literally the first rails engineer I ever met, Mhmm. Great guy.

Colleen

00:33:57 – 00:34:06

So he wrote it. And so it's really great. I mean, so you're talking about deploying Lambda functions in that way. It's like so awesome. Yeah.

Aaron

00:34:06 – 00:34:12

So Lambie is one that I've actually looked at. Is is it is Ken in our chat? I thought he was.

Colleen

00:34:13 – 00:34:26

He is, but he's not active, but he's, like, around. I think he's, you know, inactive in the chat. But, yeah, he so he wrote lamb Lambie Lambie. But it's I mean, he's has a whole series of talks on it, and he's got a great tutorial. Yeah.

Colleen

00:34:26 – 00:34:37

It pretty much it extrapolates a lot. You still have to do some stuff with the SAM CLI, but it's it's pretty hands off if you're not comfortable with, you know, serverless functions.

Aaron

00:34:38 – 00:34:54

Yeah. So this Solambi, as far as I understand it, lets you does it let you run your whole Rails app on Lambda Lambda, or does it do function by function, like, you can get out there if you want to?

Colleen

00:34:54 – 00:35:08

So I I have not tried that. I've just tried individual functions. I don't know off the top of my head, but I know it's something he's constantly I mean, for this has been around for years, so I think he's been iterating on it. So I would big. It's, so yeah.

Colleen

00:35:08 – 00:35:10

It's pretty cool. Anyway

Aaron

00:35:11 – 00:35:34

Yeah. I remember looking at this. I was looking once at we have, Laravel Vapor, which is like you can just run your entire application on Lambda. And I was trying to figure out how they did something, and I went source diving on on Lambi as well because it's very, like, as far as I remember, it handles it handles a ton of stuff Lambie does.

Colleen

00:35:34 – 00:35:36

Yeah. It's super powerful.

Aaron

00:35:37 – 00:36:02

Yeah. That's super cool. Yeah. So that's kinda I guess that's kind of the the stuff that I've been thinking about, and I would like to get whenever Sean gets back, I'd like to start, talking about the, like, the punch list for Laravel View so we can get that, like, tidied up and shipped out in the next couple of months. That would be

Colleen

00:36:03 – 00:36:04

Yeah. That'd be exciting.

Aaron

00:36:05 – 00:36:06

It'd be really nice.

Colleen

00:36:06 – 00:36:08

Right? Maybe that's a good Christmas present.

Aaron

00:36:09 – 00:36:13

Yeah. Oh, man. I hope it happens before then. What is it? August 13th?

Aaron

00:36:13 – 00:36:15

Maybe it's a good Christmas present. Yeah.

Colleen

00:36:15 – 00:36:16

I feel like that's a realistic goal.

Aaron

00:36:16 – 00:36:18

That's 4 months. Oh,

Colleen

00:36:19 – 00:36:22

man. But that would be really exciting to get that rolling.

Aaron

00:36:22 – 00:36:28

Yeah. Yeah. It would. Okay. Well, let's see.

Aaron

00:36:29 – 00:36:37

I don't think I have anything else. This is fun. I'm glad I'm glad you're you're game to do this podcast. I'm excited you're here.

Colleen

00:36:38 – 00:36:45

Yeah. It's different than my other podcast. Like, you guys are like, whatever. Someone walked into the room. It's So

Aaron

00:36:45 – 00:36:53

your other podcast chill. Software social. Tell me you've said that before, so tell me how this is different. Like, I think I know, but tell me.

Colleen

00:36:54 – 00:37:13

We I mean, we had a couple of really specific goals in mind for that podcast. And, you know, we have sponsors now and we try to sound, so we chat, but we're not like, hey, how's your back feeling today? Like, what's going on with you? How's your week bed? Because this is the only time, you know, you and I will probably chat per week.

Colleen

00:37:13 – 00:37:13

Yeah.

Aaron

00:37:13 – 00:37:15

It was like, hey, Colleen. How's it going? Let's start recording.

Colleen

00:37:16 – 00:37:25

Right. Right. Right. So it's just different. We're a little more we're not scripted at all, And we don't we don't pregame, like, so I don't know what Michelle's gonna say and she doesn't know what I'm gonna say.

Colleen

00:37:25 – 00:37:29

But we do try to stay focused on top of this.

Aaron

00:37:29 – 00:37:32

Yeah. That's where, that's where this is a little bit different, I guess.

Colleen

00:37:34 – 00:37:38

But this is cool. I dig it. This is really low. I mean, low stress, like

Aaron

00:37:38 – 00:37:40

Low stress. Chat.

Colleen

00:37:40 – 00:37:40

Yep.

Aaron

00:37:40 – 00:38:09

Yeah. There's, there's another podcast I listen to called no plans to merge. It's pretty big in the Laravel space, and it's these 2 dudes who are both, like, very good programmers. But they talk about fishing more than they talk I swear they talk about fishing and labor policy. Last week was about Kohl's Cash.

Aaron

00:38:09 – 00:38:13

They talked about Kohl's Cash for, like, 30 minutes, how they pay you to shop there.

Colleen

00:38:14 – 00:38:15

And Oh my gosh.

Aaron

00:38:16 – 00:38:34

Man, it is so funny. And I I really appreciate those episodes. I like the ones where they, like, talk about programming for sure, but sometimes they go off on a tangent, and I just it just makes me laugh so hard. So, yeah, this is a little this is a little more focused than that, but definitely less focused than software social.

Colleen

00:38:34 – 00:38:36

Yes. Indeed.

Aaron

00:38:36 – 00:38:47

So hopefully you enjoy it. Okay. Well, that that can be our exciting outro. That's how we can end the show and just say, we'll talk to you next week.

Billions with a "B"

Transcript

Not enough emails?