Episode

Hammerstone · Aug 13, 2021

Billions with a "B"

Billions with a "B"
Show notes

<div>Aaron can barely move, Colleen is working with billions of records. Sean is on vacation! Lucky Sean.</div>

Transcript
00:00

Aaron

So this is actually the first time I've been back at my desk since the last time we recorded. I've been Oh,

00:07

Colleen

my gosh. Are you serious?

00:09

Aaron

I'm serious. I've been so we recorded last Friday and then on Saturday and Sunday I was doing some stuff in the shed. And then Sunday afternoon is when my back just locked up. And I've been in bed I've been in bed the entire week. Literally, the

00:27

Colleen

entire week.

00:28

Aaron

This is the first time I've been back at my desk. I know. It sucks. So I got I got an MRI. So I've seen an orthopedic because I was the first an orthopedist because I was the first person I could get in to see on Tuesday morning.

00:45

And he was like, yeah. You need to get an MRI and then go talk to a a spine doctor. So I got an MRI, a big old bulging disc right into right into my nerve. I saw 2 spine doctors. I I guess they're called, spine surgeons, but that makes me nervous.

01:05

And they both recommended getting an epidural injection before doing any kind of surgery, which makes me feel good because that seems way more conservative than, like, taking out the disc or something. So Heather: And

01:19

Colleen

they shave the disc down. Isn't that what they do in

01:21

Aaron

the surgery?

01:21

Colleen

They shave it. Yeah. Derek Mhmm.

01:25

Aaron

And both of them said like, you know, this steroid injection may fix it altogether. It may not do a thing, though. So it's kinda I'm getting it at 3 o'clock today, so we'll find out tonight. Well, they said it could take a day or 2. So I'm really hoping that it just fixes it altogether, and I don't have to have surgery.

01:47

I'm 32. I'm not old enough to have back surgery. I have

01:50

Colleen

to ask, like, is this a chronic problem? Because my first thought when you told me was you're kinda young to have this kind of a problem.

01:57

Aaron

No. It's not. I mean, I've I've been in pain for the I've been in a different pain for, like, 18 months. My neck and my shoulder have been in pain for a year and a half, but I've been in I've had this leg pain for, like, 2 months. And I've been, like, stretching and getting massages.

02:15

And then finally on Sunday, it was like, nope. You can't even straighten your leg anymore. You can't walk. It's like, oh my gosh. Awful.

02:23

So, yeah, the one the one doctor that I really like said, you know, this this can just happen. Like, you can just be doing something, and it just like, the disc spurts out, and this just happens. Like, I wouldn't I wouldn't be depressed about it. I was like, oh, that's good. Because I feel pretty bummed about being 32 and looking down at back surgery at this point.

02:45

So Yeah. But I am just hoping that the steroid shot calms everything down. I got an inversion table off of Amazon for a $130, which is insane. So I can, like, flip over and, like, stretch my back out. So we'll see.

03:07

But, yeah, been kind of a kind of a rough week.

03:09

Colleen

Oh, that's awful.

03:11

Aaron

And the, like, the kicker is I've been having to lay on my side because it's the only way that's really comfortable. And so I can't, like, sit in bed with my laptop and, like, do stuff. So I've just I've just been laying there watching TV. I've watched the entire season of, Alone on Netflix. Have you heard of that show?

03:32

Colleen

Mhmm. I haven't heard of it.

03:33

Aaron

It's crazy. They drop people into the wilderness just to live. And then Cocaine Cowboys, I watched all of that. But it's kinda it's been kinda depressing, honestly, because I, like, can't do anything. I wanna, like, be working on stuff.

03:46

I can't do a freaking thing. So

03:48

Colleen

Wow.

03:49

Aaron

Yeah. So anyway, that's how I'm doing.

03:55

Colleen

Yeah. So mostly terrible,

03:57

Aaron

but Mostly terrible. Mostly terrible, but with a glimmer of hope today at 3 o'clock. Although I am fasting all day, so that makes things crappy too. No no food or drink past 8 AM. So I'm both thirsty and hungry.

04:12

Colleen

Hopefully tomorrow is better.

04:14

Aaron

Hopefully. Welcome to the show, people. This is a good one. Okay. So tell me, you talked to Andrew today.

04:26

What are y'all doing?

04:27

Colleen

We yeah. We're doing great. Excited to kind of get ramped up with this client and, look at some of the issues they're having. I think this is such a great place to start because they're gonna have such a huge dataset.

04:41

Aaron

Mhmm.

04:41

Colleen

It's gonna be really fun and really challenging. I think to think

04:46

Aaron

We're talking like billions. Right?

04:48

Colleen

Billions of records. Right?

04:49

Aaron

Billions with a b. Yeah. Okay.

04:50

Colleen

Yeah. Billions with a b records. And I think what what it's gonna look like, what we talked about is that the data's gonna come in, you know, kind of like a JSON dump. And then we're gonna deserialize that and put it in this events table. And then the thing is when we query when we have these nested queries, the there's concern that the joint the way we have it set up right now to just automatically do all these nested queries is not gonna work.

05:17

We're just gonna lock up our database. So we talked about kind of doing these nested queries in a way that they just return the IDs, which I don't think is gonna be an issue, but I've gotta look at it. So it's a little different than the way we have it set up now. It'll actually be breaking out individual filters. Right?

05:35

So the different models will have different filters that all come from this singular query. But the different filters

05:41

Aaron

Wait. Flesh flesh that out for me. So different models, I e a customer and a product, those are those will have different filters. Right? Show me customers that 123 and products that a b c.

05:55

Is that what you're saying?

05:56

Colleen

Right. So when you look at the way I think we're gonna be querying this data is you're saying send an email to a contact where the contact has purchased a product, and the product was green.

06:10

Aaron

K. Got

06:11

Colleen

it. But because of the nature of the size of these tables

06:15

Aaron

Mhmm.

06:16

Colleen

We're discussing and returning it in such a way that you get that we receive that query and you go down to your product. The first thing you're gonna do is query for products that are green, return the IDs, and then feed the IDs into the one level higher query in order to make it a little more performant. So that's what we're talking about, like, playing around with right now.

06:37

Aaron

Okay. This is this is I love this. This is I'm so glad you're here because this is the stuff I wanted to talk about. Okay. So tell me and I don't know how far y'all have gotten.

06:50

So Like,

06:50

Colleen

literally, this was a 20 minute conversation. We haven't looked at it yet. So this is

06:54

Aaron

So is the thought to pull the IDs into memory, like, pull them into Ruby and then bind them into a different query in memory? Or is the thought to push that all down into Postgres somehow? Because they're using Postgres. Right? Or So time stream or whatever it's called.

07:13

Colleen

Yeah. So the thought right now is to do it in Ruby.

07:16

Aaron

Okay. So the

07:18

Colleen

the, I was gonna say we are talking now about Timescale DB, not being, like, our sole source of truth. I think we had talked about this a little bit. And so that data is gonna be dumped into we're gonna go with the model domain structure. Right? So that data is gonna be dumped into the model tables.

07:35

And so, yeah, that would be so you would basically go into your product filter, pull out the IDs. That would be in Ruby that you're keeping track

07:44

Aaron

of that. Okay. That is interesting to me. That, to me, sounds less performant to pull it into memory versus doing so I know that there were some discussion. The way that we do it currently is we use independent subqueries.

08:06

Right? So we use our our refine builds up subqueries and then executes everything all at once as a giant statement, and the statement contains multiple subqueries. And so what we do by default, just as background for other people, what we do by default is we push everything down to the database layer. So we don't pull anything into memory, but we do essentially the same thing that you're suggesting where you do basically, like, a where in, and then you pass in, you know, a bunch of IDs. What we're doing is we're doing a where in, and then instead of passing in IDs, we pass in well, we have another SQL statement in there with the thought being that the engine the database engine can optimize whichever route is best to do that.

08:58

And so it can run all of those and then build them all up together. So tell talk to me about the other way. So that as background, that's what we're doing. You know this, but that's what we're doing. And so what's the thought with pulling it into Ruby instead?

09:12

Colleen

So the concern is that's gonna lock up the database. And I have no idea. I don't know yet. So this is all new. Right?

09:17

We just we just talked about this. Mhmm.

09:19

Aaron

We

09:19

Colleen

spent a short period of time talking about it. The concern is that we're gonna have trouble with that query because there will be some joins in there based on the table relationships. And I haven't looked at this model in all of its relationships, but the concern is is trying to do those joins on those multiple tables where necessary is going to give us some trouble.

09:45

Aaron

Okay. I think this is gonna be interesting because this is like, obviously, this is the biggest dataset we'll have done this with. And this is actually the first Ruby implementation that we've done, so it's kind of a it's kind of a double whammy. So by default, refine as far as I remember, refine doesn't use any joins whatsoever. We use the sub selects.

10:16

Is that Oh, no. We changed it for Ruby.

10:18

Colleen

We changed it for Ruby. Right.

10:20

Aaron

That's right.

10:21

Colleen

And I'm I'm just coming back into this after being off it for 2 months.

10:25

Aaron

I know. You gotta load it all into your

10:27

Colleen

I gotta, like, load it back in my brain, but I am relatively confident there was some relationship where we were like, it's better to you use the joints here.

10:36

Aaron

Mhmm.

10:36

Colleen

And so maybe that's part of it too, Aaron. Maybe we look at it. I mean, I think the first step here is I just have to see what do we provide out of the box for this particular filter. Right? Mhmm.

10:48

Because we what we're doing now is we're examining the relationships, and we're building up the query, you know, based on the relationship. So I need to see what we're providing out of the box. Mhmm. And maybe that's something we can further optimize instead of returning the IDs as one thing and then putting those IDs into the next filter.

11:07

Aaron

Mhmm.

11:07

Colleen

So Hello, child in

11:10

Aaron

the background. That's fine. We're we're a very casual show. Hey, buddy.

11:18

Colleen

Sorry. Okay. Visit from my child.

11:22

Aaron

Well, that's good. Alright.

11:23

Colleen

What were we talking about? Right. So You

11:25

Aaron

were saying pulling the IDs into memory.

11:27

Colleen

Right. So I think I think the first step here is to look at exactly what we're providing out of the box. Mhmm. And why are you telling me in the Laravel side, you don't use the joins ever anywhere? You In the

11:40

Aaron

Laravel side, do you wear in?

11:41

Colleen

We don't

11:42

Aaron

mhmm. We don't use a join anywhere ever. So part of the part of the thesis there is or the hypothesis there is, on the Laravel side, we don't touch so the developer provides the initial query, right, where they say, you know, it's it's the base query, like employees or act or, users or customers, basically just a wide open query. We don't join anything in because then then that means we're adding, we're adding stuff to the returned dataset. So if we were to if the developer were to pass in a base query of, like, customers

12:31

Colleen

Mhmm.

12:31

Aaron

Right, and then we were to join in product purchases, etcetera, all of that, all the way down the line, then the dataset that we return is gonna have all of that data left joined into it. And the, developer hasn't explicitly selected that data, but we have added it in. And so we have, unknowingly changed what the developer's expectation is. Right? So if the developer is saying, I want my base query to be customers, and then we return a query with customers full of products, full of variants, full of colors, full of prices, they're gonna be like, what the hell?

13:09

Why did you give me 8 tables worth of data when all I asked for was customers? Right? And we can't we can't modify the selection either because what if the developer has already modified the selection? Then we have to go in and look at, okay, which columns are they selecting, and how do we, like, how do we add or remove the ones that we're modifying. So we just leave the select alone completely.

13:35

Does that make sense?

13:37

Colleen

Yep. That makes sense.

13:38

Aaron

So whatever the developer asks for, we will give them, and we will add in the necessary subqueries to get to the place we need to be.

13:49

Colleen

Right. Yep. That makes sense. Yeah. I have to go back and look at this implementation to remember, why it's a little bit different on the Ruby side.

13:59

I don't recall off the top of my head.

14:01

Aaron

I don't either. But I do remember talking to you about why like, needing to change it to left joints for something, but I cannot remember why we did that.

14:13

Colleen

Well, this might be a good opportunity to have I mean, maybe we surfaced a good issue here. Right? Like, this might be a good opportunity to check that out and The the Yeah.

14:21

Aaron

Remember. The frustrating thing is, like, how do we, oh, gosh. I can barely move.

14:27

Colleen

Oh my gosh.

14:27

Aaron

How do we, how do we test anything against a 1000000000 records without having a 1000000000 records?

14:34

Colleen

I have no idea. That's gonna be fun. That'll be fun to figure out. I don't know. Like, how do you actually

14:40

Aaron

Do we have to run do we have to run a seed to get a 1000000000 records on your local machine?

14:46

Colleen

Oh, my gosh. My Mac won't make it. It'll just Oh. It'll explode.

14:50

Aaron

I bet.

14:50

Colleen

It'll just start smoking. There's got I mean, I don't really know enough about tables this big. I'm sure there's a way to do that. Right? There's got Yeah.

14:59

Aaron

And I don't know if there's a functional difference between a 100,000,000 and a 1000000000, honestly. The biggest tables we have at Resolute are, I think, 50,000,000, maybe 80,000,000, something like that.

15:13

Colleen

Okay.

15:15

Aaron

But, no, we don't touch we don't touch a 1,000,000,000.

15:19

Colleen

Yeah. And it's

15:20

Aaron

gonna be so dependent on their on their, like, hardware.

15:25

Colleen

This is, like, a lot of interesting challenges that this is gonna surface. I think this is gonna

15:30

Aaron

be super fun. Sucked you back sucked you back in.

15:33

Colleen

Totally is. I totally is. I was like, I could do this 1,000,000,000 records. This is so cool. I'm super pumped to see, like, to, to get into these problems and to see where this takes us.

15:45

And, you know, we'll be rock solid once we figure out everything on this side because this is data.

15:52

Aaron

Yeah. So their main source of data is gonna be events table. Is that right? And that's gonna hold, like, that's gonna hold, like, clicks and purchases and email opens and all of that stuff.

16:06

Colleen

Yes.

16:07

Aaron

And then the side the little bitty stuff on the side is gonna be, like, customers who did something in the events table. Right? So customers is, like, millions, just a few million. And then

16:19

Colleen

Just a few million.

16:20

Aaron

The events is the 1,000,000,000, and they're gonna want customers, but customers who did, you know, x y z in the events table.

16:29

Colleen

Right. That is my understanding. Is so you're looking at something like, the the sample I was given was send an email to a contact where the contact has purchased a product that was read and has not yet been emailed as part of campaign 3. And his name is like, these are crazy queries. Right?

16:53

And his name starts with c. You know what? It's gonna be exciting.

16:58

Aaron

You know what is so just kinda thrilling to hear about that sentence is if you hold constant the fact that we'll figure out the 1,000,000,000 rows, refine does that. Like we can, we can build that query.

17:09

Colleen

We can build that right now. It's pretty cool.

17:11

Aaron

Right? That's wild.

17:13

Colleen

Yeah. Yeah. It's gonna be pretty spectacular.

17:18

Aaron

That's really exciting.

17:20

Colleen

Yes.

17:21

Aaron

Okay. Well, that's cool. So you and Andrew were just, like, catching up to see where y'all are at right now?

17:27

Colleen

Yes. Because I'm just getting back into this after a couple it's been a couple months. So I just need to get the lay of the land in terms of what's the status of this database and what do we need. And so we were just reviewing some of that. And so we were brainstorming how what this is gonna look like when these events start coming in and how we're gonna save them because there had been discussions of different databases and using different, you know, engines.

17:54

But I think we're gonna go

17:55

Aaron

esoteric stuff.

17:56

Colleen

Yeah. We're gonna go, like, model based, which is what you and I had talked about ages ago, I think. I think that's gonna be great for Refine because that's gonna really enable us use the power that we've built here instead of writing another, you know, DSL on top of it. So I think we're we're all good. It's gonna be fun problems.

18:13

But, yeah, I'm just getting my head back in the space to remember what we're looking at.

18:18

Aaron

And you start for real for real on the 23rd?

18:24

Colleen

Yeah. So I'll I'll start working more with them, something like that. Yeah.

18:28

Aaron

Okay.

18:29

Colleen

Relatively soon.

18:31

Aaron

Hopefully, very soon. Okay. Do you wanna talk, contract stuff?

18:37

Colleen

What contract stuff?

18:39

Aaron

The stuff that we're waiting on.

18:41

Colleen

Right. I don't know that we need to talk about it. I think we're good.

18:46

Aaron

Okay.

18:46

Colleen

We're just waiting for it to be signed.

18:48

Aaron

Waiting on signatures, and then I'll

18:50

Colleen

get it. So, yeah, we're just waiting on that to be signed, and I don't think there's gonna be I haven't heard that there's gonna be any problem. So, hopefully, that'll happen today.

18:58

Aaron

K.

18:59

Colleen

And then you saw the thing from the lawyer about the LLC being in Texas and Let's talk about that. My god.

19:06

Aaron

So in in my in my stupor, I I read a little bit of the attorney's email, which is not a good idea. But, basically, what I gathered was they're basically saying, like, if it's a Texas LLC, which it is, then we should get somebody else to do it. Yes.

19:23

Colleen

Is that the gist? That's the gist.

19:26

Aaron

So Okay. That's a bad gist in my opinion. That doesn't seem great.

19:31

Colleen

No. I mean, it sounded to me my interpretation of that email was we don't know the Texas laws. So if you wanna keep it at Texas LLC, we cannot help you and we cannot represent you. If you wanna disband that and start an LLC in Illinois, we can help you with that.

19:47

Aaron

Thoughts before I have my thoughts?

19:51

Colleen

I I have no thoughts.

19:53

Aaron

I have many thoughts. I have

19:55

Colleen

I have Tell me your

19:56

Aaron

thoughts, sir. In fact, I have one thought, which is many. I kinda think we should just start over.

20:03

Colleen

Yeah. I'm okay. I mean,

20:07

Aaron

We would need to figure out the accounting ramifications of that, but I I kinda think at this point so Sean and I have no operating agreement. We own this LLC 5050. It's made, you know, x amount of money of which we've paid, you know, point 9 x out to you. I guess less than that, because Sean has been billing hours, and that money has been staying in the company. So and the LLC was formed by me going on to the secretary of state website and paying $300 to file the LLC.

20:46

So I'm not tied. I would like to keep the name. But if we could I think if we could just start over and have the attorney do it, because that would gain us, obviously, the attorney, which we'd need, and the ability to have you, like, written in from the start instead of, like, somehow added on to whatever Sean and I already have. I kinda think we should just disband or kill the Hammerstone Texas entity and start a Hammerstone Illinois entity.

21:24

Colleen

That certainly seems like the easiest solution. I don't know

21:26

Aaron

It does.

21:27

Colleen

To your point, there's nothing about it being incorporated in Texas that makes it extra special. And there's not it doesn't sound like there's any I mean, that seems reasonable to me.

21:37

Aaron

Yeah. Okay. Well, Sean's on vacation, and since we bothered him so much last week, we shouldn't bother him. But when he gets back, we should talk about that. And I'll ask my accountant, and he can ask his accountant named our accountant named Aaron, if there what the accounting ramifications of that are.

21:58

It'd be great if we could just, like if Hammerstone Illinois could acquire Hammerstone Texas and just close it down.

22:07

Colleen

Yeah.

22:09

Aaron

Okay. Yeah. Fun stuff. I freaking hate that stuff, man. I

22:14

Colleen

hate Me too.

22:15

Aaron

I hate the attorney stuff. I hate the accounting stuff. I am actually an accountant, and I hate it. I hate I hate it so much. It's just so frustrating.

22:26

Colleen

Yes. And And so complicated. Doesn't it just feel so unnecessarily complicated? Why does it have to be so hard?

22:36

Aaron

I don't know. We're not dummies. We could Right.

22:39

Colleen

We could

22:39

Aaron

figure this out, but why does it seem so complicated?

22:43

Colleen

Right. Seriously.

22:45

Aaron

Oh, goodness. Okay. Anything else on your side?

22:49

Colleen

No. No. I'm just getting up and running, and I'm good.

22:54

Aaron

K.

22:56

Colleen

Let's see. On my side, I've been in bed all week.

23:02

Aaron

I got an inversion table. These are the highlights of my week. Yeah. Not not super great stuff. We have so it's in it's interesting now because you're a part of like, you're a partner in Hammerstone, and we've I've had got all these other, like, Hammerstone side projects going on.

23:26

So Hammerstone is a side project, and then I have side projects for the side project, obviously, as one does. So Torchlight is one of them, and Torchlight is the syntax highlighting, API. Are you have you seen all of this? Are you familiar with Torchlight?

23:43

Colleen

I'm not actually really familiar with it.

23:46

Aaron

Great. Well, you own part of it, so I will explain it to you now.

23:49

Colleen

Please do.

23:49

Aaron

So, when I was building the Hammerstone website, I was writing all the docs for refine and everything, and the syntax highlighting options out there were really frustrating and really crappy. Prism JS, Highlight JS, there's another one. And I tried them all, and each one missed something in a novel and frustrating way, and so everything ended up, like, looking slightly wrong. And so I started writing this, like I found I found this open source library called Sheeky that does, like, basically, Versus Code compatible, syntax highlighting, but it's really frustrating to use because it's it's not client side. It it's run it's a node process, so you have to install node and do the whole thing.

24:46

And their renderer kinda sucks. Like, it doesn't have line numbers. It doesn't have git diffing. It doesn't have a bunch of stuff. Right?

24:54

So I started writing this solution for the Hammerstone website, that includes line numbers and focusing so you can, like, blur out parts of the code. You can collapse parts of the code. You can add line numbers, do all sorts of stuff. And so as I was writing that, I spun that out into a stand alone service called Torchlight. So it's now an it's now an API with a couple of different clients, one being Laravel, and then let's see.

25:26

There's a there's a markdown client for PHP. There's one for there are a couple for static site generators in PHP, so we have some specific static site generators. Jigsaw is one of them, and then statamic is a a CMS that's not quite the same, but a little bit. And so it's got all these clients. I've launched it.

25:49

I've got I've got we've got, like, 200 people on the waiting list. We've got several several people using it in production. A few people have signed up for paid plans. The Fathom analytics team is using it for their website, for their docs website. Laravel News is using it for their website.

26:12

So we've got a lot, like, we've got a lot of people using it, and it's got a lot of traction, at least, you know, from my perspective. And Andrew wants to use it in bullet train. So that was the thing I messaged I messaged y'all about last week is we were gonna have one of our chat friends, Barry. Have you ever talked

26:35

Colleen

to Barry in the chat? Yeah. He was at retreat to your friend.

26:37

Aaron

That's right. We're gonna have Barry do it, but Barry got, slammed with work. And so now we're gonna have somebody else, write us a Ruby client, and it's gonna be, Andrew's gonna pull it into the bullet train docks, so Torchlight will power those docks as well. So kind of the thought with all of that is it's like a it's basically like I don't think it's gonna make that much money. I mean, the business plan is $14 a month, so there's not there's not really a lot of room to get rich on it.

27:13

But I think there's a lot of room to get, 1, to get backlinks, because if if you use the, you know, the open source version, you have to link back to us, which is gonna be great. And 2, to get, like, to get mind share. So I've gotten a lot of people who have messaged me and been like, this is fantastic. I would love to use this. Highlighting sucks.

27:36

I wanna put it on my website. And people just start, like, tweeting about how much they, you know, they install it, they put it on our website, and they love it. And I think this has been 10 times, maybe a 100 times as valuable as trying to write blog posts about, you know, hey, we're Hammerstone. Please notice us. But, yeah, this week has been this week has been good because a couple more people have picked it up and started tweeting about it, particularly the Fathom Analytics team when they, like, when they bought when they signed up and started tweeting about it, he DM'd me and was like, dude, I didn't understand I didn't understand before, like, the value of this.

28:19

And now this is, like, this is extremely valuable and it's worth every penny that I pay, and you've done a fantastic job. Was like, oh my gosh. Nice.

28:27

Colleen

That's a good year, Aaron.

28:28

Aaron

That makes me feel so good. Like, it's so encouraging to to hear that.

28:33

Colleen

Yeah.

28:34

Aaron

So we're getting close to just opening it up, opening you know, right now it's behind it's behind a form, so you go there and you sign up for the early access list. But I think Okay. We're getting close to just, like, opening it wide up and anybody can sign up and anybody can use it. So it's been a lot of fun. Hopefully, we can get a Ruby client written.

28:55

Andrew can start using it. We can start getting some adoption in the Ruby, ecosystem, the Rails world. But this is a nice little, like, a nice little beachhead to get into people's, you know, to get into people's minds and kinda get our stuff out there. So that's what Beachlight is, and you Sweet. You own part of it.

29:18

Sweet. So if you ever see it, you gotta be a hype person because now this is this is part of your this is part of your portfolio.

29:26

Colleen

Nice. Cool. That's great.

29:28

Aaron

Yeah. So that I've been working a little bit on that recently. Good lord. There's no comfortable position. Position.

29:39

And I think that's about it. Oh, there's one other thing. Have you been do you know what Sidecar is? It's a Laravel package that I've written.

29:50

Colleen

I feel like I've heard of it, but I don't know

29:52

Aaron

So that

29:52

Colleen

it's either.

29:53

Aaron

This is and I'll be curious, like I think we've talked about this a little bit. I'll be curious how this works long term. Like, in terms of is this an is this a good strategy that we should pursue in the Ruby world as well? Like, should we should we write open source packages for Ruby or Rails as a way to gain mindshare, going forward? So that'll be something that I think will be interesting.

30:26

But what Sidecar is is it allows you to deploy AWS Lambda functions straight from your Laravel application. So you can deploy and execute Lambda functions from your Laravel app without having to go through, like, the AWS UI or API or anything. And then you don't have to set up API gateway. You don't have to set up any of that stuff. You just directly execute it straight from your Laravel application.

31:03

And so that has been another one. I actually got to go on an, a podcast recently and talk about that one. And people have been contributing, like, pull requests to that one, which is a first for me. I've never written anything where someone else has contributed to it, which is kinda cool. So that one this week, somebody added a PR to support container images or something.

31:29

Colleen

So when you say from your Laravel application, like, how much abstracting do you do from from do you, like do they have to go in and set up a SAM configuration?

31:39

Aaron

Yeah. So what they do so when you when you install it, there's a configure command. So you just run artisan configure. Artisan is our little is the name of our CLI in in Laravel. So you run artisan configure, and what it does is it directs you to the AWS IAM portal.

32:01

So it gives you a link in in your console, it gives you a link. Here's you know, go to the IAM portal, make a, set of admin access keys, and it walks you through how to do that. So, like, full privileges to everything. Give me those keys. So you paste those keys into the terminal, and then it walks you through step by step what it's doing.

32:24

And so it creates, an execution role, it creates a deployment bucket, it creates a policy, and it attaches it all to the right, I don't know what you would call the right objects or instances or whatever. And then at the end, it gives you the option to self destruct those admin keys. So you can use keys to delete keys, and so it gives you the option to say, okay. We don't need you know, side car doesn't need these admin keys anymore. Do you wanna destroy them?

32:55

And if you say yes, those admin keys are gone, and you're just left with your your sidecar access keys. So in terms of, abstraction, then all you do is you say, alright. Here's my Lambda function. The package is, you know, these 10 files in this folder of node modules or whatever, and then the handler is, you know, image dot create or whatever. And then on you just call it from PHP.

33:24

You just say, you know, image you know, create OG image, execute, and it just creates an like, it goes out to Lambda, creates the image, and comes back to PHP.

33:36

Colleen

Cool. Yeah. We have something like that in Ruby. It was actually made by, like, my very first mentor, which is kinda cool. But, yeah, we

33:44

Aaron

What's it called?

33:44

Colleen

Yeah. Lam Lambie?

33:47

Aaron

I I've heard of Lamby. Is that Yeah. It's custom ink.

33:50

Colleen

Really cool. It's custom ink. So Ken Collins, who is, like Yeah. Literally the first rails engineer I ever met, Mhmm. Great guy.

33:57

So he wrote it. And so it's really great. I mean, so you're talking about deploying Lambda functions in that way. It's like so awesome. Yeah.

34:06

Aaron

So Lambie is one that I've actually looked at. Is is it is Ken in our chat? I thought he was.

34:13

Colleen

He is, but he's not active, but he's, like, around. I think he's, you know, inactive in the chat. But, yeah, he so he wrote lamb Lambie Lambie. But it's I mean, he's has a whole series of talks on it, and he's got a great tutorial. Yeah.

34:26

It pretty much it extrapolates a lot. You still have to do some stuff with the SAM CLI, but it's it's pretty hands off if you're not comfortable with, you know, serverless functions.

34:38

Aaron

Yeah. So this Solambi, as far as I understand it, lets you does it let you run your whole Rails app on Lambda Lambda, or does it do function by function, like, you can get out there if you want to?

34:54

Colleen

So I I have not tried that. I've just tried individual functions. I don't know off the top of my head, but I know it's something he's constantly I mean, for this has been around for years, so I think he's been iterating on it. So I would big. It's, so yeah.

35:08

It's pretty cool. Anyway

35:11

Aaron

Yeah. I remember looking at this. I was looking once at we have, Laravel Vapor, which is like you can just run your entire application on Lambda. And I was trying to figure out how they did something, and I went source diving on on Lambi as well because it's very, like, as far as I remember, it handles it handles a ton of stuff Lambie does.

35:34

Colleen

Yeah. It's super powerful.

35:37

Aaron

Yeah. That's super cool. Yeah. So that's kinda I guess that's kind of the the stuff that I've been thinking about, and I would like to get whenever Sean gets back, I'd like to start, talking about the, like, the punch list for Laravel View so we can get that, like, tidied up and shipped out in the next couple of months. That would be

36:03

Colleen

Yeah. That'd be exciting.

36:05

Aaron

It'd be really nice.

36:06

Colleen

Right? Maybe that's a good Christmas present.

36:09

Aaron

Yeah. Oh, man. I hope it happens before then. What is it? August 13th?

36:13

Maybe it's a good Christmas present. Yeah.

36:15

Colleen

I feel like that's a realistic goal.

36:16

Aaron

That's 4 months. Oh,

36:19

Colleen

man. But that would be really exciting to get that rolling.

36:22

Aaron

Yeah. Yeah. It would. Okay. Well, let's see.

36:29

I don't think I have anything else. This is fun. I'm glad I'm glad you're you're game to do this podcast. I'm excited you're here.

36:38

Colleen

Yeah. It's different than my other podcast. Like, you guys are like, whatever. Someone walked into the room. It's So

36:45

Aaron

your other podcast chill. Software social. Tell me you've said that before, so tell me how this is different. Like, I think I know, but tell me.

36:54

Colleen

We I mean, we had a couple of really specific goals in mind for that podcast. And, you know, we have sponsors now and we try to sound, so we chat, but we're not like, hey, how's your back feeling today? Like, what's going on with you? How's your week bed? Because this is the only time, you know, you and I will probably chat per week.

37:13

Yeah.

37:13

Aaron

It was like, hey, Colleen. How's it going? Let's start recording.

37:16

Colleen

Right. Right. Right. So it's just different. We're a little more we're not scripted at all, And we don't we don't pregame, like, so I don't know what Michelle's gonna say and she doesn't know what I'm gonna say.

37:25

But we do try to stay focused on top of this.

37:29

Aaron

Yeah. That's where, that's where this is a little bit different, I guess.

37:34

Colleen

But this is cool. I dig it. This is really low. I mean, low stress, like

37:38

Aaron

Low stress. Chat.

37:40

Colleen

Yep.

37:40

Aaron

Yeah. There's, there's another podcast I listen to called no plans to merge. It's pretty big in the Laravel space, and it's these 2 dudes who are both, like, very good programmers. But they talk about fishing more than they talk I swear they talk about fishing and labor policy. Last week was about Kohl's Cash.

38:09

They talked about Kohl's Cash for, like, 30 minutes, how they pay you to shop there.

38:14

Colleen

And Oh my gosh.

38:16

Aaron

Man, it is so funny. And I I really appreciate those episodes. I like the ones where they, like, talk about programming for sure, but sometimes they go off on a tangent, and I just it just makes me laugh so hard. So, yeah, this is a little this is a little more focused than that, but definitely less focused than software social.

38:34

Colleen

Yes. Indeed.

38:36

Aaron

So hopefully you enjoy it. Okay. Well, that that can be our exciting outro. That's how we can end the show and just say, we'll talk to you next week.

Details
Show
Hammerstone
Date
Aug 13, 2021
Role
Host
All podcasts
Podcasts
Newsletter

Every week: the best things I've found and what I've been thinking about.