Transcript#

This transcript was generated automatically and may contain errors.

Welcome to The Test Set. Here we talk with some of the brightest thinkers and tinkerers in statistical analysis, scientific computing, and machine learning. Digging into what makes them tick, plus the insights, experiments, and OMG moments that shape the field.

In this episode, we sit down with Alex Hillman, author of The Tiny MBA and owner of the co-working space Indie Hall in Philadelphia. And I'm in Philadelphia, I go to Indie Hall, I'm just a really big Alex fan. We talk to him a bit about marketing to open source communities and how that's a healthy thing, and also about his use of AI as part of a lot of his community building work. So with that, super excited for this episode, Alex Hillman.

All right. So Alex, welcome to The Test Set. So honored to have you.

Glad to be here. Thank you for the invitation.

Yeah, no, no, I'm pleased as punch. And I think just by way of introduction, so you're Alex Hillman. Importantly, we're both from Philadelphia. You're a co-founder at Indie Hall, which is, as I understand it, one of the first co-working spaces in America?

That's right. We were the first in Philadelphia, one of the first in the world.

Word. That's incredible. You co-founded Stacking the Bricks, which includes the book Tiny MBA, and also more recently, you've really been cooking on a personal assistant that you called JFDI, or Just Frickin' Do It, which I'm so excited to get into a talk about. I'm just going to introduce co-hosts really fast. So joined by Hadley Wickham, who's chief scientist at Posit, and Wes McKinney, who's a principal architect at Posit, and has really been cooking on AI tools as well. So excited to have everyone here.

I think everybody's been cooking on AI these days, so I'm excited for the conversation since I've been following some of Alex's work from afar and have done my own kind of mini assistant work. Certainly nothing to the scale of what Alex has done or what people in the open-source community have been doing, but definitely an interest area like personal productivity with AI.

Yeah, same. I feel like what's so interesting about you, and maybe for context, I'm a member of your co-working space. I always find it so interesting all your community organizing and wrangling, but I thought maybe like a fun question for everyone, and especially like Hadley and Wes, to just kick it off is like, have you all read books like the Tiny MBA? Because I find open sources interesting where you're also kind of like putting a tool in people's hands and trying to get it out there.

I mean, I've read some books on like company culture and startups and things like that. So like, you know, the lean startup and I like Ray Dalio's principles. Ray Dalio is a little bit of a divisive figure, but actually it's hard to apply all of his principles. Like it's a little bit hardcore, but I feel like some of the basics like radical transparency and people like being honest and saying what they mean and being willing to like give feedback. And it's pretty interesting. Another book that I like that's a little bit of an MBA book is a book called The Culture Map, because I've worked in like very multicultural companies and teams, and it helped me realize that like the way that I approach work and business is very American.

I was going to say, you know, that we can come back to the book question in a second, but one of the things that's interesting is, you know, so we started this co-working space, it'll be 20 years ago this September, but there was a good solid era while co-working was going from being sort of a niche fringe thing to more mainstream.

And in part because we were early, also in part because my background was in open source, my philosophy was like, well, it wasn't my idea. I borrowed from a bunch of stuff. Let me share everything that I've learned and put it back out there. Created sort of like an open source, but not for software cycle that put us on the map internationally, even though we were only one location in Philadelphia and we've worked with, I've worked with people creating independent co-working spaces on every continent except for Antarctica.

But the interesting and completely unexpected thing for me was how that became a gateway into exactly what you were just describing, Wes, which is, you know, co-working is sort of this microcosm for work culture. And I'd argue in its best state, it's a microcosm and a laboratory. It's not just what work is, it's what work can be. When you kind of play with positional power, I don't want to say remove it entirely, but you get a sort of a different scope of it.

And when I got to work with other people creating similar work environments where a lot of the hierarchy, not even hierarchy, just where the infrastructure was either different or removed, I kind of got to see a part of the way I think about it is, you know, you do have that American, specifically North American, specifically U.S. based hyper-transactional version of work and business, sort of at the core of a lot of the culture. And at the most polar opposite end of the spectrum, a lot of Asian cultures being more collectivist at their core and how that shows up in business and work relationships and communication and all those kinds of things.

And even just like having to translate some of the workshops and realize that things I would have to teach Americans was actually preceded by having to help them unlearn certain things. And then I would go run the same workshop in Indonesia and I'd say a thing and they'd be like, yeah, yeah, we know that, that's that's normal here. And it was like this interesting to like shortcut through the unlearning into the how do we amplify the effects of this particular local cultural nuance.

Some of the most fascinating work I've gotten to do is the furthest from home, but also in this weird way was where I felt most comfortable. Like I've always felt like kind of a weirdo in the American business culture because I care so much and almost hyper-prioritize relationships in the other direction and feel like a bit of an outsider in that way. And then, you know, a bit less in European cultures and even less so in in Asian and South Asian cultures, obviously painting on broad strokes. But to get to see that up front through that kind of unique lens is super, super interesting.

Marketing for people who hate marketing

Yeah, it's super interesting to hear. And one thing I noticed, too, was that people like Julia Evans had taken your workshops like 30 by 500 and mentions like, oh, what I learned about marketing, I learned from Alex Hillman. And I'm just curious, like what your kind of pitches to people like that or what how you kind of explain the need to to people like that.

Yeah, I mean, the audience for Stacking the Bricks, depending on who I'm describing it to, I would say it is marketing and sales for people who are allergic to marketing and sales. And there's a pretty high overlap between that and people who are traditionally engineers, programmers, artists, typically very good at what they do and pretty bad at selling it. And the way that we've framed it, that I think it's landed with folks like Julia and many of the thousands of students we've worked with, is we very intentionally call out why people are often allergic to these things.

And it's because their experience with marketing and sales is often like deception and manipulation and things that would erode trust. And our entire approach and philosophy through everything we've done from the start is it's about building and more specifically earning trust first and then reinforcing that trust over time. And then what that trust allows you to do in terms of making offers and communication.

You know, again, even when we are doing a sales pitch, the goal is not to persuade you to do something you weren't otherwise going to do. You know, there are manipulative people in this world, of course, but I always tell our students, if you're worried about it, you're not going to do it. You're not that good at it. I promise. It's going to take a lot of work to get so good at manipulation that you do it by accident. But if you think about persuasion less about getting somebody to do something that they weren't going to do and instead helping them make a decision that is in their best interest, and if you do things right, is in your mutual best interest, then good news. That's sales. That's marketing. That's really all it is at the end of the day.

But if you think about persuasion less about getting somebody to do something that they weren't going to do and instead helping them make a decision that is in their best interest, and if you do things right, is in your mutual best interest, then good news. That's sales. That's marketing. That's really all it is at the end of the day.

And the only other thing I'd add is I think what I just described is more the sales end of the spectrum. I think people think of marketing as broadcast. It's I got to talk, talk, talk, talk, talk, talk, talk. And that's a part of it. But I think the part that people overlook because they're always on the receiving end of somebody else talking is how much of marketing is actually listening. And at the heart of the course that we teach, all of our material, my book is really a bunch of different systematic and structured approaches to knowing what to listen for and to do like a mix of pattern recognition and deductive reasoning from high volume listening.

And people don't think about they think about market research, but they think of that as like surveys and focus groups. And we take a totally different approach that I think is really important. I think it both resonates with and plays to the strengths of the systems minded software engineer, the perhaps introverted or shy communication skills that that or just personality traits that people are coming to it.

If I'm somebody who does not consider myself a boisterous extrovert, how am I supposed to compete in a noisy landscape? Might not be a thing people say out loud, but it's certainly a thing that people think. And I think that part of the reason our approach has resonated so much is because it acknowledges that and says, not here to change you. If anything, I want to play it to the strength that you do have and help you see them as a lever towards these particular outcomes, which, again, is mutual gain between you and your customer. And the way you get there is by earning and keeping their trust.

Spending and investing trust

How do you think about like sort of spending trust, right? Like it's definitely something you can kind of say, like, I have accumulated all of this trust in the community and now I'm going to like deliberately like I'm going to do something deliberately that I know is going to erode trust, but like I believe it's worth it. Like, how do you kind of think about that?

That's a really interesting question. So, I mean, part of it is I try not to erode trust or more specifically of what I'm doing is going to erode trust. I'm going to kind of flip your question around and I'll ask the question, what are the things that typically erode trust? And I think it's one of two things. It's either I feel like you're trying to pull one over on me, like being deceitful in some way. And again, I think most people have experienced that and don't want to do it. But if you don't want to do it, the odds of you doing it by accident round to zero.

I think the more common thing is is that people worry about the message or the request or whatever it is being for everyone, and they end up with a pretty and in order to do that, they end up with a pretty generic and fairly ineffective in all ways message. Right. And instead, part of the philosophy that I approach it with is to think of it almost like cycling through the wheel on Price is Right, stay with me, where you've got a bunch of slices of a wheel and that wheel is your audience, and I try to speak to all of them at once.

I'm going to I'm going to have to say some pretty hokey things in order for it to be believable for all of them, and it's kind of kind of collapsing on itself. And so instead, it's like, hey, what I'm about to offer isn't necessarily for everybody. But if it's for you, it's extremely for you. And if it's not for you, my hope is that you stick around because the next time I come around, I want to have something that's for you.

I think the key piece is to be aware of what would actually erode that trust and go, do I actually have to erode trust in order to do this? And if I do, sometimes just acknowledging like, hey, if this isn't for you, I hope you stick around because, you know, the stuff you are here for is is not going away.

I guess one of the things I kind of think about is like asking people to do things that are like a little like painful or annoying, but like I like 100 percent in my heart of hearts believe you're going to be better off for doing this thing or like, you know, learning this thing that's annoying or you're going to go and change some config, you know, it's annoying. But in the end of the day, you're going to be better off. Like the question I ask myself is how confident am I that the vast majority of them are on the other side of it going to say that kind of sucked, but it was worth it?

If I'm not sure, I'll look for ways to kind of de-risk that or or make it clear who it is more right for.

But also, you know, I think coming at this through the lens of an educator and we specifically ask students to kind of trust the process, I think that the thing that has made the biggest difference is the longer the longer we wait for there to be a feedback loop for them, that something is working in the right direction, the more willing they are to continue trusting the process, so I think if this is going to ask for a spend, I want there to be a pretty quick feedback loop for them to know, OK, that spend was worth it. Right. Or it's not. And that's OK, too. And I can bail. But the longer you kind of drag somebody along for them to be like, man, I wasted my time.

Um, I think that's if I think about places where people spend trust, it's usually in stringing people along so long that by the time they make the offer, they go, well, if you had told me that up front, I wouldn't have hung out the whole time.

Where's the erosion of trust in that situation for you, like the situation you're thinking about?

Like like in that case, I think, you know, I think like maybe a better framing for this is kind of like investing trust, like you're kind of asking them to spend or, you know, take your word for it and they're going to be better off because I think if you do that like too often or the payoff's not worth it, that also just kind of drives down the trust. Like you've asked them to give you a bunch and you haven't given them what they expected. You know, they're going to be less likely to to go on a limb for you next time. I'll also like personally, I don't mind overinvesting up front where it's like like I'm at the point where I want to have so much trust that by the time I ask you to do something that's a little frictioning that you're like, well, you haven't taken me too far off the rails before, so I'm on board.

And I feel like, again, there's a lot of different ways to get there. And if I'm totally honest, I think maybe I overdo it sometimes, but I feel like that puts me in a position where the bank, you know, the trust bank accounts to sort of torture this metaphor is is well invested where that might feel like a big spend in a moment. But if I zoom out to the big picture and go, I've actually earned quite a bit more than I'm asking for.

And also the reminder that like so long as that cycle continues long enough, it is like in like earning interest in investing it compounds. Right. So, you know, our case, you know, if I'm want to put this in marketing and sales terms again, like. Sometimes if we're running a product launch where we're asking folks to, you know, read a bunch of emails to figure out if it is the right thing for them or watch these videos or whatever it is.

It is pretty rare for our students to see our course go through sort of our initiation, onboarding and launch stuff the first time and buy, not because they don't think it's for them, but because they are not ready to buy yet.

The most common thing we hear from people who join our program is they're like, I've been reading your stuff for two years, which means to me I need to have two years worth of stuff lined up for somebody to experience and earn for me to earn the credibility and trust that when the time comes for the big ask that the ratio and proportion is so in my favor that I don't even have to ask myself the question of, am I asking for too much?

The most common thing we hear from people who join our program is they're like, I've been reading your stuff for two years, which means to me I need to have two years worth of stuff lined up for somebody to experience and earn for me to earn the credibility and trust that when the time comes for the big ask that the ratio and proportion is so in my favor that I don't even have to ask myself the question of, am I asking for too much?

And that feels to me like that's where like a good sales, like a genuine good sales person would like help that person kind of like pull the trigger after three months, right? Like there's something, there is that like, yeah, that bad sales technique, but yeah, not the try, I'm trying to scam you, but I am genuinely like in there with you. I believe this is going to help you get you over that like hump of, of, of taking that, you know, scary action.

Community, email, and the people workflow

Maybe just to tie some of this together. I know, uh, as a fellow Philadelphian that you ran a meetup, the meetup mashup that had like almost 20 meetups, I think, um, including some, I love like Code for Philly, but also some, I learned about like Philicon Valley, which I found to be just, uh, the most beautiful name. Um, And that crew is just so good. They're the energy they bring to every room is, is, is phenomenal. I want a piece of that. Yeah.

Um, I, I'm so curious, cause I know that you have been so involved with different communities and I'm sure a lot, so much of this involves like email and getting in touch and like wrangling, not wrangling people, but certainly Not wrangling people. That's a big part of my job.

Part of the, um, how they wrote about that, but you know, part of the data science workflow and the people workflow. But, um, I I'm really curious, like, yeah, maybe you could say a little bit about, um, that those types of situations where you're doing wrangling and your, your system that you've been working on with like Claude Code to kind of like organize a lot of your interactions and emails. Cause I think, I think some of this will be really juicy to Wes who's working on like email tools and, and a lot of other stuff.

So we zoom out a little bit and then kind of make, make our way to that specific question because I think it is relevant. So, um, uh, first I haven't said it out loud, maybe it's been implied. I have a software development background, but I've never been a programmer professionally. Um, I've done some like front end development, HTML, CSS, JavaScript. Um, I did that professionally, but I always needed software engineers in my life. Um, to implement anything beyond my own core skillset and, uh, between or like early use of tools like Cursor and then Claude Code, they initially kind of showed up as an extension of that experience where before I could kind of architect stuff because I'd spent so much time working with really talented developers to know enough of not just the language and jargon, but actually like systems.

I learned systems design through actual like scale project implementations across lots of industries over my career, but I always needed another person to, to actually scaffold it out. Right. Um, even if the, as the tools got better, I was like, this is not a great use of my, my time, my, my skillset, the ramp up for learning. I just couldn't get over it.

And I always felt like anytime I wanted to build something or get, get a thing out of my head and onto a screen that was actually like functional systems software. Um, I felt like I was working with 11 mitts on, I would need somebody else to implement it, or I'm just kind of fumbling around with, with two giant thumbs. Right.

Um, and so my, my, and well, my initial entry into like AI software engineering was this again, sort of weird sideways in, uh, the other thing is the fact that like my goal was not to build software. My goal was to simplify and improve parts of my life and my workflow. Um, as a business owner and as a community builder. And so communication is a huge piece of that.

Building an AI executive assistant for the inbox

And the, one of the earliest, you know, besides hooking it up to my inbox and being like, can I teach this thing? How do you run my inbox? I think the most common thing people jump to is can I get it to respond to emails for me? And I was very early, a hard, no, I don't even want that. I don't want to respond to emails. I don't even really want it drafting emails. Um, that was not my goal. I'm pretty good at that. It's all of the stuff that gets in the way of doing that.

And so did a bunch of things in my inbox to have it, um, learn what belongs in my inbox, what goes straight to labels. Basically I had a pretty robust Gmail filter system that I had it download, refactor, reupload, test, and then use everything that it learns to do. So like my email filtering is actually almost entirely deterministic, but it is the deterministic system is managed by Claude Code. Um, and that was like the first time I gave my inbox as a major communication source, um, and these tools kind of an environment to work in together.

The title all the way back to the question about message vault and sort of where email and communication fits in. I was like, okay, I'm going to use Gmail. I'm going to use Gmail. I'm going to use Gmail. Communication fits in. I was like, okay, this isn't so much about keeping the inbox clean because I would kind of have that done. And I'm not even sure that I necessarily want it to like do my inbox for me.

But I realized that my inbox is this incredible source of information about me, about my work, about the people that I worked with. And the second system that I built for my executive assistant was a relationship manager tool, because that is the through line of all of my work. I don't think of anybody as a customer or a client. Um, within Indy Hall there, when people refer to them as customers, I'll always correct them to the members. Even if there's a transaction, the relationship prioritizes above that.

And so my thought was how can, if this system is successful, how can it help me show up better for the people in relationships that I have in the ways that I struggle to, which leads me to the fact that I hate every CRM that exists on, on, on planet internet. Um, cause I think CRMs are designed for a sales pipeline to get you from lead to purchased or qualified somewhere along the way, and I just don't think of people that way.

But I do want help noticing when, you know, a relationship that was warm is maybe going a little bit cold. I don't need it to do a thing. I don't even really need it to judge, but it's like, Hey, is there a Delta in the pattern here? Is this accelerating or decelerating? And just flag that for me and let me make the decision or if you can infer other things.

But I do want help noticing when, you know, a relationship that was warm is maybe going a little bit cold. I don't need it to do a thing. I don't even really need it to judge, but it's like, Hey, is there a Delta in the pattern here? Is this accelerating or decelerating?

So the starting point was, how do I start, how do, how do I build a dossier of the relationships that I care about? And I've tried a bunch of tools. I think the app that came closest for me was clay. Um, which I know has evolved a bunch, even since I used it, but even it had the same problem that I did, which was the first thing it does is ask you to like import all your contacts from your phone. I'm like, well, I don't want to import all of my contacts from my phone. I want to import the people that actually care about.

And so the intersection of these things was how do I figure out who these people are and I'm now teaching at my inbox. I realized if I send you an email, that's a pretty good clue that I care about you in some capacity. Receiving in in inbound mail is a useless signal in basically every way. But if I send an email, that's a pretty good clue. And so I basically taught the system to use that as a hook.

And then I'll have a job that runs and it looks at over the last period, all of my sent emails. It intelligently looks at participation. If there's like, if it's multiple people on a thread, it's going to filter out people that didn't actually participate in that thread, you know, they just so happened to get an email from me, but if I'm sending an email or responding to somebody, that's now the hook. And then it uses their name and email address to search forward and backwards in my inbox and build a, a context file effectively for what it can figure out about our relationship.

And they all start the same way with basic contact information, some meta around how, and when we met, were there people who introduced us? Do we work together on certain projects? Um, have we done business together or not? Do we have overlapping friend groups, um, and things like that. And then, then the robot kind of gets to freestyle. After that front matter.

And so depending on the nature of the relationship, this dossier can go anywhere from 30 lines of text to I've got people like, you know, Michael, our mutual friend, Chris Alfano, and I've been friends for like 17 years. I had to do a deep research. Basically I built, I built the, I built it as a sub-agent. And so there's a few different flags for depth and how to do the deepest research it could. And it pulled up stuff that I completely forgot Chris and I had worked on together. And like, there was so many like moments and memories.

So it was this cool, like trip down memory lane, but where that has come in handy is now when I'm interacting with my executive assistant on anything else in the system, that could be something as simple as I found a cool link on Twitter. And I, I want to bookmark this for later. When I drop it into the system, it'll automatically scan the entire system for related and relevant projects that we're working on that it might be useful for. But it also scans my people database.

And it goes, Hey, that thing we just filed, here's three people that might find that interesting. Do you want to send it to them? And it doesn't do the sending, but it is bringing that stuff to the surface. And I don't know, for me, sharing links is definitely a love language. And I don't think the person on the receiving end cares that I had a little bit of like remembering support to be like, Oh, this person would actually really like that. And then I still manually go and I text or email the link, whatever it is.

But using, using the depth of and the scale, I've been using the same email address for almost 20 years, being able to build these really robust profiles that still are, um, you know, it's, it's, it's not pulling in like sensitive project detail and stuff like that. It's like, I can look at any profile and I'd feel comfortable showing it to the person. That was also like a, like a, like a values and ethics guardrail is, is this going to build a profile that if, if somebody found out I had this profile of them, they would be freaked out.

Um, and so like, I think while it is, while it is deep, it is predominantly based on my communication. And so it's not like building a profile of their personality, their words, all of those kinds of things. Um, but instead what our shared relationship is, uh, has been, been super powerful.

Twenty years of email as exploratory data analysis

I mean, it's such a freaky type of data analysis too, that you do have with 20 years of email, like, uh, analysts might be tempted to like produce a bar chart, but actually having a written summary of 20 years of communication is such an interesting type of like exploratory data analysis.

I mean, for me, it, it, it like, I have, I'm like endlessly frustrated with, with like Gmail as a, as a product. Cause I feel like you don't even, you don't even need AI to make a better analytical, like called an analytical experience for email. Like just like, just to be able to have like a condensed timeline of your correspondence with a single person and the ability to like look at the email attachments that you exchange with that person would be tremendously useful and does not require, does not require any AI.

Like I was just looking in my, in my own message vault and I have something like 200,000 email attachments that have been, you know, sent and received. And I think some of those are like tracking pixels and stuff like that. Um, but, but like, you know, my life and like my, a lot of the. A lot of the — since like I've had many professional relationships and a lot of business relationships and a lot of those relationships come down to like exchanging documents back and forth. Like anytime I invest in a company, like there's a whole bunch of PDF documents that get exchanged and like just like the burden of going through and like finding the right PDF document. Like where is that one PDF document that I was sent? And it's always buried like somewhere in the middle of a thread, which Gmail makes even harder to find.

And if you try to search for it in Gmail, like, you know, there's some kind of Gmail ninjas that know the right like search incantation to find, you know, the right file type and they know exactly what to type in the Gmail search box. But like after 20 years, like I still haven't learned that magic syntax, which means that it's like not a good — It doesn't exist. Yeah.

And so, so that was like, I had definitely like a, like a, like a groundswell of frustration that led me to be like, like, let me get all of my email out of Gmail and like create a system where I can like, you know, essentially synthesize and reconstruct like these relationships, get all the email attachments out. And so now it's like wonderful to be able to ask Claude Desktop, like, hey, could you find this PDF from 2018 in a matter of seconds, like it will pull up the document for me. And so it's just like, Google, like, couldn't you have done that? Like, you know, and you have any, have any engineers lying around that could have built that feature? Like, I just don't, I just don't get it.

I mean, so, I mean, I have a question for you about that. And maybe, maybe I'm kind of curious if you've been paying any attention to other similar

projects to MessageFault, I mean, you mentioned that you're working on, on like an iMessage text message component.

One of the things I'm most excited about in this current era is, is sort of people waking up to the value of like sovereignty over their data. Yeah. Yeah. Yeah. I mean, I'm, I'm really excited about the local and private, you know, applications and like the, you know, because a lot of people are talking about the death of software as a service, but actually like, it's not, not that people are going to build new, like replacements for software as a service, but actually like software as a services, like software as a service as a delivery model. It may be like, may be dying, like actually like, like probably for most businesses, like most applications, business applications can just be a SQLite database. Like you don't even, may not even need Postgres.

It's like how many, you know, maybe if there's just like one, you know, one writer to a SQLite database, like holding a mutex, you know, maybe that's like, maybe that's enough. But for a lot of like business productivity applications, especially like applications that serve individuals, like, you know, there's a lot of products that I use that have, where I don't collaborate with anyone, like only I'm, I'm the only user, like I don't share anything with anyone. Like there's no reason that that can't be like an application with a SQLite database and that, and so like, you know, I've probably built in the last six months, like a dozen projects, like material projects that, that are like basically a productivity application with the SQLite database. And they, they work perfectly fine. Like they run on, you know, a Raspberry Pi in my closet, on my tail scale network. Like I can connect to it from my phone, you know, from anywhere in the world and it runs just fine, you know, and you know, SQLite's powering the messages, like the chat applications on your phone, like iMessage has, has SQLite inside of it, Apple Contacts, like I'm sure all the Google versions of these things are all based on SQLite.

So it's, it's, it's pretty interesting, kind of that, you know, that, that trend towards that type of like local private, you know, thing, and you can just build these applications really fast now with AI. But I'm also like, I don't know, like, I mean, people have one kind of maybe this is a little bit of a hot take. And I'm curious what you think. But I, there, everyone is building like tremendous amounts of software generating tremendous amounts of code with, with Claude Code these days. And yet, like, there actually aren't that many new things that I'm using. And maybe I can't tell whether it's like, I'm not like not finding things that are useful or like, maybe I'm just like lack of curiosity.

But like, you know, I've been putting, I've been building stuff for myself. And then if I think something is useful, I'm putting it out there into the into the world for people to use that they find it useful, then that's great. I think message vault is something that clearly resonated and a lot of people find useful. But I'm, you know, people are, you know, people are singing, you know, praises for like how productive Claude Code is making them. And yet, like, you know, there's a ton of AI slop on GitHub and like, open source projects are just drowning in slop and like a lot of open source maintainers are like disabling pull requests, like disabling issues, because they just can't anymore. And they're like, you know, it's it's become like very, like, you know, very Wild West and in a sense, and you know, maybe the more optimistic view is that eventually things will solidify and then you know, there will emerge like a wave of new, like, high quality software.

So maybe it just takes like somebody like yourself is just like really focused and passionate about making something that's like, really, really good for yourself. And, you know, whether it's, you know, for other people, or if it's just for you, like doesn't really matter as long as it makes you more effective, it makes you more productive. Or let me let me try to reflect the question to be sure we have it. Is it? That's kind of a rant. I guess that's kind of a rant, I guess. Yeah, no, no, no. I mean, because I mean, I'm with you 100%.

Locus of control and the Wild West of AI code

I think there's a lot of like productivity performance, especially on social media. And I like I write down a large percentage of and a lot of people get met with a mute button for it. I feel like you can kind of tell. Now, I'll also say I have this kind of working theory that while there is a clear delineation between like good code and bad code, I think just everybody hates everybody else's code regardless of who makes it. And I think there's there's there's a unspoken thing happening where people are like, my tolerance for other people's software is so low that I'd rather have a shitty version that I made with the help of my robot than deal with somebody else's bad mistake. So I think it's about like the locus of control more than it is about the quality of the product in this very moment.

my tolerance for other people's software is so low that I'd rather have a shitty version that I made with the help of my robot than deal with somebody else's bad mistake. So I think it's about like the locus of control more than it is about the quality of the product in this very moment.

And I think you could apply that kind of answer lens to a lot of things. Why is this weird behavior happening? Because people crave control. And that's like I think that is the best, I think, psychological slash philosophical answer to it.

I also think that we are I mean, to your point about the Wild West, it is both the most exciting and the most frustrating part of the whole thing, because it reminds me a lot of early Internet. You know, for me, I'm thinking back to like 2002 to 2009, where people like big companies like Yahoo were building things like Yahoo Pipes as the precursor to Zapier and basically encouraging you to smush websites together that did not give you permission to smush them together or because they had an RSS feed, they effectively did give you permission to smush them together. And then we kind of drifted away from that. And I feel like to a degree, we're drifting back to it.

The difference is, is we've spent the last decade or so operating in a more like iOS version of the Internet where things that people have more people are using the Internet. I think that's a good thing. Most of those people have no idea how the Internet delivers or does what they're doing. And to a degree, I don't think they should have to in order for it to allow them to accomplish those goals. But I think the downside is, is we've now got a bunch of people who don't know how any of this stuff works and to a degree, don't believe they have to care. And maybe they don't, but they have actually the same power that you and I do. And that creates a lot of really interesting friction and tension, both at the cultural level, as well as the, hey, like what you're doing actually does have effects on not just the industry, but the people that are building and maintaining the tools that you don't even realize that you're using.

I think this is especially true of the open source world. I think people, a lot of the people that are new to it and we can just like even just zooming in on the world of open source and like I simultaneously have published more open source software in the last four months than I have in the last 14 years. And that's some of the most exciting experiences that I've had on the Internet in a really long time. And I love that. And I love that. And I want that for other people. And I had the early experiences like you were saying before, Wes, is like you put something out there and it becomes even moderately popular or successful. You end up with a pile of slop PRs from people who are even less experienced than me thinking that they're trying to be helpful and making more work than it's worth.

And that just feels like a messy thing that I don't know if it's infrastructure, cultural education and values, or my belief is it'll end up being some combination of the two. I think we were dealing with cultural norms colliding, perhaps, is people who are used to the lockdown of iOS style software and even SaaS to a degree where like it doesn't work the way I want it to and I'm out of options until they make a change to now I can make it do anything that I want, even if it kind of sucks and breaks all the time. And that's what people at least think they want in a short term.

I don't know, maybe the big question that I would riff on your question with, Wes, is how many people that are building those tools and talking about it are still using them two weeks, four weeks, eight weeks, 12 weeks later, let alone a year later. And I'd be willing to bet the vast majority of them went back to old stuff because they were building images in their head that showed up on a screen that is very different from a workflow that is integrated into your work, your life, and actually generates more value than it costs to put effort into.

Right. Yeah. Yeah. I mean, I think that there's there's a tremendous amount of like, you know, vibe coded, effectively abandoned where where maybe you have a project that is like on the order of, you know, 100 prompts or less. And it got you to like a prototype of something that looks interesting, but maybe if like you took a healthy amount of time and exercise like good judgment and taste and really like dug in on the details, like the design and the architecture and the project scope, and you cut out the stuff that's not needed and you build like a really you can build something that's really useful and compelling.

But now like when I look at a new piece of software on GitHub, like I actually look at the number of commits in the project to try to judge like how much how much time did the person spend on this. And I think based on the number of like, you know, I roughly look at like a like a commit like I assume that roughly one commit and it's like one prompt and like sometimes people will have multiple prompts and that will generate a commit but you know, on the order of that and so like, you can look at the number of, you know, commits slash prompts and get a sense of like, did they spend three hours on this? Did they spend 10 hours or 100 hours and so something where, you know, and again, like and is it is it somebody who's built good software projects in the past so based on like, how much time did they spend on it? And is it somebody who has like a track record of building things that are good, like actual good software, especially pre AI, I think are become pretty strong indications of whether like you're looking at something that is, that is useful.

But but I even I found it really interesting. Like I started releasing, you know, AI software projects that I built with AI, I think starting in last October, the first one that I put out there where I was like, full disclosure, like I made this with Claude Code, like, don't judge me, like I didn't write the code Claude Code did, but I wrote the prompts. And and I, I found it interesting, because there were a lot of people that that began to regard the things that I was putting out there with some skepticism where they were like, is this a real project? Like, is Wes going to lose interest like next week and just move on from this? Because a lot of people are just throwing code over the wall on they're putting up, you know, a 30 prompt project on GitHub, and like, hey, look at this cool thing I made, like, then they move on to the next thing.

And, like AI has given us the worst ADD of all time, essentially. And I, myself, like I've fallen into that trap a little bit, but like, you know, but I think sometimes like, you know, like I started building, I started building Robo rev, like a code review system almost three months ago, and I still work on it almost every day. And I talk to people, and they're like, you're still working on that? Like, I'm like, No, it wasn't. It wasn't just like a vibe could have prototype like I this is serious. Like I use this. This is software that I use every single day. And like, I develop it every single day, because it is like become, it's important to me. And but like, not all, you know, not all software we're seeing is where new software we're seeing is like that a lot of it's like, you know, yeah, people are having fun with Claude Code. And that's totally fine. I don't judge it.

But But yeah, like, I think it's created this problem where like, people are like, people's eyes are glazing over with like, 100s and 1000s of new projects that they're seeing every day. And they just don't know, like, is this a real project? Should I pay attention to this? And a lot of people are, they're just like, their brains have turned off and like, you know, are not engaging, because there's just too many new pieces of software to look at.

Returning to long-running projects

Yeah, I'm curious what those two questions, how they like, have you noticed, like, in the past few months, has the way you viewed new projects changed? And what's your perception been of? Will this project exist a year from now?

Yeah, I mean, interestingly, in some ways, I've had kind of not not in the tools I use, but in the things I've been using code for, for almost the opposite, like, I've been coming back to packages that I've been working on for 10 or 15 years, that have accumulated a bunch of stuff over that time where like, the cost benefit ratio has never been worth like fixing these issues. But now like the cost has dropped so much that I've been having a lot of fun just like I've been working on Roxygen 2 lately, which is a documentation system for R packages. Like I've closed like 150 issues in that and closed them by fixing them, not by like saying won't fix. By and large, yeah, I like that I've mostly, mostly fixed them in a way that like just that that like feels really fun and empowering to me and feels like I'm like, right, like that's making the software better. Because I'm just grinding down all these little edge cases and like toe stubbers that people used to hurt.

I think it's helpful to hear because like, like that project is, I think it's a safe to save like Roxygen 2 doesn't necessarily spark joy in you in the way other projects do, but you feel some responsibility or, or it's undergirding a lot of things. Yeah. Like it's infrastructure. It's not like sexy. It's not fun. Like people by and large don't come up to me and be like, Oh my God, Roxygen 2 changed my life. Um, but yeah, like it's, it is, it's clearly like really important and I get a lot of satisfaction from just like making it incrementally better.

I mean, I think what you're describing is also, there's like almost two, there's probably more like a spectrum than two classes, but I feel like there's infrastructure stuff where it's basically plumbing and I just want it to work. And you know, if an edge case pops up once or twice, I'm like, well, at least I know what's there, right. I can either avoid it or build in a workaround that the need to fix like that cost benefit analysis that felt that. And the other end of the spectrum, I think is sort of more like what you're describing Wes and where I've been with, with my systems is it almost feels like a hot rod where I do actually like I'm drawn to work on it and to see and experience the improvement because I am the primary user.

And there's also something, you know, I feel like, I feel like most people who are, are not just software engineers, but makers in general, there is just the satisfaction of making stuff. Right. And I, I didn't realize how much I was missing that until I kind of got back into it. And if anything, the real challenge has been like figuring out the balance of those things where I'm like, all right, I want to build infrastructure tools that don't just make my life better, but also make like my teammates life better or make us more consistent and those kinds of things. And I don't, if anything, I don't want to tinker with those. I want them to be invisible, the best, that's the best version of invisible software, right?

It just does its job. And then I've got the thing that I actually work on every day. And every time I feel one of those little paper cuts of a rough edge and I go, well, I could file a bug or I could fix it now, how does it fix it now?

And there's an interesting discipline to be like, no, no, actually file a bug and then like make a session where you're going to go burn through a bunch of those later that I've had to, I don't know, somewhere between relearn and maybe learn for the first time is something that a more experienced engineer is approaching differently.

I've been thinking about that, like that spectrum and quite a bit and how I think that the majority of that sort of performative productivity software stuff is really just everybody getting a hot rod for the first time. And it might become something that they want to tinker on. But I think people are maybe mistaking one for the other.

Yeah, I mean, I think I think what's my my like my thesis on this is that I believe that what AI and coding agents in particular has revealed is that is that coding itself was always something that was a bit of a like a bit of a mechanical act. Once you have a clear picture in your mind of what of what you want to build. And so you write down essentially you have a mental model of what you want to build and then you think, OK, I need to write these tests and I need to set up like these, you know, these structures and these classes and write these functions and things.

And so like that, that like mechanical execution part of of software engineering is now like can be entirely delegated to the agents. But what's left is the thing that was always the hard part, which is deciding what to build and what not to build and having like the clear conceptual model in your mind of like the system and how it should work and like what are its responsibilities and like what's the minimum viable like functionality to be to be useful and then designing it in a way that is like pleasant and like joyful to use.

But what's left is the thing that was always the hard part, which is deciding what to build and what not to build and having like the clear conceptual model in your mind of like the system and how it should work and like what are its responsibilities and like what's the minimum viable like functionality to be to be useful and then designing it in a way that is like pleasant and like joyful to use.

And so I think like, unfortunately, you know, the if you just let, you know, Claude Code or codecs rip, it's kind of like going out into the into the desert with a with a speed racer and just like pushing, pushing on the accelerator like you're going to go really fast, but like, you know, where are you going to where are you going to end up exactly? And so kind of without like a, you know, like an operator in the loop to like to steer and make those decisions and decide what to build.

Like you could set down two people with completely different backgrounds to work building the exact same thing and, you know, a day later end up with two things that are completely different in terms of like, you know, features and usability and just overall overall quality. It's like same same exact AI models, like very different, very different outputs. And I think that's something that's being lost in the dialogue a little bit because like, I think I think, you know, if you just read the tech media, they're like, oh, AI is transforming everybody into coding superheroes. But like, as far as I can see, like, that's not the case.

No, and I have a lot of exposure to people who are like on the edges of tech and tech companies but are not software engineers themselves. And I think the gap it's interesting because I think two things can be true at the same time. One is I think somebody with the curiosity to learn and the willingness to push through some of the struggle and the willingness to like build stuff that they throw away as a learning project to build the thing they actually want to learn. I think that that bar has been lowered and there's a lot more people who did not think of themselves as programmers or software engineers because they had no training or experience are capable of experimenting and learning more faster.

But I see way more people who look at this and go, I am I don't even know how to think about like, I don't know how to think the way you all think.

I have this kind of working theory. I'm going to test this out. I'm curious how this lands for you.

I think that software engineers and developers more broadly, I think, have a range of skills and traits that we're known for, some more complementary than others. I kind of want to figure out, like, what are the complementary ones that you can learn to think like a programmer, but just the good parts is kind of the way I think about it.

So systems thinking, task decomposition, diagnostics and debugging. I think those are skill sets that are widely universal and would make anybody in any field, creative or otherwise, better at their job, probably more successful, probably more satisfied. But we relegate the training on those skill sets to certain domains, including engineering.

And so what I'm not seeing, everyone's like, yeah, just fire up Claude Code and tell it what you want to your point doesn't really it may get you a feeling like you've gone very far, but you're going to hit a wall pretty quickly.

And I'm curious, if you were going to, I imagine you guys have been in this career for as long as you have to taught a lot of junior developers. What kind of resources do you point people to when they're like, I'm not just learning how to write code, but I'm learning how to think like a software engineer. And I'm also curious if there's any core skill sets that I didn't mention that really jump out is like, now, if you're going to learn a thing, learn these. And here's some great places to pick that up, because I guess I'm trying to kind of compose that because I find myself, those people are coming to me and being like, you're not an engineer, but you, you know, this, can you teach me and I want to kind of be a bridge, like I can speak both languages.

I can be a bit of a translator, but I can also distill best practices from both directions and hopefully help be helpful in both directions. So, so I'm curious what you're going to like experiences or exercises or, you know, books, specific people that you find, like really resonate with people that are new to those kinds of design principles. Cause I really do think that's, that's where the strength and leverage is.

What junior engineers should learn

Yeah, I certainly don't have any answers. I mean, I've been thinking about this from the side of like the data scientists to like, if we kind of assume, which I think is a fairly safe assumption now, like the ability to write code as much as valuable than it used to be, like you still, you're going to still need, like, if you're constructing a visualization, you still need to understand like how a visualization works, what are the parts? So you can like precisely, so you've got the right language to express that.

And, and more holistically, like, I think a really important part of being a data scientist is like skepticism and like traditionally, like the way you learn that is just by doing a bunch of stuff and like getting it wrong. Like, how can we kind of like shortcut that now or like speed up that process? So you don't have to climb up, you know, all the same steps that we did. How do we kind of give you those, like those, those like mindsets and like attitudes more than like kind of knowledge and skills.

So yeah, I would, I'm super interested. Like, I think there's a lot of like, it seems like a really scary time to be like a junior engineer. I think it is a really scary time to be a junior engineer, but it also seems like we, we can overcome that. We can figure this out. Like what are the new core skills? I was going to say, it feels like the door has been kicked down to be like, okay, if you're getting in now, the things that the teachers are telling you to learn probably have changed, but we haven't entirely figured out what to put in its place.

I mean, I mean, I think like the, I expect that the computer science curriculums are going to be become much more focused on like studying less about like to use the analogy to like English and English literature and things like that, like more like analysis of things that have been written as opposed to writing itself. Cause like we aren't, we aren't writing anything anymore. Like I haven't written a line of code in six months, like no joke, aside from like some bash commands and things like that. But, but, but basically like rather than learning how to write Java or how to write Python or how to write C++ or whatever, that we're instead going to be studying like software architecture and design patterns and, and things like that.

And, and, and especially like looking more like structurally, like just more abstractly at like, you know, diagrams of like the way that the software should be layered and how, like what should be the responsibility of different classes and layers of the system so that, cause essentially like we aren't going to be implementing the details of what the components, like how the components work, but rather like we need to have, we need to have the language to be able to describe like the different parts of the system.

And from now, you know, using coding agents, you know, all day, every day, essentially for, for many months now, like I am impressed that they do use a lot of technical language to describe what they're doing. And if you're using like, like a spec, like a spec process framework, like superpowers, if you go and actually read the specs that superpowers creates prior to implementing something that uses very technical language to describe like the design of the system that you're building. And so if you don't have the basis to understand, like, what is a model? What is a view? What is a controller?

Like, what is the data plane? Like, what's the, you know, like, what's the IO plane? Like, what does async mean? Like to have like an intrinsic understanding of when, when like the agent is going to ask you questions, like, how do you want the system designed? And maybe it's inclination, its initial inclination is the right one. But a lot of the time, it's not like it will present you with three questions like, okay, for this layer of the system, like, how do you want to design and it'll give you three choices. And if you don't, if you can't tell them apart, like, why would I choose a B or C?

I mean, how can you if you Yeah, if you have no basis to understand even what the words mean, how can you decide? And so I think, you know, I expect that still, like, these systems are still going to present like the aspects of software design in a way that is like fairly technical and requires like some, you know, software engineering and software design, computer science background. And so I think computer science curriculums and like junior engineers need to invest in like, you know, just the basics of learning all that language so that, you know, when the agent asks you a question, like you, you know, you know what the words mean.

Going back to fundamentals

To tie this back to an early question from you, Michael, about business books, and I think this is probably fairly relevant here too, is like, I haven't, I stopped reading new business books, and I like business podcasts, like I've tuned most of it out, because I haven't heard a new thing in a very long time. There are specific voices I listen to, but the broader media landscape is just regurgitation.

Most of the things that are meaningful today were written 20, 30, 40, 50 years ago, because they're written about humans and human brains, which, while there's certainly been transformations, understanding those things is really at the foundation of everything that comes to like good experiences, usability.

You know, if you're getting into this now, whether you're a brand new engineer trying to figure out what to learn or someone who has decades of engineering experience and trying to figure out what translates, look to books that were written 20 years ago. Fundamental engineering principles, we had to figure out in order to build giant tech companies pretty early, and the details have shifted for sure, but the fundamentals haven't changed very much.

turns out that it generates way less garbage code, the entire system becomes more maintainable, and if you need to collaborate with any developers besides yourself, those problems become inherently solved rather than solved just for agents. And so I think it's like, it is this interesting tension between brand new frontier and, no, we shouldn't throw away the fundamentals. We should maybe choose our fundamentals intentionally. But like those fundamentals can serve us well.

If anything, they might even be a significant, like a fulcrum in a way where like you having those engineering skills is what will make sure that using a coding agent doesn't generate unmaintainable garbage, doesn't make your co-workers lives a nightmare, doesn't put you or your users or your career at risk. So it is interesting, you know, what are the layers of that that will permeate outside of engineering culture and into broader creator culture as people are using these tools for things to make things other than software.

The bothness of it

I don't know what that looks like, but that's the kind of stuff that I'm paying attention to as someone who like, again, is, I feel like I'm, I've got, I feel like I'm standing on the border of two countries with one foot in each side and being like, it's not that I am in neither, it's that I am actually quite literally in both. And I can't remember a time career wise for all the multifacetedness of my career that I've felt that experience so viscerally because the bothness of it seems to really matter.

And the bothness almost seems to be like, it's so, it's more valuable today than it ever was before. Like I think that the people with those kind of, like if you, yeah, like as the cost of code producing code decreases, like the value of like, what, what should we do with code? Like that increases.

And the bothness almost seems to be like, it's so, it's more valuable today than it ever was before. Like I think that the people with those kind of, like if you, yeah, like as the cost of code producing code decreases, like the value of like, what, what should we do with code? Like that increases.

And also what, what shouldn't we, I mean, I think your point before about like being skeptical, like, I mean, there's a lot more that can be said about, you know, broad cultural shifts away from people being skeptical of the information in front of their faces. And then the terror that then the landscape we're in right now, we're like, it's more important to assume that things aren't real and then prove them real than the other way around. We're not built for that. And we certainly don't have like tools or training for it.

So we're in this phase where I feel like any of us that are on the bleeding edge, like we're building a muscle where, you know, I'm working with Claude Code and it'll say a thing and I'm like, I just don't think that's true. But I think for a lot of people, they don't have that muscle to say, I don't think that's true. They just accept it. And, you know, again, zoom out to the rest of the world and how many times a day people could look at a thing and go, I don't think that's true, but instead accept it at face value. And how

much of the world's problems can be traced back to that? I don't know. I feel like if we solve this, we solve lots of things. And I think it makes it a worthy pursuit to remind folks that like asking questions and being skeptical might annoy your coworkers, but doesn't make you, it doesn't make you better for not asking the questions of anything like science. Again, as it goes back to like so many people misunderstand science and experimentation is like proving things true. It's like, no, science's job is not to prove things true. It's to prove things false so that only one truth can remain until we have new information.

At least in America, most people like either missed that science class or plainly forgot it. But it applies to, I think, again, it's sort of at the root of things like taste and judgment that feel squishier, but at the heart of it, it is the, somewhere between the willingness and the ability to ask the right question at the right moment. And if you don't know the answer to have a system to find it.

I think that almost, that reminds me too of something Wes said before we started to like reviewing PR is that maybe like one thing he helps bring to some of the tools is like a certain level of perfectionism that I wonder if that also entails some like question asking that as a virtue, like a person with, with like really high standards for their tools or something, then also are kind of compelled to ask questions about what's going on with their reviewing PRs or alternative options.

Yeah. I think people just accept things like that's the way it was. That's the way it has to be. And I think there's something magical about having any creative skill to be able to imagine a possibility beyond what you've been presented. Like that's a skill. I think everyone can practice and learn. It's a learnable skill. I don't think any of us are, are, are born with that versus another one who's not, I think people have it beaten out of them or people have it enhanced. But, but yeah, I mean, it's, it's another side of the same coin is to look at a thing again, like my tolerance for other people's decisions, I think. And so I'll give you a concrete example. Like there's some, some bad, you know, administrative software user, like the membership software that we use at Indie Hall, Michael, you've heard me complain about it endlessly. And we have a couple of other tools that like somewhere between I can't get rid of, or it would be very difficult to get rid of.

And now I'm looking at them going, do other people who use and pay for this software just not realize that it could be better? Like, am I, is my demand that out of line or more specifically is that most people's bar is just so low. And I mean, it could be both, but I don't know that that's, I feel like there's something in there that, that is both confusing and frustrating to realize. Sometimes software companies are being careless and negligent, but sometimes I think in these particular cases, if their user bases are non-technical in particular, the bar for, well, that's just how it is because they don't even realize how it could be just goes to the floor.

The catharsis of building your own tools

I mean, I don't know about, I don't know about you all, but I think for me, like, like one of the feelings that I've had, like, especially over the last six months is this, like, it's this catharsis of being able to assuage pain that I've experienced over such a long period of time. Like I've been, I've been, you know, I graduated from college almost 20 years ago. And I feel like I've used so many software products over the years and like as a perfectionist and somebody who's really passionate about usability and designing tools. And I know like Hadley is like the ultimate like programmer ergonomics, like, you know, like famous for, for his, like, everyone loves the Hadley verse and like the tidy verse because of his ergonomics. Like that's why they come, that's why they stay. And so like whenever I would use a software product and it's like badly designed or it seems awkward, I'm like, somebody sat down and they wrote, wrote this code and they decided that like the state that it is in right now is okay to ship.

And so, and now like to be able to take things that we, the software that we've used in the past and to say, like, I can reject this piece of software and make something better and I can do it right now. Like that is like this tremendous cathartic feeling of, of being able to like, yeah, essentially do something. And in the past, like you couldn't, you couldn't do that because you would, you would have to like raise money and like hire a team. And like, you know, it was just this whole, just heavy lift to, to be able to do something about, about anything. And so you would have to like pick and choose your battles of like, oh, we have to accept this media piece of mediocre software. Um, because like building something that's better, like just replacing it or spilling something that's a little bit better is just like would be an impossible lift would cost, you know, a million dollars and six months or two years or whatever. And so like, yeah, just to not have to live like that anymore. My gosh, like it's like life-changing, like it's a tremendous time that we're living in, but you know, I, I think once we get through like this catharsis of like, we can build the things that are good and like, we do not have to accept the mediocrity of, you know, software and vendors with bad taste anymore.

Like that is like this tremendous cathartic feeling of, of being able to like, yeah, essentially do something. And in the past, like you couldn't, you couldn't do that because you would, you would have to like raise money and like hire a team.

Like it's, uh, it's amazing. It's truly, I think it's a really interesting question. Like some of that, like, yes, like there's a lot of shitty software out there, but there's also a lot of it. Like that software is not right. Like for me and like that there's definitely things I'm like, this is how I want it. And I a hundred percent accept that probably no one else in the world, like either has this exact problem or this or wants this exact UI. So I think it's going to be really interesting to see like how much, like just what, what happens as this, this explosion of kind of like personal, like user interface, like, are we going to, is there just going to be this massive diversity of tools? People are mostly creating their own tools for themselves, or are we going to start to see it kind of like drifting back to towards standardization?

I like looking to other industries for clues. And I feel like this is maybe a weird comparison, but I feel like the arc of fashion is maybe an interesting one to look at where it spent most of its history bespoke. And then for a relatively short period of decades became mass produced in the interest of reducing costs and increasing accessibility. And now we've kind of found our way around and there's like some interesting options in the middle where I can buy a mass produced thing, but then take it to a tailor and get it personalized. And I'm, and this is like some of the space that I feel like I'm playing in where, you know, for instance, you know, I'm not going to call them out by name, there's a piece of software that they're a source of revenue for us. So giving them up is very difficult, but their software is objectively some of the worst software I've ever used in my entire life and creates a lot of frustration. And my business partner and I complain about it to each other all the time. We have nobody else to complain about it with. And I got frustrated and I said, you know, my, my Claude code robot has access to a web browser. Maybe I can just teach this thing how to log in for me and do the handful of painful, frustrating tasks. So we only have to even log into the software to do edge case stuff. The vast majority of what we can do from a custom UI.

And I basically built like this, this veneer layer that lets us report information that is useful to us in a way that is way easier to look at and only tells us what we need. And when I push a button, it may not respond instantly, but it throws it into a job queue. And then a little robot, you know, logs in, walks around, does what it needs to do. I have it taking screenshots and writing a written audit trail of what it's doing. So when there's mistakes or problems, I can, I can audit it. It is self-learning. So when it bumps into edges, it can report, say, Hey, I found this problem. Here's what I did to fix it. But you might want to look more closely, those kinds of things. And there's so many things like that, that like my goal is not to build software. And my goal is to make a problem that was a daily or weekly, like complaint session vanish as entirely as possible. And inside of six months, I can point to like, I'd say once every other week or so I can go to Adam and say, Hey, that thing that was a problem doesn't, the problem doesn't exist anymore. And to the point you both made is like that, that feels really good.

Agentic coding for smart home control

Another one that was pretty funny was, so we use smart thermostats and I feel like the intersection of like agent, agentic coding and smart homes is both a very exciting and kind of terrifying land to be exploring. But we use Ecobee thermostats and we had a problem where a particular, the problem we can't solve, like the duct work is already run. But if I've got a meeting room that heats up and running the AC full tilt to cool it down is going to freeze out the outside space. I need a little more control to A, monitor the temperature and to cycle it between cooling and not. And doing that manually was literally impossible with the app. Even if I could sit there and press the buttons for the six hours that the person is in the meeting. And so I was like, Hey, Claude, does Ecobee have an API? And it's like, well, they do, but they stopped giving out developer keys a couple of years ago and people are not happy about it. And I was like, that's understandable. And then I said, well, can you log in with the browser? Cause you've done that before. It's a gadget, need your credentials. And so I store their credentials in my little credential store and it goes and does its thing. And it's like, here's all the buttons I can press for you. And I was like, cool. While you're in there, are there any internal APIs that you can access now that you've got a valid session? It's like, yeah, all of them. What do you want? And so it was basically able to build a CLI for a tool that didn't have a CLI, have granular control down to satellite thermostats that you normally get the thermostats making choices based on the average of multiple sensors. I can drill down to a single sensor and have it write a little JavaScript loop that just checks that sensor every five minutes to see if the temperature is increasing or decreasing. And based on what's happening, cycle through a couple of different cooling profiles until it solves the problem. And if it can't solve the problem within 30 minutes, it sends me a message and says, Hey, there's a problem. That was a pain point. That was an entire constellation of pain points that I solved in about 30 minutes while they were happening.

I love that. Essentially we've talked about like replacing the app itself. And in this case, it's like replacing yourself as the user. There's like two ways to swap something out.

Karaoke as community

But I figured I didn't want to end without some fun facts that you said. So I learned something a bit surprising about you, which was, and honestly, I'm disappointed that I've never been witness to this, but you're apparently a karaoke buff. Is that? I have a bit of a history with karaoke. I would say I'm not a particularly good singer, but karaoke is my sport of choice.

That just makes it more intriguing. Yeah. What are you competing on and are you winning? You know, so I have competed. So, I mean, it started as a casual hobby. Our local watering hole, National Mechanics is just an amazing bar and restaurant that is a second home for Indy Hall. In a lot of ways, Indy Hall wouldn't exist if that bar hadn't. And I just became a regular at their karaoke night. And a community kind of grew around that. But in 2009, I think it was at South by Southwest Interactive Conference, there was an interactive agency that used to be called Happy Cog. Happy Cog might still be around. I'm not actually sure. But Happy Cog was hosting a big karaoke competition. They did a pre-festival bracketed like playoff on social media to even have a chance to perform. And so if you had a decent sized Twitter following, you had a bit of a leg up and I was pretty active on Twitter and was able to get placed. And then I placed second, singing Beastie Boys, Fight for Your Right to Party in front of about 800 people at South by Southwest Interactive. And had one of my internet heroes come up to me the next day and the convention center and say, he's like, you're Alex, right? And I was like, yeah, he goes, you sang Beastie Boys last night, right? He goes, that rock man, and then walked away. And I was like, I'm good. So that's karaoke. Placing in a karaoke competition. I'm pretty proud of that.

The other side of the karaoke experience is that I also inspired by South by Southwest shenanigans. Me and four friends turned a Winnebago into a rolling karaoke lounge and drove it 4,000 miles from San Francisco to Montreal using our network of coworking space relationships at the time as hubs to land in those cities and throw a pop-up karaoke event in each of those 13 cities. It's the closest I've ever been to being in a band and going on tour. It was so much fun and I still like karaoke and I still like those friends, but this is not as abstract as it might sound. I actually think of karaoke as an active community.

I went to a friend's birthday recently who had rented out a karaoke room in Chinatown here in Philly. And I was like, I'll go because I want to hang out with my friends, but I prefer a room full of strangers than a room full of friends for karaoke. And for me, the reason for that is I think most people, if they've had a negative karaoke experience, they think that it's somebody just like singing really badly or just like ears grating. And I'm like, no, most bad karaoke is not bad singing. It's selfish karaoke. It's about song choice. It's about choosing a song that nobody wants to hear, but you, it is the inconsiderate karaoke versus if you're nervous to sing or you want crowd feedback, song selection matters a ton. So getting a sense of the room and, and inviting people into your song and making song choice, something like, well, what do they not just, what do they want to hear? What would they sing along to as part of my karaoke philosophy? And, and perhaps just part of my, my community and business and education philosophy is like, I'm not going to make you do a thing you don't want to do. What would you want to sing along to? And how do I create an environment where that's possible?

most bad karaoke is not bad singing. It's selfish karaoke. It's about song choice. It's about choosing a song that nobody wants to hear, but you, it is the inconsiderate karaoke versus if you're nervous to sing or you want crowd feedback, song selection matters a ton.

Karaoke is weirdly a through line through, through all of those things. Yeah. I appreciate you making karaoke a form of community. I'm the opposite. I find karaoke is adversarial, you know, I'm trying to work against the best interests of the people, but to each their own, you know, we'll go out, we'll go out sometime. We'll figure it out.

We'll figure, we'll figure out song selection for you and we'll, we'll turn you around. Yeah. We'll, we'll report back. Yeah. Hadley and Wes, do you have favorite karaoke jams? What's your relationship with karaoke?

I do love, I do love karaoke. I'm not, I'm not good. I mean, so I'm like, you know, one of those enthusiastic, bad karaoke singers, but I was when I was in my, when I was in my 20s living in, living in New York, like we used to go to go to K town and go karaoke on a regular basis. You know, when life was, life was simpler.

She was, it was at a place called chorus where the guy would occasionally come out with the saxophone. Uh, I'm not sure about that place. Yeah. Just to do like, you know, private room, like private room karaoke. Maybe I've probably been to chorus actually. There's, there's a whole bunch of them there on, you know, 32nd street. So yeah, good, good times.

Hadley, have you jammed out to karaoke before?

I'm just, I don't know. I hate being bad at things in public. So I have been like tempted to like, get like a singing coach specifically. So there could just be like one song I could belt out at a karaoke. And then people will be like, Oh my God, Hadley is an amazing singer. But it would just be like that one song, that one song.

Like there are songs that I choose because they're fun to sing their songs that I choose because I think the audience likes them and their songs to choose because I'm like, I know I can nail this song. And that's a much, much, much shorter list. I do think it is a valuable life skill to have that one song, even if it's not with a vocal coach. I think it's to your point, Wes, it's about more about the confidence is like, are you able to show up confident and enjoy it? Then everyone's going to have a good time.

No, that's, that's a great, I feel like take home message. I feel like that's the end on. No, Alex, it's been so nice. And, you know, having a fellow Philadelphia and it's such an honor. I feel like I should drop every Philly thing, all the Yos, the Johns, the water, we'll grab a water ice after this. I'm not going to explain what that means. They'll figure it out. Ask, ask Claude Code what Philadelphia water ice is.

Yeah. Really appreciate all your wrangling community and, and coming on. Yeah. Great. Thanks. It was really nice to meet you guys. And thank you for having me. It was great.

The Test Set is a production of PositPBC, an open source and enterprise tooling data science software company. This episode was produced in collaboration with creative studio Adji. For more episodes, visit thetestset.co or find us on your favorite podcast platform.