Transcript#
This transcript was generated automatically and may contain errors.
Welcome to The Test Set. Here we talk with some of the brightest thinkers and tinkerers in statistical analysis, scientific computing, and machine learning. Digging into what makes them tick, plus the insights, experiments, and OMG moments that shape the field.
Michael's traveling at the moment, so you're stuck with a producer's voiceover this week. On this episode, The Test Set crew jams with Caitlin Colgrove, CTO and co-founder of HEX, the data platform that somehow counts a sweet green head chef among its power users. Caitlin joins Michael, Padley, and Isabel to make the case that data teams aren't going anywhere. But right now, their job is changing from writing SQL to vouching for the answers. We hope you enjoy.
Caitlin, welcome to The Test Set. Thank you.
Thanks for coming on. Just by way of introduction, you're Caitlin Colgrove, the CTO of HEX. And I'm Michael Chow. I'm the host of The Test Set. And I'm joined by Hadley Wickham, chief scientist at Posit, and Isabel, software engineer extraordinaire at Posit. And we're so excited to have you on.
I feel like HEX is everywhere today, and I'm constantly trying to explain to people what HEX is and why they should use it. I know HEX is included in cloud.ai now. Maybe do you mind just explaining a little bit about what HEX is and what it does?
What is Hex?
Yeah, totally. HEX started when we started as founders. We were basically, we'd all left Palantir and we're kind of out in the world looking at how people were working with data. And what we discovered was that there were a lot of new patterns for working with data that were emerging. People were a lot more technical. There was a lot less clicking of buttons and a lot more writing of code, SQL, Python, all of that. And the tools out there were Jupyter notebooks. Those are very popular. This is 2019 or so. But in order to actually sort of string together a full enterprise data workflow, like to actually do something useful with your data, people would be doing these ad hoc analyses in Python, SQL, do a SQL in a notebook. You still had to do the backticks. It was like really not native.
And then that was in a notebook and it was a locally running notebook with your own Python environment. And okay, so maybe you do this whole analysis and what are you going to do? Are you going to email that notebook to the CFO? That, you can't do that. So they would take these notebooks and they would, okay, they're like, okay, we're going to turn it into a Python script and then we're going to run the Python script on a schedule on AWS. And then we're going to dump the output into Looker so that the CFO can look at it in Looker so that they can take a screenshot of the chart and put it in the board deck. And that was the workflow. And so Hex was really born of this idea that there just has to be a better way, that there needs to be first-class tooling for this new era of data work. And that was the original product that we built. And then obviously like, you know, in the past couple of years, AI has become a huge part of that. And I think even though some of the technology has changed, the underlying story is still true. It's people who are working with day in and day out. What are their workflows? What are the best tools for them? And Hex should be that.
I've seen so many engineers in Python use like Pandas read SQL, that classic, like Pandas dot read SQL with a string of SQL in it, which feels like maybe a little bit unnatural. More than a little bit unnatural. Yeah, yeah, yeah, yeah. And that scheduling dynamic, just all these bits of duct tape kind of like in the process. I'm really curious, I know like with the anthropic integration, like what does that look like for a person to use Hex?
AI and the Anthropic integration
Yeah, so I think this is one of the big ways that work is changing. Pre-AI, you basically had these big heavyweight vertical SAS tools. And depending on what you wanted to do, you would navigate to HubSpot or you would navigate to Salesforce or you would navigate to Hex because you're like, I want to do something with my marketing data. I want to do data science, data analysis. And that would be where you would start. And so you'd have these big full featured front doors to these vertical applications.
Today, everybody just starts by like asking Claude or ChatGPT a question. And we've actually had some really, really interesting conversations with the Anthropic team and the OpenAI teams about how they're seeing the changing and sort of the emergence of these new workflows. And they really view a lot of these chat applications as a front door. They're not going to go and build a first class data science agent or a first class customer service agent or all of these things. They want to be the platforms on which those things run. But still today, it's very sticky, these habits of just ask your question to Claude.
And so what Anthropic released, and ChatGPT has sort of its own version as well, is this plugin architecture. And it's based on MCP, as you might imagine, but they've enhanced that with more UX around it. So now if you hook up your MCP to Claude, you actually get a interactive Hex experience in Claude. And Claude, you know, the models are actually smart enough to know how to route. So if you go and you have Hex wired up as your sort of data science backend, and you ask a question, Claude knows that the Hex plugin is the plugin to use to answer that. And so it'll actually go and it'll go kick off sort of a sub-agent architecture where it kicks off an agent in Hex, which will go look at all your data. It'll go look at all your semantic models. It'll do all this stuff and it'll come back. And it's actually really nice. You get to sort of see the thinking tokens. You can get an interactive set of charts back. And then if you really want to go deep, or you want to go build a dashboard, or you want to do something further, you can kind of click through into the underlying Hex project that it's been building up behind the scenes.
What's a question I could ask that might produce a Hex widget? Like if I wanted to check the weather in different cities, or like, let's say we just had, yeah, like... What's something you have used Hex to answer about Hex?
Well today, so Hex is typically most useful for the types of enterprise data workflows where it's all governed in the data warehouse. You've probably done some modeling on top of it. You probably have some dashboards, you know, maybe you have your KPI dashboards or whatever. It's like sort of classic data workflows. And then you can start to pull in other sources, obviously via MCP and other things. But it's very similar to, you know, like, you know, pre-AI, the types of things that you'd be managing with dbt and Snowflake and all those things. So it's usually business data. And if you're syncing other sources, those are to enhance the questions that you're asking about the business data. So these days, I'm asking a lot of questions about how much we're spending on Anthropic. Where are those tokens coming from? How are people using these products? And so that's a very, it's a sort of, I would call a combination kind of finance product analytics type workflows. And so basically what we're doing is we're piping all of our logging around the usage of Hex's AI into Snowflake and all of those like fine grained token counting and, you know, attribution and where those are coming from. And so I'm spending a lot of time looking at, you know, where are we spending the most tokens? Are there places where we could be being more efficient? We also have a lot of data around, you know, our internal evals and performance. And so are there areas where we like should be using Opus and we can't actually drop down to something like Sonnet or Haiku? So I spent a lot of time looking at that in Hex.
Confidently incorrect — model behavior in data work
We know that like models can be like pretty sycophantic. Like how much do you worry about like when you come in with a question and there's like an sort of an answer that like you want to hear? And then like the, you know, do you worry about you kind of accidentally steering the model in that direction that just reinforces like what you want to hear?
Yeah, I think, I think that's actually one of the bigger problems, like model behavior is one of the bigger problems in sort of agentic AI for data science today. And it's a combination of that. It's, it's, I think it's a less of the sycophancy side of things that we see, but we see a lot of confidently incorrect. So the models will, instead of doing what even a relatively junior data scientist would do, which is you start looking at the data, you start exploring, you get an answer, and then you check that answer. Maybe you look at some other, you know, sort of other sources to see if you can corroborate what you're seeing. Models don't do that at all. They find a table, they're like, ah, this table has a column that looks like it should be useful, and then they just go. And so that's one thing that we're thinking a lot about is like how do we give the models, how do we push them to explore the space more? And how do we push them to sort of self-validate in, in a way that more likely will, because often, and you see this even with humans, right? If, if you start at different points in the data and there's different ways and different assumptions that you might make about how to answer a question, you can end up with just wildly different answers. And you kind of want to look at all of those and understand, okay, well, what's, what's the actual sort of truth here?
I think it's a less of the sycophancy side of things that we see, but we see a lot of confidently incorrect. So the models will, instead of doing what even a relatively junior data scientist would do, which is you start looking at the data, you start exploring, you get an answer, and then you check that answer. Maybe you look at some other, you know, sort of other sources to see if you can corroborate what you're seeing. Models don't do that at all.
There's a lot of knobs to turn when making sure that the model is, you know, being a little bit more creative, maybe thinking a little bit harder about the answers. What are maybe the knobs that you're thinking about in HEX to make sure that this is being solved?
Yeah, I think the models these days are very, they're, the reasoning steps are actually quite good and quite solid. And it, you know, if you rewind a couple of years and hallucinations and just like them not paying attention to the things that you want them to pay attention to are huge, huge problems. And even just keeping them on task. These days, I actually find that they're quite creative in taking the pieces of information that they have and putting them together into an answer. Like that's actually, it's actually like really impressive when you read their thinking traces, how much they're able to like hit a roadblock and overcome it and like cycle through that in order to get to an answer. Where I usually see the problems happening is, as I was saying before, in them having not sufficiently explored the space. And so they're very, very good at coming up with an answer based on what they have, but if they don't have all of the pieces that you need them to have, then that's where they'll, you know, that's where they'll kind of fall on their face. They may even have an answer that looks plausible, but it's actually not, not what you want. Or maybe they'll, you know, they won't, they won't even get to an answer. So where we, what we're thinking about is that exploration phase. And you, and you see this in, in other modes too, like in coding, you have like a, kind of have like a plan and design phase and then an implementation phase. That's something that we're thinking about how we bake into our models where you don't necessarily just like start from the question and have them go at it. You actually kind of push them to look harder for things before they come back and start to actually then go and do the analysis.
A closer look at the Hex notebook experience
Just to be sure, I'm, I'm going to try to paint in exquisite detail what it looks like to use Hex just because it's, it's such an interesting, cool experience. But I also think some people, when I talk to people, some people listening, I think might have trouble if you've never used Hex, like just conveying it that, and correct me for anything I get wrong, but if I understand, it's kind of like you have often like data in the warehouse and say like an analytics engineer, or some people have tried to transform and kind of curate what you need. And you might even be a person who doesn't do a ton of data analysis. You know, you might be working in like finance or not, no shade on finance or like another area and essentially you go into this notebook that's wired up to your warehouse. And right now you're able to ask an agent essentially a question. And then what happens is it starts populating cells that run and produce outputs. And some of these outputs even are like filters. So it might be like a dropdown to select like something you're interested in, or to filter a company that you might want data on. And then essentially like you can adjust the filter and everything else in the notebook will kind of update. So you can sort of like build this notebook and explore your data at the same time. But then what I think is also interesting that I just want to make sure to mention is, if I understand you also have an app mode, is that right? Where you kind of shift from seeing the code in the notebook view to the app mode where the code sort of like goes away or is more hidden?
Yeah. Yeah. That's all totally correct. I think you can think about it in Hex as the backbone of the product is this notebook. And we were just chatting about this earlier. That notebook actually under the hood is essentially a graph of transformation nodes that we can run behind the scenes. And then you can take that notebook, which we call project in Hex, and you can do a bunch of different things with it. So you can manipulate it with the agent. So the agent will actually like edit that graph. It will build up that graph. And you can do that, you know, just to do sort of some exploratory analysis. But often what you want and often, and this is actually one of the huge pain points that Hex was solving in the early days is, you know, data workflows don't end with getting a number. They end with making a business decision. And so the way you get from, I have an answer from the data to we've actually decided to do something. There has to be sort of a collaboration and presentation and sharing step. And so what we've done with Hex, it used to be excruciatingly difficult to go from those analyses to that, you know, screenshotting charts into PowerPoint and things like that. And so what we've done is we've actually layered a presentation layer directly on top of the notebook. So it's exactly the same. And, you know, it's a typical more dashboard like experience, I would say. We call them apps because they can be more powerful under the hood than what you would expect from a traditional dashboard.
And then you can also really extend this. So this has actually like been such a versatile paradigm for us. So Hex has the notebook and the notebook agent, which is really geared towards technical people. It's people who want to be in the code, even if the agent's writing it, they want to be looking at it and working with it. But there's a lot of people in the organization who don't want to look at the code, but also still want to be able to take advantage of the power of the Hex agent. And so what we have is we have our conversational agent, which we call threats. But under the hood, it's actually the same as the notebook agent. And so what it's doing… I love it puts a different hat on. Yeah, it's a skin. But it's actually building up a notebook behind the scenes. And this is really, really powerful because it means that if you get to the end of this conversation and then you want to do something with it, maybe that thing is share it. Well, great. You're in a notebook. You can make the same types of dashboards, things like that, that you could before. Or maybe you're like, hey, I don't know if this is right. I actually want to send it to the data team to check my work. Well, the work's all there in the notebook behind the project.
Yeah, Hadley, I know you… In R, it's super different, right? Like the analysis, the kind of workflow versus a notebook where you're sort of like, this workflow is so great. If I'm understanding, it's kind of like, whoops, I made a dashboard. You do your notebook stuff, but then you have a dashboard at the end.
Yeah, I think it's really interesting because the sort of heritage of R is this like REPL, this run, eval, print loop where you're in the console trying stuff out. And that's kind of your primary means of exploration and experimentation. And then as you discover things, you kind of believe to be true. Like that's when you record them into a script. And that's kind of what led to R Markdown and Quarto, which is like, you've written this mingling of pros and code, like much like a notebook, but you then you run the whole thing and that produces your output. And that has some like big advantages, like kind of pre-reactive notebooks, very easy in a Jupyter notebook to kind of get into this state where you've like run some cells in like different orders and there's no way to reproduce what you've done. Like no way to have that problem with R Markdown because you always run it from scratch. Like there's a different set of problems, obviously a different set of trade-offs, but like that's the one problem you never experience. And then I guess there's often a dashboard phase after.
Yeah. And so the thing that you produce might be a document, it might be a dashboard, it might be a presentation. Like I think that that same like underlying thing and you're putting on like different hats for different people. That's just, that's like such a valuable metaphor. I mean, we kind of often joke in engineering, I guess, like, oh, that's a DAG. That's a directed acyclic graph. Like everything's, oh, that's an immutable DAG. That's a reactive DAG. Like, oh, wow. What a surprise. Like we've created a DAG and it turns out to be the right model. It's just. And sometimes so at, obviously the hex notebook, as we've been talking about behind the scenes is DAG. And as the, as the cells within the notebook got more and more complicated, we started to realize that the cells themselves needed to be DAG. And so then you have DAGs and DAGs and DAGs, it's DAGs all the way down. And actually, you know, what is a DAG of DAGs? It's just a DAG. Yeah. It's kind of like every, it feels like every problem with an LLM you solve with like another LLM, like, you don't know if your LLM is doing a good job or not. Well, let's have an LLM, a different LLM judge whether it's doing a good job. And it's, yeah, it's like turtles all the way down.
Anthropic and Sweetgreen as Hex customers
And another thing, just, just to not to keep buttering up Hex, but OpenAI, they released their, one of their reports was using Hex. Is that right? Like a state of data? Uh, probably Anthropic. Anthropic. Okay. Yes. I gotta be careful about mine. Yeah. Do you recall what that was? There was like a… I don't know specifically what you're referring to. There's a few times where we've seen Anthropic kind of use Hex in there. It like pops up. Yeah. Because, you know, they're, they're Hex customers. Yeah. And sometimes it's fun to like watch their live streams and like spot the Hex charts. Yeah. Yeah. For Hex. Yeah. Yeah. That's cool.
And I know, I know you mentioned, um, Sweetgreen as an example. Oh yeah. This is a recent one. Um, Hex, Hex is like the whole spectrum of customers. You know, it's like very traditional data science all the way to some of these use cases that you would just never imagine. Um, so Sweetgreen just signed on as a Hex customer and this was the wildest one that I've, I've heard in a while. They use Hex to monitor the temperature of their ovens so that they can correlate customer complaints about the temperature of their food to, you know, what's happening on the ground in their restaurants. It was wild. Every now and then you come up with, you hear some of these things from customers. It's actually one of the really fun things about building a product that's so flexible and so horizontal. Did the head chef have like a lot of data experience? Like what's… You know, I'm not, I'm not sure. Um, I, you know, I, I actually haven't met him, him personally, but this is one that we heard kind of from our field teams.
Making data accessible to non-experts
Yeah. Like one of the things I've sort of been, you know, thinking about like how, like as these, you know, people who never would have, you know, worked with data in this way before, cause they couldn't program like, and now they can because they can write in English. But, you know, there's a lot of like vocab and like ways of thinking about data that you don't, you, you don't have and you can't express in everyday English. Like how do we start to like train those people up to give them like the basic kind of like that kind of gut feel for like, I should be a little bit skeptical here or like, oh, what I, what I need right now is like a smooth or like I need to tidy data or how do you all think about that?
Yeah. So I think this is an unsolved problem broadly. Um, and that I think building these agents to behave like a data scientist, um, truly, I think we haven't gotten there yet and there's a lot more work to be done there, but I actually think that this is one of the big breakthroughs broadly in terms of, um, uh, you know, AI for data and really enabling a much broader spectrum of people to work with data because yes, there's, you know, the actual, can you write code aspect of it? But if you look at sort of the prior generation of data tools, even ones where you don't have to write any code, people still don't use them. Um, like the lay people in the organization do not use them. Like we talked to a Looker, uh, an ex Looker customer, another hex customer. And they said, you know, they bought Looker for everybody. And in practice it was like, they looked at the last 90 days and only two people in the company had made like new dashboards in Looker. Um, and when you like unpack that a little bit, it's just buttons, right? You don't have to write code. How hard can it be? It's actually that doing data work is just inherently very complex. Um, and so none of these tools code or otherwise, uh, more walkup usable. There was just too much foundational knowledge that you needed in order to be able to be effective in the tool. And so one of the big breakthroughs besides the, you know, LLM writing code bit is actually imbuing the agents with that knowledge.
And so one of the big breakthroughs besides the, you know, LLM writing code bit is actually imbuing the agents with that knowledge.
Um, and today that's already a lot of what we're doing with the hex agent is like teaching it, not just like, you know, how to write code, but you know, how to work with the data, you know, the, your tribal knowledge around the organization of what data is where all of those things you bake into the agents. Um, and so you're starting to see this pattern across, you know, across agents in general purpose, uh, models with skills and things like that. Again, I don't think we've totally nailed it because I think even with a skill, um, some of these more specialized domains like data and data science, the models haven't been post-trained on it as, you know, as rigorously as I have been with coding. I think there's like a, you know, still a fair amount of headroom, but it's like, even if it's, you know, 60, 70%, it's still massively better than someone who's never really thought about how to work with data before. So our internally at hex, some of our, we have, we have this feature that we call guides, which is basically basically skills. Um, and some of them are like, you know, this is how this data set is shaped and things like that. And some of them are, you know, this is how we do data analysis at hex. These are the types of things that you should do. Um, and the models are pretty good at adhering to it today. But again, as we've been talking about, if there's any kind of like judgment or anything that they would need to off-road on that, that they're like much, much less good at.
It is. And like, I think one of the things a good data scientist has always done, like you come with a question and then the data scientists like helps you turn that into a question that you can actually answer, like with the data you have. And like, you know, if you just go into the data warehouse and you just start asking questions, you're like, you, you just don't even know like the right questions to ask. And that's such a valuable kind of, and we're starting to get that a little bit with LLMs, I think they can at least help you brainstorm and kind of say, oh, hey, you could ask this instead. But yeah, it's, it's, it's interesting, interesting time to be a data scientist.
Agents learning from existing data work
The other thing that we do in Hex that I think is very powerful, um, is the agents have access to not just the underlying data warehouse, but the other things that people are doing with the data. Um, and so some of this is going to be garbage, just like, you know, people not knowing what they're doing, but, um, you know, you may have that company dashboard that gets looked at, you know, thousands of times a day or whatever. And that is actually a very rich source of context for the agent for how you should do something. And so when a user comes in, it's like, I want to know something about revenue. They may not know anything about how we actually calculate revenue, but that's actually encoded in various places in your data workspace that the agent, which is, yeah, which is because I mean, that's also how humans work, right? Often that's not documented. You have to go to another successful project and like copy and paste that code. Yeah. Yeah. Yeah.
That was an interesting point. I think I saw a poster recently. It was like this idea of maybe like some of your best documentation is just the other projects around and that the agents it's, if I understand y'all made it so the agent can essentially like look across projects so that if it doesn't exist, like in the documentation or in the models, it can sort of figure out how someone else maybe like went off road or use some kind of like knowledge in their head, um, to do something. Yeah, exactly. I mean, the idea here is it's always been extremely difficult and basically impossible for humans to exhaustively and thoroughly document their entire data ecosystem and then maintain that and keep that, keep them up to date. So what are the other places that you can go to understand, um, you know, how to use particular parts of data or how to do particular types of analyses. And, you know, it's, it's a range because when you have, when you have the documented context, um, for the most part, that's like highly trusted and that's something that the agents can just rely on. But then everything else is this range from, you could go all the way down from like, oh, I'm just going to like look at the SQL queries that were executed against this table all the way up to, um, you know, you have a project that, you know, is highly used and maintained by the data team. And in Hex you, we have this mode called endorsements where basically the data team can put a tag on it. That's like, this is like a blessed gold star. Yeah. Essentially. Yes. And all of this is to help guide the agent towards, you want it, you want it to be able to take advantage of all the context that it's able to, but you also want it to be able to distinguish between the highly trusted parts and then the less trusted parts. Cause I think that that is like one of the risks of agents because they can be so much more productive. You just get stuck in this loop where they kind of get worse and worse and worse and a greater and greater purdue proportion of your internal dashboards and stuff are written by agents and they're just learning from other agents who made mistakes like 15 months ago. And yeah, like that, that kind of idea of saying like, we also need to give feedback. Like these are the really good things that like copy from that, not from.
Context studio and managing agent knowledge
And, and this I think is one of the key sort of new jobs to be done for the data teams, if that makes sense. Um, and a big part of, of Hex that we haven't really talked about yet, but is relatively new, um, post agent is what we call the context studio, which is basically about observability and management of all of the context for all of the agents. And so you can see what people, you know, what the agents are actually doing for each thread. You can actually see what context the agent is using. And then we have a second, uh, sort of like a separate set of agentic workflows around managing the context around extracting the pieces out of these things. Um, and then putting that, feeding that back into the like highly trusted, highly governed context to kind of avoid this like spiraling problem of, you know, the agent just kind of deciding that that's the way to do something. And then that producing, um, sort of being a self-fulfilling prophecy over there, you can actually like break that. You can see that it's actually doing something that's, that's incorrect. Um, and then update your human managed context, um, which is prioritized, you know, more highly above the kind of ambient context for the agents.
Building for humans and agents
I'm really curious. I read recently that now more agents are deploying code on Hex than humans. Um, how do you manage as someone who obviously cares so much about their users and like is giving people such a beautiful suite of tools, like what are you making for agents versus what are you making for people nowadays? Yeah, it's a really good question. I think in the early days of building the Hex agent, it was very clear that, um, the tools that we'd built for humans were actually also very good for agents. And so there was a lot of overlap. Um, you know, we had this modular notebook structure, looks a lot like tools that you can give to the agent. Um, and the agents just took to that very naturally. And that was like the first generation I would say, where it was pretty, pretty one-to-one. But now that we've had this sort of crossover moment, um, of more code is being written in Hex by agents than by, than by people, you start to think about what that implies for the rest of the system. And you still have, you still want the human interfaces because there are still times where people are going to come in and tweak things. And, you know, it's, it's not, it's not fully, fully, um, be a natural language. Like sometimes typing a whole sentence is just a very inefficient way to do something that's, you know, as opposed to clicking a button.
typing a whole sentence is just a very inefficient way to do something, as opposed to clicking a button. But at the same time, knowing that over time, more and more of this is going to get produced by agents, it actually gives you a lot of opportunities to build things that are useful for agents, but less useful for humans, but are very powerful for the agents. And so, you know, one thing that we've thought about in HEX is, could we give the agents slightly different — all of our cells are built to be, you know, they're very sort of easy to inspect, you get a little preview, you get all of these nice UI features. Agents don't need that. And actually, they're quite expensive. So if an agent is building up a project, could you give it a slightly different set of tools that would have different characteristics? Maybe less data warehouse load, maybe it runs faster. Maybe it's just, honestly, sometimes agents like to work in a different way than humans. They've been trained on a certain distribution of data, and it's not exactly what our human brains have been trained on. And so I do think that there will be some divergence there. But the power of HECS is that these things are shared across — they're interoperable. And I think that will continue to be true for a while.
One of the things I really like about the context studio is it feels like HECS is sort of thinking, well, your job of what a data team or data science do is changing. This is going to be one of the things where you give value to your organization, as opposed to like there's also a lot of startups out there today, which their pitch is basically like buy us and fire your data team, which, you know, as a data scientist doesn't feel very nice.
Well, we were talking to a head of data, and he said something that was really interesting. And I think when you frame it in this way, it made it really clear to me why data teams are not going away. He said, my job is to vouch for the correctness of the data and the answer that you get.
He said, my job is to vouch for the correctness of the data and the answer that you get.
He's the head of the data team, and obviously that's a slightly different role than ICs on the data team. But when you frame it like that, you're like, oh, yeah, actually, in the abstract, when you don't have people clicking buttons, that is the job of the data team. The job of the data team is to make a system that produces correct answers.
And today, a lot of that has to do with, instead of being the person who writes the sequel that is correct, it doesn't have bugs in it, how do you set up the context layer such that the agents can arrive there themselves? Yeah. I mean, the depressing thing is, definitely in corporate America, having the person to fire when something goes wrong is important. I mean, even more positively, someone has to own the stuff and be like, yeah, it's my reputation on the line that this is correct. You can't just distribute that to a bunch of nebulous agents that no one really understands.
Ownership is not just like, it's my reputation on the line, but it's like, you know, my job is to make sure that this functions. And because it's, you know, in organizations like this data is being used for, you know, existential, answering existential questions, like the business needs someone to care that that is correct and to, you know, build the system that is correct, rather than just kind of assuming that everything's gonna be fine and figure it out.
But it does feel like data teams are going to get more unhinged analysis from execs. That's also going to have to be part of their job. Someone has done something that they think is really cool, and it is really cool that more people in the org are interacting directly with the data. But the consequence of that is more people are going to be making mistakes and the data team is going to have to walk them back.
Yeah. You already see this. You see this in software engineering, like, you know, a CEO will come in and like, you know, use cursor or whatever to like vibe code some insane thing and then like make a pull request because like all the like traditional barriers to that happening. And then, you know, the engineers are kept to code and like review this thing. I do think you'll start to see a similar pattern like that in data where all of a sudden people have all of these really powerful tools at their fingertips and they don't know how to use them.
And I think there's a couple of things there. One is, you know, in Hex, we're thinking a lot about building review tools, both, you know, manual and also agent assisted review tools so that that process is as painless as possible. But I also think it's just another indication of, you know, where there will still be value. Like there are there are times where you actually want an expert to do the analysis. There's many, many ways to do the wrong things with statistics. And if you if you have a very important decision, there will still be a time and a place for people to kind of do those things and not just have someone, you know, fire off an agent query and and, you know, not not be able to really think critically about the result they're getting.
There's many, many ways to do the wrong things with statistics. And if you if you have a very important decision, there will still be a time and a place for people to kind of do those things and not just have someone, you know, fire off an agent query and and, you know, not not be able to really think critically about the result they're getting.
It's funny you say that because our CEO got in trouble with our IT department for an app he vibe coded with HR. I think it's so like maybe empowering and lovely to think about, you know, data scientists are coming with all of the the muscles for, you know, really productive AI usage like data scientists do a lot of experiments. Data scientists know you look at something, say, wow, this is bad. Like let's scrap it all and start again. So I think there's something really interesting and especially with the aid of things like review tools built in the loop, you know, no better time question mark to be someone who's writing code to do data science.
And I think there's a lot of hype around AI engineering and things like that. And we internally at Hex have an AI engineering team. One thing that I've observed is sort of the lines between our AI engineering team and our data team really blurring and starting to overlap. And so, for example, a lot of things that we'll do internally for our ourselves to build our own product and build our own agent, we'll build, you know, we have evals, we have various other things that we've built. You know, we will look at those and be like, oh, we should actually take those and put those into the product for our customers. Because these workflows of, you know, building a highly trusted agent actually, you know, converge over time.
Who runs Hex's internal context studio? Who's the person curating that? Yeah. That's our data team. Nice. Yeah. And they are like our number one dog fooders. Yeah. For obvious reasons.
Do you see like the lines blurring elsewhere? Not just because, I guess like in our, like we're starting to see like designers who previously would create, you know, like a beautiful thing and turn it over to engineers to implement. Now they can give a work, deliver a working prototype or like product people who previous were like, hey engineer, can you implement this? They just go away and do that. Like are you seeing that blurring of lines in your engineering work too? I would say absolutely.
But I think it's a little bit less dramatic at Hex than maybe at other places because our org has always kind of been designed like this, if that makes sense. Most of our PMs used to be technical, like actually hands-on technical in some capacity. All of our designers, we like give them an interview that's designed to be like, can you do light, you know, CSS and React work? Like we don't want people to just be completely boxed out of a core part of the technology. You know, our front end engineers and our backend engineers kind of flow back and forth. Everybody has their spikes, you know, it's not like we're expecting, you know, a backend engineer to be, you know, deep in React. But we don't want people to have these really hard boundaries because that can often create just like a lot of really annoying inefficiencies in the organization.
And so, yes, we are seeing that blur. We're sort of seeing that intensify, I would say, like, you know, designers and product managers are like, can go much deeper kind of in what they're able to contribute than they used to be. But, you know, we hired a lot of people who are excited about that. So I think that's like just strictly a good thing.
Hex's style and brand identity
Yeah, it's interesting. And maybe this is a nice segue into a less kind of AI, more style topic. But Hex has incredible style. Hex is just cool. I don't know how to explain, like, you go to the Hex website, and it's been like this for years and years and years, I feel like. The website, cool. The booths at conferences, unhinged. In a good way. In a good way. I'm sorry. Like a retro dining booth. Hex is throwing like magic shows. I don't know how to ask this. How did Hex acquire such immaculate style?
Well, I'm going to have to give some credit to some other people in the organization for that one. My co-founder, Barry, the second person we ever hired was Adam, our head of design. And I think, you know, those two have been kind of the masterminds. So Annie, our head of brand design, have been the masterminds of a lot of these things. So in some ways, it's down to specific individuals and having particular individuals who have these kind of off-the-wall takes. But I think also it's about the culture that you build as an organization and who you find and us trying to bring in people who are creative, who don't take themselves too seriously. And when you have some of those things, it creates an environment where you can be generative and think differently like this. And yeah, so it's both. There's like some very, very talented people that we have and also, you know, the team that we built around them.
Yeah. I really like the Hex. Like the Hex color scheme is like so, like it really stands out and like tick. No one is like pink and cream and brown. Like that's, I mean, probably now that's the next trend, but like, you know, I kind of love, like we spent a lot of effort at Posit coming up with our logo and stuff, but just like blue and orange is just like such a like traditional kind of tech color scheme. It's yeah. It's just fascinating to see like as we grow as a company, how these kind of forces of like unification and kind of corporatization, like pull on you towards these, just like this is like the safe, safe choice.
And yeah, and some of that just has to do with, you have to scale your content and branding engine. And so early on, a lot of creativity can actually just come from particular individuals. And you almost always get certainly a more like unique results when you're, when you're doing that and you're doing that in small teams. And then you get to a point where it's like, oh, you have to write all of this content and you have to put out things on Twitter and LinkedIn to get the, you know, SEO or AEO or whatever we're doing now. And you just, the amount of stuff that you have to produce can't bottleneck through your main creative engines. And sometimes you can scale, like you obviously scale your brand voice and your brand styling and things like that, but you can't do it perfectly. And so I do think that, you know, we're, we're trying to fight really hard against this and trying to like balance the two, but you do sometimes sort of inevitably have feel this like regression towards the mean pole, just like the more and more stuff you have to do on the branding side.
Metrics, altitude, and the CTO role
Yeah. And it's sort of also interesting just as the company grows, like the, the amount of visibility you have within all of the parts of the organization. And this like sounds weird coming from a data person, but like you are forced to start developing like metrics so that you can get some oversight and then like metrics are good, obviously, but they're also bad because they kind of like trap you into these like predefined, like how, like how do you like wrestle with that as like clearly a data driven company, but you don't want to be like too data driven at the same time.
Yeah. I think for me, a lot of it, a lot of it has come down to being able to sort of shift the altitude at which you operate and then knowing when is the right time to be at what altitude. So obviously it's extremely important, you know, as a senior leader of a company to be thinking about strategy and long-term time horizons. And I spend a lot of time, obviously we spend a lot of time talking about AI. Our AI strategy is where I spend a lot of my time thinking. And then, you know, you're intermediated through layers and layers of organization. And sometimes there's something down at the bottom that's like not going well. And if you've built a good organization, you'll know about that most of the time, but sometimes you don't. And sometimes you just, you see things in the data or you see things, you know, that don't feel right.
And I think for me, cultivating that kind of like instinct of, hey, this section feels like it feels like there's something not working here. The vibes are off. And being able to just do a deep dive on that. And in order to be able to do that, you have to know enough about the things that are like happening on the ground. And so for me, you know, when I first started really building up a bunch of our AI programs, I spent, and I still do actually, like I read the papers, like I spent a bunch of time in the technology and that's to both to like be able to see the big picture, but also when something's not right on the ground, to be able to kind of go all the way down and like help debug, you know, where something is breaking down.
Yeah. As CTO of Hex, I'm so curious, what is your, what does that look like? Like what's your day to day look like? It's a pretty big range of things. I think one of the funny things about being CTO is sometimes I feel like my job is more about humans than it is about technology, even though technology is in the title. And so a lot of it is about what I'm not sort of thinking about the AI side of the AI technology side of things. A lot of it is about like, how, how are we running a functional technology organization? And there's engineering, there's data, there's, there's our survey, our research side, there's security, there's all of these things involved in that. And a lot of that day to day is actually not necessarily like making technology decisions. Like I've hired smart people, if I've done my, especially in areas like security where I'm not the expert, if I've done my job right, I've hired people who are better at making those technology decisions than I am. It's about, you know, setting up the humans to do their best work.
I think one of the funny things about being CTO is sometimes I feel like my job is more about humans than it is about technology, even though technology is in the title.
But as someone who's like come up from the kind of technologic, like the technical side, like how do you, like, do you still, is there some amount of programming like you just want to do to kind of like scratch that internal itch of like being? Let me cook. Yeah. Yeah, for sure. A lot of it I try to do on sort of side, less, you know, mission critical, critical path things right now. Some of it is actually fun to get.
Background in NLP and rediscovering AI research
So when I was in college, my concentration was, was on called ML. And then I did my master's in NLP. And so I've always really been interested in this side of the house, but this was over 10 years ago. For people who might not know, NLP is natural language processing. And it's all about. It's basically all of, it's like pre LLMs, pre generative AI, how you operated with natural language in, you know, in a statistical sort of modeling sense.
And actually a lot of the techniques back when I was in college were sort of hilariously primitive versions of what we have today, if that makes sense. So, but, you know, it was, it was pretty clear, this was 2012, it was pretty clear that this was like the stuff that they were doing was like kind of neat and you could like see some interesting properties of language with it. But like your ability, you couldn't generate text, like your, your ability to actually do things like machine translation was like a big area of study back then. And it was like, you know, comically bad, you know, it was like, you could get the gist of what the, what the text was saying, but it wasn't anywhere approximating real speech. And it was pretty clear to me that like the techniques that we were learning about as state-of-the-art at the time were like, not it.
And so I kind of like left that and went into just core software engineering because I was like, I want to do something real. I want to do something that's like practical. And then sure enough, a few years later, Attention Was All You Need was published and it's kind of like off to the races from there. But so, so, so sometimes it's me getting hands-on and building things. And sometimes it's me just like getting back into that has been really fun and spending some time, not just like reading blogs, but like reading the actual underlying papers and like understanding, understanding the research side has been, has been like super fun over the last couple of years.
Yeah. Oh, that's interesting. So you've been kind of diving back into NLP. It kind of like reignited. Yeah. I mean, well now, now it's just, you know, all generative AI and it's like, it's sort of a blurred line. Large language models do so many more things other than just NLP. But yeah, a lot of, a lot of it is like, again, at the foundational level, like, you know, similar techniques. It's all just matrix multiplication under the hood anyway. It is interesting. Like, you know, like named, named entity recognition used to be like this really challenging thing. You have to have a massive tech stack specifically for that. And now you just like throw it at any random LLM and it does a good job. If I'm a barbarian, what's named entity recognition? Like, you know, if you've got like a blog post and you just want to extract all the people's names that were mentioned in it, like that's, you can't do that with a regular expression.
One of the hard problems is naming things. Yeah. Extracting names from things. Like I remember, um, a very long time ago, Hilary Mason had this really cool blog post where she scraped like 200 chocolate chip cookie recipes and then created like the average chocolate chip cookie recipe, which she then made. And like back then that was like, that was a lot of work. I was like, wow, that is so cool. And that is now something I could do today. And you know, like 30 minutes with, or less than that, you just ask chat GPT for a chocolate cookie recipe. And that's basically what you're getting.
Ben Stancil mentioned something about this too, where like, and having to do with data work, like support, if you have like thousands of support tickets, you know, as a data team, you might have used to go in and tried to analyze it and maybe like calculate a number from support tickets. He mentioned, yeah, like nowadays you might even just dump your support tickets into like chat GPT or something and just ask like your question directly. Whereas before we were trying to roll up numbers to try to kind of. You have to like take the unstructured thing and then, you know, run some models over it, maybe some classifiers, maybe some sentiment analysis, maybe you're like extracting specific signal from it. And then you get this like very, very limited structured representation of it, topic, you know, topic clustering or whatever. And then you like try to figure that out, figure out what's going on based on that.
And some of that actually is still very complimentary, I would say, because, you know, LLMs are non-deterministic and they're not very precise, especially for things that are like if you're just dumping a bunch of unstructured data into them for things that are require, you know, numerical calculations and things like marrying those two things can be challenging. But for things like, you know, what are the most common, you know, user complaints? I've done this in Hex on Hex data. And you can it's actually like very, very fast and very powerful to pull out that kind of signal. Are you saying like user complaints? You can get surprisingly far by putting a lot of the feedback in and just asking.
Just get straight to the kind of where you're going. And you probably want to move into the more structured realm when you're getting to the point where you're like trying to be like, OK, well, what fraction of users are asking about this? And, you know, how does that compare to like time to resolve the tickets and things like that? Then you probably do want to start to shift into like a more classically structured data. Often you just want like imagine like someone with all the support tickets and they're just going to give you the vibes like that's super useful, even if they can't tell you, like 25% support tickets are about blah.
I like the alternative data science workflow. If you have this like from like R for data science, you have this workflow that's like import, tidy, transform. I like the alternative, which is like copy paste. Just give me the vibes and then you like. And then you can iterate. But I think I even find that with like some of my projects where I'm just like, hey, Claude, just Claude Code, just do this. Like look at this transcript, do the stuff to it. And then I'm like, OK, this this it feels like this has teeth. And then I'm like, OK, now Claude Code, write the code that will do that for me, like reliably and reproducibly. Like that's such a just to be able to do that very, very quick initial prototype to say like, is there anything here that I should bother exploring further?
On success and the co-founder mindset
You have come from, you know, co-founder, your CTO. I'm really curious. Like what does success look for you look like for you, especially, you know, in the age of AI? Maybe how has that changed throughout your career?
Yeah, I think starting a company has really forced me to like disentangle myself from classic indicators of success, if that makes sense. And there's a couple of reasons for that. One is you'll just drive yourself absolutely insane. You're like that company has how much revenue and they raised how much like you just like you just can't compare yourself. Like a lot of us, you know, come up through school. You're basically like ruthlessly, you know, trained to optimize this like one metric and like compare yourself against all the other people to get into. And like you just have to just let all of that go. Like you just have to let the numbers go and focus on, you know, what you can control.
And I think the other thing about being a co-founder that I found, I found that the hardest part about being a co-founder is that you are, you know, the job changes every six months. And so what that means is just as you're getting your feet under you, you have to do something different. And so you're always in this mode of like, I'm not good at the thing I'm supposed to be doing right now. I'm learning how to do this. And for someone who, you know, generally, you know, had a relatively successful career prior to that, you're like, you're used to operating in areas where you're good and you're strong and things that you were hard for and you interviewed well for and you're like great at software engineering and being a co-founder is just like not like that. And so all of these like classic ways of being like I am successful just kind of go out the window.
And I think for me, I just kind of keep coming back to a couple of different things. One is, are we building something that, you know, is doing good for the world? That is a useful contribution. And then am I enjoying what I'm doing? Is this a good use of my time? And as long as I feel like and, you know, you don't have to enjoy everything that you're doing, but like sort of in the macro. And as long as you feel like those two things are happening, then I think that's like a pretty it's actually like it's quite rare to really be nailing both of those things. And so that's kind of how I've been thinking about my own personal career success.
And as long as you feel like those two things are happening, then I think that's like a pretty it's actually like it's quite rare to really be nailing both of those things.
Do you feel like there was one pivot that was like this is the hardest thing or this is the easiest thing? Was there one pivot that felt like it hit harder than the rest?
I don't know. I think in some ways I kind of from the beginning of Hex had to make that pivot even just to start the company. And it was a bit of a sort of mental hurdle to get over because prior to that, my career had been extremely linear. It was like I see and then manager and then manager of managers. And then like, you know, you just kind of like walking up that and taking a step away from that. And and also candidly, like, you know, I went into Hex being like, I don't know if this is going to work. Right. You know, most startups don't work. We may all be back on the job market in a year or two, and that would feel like failure.
But I think getting kind of getting past that and being like, you know, is that the worst thing in the world? Like, we try something and it doesn't work out like maybe you're like slightly embarrassed. I don't know. Like, what is the actual negative outcome there? And kind of pushing through that feeling of like, oh, what if it fails? Well, what if it does fail? So what? We all go like we're all software engineers. And I think that was probably the biggest mindset change and even just jumping off and then, you know, having to kind of repeatedly remind myself of that over the years.
One of the frameworks I found really useful is like this idea of maximizing versus satisficing. Like, I think I feel like, you know, data people, people have had good grades at school, like your natural thing is to always like find the best, like maximize. Like I find this a lot when I'm trying to like buy a hotel somewhere, like I do so much comparison shopping, because I'm trying to find like the best combination of location and price. Whereas the alternative approach of like satisficing of just finding the first thing that's going to be like good enough. Like, that's such a powerful like mindset change. Like you don't have to be like, not everything in your life has to be the best. Just like finding the things that like make you where you'll be like happy and satisfied.
There's an interesting story of like the, you know, step one to two feels hard, but what was way harder was step zero to one. And you know, what mindset shifts do you have to do to even get to a spot where you're at zero versus, you know, continuing to just go like, oh, well, I'm at one, I can go to two, I can go to three. It's like the same with like growing a team, like the first person, like going from zero to one is really hard. And one to two, that's not so bad. And then it just gets kind of get on the downhill slope a little bit.
Giving engineers permission not to be miserable
This is a pretty common piece of coaching that I give to engineers, which is, it's basically telling them, like giving them permission not to be miserable. I love that. They often, sometimes in engineering, sometimes again, sometimes you got to do what you got to do. You know, sometimes someone like we had an engineer who moved into an engineering manager role and was miserable. They hated it. And I was like, you can go back. Like we actually intentionally have set up our work structure such that like senior engineer and engineering manager are the same, no pay difference. They're just the same level, just a different title, different role.
And coaching people through their first moment of, I tried it and I didn't like it. And I didn't like make it all the way to something that feels like a success moment, but that's okay. I don't have to get there. If I'm really unhappy with what I'm doing, I can just stop. And that, especially like we've been talking about for people, high achievers that can feel like a failure because sometimes you're in a, you're in a spot where you're actually not doing a great job at the thing that you're supposed to be doing. And you feel like you have to push through to get to a point where like you've made it, but I don't know. We only, you only live once, right? Like, you know what you do with your time matters. And there's only a certain amount of like pushing through.
And it really depends on like what the underlying reason is that you're not happy. There's like lots of different reasons, but usually I see this in a case where someone just like, they don't like the role. And that's not a failure. That's just like, you would rather be doing something different. Obviously like there's differences where, you know, you just have, sometimes you do have to power through like tough times in the company and you just have to like make it out the other side. But those are temporary things. Like the role doesn't change. The role is still the role. And so if you don't like it, don't do it.
You mentioned like the kind of uncertainty of starting a startup reflected in this person's shift to engineering manager. You mentioned, you know, the fear of starting a startup and thinking like, what if it fails? And then being able to say like, so what? Like I tried it and I'll still have other things to work on. I'll have learned and grow. I think that like mindset shift between like this experiment failed versus I'm a failure. Like that's a really, really big difference. And to coach an engineer on that, that like jumping from to being an engineering manager is a similar like experiment as starting a company. Does it make sense for that person? Maybe it is probably as big and as scary of a jump I'd imagine to try this like new way of operating and that you might not know how you'll do.
I think that like mindset shift between like this experiment failed versus I'm a failure. Like that's a really, really big difference.
So it seems like honestly, so it has got to be so relieving or helpful to know like it's an experiment and you can switch back. Yeah. And now that we've had, we've had this happen in various parts of the organization a couple of times now. And again, it creates this permission structure for people to just try things. And that obviously has like huge positive externalities when people are like less afraid to go out on a limb and try something new. I mean, again, being the first person to go from being an engineering manager back to senior engineer, it's like super scary. Like if you're the fifth one, like, yeah, that's a good point. It kind of like snowballs, I guess.
It does feel like right now, like being, like creating an engineering culture where people are excited to try experiments, even knowing they're not going to succeed. Like that's like particularly today with AI, just so, so important.
Unwinding with Soulsborne games
We went through Hex, life as a CTO. How do you like to unwind at the end of the day? What's your move? I mean, we do a lot of the normal stuff. I also have a toddler, so not everything is unwinding. Yeah, yeah, yeah. Winding in different ways.
Once he's in bed, though, actually my husband and I have been, we've been playing through together all of the Soulsborne games, which might be a little bit weird for an unwind session, but it's like, it's a really good way to like, you know, release some tension, you know? We like, I hadn't played them before, but like, I got started with Elden Ring, and then we like, played all the way through Elden Ring. And then the rule is, when you die, you pass the controller. So you get a little bit of a break. Like, you die, you pass it, and then you get another turn. Do you have different specializations or jobs in Elden Ring, would you say? You and your husband?
Yeah, I will say he is much better at sort of like, learning the timing of the boss mechanics. So he's typically like, he's gonna be a little better than that. I tend to be a little bit better at the like, mobs, because I'm less reckless than he is. He just like, charges in and just get like, totally annihilated. And I'm a little more strategic in how I like, do the placement of the, yeah. My partner and I played a lot of Elden Ring, and I am very much so that avoid the mobs, run away, like, learning that you don't have to fight everyone, didn't realize that was an option in Elden Ring. Life changing, life changing. It's an option in real life too.
Yeah, I guess for the whole table of Elden Ring players, when you're playing with a person, is this a collaborative, peaceful act, or? An act of violence. You know what I'm saying? Well, we actually stopped playing real co-op games together because they got too intense. And we were like, or it's just like, we've just different play styles. Again, like his is very like, you know, just like rush straight at it. And mine's a little bit like, more sort of like, careful exploratory. And so we were getting really mad at each other. Like we tried to play Baldur's Gate co-op together and just like, that did not go well. And so I actually think that this model works a lot better. Like one person is in control at any given time. I feel like that's a hot tag.
I love Baldur's Gate because split screen, you can almost have a co-adventure, like, or completely different adventure. Oh, you can split off. You're saying you took the essence of playing separately, but within a cooperative game. Yeah, pretty much. You're like, see ya. And then the screen just splits and you never reunite. Yeah. Sometimes. And then it's like, oh no, I'm dying. Can you come help? Yeah. And then she'll pop in and have a good, yeah, everybody for me. That's incredible. It's a good life.
Caitlin, really appreciate you coming on the test set. Thank you so much for having me. Yeah, no, it's been great. I feel like so much to think about. I honestly, I hate to admit it, but I feel like I have to. I hadn't learned about Context Studio, but it is going to live rent-free in my head for the next probably all my life. We'll hook you up. I would love to, I mean, always showing Hex to people. I feel like, I don't actually, what you said about Context Studio really kind of blew my mind because I do like a lot of software engineering on data tools,
but that strikes me as such a unique, maybe not totally unique problem across a lot of domains, but for data people that need to like curate and stamp and endorse.
It's honestly like this moment in some ways is like a match made in heaven for data people for this workflow, because this is the workflow that data teams have been doing forever, but it's just been so painful and laborious for relatively little impact. Like if you think, or I mean, obviously there's impact, but like the amount of impact you get out of like all of this labor just sometimes doesn't feel worth it. Like, I don't know if you all remember, or I'm sure you remember like when semantic models first kind of became a thing, we just didn't see a ton of adoption of semantic models. And it wasn't because people didn't want them, it was just because it was so painful and took so much time to build them out. And at the end it's like, okay, yeah, only two people are going to be building your dashboards, right? And now today that's like totally been inverted. Not only can you take constructs like semantic models and make them available to everybody in the organization in a way they can use, but actually also you can do the curation and you can do the governance much, much more easily with sort of agents yourself.
It is a pretty incredible that like it's both the cost has decreased dramatically and the impact has increased dramatically. Like it's pretty unusual to get both of those at the same time. It's pretty cool.
It is a pretty incredible that like it's both the cost has decreased dramatically and the impact has increased dramatically. Like it's pretty unusual to get both of those at the same time.
Yeah, totally. Yeah. So a lot to check out. Yeah. Can't recommend Hex enough. I feel like also giving software engineers permission to like try things and fail, I think so relevant, especially with AI where there's a lot of experimentation going on. And then I'm putting in my pocket Elden Ring co-op strategies. I'm going to find peaceful ways to play with my partner. But yeah, thanks so much for coming on. It's been so nice and been great having you.
Awesome. It's been fun. Yeah. Thank you so much.
The Test Set is a production of PositPBC, an open source and enterprise tooling data science software company. This episode was produced in collaboration with creative studio Adji. For more episodes, visit thetestset.co or find us on your favorite podcast platform.