Transcript#
This transcript was generated automatically and may contain errors.
Hey there, welcome to the Paws at Data Science Hangout. I'm Libby Herron, and this is a recording of our weekly community call that happens every Thursday at 12 p.m. U.S. Eastern Time. If you are not joining us live, you miss out on the amazing chat that's going on. So find the link in the description where you can add our call to your calendar and come hang out with the most supportive, friendly, and funny data community you'll ever experience. We can't wait to see you there.
With that, I am so excited to introduce you to our featured leader today. His name is David Brodkey. He is a data science manager at Intuitive Surgical. David, it is so nice to have you here with us. It would be great if you could introduce yourself, tell us a little bit about what you do, and something that you like to do for fun.
Awesome. Well, thanks, Libby, for having me. Really excited to be here today. Hi, everyone. My name is David Brodkey. I'm a statistician by training, and I've been working at Intuitive Surgical for almost seven years now. We are a large medical device company who makes surgical robots and accompanying products. We're mostly known for the DaVinci surgical robot, which is a system that allows surgeons to control a robotic system that's performing surgery on a patient. If you've ever seen the video of doing surgery on a grape, that was promotional material that they did with our machine a number of years ago. I've been in our global operations org for a little over five years. This is an org that supports manufacturing, supply chain, logistics, and planning. Everything about our released and approved products in the market, the ways that we need to actually produce them, get them out there, keep everything flowing smoothly.
Right now, I work in a data science management role. I've kind of slowly transitioned from being a pure IC to hybrid to now it's mostly management and still squeezing out a little bit of time for IC work. I think it's a really fun role. We cover a plethora of domains, and we really use a broad spectrum of data science techniques and methods, which I'm excited to talk about today. I think the favorite one that I would plug would definitely be Monte Carlo simulation. I love using that, and we've used that in a number of different domains and applications with really good success. Then as for something personal, I'm a really big soccer fan. I grew up playing. I still play pickup. Then I love watching the Premier League. I was talking with Libby. I have a personal data science project that we may hit on where I've built a Shiny app and some logic to try and calculate trips for me where I can kind of optimally find routes. I've used this to visit England a few times and go on these amazing trips.
The Premier League trip optimizer app
Are you able to hit multiple games happening in one stretch of time in the best location for them to happen? Tell me a little bit more about this app.
Yeah, I mean, so you get all this data about these are the days that the games are happening and where they're happening. What we've done is we've flown to England usually around December when the games are really jam-packed and gone to a bunch of games. Last year, we were there for 12 days and we went to nine games all around the country. Then a few years before that, even slightly more intensely, we did nine games in eight days. So every day a game in one day, we had two. So I've used this for some other applications with friends, but the idea is you get all this data about where games are geographically. You need to compute distance matrices to see how far all the stadiums are from each other. Then you need to try and plan out what kind of trips are optimally possible.
All right. Well, we already have two questions in the Slido and they are the same question. And I see people asking in the chat. So we're going to round this up and I'll ask David together, who is your favorite Premier League team?
Manchester City. When I started supporting soccer, my closest friend at the time was a Manchester United fan. So I kind of picked Man City to be contrarian to him. So I've been supporting them for, oh geez, let's see, it goes back to about 14 years now. So yeah, a really good time. Nice. It's been some amazing periods.
We're going to hear in the chat whether everybody agrees with you or disagrees with you.
Operations data science at Intuitive Surgical
Okay. All right. The Slido is now open for all questions. I would love to ask David just some context building questions about what you do so that we can all kind of get oriented around it and we know even what we can ask you. You work for a medical device, surgical device company, but you are not a doctor and you don't work with clinical trial data. Tell me the sort of area that you work in and the type of data that you work with, and also the tools that you use so that we can orient ourselves.
Yeah, no, that's a good call out. I don't know a lot about kind of healthcare data and things like that. Intuitive is a very large company. I think we're somewhere around 15,000 employees. And so there's analytics all throughout the company. And so I'm a part of an analytics org specifically supporting operations. And so I mentioned before, but manufacturing, supply chain, logistics, and planning. And then within that, I lead our data science team. I'd call our mandate is to come up with advanced analytics solutions that somehow optimize the business and try to improve things. So for our supply chain org, this could look like identifying parts that were at risk of going low on inventory and trying to proactively identify those so we can handle that. For our logistics team, it could be extracting information from shipments documents in a programmatic way. For our planning team, it's helping to manage our algorithms that are predicting how much we're going to sell or how much we need to produce of different parts so we can have really smooth, stable plans that are going to allow our operations to run smoother.
For a tech stack, we have basically all of our data is in Snowflake. So we're using SQL to access that. We are historically a very R-heavy team, although I'd say we're somewhat transitioning to be a little bit more bilingual between R and Python. We use Posit's commercial products. So Posit Workbench is where I'm typically working for dev, although I'll say that my IDE situation has been rapidly changing in the last year with all these new AI tools and AI IDE add-ons. And then we publish all of our stuff in Posit Connect. So most commonly, a Shiny app would be our end product for being user-facing. It really lets us do whatever we want to do. Although there'd be some other things like sometimes we'll have scripts that are scheduled that are going to send out emails, or even it's just a part of a data preparation that's then later consumed by a Shiny app.
For a tech stack, we have basically all of our data is in Snowflake. So we're using SQL to access that. We are historically a very R-heavy team, although I'd say we're somewhat transitioning to be a little bit more bilingual between R and Python.
Nice. Okay. So when you say you're end users, who are your internal customers? Yeah, that would be anyone inside of operations. So you could have, for example, in our supply chain team, maybe our buyers who are responsible for actually purchasing the materials, right? So we purchased thousands of raw materials. There could be someone who's usually specific to an area, like I purchased this kind of electronic components, and you're responsible for managing inventory levels. And so we'll do something like say, hey, we're kind of nervous about the inventory levels of these parts. We really should take a look at it because we have this model running that flagged it to us, right? And here are the parts that you really need to go take a second look at. Or our planning team to say, you know, here's an application where you can view all of our forecasts for how much we think we're going to need of these different components. And then they can also have interaction with it. So it's everybody within the operations org. We really don't have anyone external using it. It's all inside intuitive.
Forecasting and supply chain challenges
Nice. Okay, cool. Yeah, I wanted to give a little bit of background for anybody who is not used to operations speak, there's all kinds of different end users inside, because you've got to forecast and look at supply on the inside, all of your supplies, and all of your supplies and your raw components might have suppliers, multiple suppliers, maybe multiple different prices, maybe different countries they come from, maybe there is a supply shortage in one country, and you have to switch suppliers. There's all kinds of juicy problems that go along with operations. I'm always trying to shill for that a little bit because it was the smallest major at my college. It's what I did in undergrad. And I think more people should consider studying it because there's such cool problems in there.
And you also do forecasting. So what kind of what other kind of forecasting do you do? Do you do supply forecasting as well as like industry demand forecasting that's more external? So we've done some service forecasting is something that I've owned for a very long time. And this is where we have these very large medical device products when something goes, you know, either something isn't working correctly, or just we're flagging something needs replacement. We have service parts where we're swapping out components. And for a very long time, I've been the owner of our service forecasting where we're trying to predict how much we're going to need of components as well as where we need to stock those around the world because they need very quick servicing. I've done some more consulting for our commercial tools on sales and build plan forecasting. So we're saying this is how much we're going to sell of this product, or this is how much we need to build of this product by period of time. And, you know, just with the size and scope, it's really important to get those forecasts as close as possible, because you need to plan for headcount, you need to plan for resources, you need to plan for space, things that if you miss on it, it's not easy to make up quickly.
Deciding when to use advanced vs. simple approaches
We have a great question in Slido. Prem asked a question that I think is fantastic, even though they said no need to answer straight away. This is a great question.
Yeah, that's a great question. And I think one thing that has always been a struggle in my career is a lot of times stakeholders come to you with a solution in mind, which is usually like whatever is the current hot thing. So for a very long time, the solution to everything was machine learning. And now for the last year or two, the solution to everything that's being told to us is AI. I think we always try to start as simple as possible, right, and try to build up slowly. So a lot of times if we're venturing in something that's like really new ground, where we don't even have like the right data set up or the right process, we'll start simple and then we'll try to add on complexity as we derive value. We definitely have a very much of a discovery mindset where we try to test things out, POC things really quickly, get feedback loops, and there's stuff where we absolutely look at it and say like, this is the right approach, or this isn't the wrong, this isn't going to yield that much value. So just really quick experimentation and being okay to let go of things. Those would be kind of the two main things I would say.
I think we always try to start as simple as possible, right, and try to build up slowly.
I think that's a great question, Prem. Let me know in the chat, everybody, if you have some litmus tests or some heuristics that help you decide when you want to have a bigger solution that's more complicated or when a simpler approach will do. I saw a reply on Prem's question that says I would always go for the simplest approach. I tend to be the same way. I feel like model parsimony was like really, really hammered into my head when I was learning statistics, and I think that I use that approach in business questions as well.
A typical week as a data science manager
All right, we have a question from Natasha. It has an asterisk in there, so I'm going to read it for Natasha, and it says, what does an average day or week look like for you? What areas, like programming, reporting, meetings, do you split your time between? Yeah, I can answer for myself, and also I can comment on maybe my team because I think for a lot of folks that's an interesting perspective as well. I would say over a typical week, I'm probably spending nowadays maybe 25% of my time like actually writing code and solving problems, and that would probably be about, I would say a small sliver on that is kind of maintenance of existing projects and just anything that's coming up of, okay, there's this new change in our data structure, something like that. A much bigger one would be people are asking for some kind of enhancement to it, and I'm trying to just keep things moving along, and then the third piece would be like something just kind of totally net new that I'm experimenting and trying out.
I would say for me then probably about another quarter of my time, it would be purely just working with my team and supporting them, so understanding what they're working on, brainstorming on solutioning, trying to come up with, you know, ways to get past roadblocks. I put another quarter is probably working with stakeholders, trying to really understand what their problems are, trying to get feedback on things we have, working back and forth, and then I probably put a quarter on about, I call it more administrative tasks. So, for example, right now I'm doing some stuff with hiring, or we have goal setting, or working with leadership to just get a line on roadmaps.
That's becoming a bigger and bigger part. If I flipped over to my team members who I'd probably, you know, say for them, it probably looks like maybe two-thirds of their time is actually working on solutions, and we try to be really protective of that focused development time. I try to hold off on meetings and discussions and keep it as thin as possible, and then the other time I'd say, you know, split between internal and external discussions.
Awesome. I like that you said a full quarter of your time is helping learn what problems are and helping dissolve roadblocks, because that's a big chunk of time.
Did you have any difficulty when you went from IC to manager in making that move and having your time spent differently? Did it feel different? Did it feel less productive somehow?
Yeah, I would say there's definitely days where I sometimes don't internally feel like I did a lot, but then I have to step back and say, okay, I actually did, I'm just not measuring it correctly, where I'd be like, you know, I didn't write a single piece of code today, but I say, well, you know, if I helped this person and this person and this person with their problem, and I, you know, worked on this, you know, getting in a tool that's going to be really impactful for our team and important, and I connected with these two stakeholders to help brainstorm on their problems, sometimes I get home and I say, okay, I didn't write any code, I feel like I didn't do a lot, but then I have to look back and say, no, I'm really still contributing to the broader effort. For myself, it was a very gradual transition. So I started as a pure IC, you know, it was a relatively greenfield space for us. In data science, we had a lot of experience in what I would call more data analytics or BI analytics, and we still have a significant footprint in that where we're doing more descriptive analytics. But this idea of data science, advanced analytics, more getting ahead of things was relatively new for the organization at the time. And so, you know, it went well, they said, okay, you know, we want you to mentor an intern, mentor a contractor, okay, we want you to make a full-time hire, now we want a second full-time hire. And so it's kind of just gradually gone, which I think has really been something I've been really fortunate about. I've seen people who have done, you know, these kind of hard cut transitions, which I think would be much more difficult, and I've stayed away from that.
I always feel like whatever I'm not doing is sometimes the thing that I'm eager to do. So I remember when I was kind of just doing pure coding, I'd be like, I want to discuss this, I want to brainstorm on problems. And now the thing that sometimes I'm most squeezed for and most missing is just actually sitting down and writing code for a problem. But, you know, there's a lot of great stuff to enjoy. Of course, and now you get to enable other people to to do the cool stuff too, which is, I find a good feeling.
Contractors vs. upskilling
All right, we have an anonymous question here in Slido that says, we are doing more AI projects at work, but the issue is that we need to finish them and spend the project money by a particular date. Can anybody else relate? The dilemma we have is to get a contractor quicker, but more expensive, or upskill someone internally, which is potentially slower. What would you do in this situation? And there is a reply on here that says, sometimes companies go for contractors because they feel they're actually outsourcing risk. If it goes wrong, you just blame the contractor. So what do you think?
So we've done some, you know, either contractors, interns, things like that. I've always looked at it as whenever we've had like an intern or a contractor, I would say that the majority of the value has not been from the work that they've done during that period, because the reality is it's like pretty complicated to get up to date on our company, our tech stack, and what they're doing. And so I usually find that those kind of individuals are not able to make, you know, gigantic contributions compared to our existing employees. But the way that we've found a lot of value on that has been a really good kind of roadmap for potential hiring, for potential inclusion in the future. And so we have, you know, employees who were past interns or past contractors who transitioned to full-time employees. And so I've kind of looked at that avenue, not as much as a short-term labor supplement, because I just haven't found as much success with that, but more of really good kind of test the water for hiring path.
As for upskilling, I think that's something that we've done at times. It's not easy also. You need to have people who are really bought in and internally motivated, and you really need to have buy-in from their leadership is probably the most important, because if they continue to still get, you know, whatever their existing workload is, and they're struggling to make time for this new thing you want them to do, it's not going to succeed. So I think I would probably lean towards the upskilling if it's really about like getting the deliverable and things done, but eyes wide open of lots of challenges on that front also.
Yes. And Rachel asked a great question in the chat, which was like, wait, I'm curious about the fact that you need to spend the money quickly. So chime in in the chat on Rachel's question so that we can give context to that, because sometimes grants have spending windows, right? Like you have to spend the money in a certain amount of time. It might be a long window, maybe like two years, but sometimes you might be picking up a project that falls within a grant that actually in six months from now, maybe you're at the end of the grant window, and it's not going to be renewed, or there might be a contract. So there are all kinds of reasons why sometimes you have this short window of time to spend money. And as a freelancer, that can actually be a good thing for you if you're a data freelancer, you catch somebody at the end of a spending window, they have extra money to spend, they can put more towards a project or something. So I've been on either side of that.
Thank you for that fantastic question, anonymous asker, that was great. And thank you, everybody, for hopping in with lots of context on that question for Rachel, you guys are the freaking best.
Forecasting: too high or too low?
Okay, okay, okay, okay. Let's get more slide questions. We have some really great ones in here. Zach asked one that I like, it has a little asterisk next to it. And it says, although you want your forecast to be as close as possible, would you rather a forecast be too high or too low?
Um, so great question. A lot of times that we talk about is first doing a forecast where we're just trying to get it as mathematically close as possible. Like this is just our estimation. And then sometimes, you know, depending on what we're doing, we can also attach confidence intervals to that. And then completely situationally dependent, we may say, okay, this is our prediction, but we are going to, you know, bias it one way or another because of the downsides of certain things, right. And that all comes from really understanding the business question of what are the negative implications of getting it wrong in either direction. You know, sometimes we try to quantify that with, you know, especially like dollars for a comparison, because you can try to translate a lot of things to dollars format and say, this is our negative implication on either side, and then say, this is what we're going to bias it.
But we usually first start with, you know, what is our really best prediction. And I'll say there's been times where different teams have different motivations for how they want forecasts to look, right, and what ways they kind of want it biased. And so, like, for example, I've seen times where maybe our sales team wants to give our sales representative something that they can have a good chance of beating, right, you don't want to, like, really make it too harsh of R, you kind of want to keep positive momentum going. But if they're consistently beating it, and then you're using that, it's no longer really a valuable forecast for, like, operations perspective, we're like, well, give us the true one, right. So I try to think about it both ways. And ideally, we try to forecast with confidence intervals when business is able to accept those kinds of nuance and the methodologies we're using work for it. And then sometimes we kind of have different confidence levels that we want to pull out.
Fantastic. I really like also, when forecasters do like a pessimistic, more probable optimistic, right, so you can get a little, a little spread.
All right, Zach, thank you so much for that question. And there was a reply on there, by the way, from Saptarishi that said, hey, maybe add confidence intervals. So great suggestion.
Thank you so much, everybody for asking questions. As a reminder, we're at the halfway point, you can ask questions in Slido. Rachel is putting the link to the Slido in the chat every once in a while for the people who are still kind of trickling in as their their half hour meeting ends.
Putting guardrails around LLM workflows
So he says, in the midst of the LLM craze, how do you set up a project that would utilize some LLM capabilities? Instead of throwing everything but the kitchen sink to AI and getting unreliable results, how do you put guardrails around your data and workflows to utilize Gen AI more effectively and reliably?
Yeah, no, that's a great, very timely question. And we're definitely integrating LLM tools a lot more. I was just looking yesterday with a team member on a problem where they're working on, which is, you know, at a high level doing this kind of deep analysis with LLMs where we want to check all these different data sources, and we want to come to a high level conclusion. And what he was doing really well, which we talked about, and I was really excited about, was he was mixing in purely deterministic steps with LLM steps and scoping them as small as possible. So there were things like he would execute a query. And then based on the results of that, it may just decide, I don't even need to call an LLM for this check at all, right? I've ran this query, there's nothing noteworthy in this. There's nothing noteworthy in this data set that needs an LLM to look at it. So we're just going to skip that step entirely. Or we're going to scope it down, you know, we've now found this interesting thing in the data set. We're doing this in kind of mass and we need an LLM to provide some level of analysis on it, but we're going to give it as small a thing as possible and see what it gives back to us.
So that's the biggest thing we're trying to do is pull things away. We've had really good results with things like we're setting up tool calling and things like that, where we're just trying to scope things down and make it deterministic where possible. So we're just cutting out as much variability as we can. Lots of work on testing always whenever we have these LLM workflows. It's gotten much better at constraining formats of outputs. I remember when I was first using them to generate stuff, I was like, please, please, please follow this format or it will break, you know, return it exactly like this. And now there's great tools where you can actually forcibly constrain it to have a certain kind of output. So that works great. Or you could just throw a deterministic check on top of it. So as much determinism as possible.
He was mixing in purely deterministic steps with LLM steps and scoping them as small as possible.
As much determinism as possible. Okay, fantastic. And I want to shout out that we did have Leslie Emery join us at the Data Science Hangout not too long ago. She is at Bristol Myers Squibb and she talked about reproducible workflows with LLMs and I think it's what her PositConf talk is about this year. So if this is a topic, Saptarishi, that you are interested in, make sure you check out her talk. Join us at Conf. If you can't go in person, join us virtually. It's super accessible and then things will be up on YouTube many months later after Conf, but that will be a good one to look out for.
Instrumentation and data capture at Intuitive Surgical
Thank you so much for your question. Tim, Tim Golden, alumni here at the Data Science Hangout has a question and I don't see an asterisk. So Tim, could you ask yours live? Highlighted it in Fido. Yes, yes. So I've for the last 30 years done data science, well actually 15 years in data science, in large law firms and I've had to do data analysis on medical device litigation. To what extent are you instrumenting the instruments that you produce so that you've got good data to make sure that the application of these instruments in a medical setting is defensible?
Yeah, great question. One thing I would call out is I'm not on the medical side or I'm not on the kind of software development side, but I do know and certainly utilize some of the logs and traceability that our software engineers primarily producing these products put in there. So we have very extensive logs of what's occurring on our systems, some of that that we use. Same thing with our instruments. Even our instruments, there's things like are set for a limited number of uses. So you would have an instrument that attacks us like monopolar curved scissors that are reusable so you can use it and then you can sterilize it and use it again. That has chips that track lives in it. And so when you attach it to the system, if you've used it for say the 10 number of times, that it's been cleared for, it won't even let you use it, right? So there's a lot of these controls in place. Being a very tech forward company, you know, our entire system is running on, you know, software and grabbing all this data that it can. We have our new system, which has gigantic amounts of computes in it. So I would say that there's gigantic amounts of logging and tracking going on. And I use some of that for some of our analytics purposes.
And then there's some things that are recorded during surgery as well. Like video data is something that we have something called the MyIntuitive app, where surgeons are able to watch back videos of portions of their surgery to learn from it. So I think it's probably one of the most data capturing processes that I've seen. I realized when I asked the question that there's, this is being recorded, and there's probably no good answer for you to give, given this, without getting some level of call from someone else in the organization. Lots of data in lots of places. I would say it's the biggest thing. Lots of data everywhere. Thanks.
Thanks, Tim. All right. I wanted to like come back around for just one second to the AI and LLM
question, because I saw Mike and Rory mentioned in the chat that they are using Posit Assistant, Posit AI. Is that part of your workflow? I know that a couple of people have been saying that they have been using it at work or using it on their own to test it out.
Yeah, I think the IDE and IDE plus AI Assistant journey has been like all over the place for me. I'd say I had like 10 years where basically all I ever used was RStudio. And then for a little while, it was RStudio plus like an LLM on the side that I would copy paste back and forth. And in the last, I don't know, year, year and a half, I've tried a lot of things. I've tried Positron. I've tried, we have some internal tools like a tool called Windsurf, which is a AI development platform. I've tried it with Cloud Code, Cloud Code plus Windsurf, Cloud Code plus VS Code, Cloud Code plus Positron, Posit AI, I've used personally, but we don't have that set up at work. I think my favorite right now is Posit AI, because it's just so fine tuned for the data science workflow. And I'd love to get that set up at work, which I'm hopeful we will shortly.
There's some things where we have to be on the latest version of the commercial products of Workbench and Connect. And it like just got released in the last, I don't want to get this wrong, but very recently. And so we're a little bit behind on that. And then we need to set up like our API keys correctly. But I've used that personally, they have the, you can do like a free $5 in credit, and then you can buy a month for $20. So I've used that a couple times, and really liked it. So everything's in flux. Yeah.
Everything is in flux. You said we're behind. I'm like, I feel like there's no way to not be behind right now. Like everything is moving so quickly. Even inside of Posit, I work at a company that is actively developing products that enable data scientists with AI, alongside AI. And I am constantly like, oh, am I behind? Am I just like two steps behind what they're already like producing the next thing? So I don't think that you are. There's always something new happening. And Tim, thanks for shouting out some packages there for us. Everybody go check those out if you would like to as well. When you were saying in the beginning that the IDE game is changing so quickly, I was like, oops, sorry, not sorry. We're part of the problem there, aren't we? Like giving you more options. Okay. Fantastic questions. I saw one from Dev and then also someone followed on to Dev. I think it was Ken. I think it was Ken. So let's see if we can get those together. Dev, do you happen to be able to unmute and ask yours live?
Estimating timelines and managing stakeholders
Yes, sure. So hi, David. Hi, everyone. I wanted to ask, I've been working 10 years in the industry and besides the technical aspect of data science, one of the things that I've always found challenging is estimating the timelines for a project and how to communicate that to the stakeholders. So just wanted to check in the era of AI, like whether a project's the number of requests coming in are increasing and the speed of work is genuinely also increasing. You can do more, but how do you still estimate timelines accurately, especially in a health tech environment where edge cases and validation, how do you carve out time for them and how do you estimate project timelines in today's times?
Yeah, great question. I'll give kind of a cop out at first and say, I try as much as possible to give people very loose things and to not to commit to specific things. And I have a lot of stakeholders who I've known for a long time who kind of just trust that we're going to do a good job. We're going to work as fast as we can and we're going to hit what we're going to hit. When I have stakeholders who really want to pin me down to timelines, what I usually end up doing is setting things more conservatively because, you know, if I have like my median estimate in my head of what it'll take, I don't want to give a response that I think there's a 50-50 shot we're going to have to go over. So I end up setting things fairly conservatively.
I will say for them that usually ends up resulting in them getting things slower than they would have if they had just not tried to press me down for timelines. Because then if I have something and I have time, I may kind of put it on pause and wait for something else. That's a skill that I've definitely learned recently is just because you have something done and ready doesn't mean you have to send it off the second you have it there, right? If you've kind of communicated with this stakeholder this is what it's going to take. We will have, you know, things that are sitting and so we can kind of deliver things in a more steady process. You know, we have one project where we're trying to ship out like meaningful enhancements on about a two-week cadence.
And so we certainly have at least like our next two releases kind of basically done with development and we're trying to be kind of very steady in what we put in for testing and then release to production. Other than that, I'd say, you know, it comes with experience and learning what are the nuances and what are the things that you're going to hit. You know, trying to be protective of those things and explaining it. The kind of pure development is usually not the biggest thing. It's what are the exceptions and do we actually have the data that we need? Do we have access to it? Do we have it in a reasonable format and structure? Those are the things that can just really extend timelines out like crazy.
The kind of pure development is usually not the biggest thing. It's what are the exceptions and do we actually have the data that we need? Do we have access to it? Do we have it in a reasonable format and structure? Those are the things that can just really extend timelines out like crazy.
I think we're all agreeing in the chat that this is a really, really hard thing, Dev. So at least you're not alone in the fact that it's really, really difficult. I love like Renato says, yeah, under promise, over deliver. I like the reminder that just because you have something doesn't mean you have to send it out right now. For me, that's a lesson that I have had to learn over time with emails. Hey Libby, just because you get an email doesn't mean you need to answer it immediately, even if you can. I frequently need to at this point in my career, in my life, let an email simmer for like a day or two so I can batch it with other emails and do them together because they're the same type of thing.
But if you are early in your career right now, just know that if you answer emails immediately, right away, 20 minutes after someone sends them to you, you are building a standard that is unachievable long-term. This is not something you can keep up. Don't do that to yourself. 24 to 48 hours is generally okay unless you have an SLA, which is a service level agreement that is different than that. If you have an agreement with a client that you have a quicker turnaround time than that, by all means, observe that. But don't do that to yourself. Free career advice for Libby, everybody from her mess ups.
Lightning round questions
Okay, so we have so many questions in Slido that I'm wondering if you would be willing to do a little bit of a lightning round, like let's see if we can get a few questions in where we go as fast as possible. And this is where I also would love everybody in the chat to chime in with your answer too, because it's always fun seeing that. Okay, so the first one, Nelly had asked it and it has a little asterisk, so I will read that for her. It says, what pieces have you found are better done by a human than by AI? What do you do with your team to ensure the human aspect is still important?
The server logic, especially like in an app, knowing the actual computation is something that I'm very hesitant to give up control of. The UI is something I've been a lot more okay to let go of and say, I can read that, it looks reasonable, it looks fine, let it go. So I'll give you a quick one on that. Server logic, okay. I get nervous if I can't understand something that something wrote and then if somebody asked me a question, if I couldn't answer it, I'm nervous.
I love it. Okay, that was a two-parter, Nelly. We might have missed the second part. We have an anonymous question that says, how much does medical or engineering knowledge factor into your role versus pure data science? I'd say you build it up over time. I don't think we expect anyone to have any kind of specific domain knowledge coming in, but knowing about the products and processes of what you're doing really enables you to be successful. And so you really need to be curious about it to succeed. Not so much medical, but some engineering for some on the manufacturing side. Yeah, some engineering, that's a good point too. I think everybody gets nervous that they don't know everything about a domain when they're applying for a job. You learn over time. You really, really do. And I think that people understand and expect you to learn over time.
Fantastic question. Let's see. We have another one that's anonymous. It says, do you deal with analyzing freeform text that's generated from your process users? If so, what have been your most effective tools for re-engineering or processing your freeform text? Any freeform text for you? Yeah, definitely. Somewhat. Yeah? Somewhat. Like one example, so we have an internal request system where people submit what they need. And we have an automation now where we try to route it to the right person in our 30-person org of analytics to see who's the right person to try and tackle it. Anything freeform text has definitely moved to LLMs in the last few years. A long time ago, I did some more like NLP things, but I feel like LLMs have really taken over anytime there's freeform text.
Yeah, I'm going to second that. I work with freeform text as well. I also work with semi-structured freeform text, which is confusing. Ask me later. And I use LLMs for that as well. I love NLP. There's so much that you can do with basic NLP that you don't need an LLM for, but whenever it's super freeform text and you have a very specific task, LLMs are really, really great at that. Thank you, Anonymous Asker. That was a fantastic question. Okay, advice section.
Advice: getting involved in AI projects
This question is anonymous and it says, I am keen to upskill in AI. I contact people in my company who are working on projects and offer my help. They're happy to talk to me about their projects, but I'm finding it difficult to get involved in their projects from the design and coding side. Do you have any advice for me?
I have one last thought on the last one, so let me go to that for one second, then I'll bounce to this. LLMs are really good at writing a ton of regex for you. I'll have an LLM write hundreds of lines of regular expression and use that. Thought of that one. That's a tough one. I would say that's something I've seen people succeed with is reaching out to people and trying to volunteer their time. I would say if you're first getting involved, I would not be super picky about what you're doing. If it's not the most complex side of it, if they need help with something that's a little bit more on the side, I would be totally willing to do that. You can then kind of broadly learn about what's going on and hopefully over time, uplevel your complexity and get to a little bit more heart of the problem.
If you're kind of just starting with something or just offering your advice to someone or trying to collaborate with them, being a little bit flexible on what tasks you're willing to take on is a lot more probably likely to succeed and then building up trust with them. Rather than trying to say, let me help with the absolute central core of your really important thing where they're probably going to be a little bit more apprehensive about someone totally new. I love that. Yes. Build trust. That is very, very important.
Tim Golden had a reply on that one, which is an example of how you can build trust. Tim said, find out the language or platforms involved. See if you can do some hello world projects on your own. That is like beginner projects that help you build foundational skills in something. Then you can show those as examples next time you're asking if you can help so that you can sort of show that you're interested, prove that you're working on it, stuff like that. That can help you build a little bit of trust. I wanted to say that Kenneth had chimed in in the chat and said it also could be a permissions issue. You could offer to QA, which is like help do quality checks or help with the presentation review if you don't have permission to be in on that more nitty gritty foundational side. You guys have the best suggestions. Thank you, everybody.
Advice: getting your first data science job
Oh, another advice question. Anonymous, it says, if you had recently graduated from a degree in data science, how would you go about getting your first job today in 2026? No pressure, David. Yeah, no, it's tough. The biggest thing I would say is to find creative ways at the beginning to kind of make inroads, to make connections. And I'll tell two anecdotes on that.
When I just was finishing my freshman year of college, I really wanted to do like an analytics internship. I wasn't having success finding one that would take a freshman. And so I reached out to a bunch of like local nonprofits and said, can I come volunteer? Do you have analytics work they need help with? And so I worked with three nonprofits for the summer, split my time between them. And then I was able to highlight on my resume, like all this, you know, really great experience that I had. And that helped me get an internship with intuitive.
I've seen a lot of folks where they take roles that are maybe not 100% a match for what they hoped for. But it's a way to break in and start doing things, you know, starting on the analytics space, maybe. And then, like the prior commenter mentioned, you know, offering to work with people who are doing the work that you really want to aspire to collaborate. There's someone in my team who basically followed that path. They were in more of an analytics role. They were really interested in data science. They said, hey, can we collaborate on some stuff? They worked with their manager to make it happen. They, you know, definitely worked extra for a period to kind of dual do these things. They did a fantastic job and we hired them into our team. So being flexible and being creative at the beginning would be the biggest thing I would say.
So being flexible and being creative at the beginning would be the biggest thing I would say.
Excellent advice. We don't even need to ask you a career advice question today because I feel like we've asked you three already. I am going to add to this and say there are so many tools that people use in the workplace that you can learn to use for free, especially our products because they're open source. So if you're like, oh, well, I've never used connect, go get a free connect cloud account, right? And start building some shiny apps. David builds shiny apps and his team builds shiny apps, right? You could both build a shiny app and deploy it to connect cloud and kind of show that you already have that skill set without needing to like pay for a pro license. There are a lot of different ways you can do that if you Google around, including I can investigate this SQL database with a free license or whatever it is. Do as much of that as you can in exploring so that you don't feel limited by these tools that feel proprietary. David, I know that you said you all use connect. And I'm having a personal
conversation with one of my friends lately around trying to get people to use the things that get built that they spend a lot of time building. Do you have any advice around how to get people to use the things that you build and how do you connect your users with the stuff that you guys build? Because I feel like you have had a lot of progression with getting things implemented, new tools like connect and shiny apps and stuff inside intuitive. Yeah, we've been using connect for a little bit over six years. It was there when I joined. So it's been a big part of what we've done for a long time.
Two things on that. One, I would say, I think it is important to somewhat separate out your kind of like personal feelings about a project from its impact. Because I have absolutely had projects that I was like, this is the coolest thing. I'm incredibly proud of it. I worked on stuff I really felt good about. And I even had like a huge one that just leadership decided not to implement for different reasons outside of my control. And I've had some projects where I wasn't super emotionally invested in it. It was like to me fairly simple, a quick thing, and it just totally took off. So and I've seen some people sometimes really struggle with that where they work on something and it doesn't get the traction they want. It could be different for different roles. But for our role, there's definitely like an acceptance that it's not always one to one, right. And so our products, you know, we don't judge people based on that always, that's certainly a factor.
I think it is important to somewhat separate out your kind of like personal feelings about a project from its impact.
One huge thing I would say is to not overbuild at the beginning. We really try to just start with an MVP and keep it as small scoped as possible and see usage and see utilization and build from there. Connect has like fantastic usage tracking. So we use SSO for all of our logins. So I can see exactly who signed in when with what their name is, what their time is, how long they spent on it. And Connect also makes that data available via API. So we've invested the time of, we've built our own analytics on how people are using our applications and products where we pull that data from the API, and then produce apps that consume it. And that guides a lot of our decision making. And we've said at times, like, you know, we saw more potential for a project, but the truth is, it's not getting usage, and we're not going to invest time in it. And we've had times when we're talking to like our leadership in an area, you know, and say, the planning leadership says, oh, this is great, we need to spend more and more time on that. And we say, hey, here's the data, like, the people in your org aren't using it, they don't want to make it a part of their workflow. So, you know, we want to rectify that before spending more time on it.
That was my next question was like, does this development pressure or the ideas, is it coming from the top down? Is it coming from users asking for something? And sometimes that could be, that could make all the difference. However, I have seen like end users ask for something that, again, that they didn't end up using, it just felt like a good idea. But maybe the modality was wrong. You know, who knows? Yeah, we definitely say like, we're, you know, we try to start small, keep it small. As we see usage, as we see buy in will grow, right? And really thinking about what is an essential for an MVP for a starting point, and what's something good to add on later. Because if you if you start building like the perfect thing, number one, you know, you may be overbuilding to where it's not really needed. And a lot of times, the direction of our products changes pretty drastically. Like we'll build something thinking, this is how we think users are going to use it and the value they're going to get out of it. And then sometimes they come back and say, actually, you know, it does this other thing on the side that to you was more a minor piece, but to us, that's what we really care about. And so we have to course correct.
Prioritizing projects and managing requests
Excellent, excellent advice. Okay. Do we have one more question that we can fit in? Maybe one more. Let's see. We have, oh, we have Ken's question that was tagging on to Deb's question. Ken, do you want to ask that one live in our last little five minutes? I can highlight it in Slido for you. Ken Roosevelt.
Hey, how's it going? I think this is my first live question. So tagging on Deb's question, with competing interests and limited resources, as a service department with customers in the organization, how are you prioritizing projects? Is it you? Is it a management team, leadership?
Yeah, great question. It's a mixture. You know, there's a few kind of philosophical things we've done. So one is we have like kind of a dumping ground for ideas where we just put in anything that comes in and then the ideas and that can be from other people or from us internally. And so we actually have like a GitHub repo where we create an issue for every idea from anyone. We have a standing once a month call with our leadership where we go over what are our priorities for the month. So that could be continuing on existing projects or new ones we're picking up. I would say that I drive a fair amount of it, but then keep leadership both in touch. And then on key decision points, especially, we'll talk with them to get feedback. And so my broader leadership, and then they connect that with even beyond that, trying to make database decisions. So we're saying, OK, this application is really getting value, getting usage is what we're seeing. We want to lean into it. But there's also kind of our gut intuition of just like, what is the potential here? Is it really going to have a larger impact? So the biggest things I would say is just trying to be very clear, trying to get things out of people's heads and documented on paper. So you're not carrying that mental load. And so it's very traceable are some of my biggest philosophies.
And do projects come to you from management or do you triage them and then put an analyst on it? It's all over the place. So we get requests from leadership. We get requests from more of our business teams. We have some of our most successful projects have been one that we've been we've created on our own. We've seen like, OK, they asked for this, but we have this other idea. And so it's a kind of a blending of that. We also have a structure where we have data scientists. But then in my kind of sister team, we have data analytics who's doing more descriptive work. And so we partner with them a lot to see when things are kind of reaching a maturity standpoint. So it's all over the place. It's very loose. It takes a lot of time and energy, but just trying to keep everyone in the loop and trying to keep things documented.
Favorite tools and what's exciting in data science
Well, thank you. Thanks, Kenneth. OK, we technically have time for one more question. There is another small one on the Slido that is anonymous, and it says, what are your favorite data science tools, packages, tutorials, softwares, books, etc.? I will extend this out and just say, like, is there stuff that you're excited about right now in data science?
I'll plug again. I love Monte Carlo simulation. I've had really good success with that in a number of domains. And it's like one of my favorite things to do. And it also just really matches with my mental model for processes. And so we built out across a lot of domains, really successful things with that. Favorite packages and tools, I would say, you know, there's some new ones that we're really loving, like Elmer and ShinyChat have been really fun. Some of our more classic ones, I would say all of the tidyverse for sure. There's some really great like shiny extension tools, like shiny widgets, I would say is a big one. Let's see other things that Blastula, like anything that just makes processes smoother. All of our database connection tools. I think, yeah, integrating LLMs in a controlled way for different things has been really exciting. And we have some things that are looking really promising there. But it's obviously like constantly changing all the time. As always, as with the LLM and AI stuff, right?
Integrating LLMs in a controlled way for different things has been really exciting.
You mentioned Blastula. This is a good time to say we're missing Isabella or Isabella would like normally quickly be like Blastula link. I'm trying to keep up with Isabella. It's so hard. Isabella is so much faster than we are. Go check out Blastula. If you haven't used it, this is a Rich Iannone package. If you know Rich Iannone, he makes point blank, great tables, two packages like the GT package that you have probably heard of and used. He also makes Blastula which helps you send emails. And if you have connect, PositConnect like David has, you can also schedule emails inside of PositConnect. You have so many ways to like get the thing done that you want to do. All of those are fantastic packages. Yeah, Zach says Isabella is sonic. She is. We miss you, Isabella. We'll see her soon.
Also, Saptarisha wanted to call out, hey, if you have a university email, a .edu, you can access O'Reilly through that for free often. And I will also say if you have a university affiliation, log into your library and you can often access papers, books, tons of material for free. Go do that while you still have that .edu email address access, okay?
All right, David, this was so much fun. Everybody in the chat agrees this was fun. We had a fantastic time. I hope you did too. Thank you so much for having me. Really appreciate it. And yeah, hope people got some interesting stuff out of it. Yes, I think that we all did. Everyone's applauding. Thank you everybody for hanging out with us. We love spending time with you. We will see you on Tuesday at the Data Science Lab, pos.it.dslab. We will see you next week at the Data Science Hangout, and we cannot wait to see you on the Discord channel. So make your way there, because we're there, you know, all week long. You don't have to wait till Thursday. If you want to save the chat, there's three dots at the top right of your chat panel if you're in the Zoom app, at least, so you can save the chat. And I think that that's it for me. Go sign up for PositCons, because I want to hang out with you. Okay, bye!