Transcript#

This transcript was generated automatically and may contain errors.

Hey there, welcome to the Posit Data Science Hangout. I'm Libby Heron, and this is a recording of our weekly community call that happens every Thursday at 12pm US Eastern Time. If you are not joining us live, you miss out on the amazing chat that's going on. So find the link in the description where you can add our call to your calendar and come hang out with the most supportive, friendly, and funny data community you'll ever experience.

Can't wait to see you there. I think it's time to introduce our featured leader today who also is joined by a pet. Leslie, would you like to introduce yourself? We are joined by Leslie Emery, who is Senior Principal Data Scientist at Bristol Myers Squibb, which we will be calling BMS, I think.

Yeah, thanks Libby. Definitely BMS all the time. So I have been working at BMS for about five years. I'm based in the Seattle office, and that's where I did grad school and worked as a research scientist afterwards as well. And at BMS, I work in the research organization, and specifically in the Informatics and Predictive Sciences team, which is, you can basically think of that as like computational biology or computational science. And then the smaller team that I'm in, Knowledge Science Research, we are focused on helping the other computational scientists get their work done in a more efficient, more reproducible way, and helping to bring together a bunch of different data sources from across the company to make it easier for computational scientists to work with our internal data.

All right. I feel like I have so many questions already, but let's hop to the most important Leslie question. What's something you like to do for fun? Yeah, I have a lot of hobbies. The biggest two are probably knitting and reading. I'm in two different book clubs and spend a lot of time reading science fiction. I just voted to nominate for the Hugo Awards this year, so I'm excited to see how that goes. And I've always got probably at least three knitting projects going at a time.

Well, we have book fans and knitting fans in the Data Science Hangout crew. I am a big data book person. I love science fiction and fantasy books, and we could get stuck on that topic very easily. So I'm sure there will be, if you have a question about Leslie's favorite books coming up, put that into the Slido.

Data harmonization in pharma

All right, Leslie, I would love to get a little bit more context around the types of data that you see and the types of problems that you solve with that data. And I know that you have a PositConf talk that got accepted, so we will be hearing from you live in September, and it's on this topic. So would you like to give us a little info on each of those things?

Sure, yeah. So at BMS, we have a lot of internal data from clinical trials that are going on, and then also more experimental research that's being done, either basic mechanism of action kinds of things or translational research on the trials that we've already done. And that's what I often help with, is people doing additional research questions on data from our clinical trials. And the data that we have from clinical trials, it's designed for the clinical trial first and foremost. So the data that is going to be the primary outcome in a trial is really well formatted, well curated, easy to work with. But experimental biomarkers, for instance, new assays that are just being developed, things like that, it's not as uniform in the format.

And we often want to do analyses across multiple studies. And each of these clinical trials is kind of its own thing with its own team, and so there can be significant differences between them. So I help to do a lot of data harmonization, combining multiple clinical trials together, solving differences in format, data cleaning issues, and then also trying to make decisions, like if there's an actual difference between the way an assay was done in study one versus study two, how do we combine that in a way that's still valid to analyze them together?

It's a lot of the data cleaning, data wrangling work that I think there's a bad reputation for it, but it's so, so important. And I get really excited about it and trying to make the data that we already have higher quality, easier to use. And so I've done a lot of these data harmonization projects, and it's kind of the same things over and over again. Every project you have to identify which columns are compatible between studies or data sources, figure out does the content match, do the units of measurement match, that kind of thing. And it's, every project is its own unique thing, but also there's so much repetition in the kinds of things that you're doing.

Using LLMs for reproducible data workflows

And so I'm really excited about ways to automate that. And so one thing that I've been working on recently is a way to use LLM assistants for these data harmonization projects in a reproducible way. So a lot of the work that people are doing with data in LLMs, it might be through something like DataBot and Positron, and you're having this interactive conversation with your data. But then when you're done with that, it only works for that one data set you were looking at, and it only captures exactly what you did in that session.

So what I'm working on and what I'll talk about in my PositConf talk is a way to capture the output of an LLM into config files, specifying what kinds of data transformations do you need to do, what parts of your data sets that you're starting with are compatible with one another to be put together into the same outputs. So capturing all of these steps into config files on a standard data model so that you can have the LLM work on the parts that it's good at, which is making these config files, and then put it into a reproducible, deterministic pipeline, and have that part do what it is good at, which is reproducibility.

So capturing all of these steps into config files on a standard data model so that you can have the LLM work on the parts that it's good at, which is making these config files, and then put it into a reproducible, deterministic pipeline, and have that part do what it is good at, which is reproducibility.

This is one of the things that has made me most scared about LLMs in the past, is just like we can't put them into a reproducible pipeline and expect them to do the same thing. So you are focusing on solving that problem and making it so that the LLM is creating the instructions for the work that you are doing, and those instructions are reproducible, and you don't have to depend on the LLM to be a deterministic thing when it cannot be. I think that's amazing and interesting, and I can't wait to hear more from your PositConf talk.

I will not pump you for information on your PositConf talk. I will let you work on that. I know it's not done yet. But I do want to point out, not a question from Slido, but a conversation we're having in the chat. The chat is always so much fun because we have other people who do data harmonization in the chat here, and there's a conversation about OMOP. Do you know what OMOP is?

I'm acquainted with OMOP, yeah.

Okay. I'm curious. Raha says we all need to use OMOP. I will let you all investigate what OMOP is with links and stuff that are in the chat. But data harmonization sounds like a very fraught endeavor, perhaps, and something that a lot of people struggle with.

Community Q&A

Okay. So, we do have to hop over to the Slido and get started, and I would love to get started with Noor. Noor, if you're able to unmute and ask your question live, I would love that. I will highlight it, and I'm also going to stick it in the chat for everybody to see.

Hello. Can everyone hear me?

Yes.

Awesome. Hi, Leslie. Lovely to meet you. I am also a bioinformatician by training, so I'm also in that genomics field at the moment. And I often describe myself as a translational bioinformatician, partially because one of the things that attracted me to the field was existing at the intersection of multiple domains. It helped me not have to choose what I wanted to do, for better or worse. Given your role at BMS, you also seem to be existing at the intersection of multiple departments and so forth and multiple needs and challenges. What are some of the challenging aspects and rewards of your role?

Yeah, that's a great question. Thank you. So the most rewarding thing for me is to be working on something that someone else is specifically asking for and waiting for and going to be excited to use like the moment I have it ready. So I came out of academia, I did a PhD, and then I worked as a research scientist for five years, where I would be working on something for months at a time, and then or sometimes years at a time, and then publish it in a paper and put it out into the world and never hear anything back about it. So I love the immediate feedback of like somebody else at the company needs something done. And I am the person who can help them. And it's super exciting to be able to do that.

You know, I see a lot of people who they're trying to do these kinds of things themselves in Excel spreadsheets, and they don't want to learn coding, they just want the data. And so I love to be able to find somebody like that and help get them an analysis ready data set. So they're not doing like merges in Excel.

And then BMS is an enormous organization. So that was a huge adjustment. When I moved to BMS, there's so many people. So everything I do has to be a collaboration. And there have often been times when, you know, I'm in a meeting, and we're talking about some new type of thing that needs to be done a new analysis, a new type of tool that needs to be set up. And my like academia mindset was like, Oh, well, I can learn to do that. You know, I could, I could spend a month or two learning to do that and do that for us. But in an enormous company, there's often somebody else who already has expertise. So I needed to learn like how, how to instead think like, is there somebody else who already has this expertise, and maybe I can help consult with them and learn some of those skills from them. But I don't have to learn every single thing myself at BMS.

Wonderful. Thank you, Nora, for the question. Okay, we have a couple of things that I think that we can get in that are follow ups to that. And one is what LLMs have you looked into for data harmonization? Have you found different LLMs work better for what you do? And that was an anonymous question. Thank you, anonymous asker.

Yeah, that's a great question. So I have not yet gotten to a point where I am like, fine tuning things like does, you know, model version one or model version to do better does model a or model to be do better. I'm still sort of working out the basic workflow and all the different moving parts of this automated data harmonization workflow. And so what I've kind of just picked a model that others at BMS have used and liked and recommended and stuck with that. And what I've been using so far is I think Cloud Sonnet still. And there's also, you know, there's at an internal company, there are some constraints around what's available. You know, we're using internal LLM endpoints. And so, you know, sometimes we might need to wait a little bit to get a new version of some models.

I think that's a really common experience, though. I am working on the data science hangout sort of 2025 end of year survey that we did all the responses. And there was a question in there that was, what LLM do you use? What's your LLM of choice? And like, what do you use inside of your IDE? And there were a lot of responses that were like, I can't use one. I'm really, really limited by what my employer lets me do. So I think that's pretty common.

Thank you. And I think I actually we have actually been pretty lucky in that we have a lot of options and they are adding models pretty quickly. And we have a couple different ways of interacting, like the approach at BMS has kind of been like, let's try all the things and keep everybody, you know, on the forefront and then wait until we see how things shake out before we kind of converge on any single one. Yeah. And then there's always trying to keep everybody safe, everybody's information and data safe as well.

OK, a quick question we can get in here, which is from an anonymous asker as well. And this is actually the very first question that popped up in the Slido was, did you knit your sweater? Because it's lovely. I absolutely didn't knit my sweater. It's the salty air tea. That's the pattern name. A lovely, lovely thank you,

anonymous asker. I love the questions that are about are the fun stuff. So let's go with one more from Zach. He has an asterisk next to his question. So I'm going to ask it. And it's what sci-fi books do you recommend? And then also he has a legitimate question in here as well about the data stuff. Do you think I could help or hurt with the data harmonization part? And I think that this is a we've already talked about like it's obviously helping. Were you nervous about it hurting in the beginning? Yes, absolutely. And I would say I'm still I'm still cautious.

I feel really lucky in that I have already had a chance to develop an expertise in the areas where I'm trying out LLMs. And so I have the background knowledge to be able to evaluate the output that I'm getting. So I'm thinking in the context of that answer, I'm thinking more about like coding assistance as opposed to this specific LLM or data harmonization project. But there have been areas where like I used GitHub Copilot to learn to use the play rate framework to write automated tests for a Shiny app. And that's an area where I didn't have expertise. But I knew kind of the design that I wanted for some tests. And I was able to tell like all the suggestions it's giving me here for writing these tests like I don't like the way these are architected. And so having that background knowledge first, I think is really, really important.

So going back to using LLMs for data harmonization, one thing that is really key is having like an evaluation framework. So I have a colleague at VMS who is building like a custom like LLM output evaluation portal in a Shiny app. So we're really hoping to have something like that to help with the harmonization project.

And you always you are always going to need someone with the expertise in the data. So you can't just, you know, drop a couple of data sets into an LLM and say like harmonize these and then evaluate the output without having understood the data that you're starting with. And often the I think this can get overlooked when people are talking about how much time LLMs are going to save us. That you still need to put the time into LLMs. And so that's that you still need to put the time in to understand the data that you're working with. And that is something with like a hard limit. There's, you know, there's often no way to speed that up other than spending the time digging in and looking at the data.

And that is something with like a hard limit. There's, you know, there's often no way to speed that up other than spending the time digging in and looking at the data.

I agree. You have to know what good looks like, which is sometimes hard, very challenging.

Sci-fi book recommendations

What about the sci-fi book recommendation? Anything that's on this lab lately or that you've really enjoyed? Yeah, I'm going to give my favorite recommendation, which is the Murder Bot series by Martha Wells. She's got the latest one coming out in May and I'm super excited for that. And then my other recommendation is anything by T. Kingfisher. This is more fantasy, but I just had the opportunity to see Kingfisher at a local event and she's just so entertaining and writes a huge variety of books. And so I think almost anyone would be able to find something by her that you would like.

Wonderful. I will throw my vote in for Robert Jackson Bennett's Shadow of the Leviathan series, which I didn't expect to love, but really, really do. Book one and two came out. I fell in love with the characters who are kind of a fantasy version of Holmes and Watson, I would say, and I just love them so much. And the third book is, I think, coming out in August. I'm really excited about that.

Pharma data standards: CDISC and OHDSI

Let's head over to Slido again and let me see what we have here. Because there were a couple that were very specific to Pharma stuff and I don't know how far in the weeds you want to get with that, but there was one that said, since this is coming up in the chat, does BMS use data curation like what's created through CDISC or Odyssey? And if so, how does that work? And thank you to someone in the chat, probably Mike, for letting me know that O-H-D-S-I is Odyssey. Yeah, so that's a great question. I'm not familiar with Odyssey.

All of the clinical, well, most of the data from clinical studies that I work with is coming from the CDISC formats, which there's SDTM, which is sort of like the initial data format, and then ADaM. I think it stands for analysis data model. And the idea of that is it's supposed to be like the analysis ready data set and it's prepared by the statisticians.

So we do use those. And in my team and the computational biologists doing exploratory research that we're working on, we're often starting with the CDISC data formats. And they are standardized to a degree.

It is such a flexible standard that when you try to do any specific analysis, you often need to do additional reformatting. And then there are also differences in the CDISC version from study to study. CDISC is so flexible that I imagine there are different company versions of using CDISC because they're all... When I started working at BMS, the office I started at had started as a company that was acquired by Celgene and then Celgene was acquired by BMS. So there's a Celgene formatted CDISC, a BMS formatted CDISC, and then any other smaller companies that BMS acquires, they often have their own CDISC format as well. So it's nice in that there's some standardization to start with, but to truly get to an analysis ready data set, you often have to do a lot more work. You have to converge all of those things into the one thing that you need to work with.

That's wild. And it's so much more detailed than I feel like I can understand, not being somebody in pharma. But I hope that that was helpful for our pharma folks and our bio folks who I know we have so many of them in our community.

Getting company resources for infrastructure changes

Okay. So I have another question here that was anonymous, which is how do you go about getting your company to put resources, i.e. that time you need into changing up your computational infrastructure? Yeah. If anyone has suggestions for this, I would love to hear them. Yes, let's talk about it in the chat. You know, especially at a really big company, it's really hard to make changes like that. One approach that my team has taken, I think, is to just focus on the parts that we can actually make change in. So we, like, one of my closest coworkers, Clara, she gave a talk at Posit Conf last year about an internal data standardization framework that we've developed called data as a product in R or DAPR.

And so that has been something that we have had time invested in. And we have done that by getting our closest stakeholders and collaborators on board with us spending time on that as part of meeting our shared goals. So, like, if one of my stakeholders needs to do a cross study analysis of five different studies, potentially multiple different disease types that would require a little more work to put the data together, we have their support for taking a little bit of extra time to do that in the DAPR framework because they know it'll be easier to work with and a longer lasting analysis ready data product than if it's just like a one-off data set sitting on someone's OneDrive.

Fantastic. And we do have a couple people in the chat who have talked about this or who've chimed in. Josh says, the easiest way I've had is to support true dollar value of the infrastructure. So time saved, time to insight or time to value, decreased downtime, etc. I think any time that you can quantify something like that is great, although quantification is so hard to do. And then Raha said, wait, this was my question. It was anonymous, but Raha is the one who asked it. And she says not doing this makes other things downstream take more time. So her new idea as an academic, though, not as an industry person, is to just write a grant just to buy her own time to do this and to fund it. And then it's probably different from industry. Obviously, it's like a conversation where you're advocating for yourself and sort of pitching to leadership that you need the time. Adam also said, on a very small scale, requesting extra nodes or additional memory, I try to make the time case as well by incorporating speed tests into every process without any exceptions. Thank you, everybody, for all of your feedback on that. That was amazing.

Tech stack and tools

Okay. Let's head, actually, you know what? I want to, since we're at the halfway mark, we usually ask this question by the halfway mark. And if we don't, I'd like to get it in, which is context building around what tools you use. So we talked a lot about the process of data harmonization and the challenges behind that and some industry-specific things in pharma. Are you working mostly in R? Are you in Python? Are you in SAS? What does your team use? Do you have any sort of roundup of like a tech stack that you would like to talk about?

Yeah. So I and my immediate team, we work mostly in R, a little bit of Python here and there, and there is support for Python in our larger team, the Informatics and Predictive Sciences team. For myself, personally, I used to work a lot more in Python, but there's kind of a lot of momentum behind R in my team right now. So since that's what most people are using, it's easiest to work there. I am probably working like one-third of the time in Positron now. I'm trying to use it for new projects and for older projects that I already have set up, sticking with RStudio for now. We do have the like Positron team, Positron Studio for now. We do have the like Posit commercial suite that we use, so Posit Workbench. I use occasionally and Package Manager for publishing our internal packages and making them easy to install. And then Posit Connect, we use all the time. We've got so much stuff deployed there, reports just to make it easier to share and lots of Shiny apps there.

Oh, Shiny apps. That's a great topic. I hope we get to talk about that. And if you have not ever used Connect, I realize that I think the last time we talked about this, we didn't define what Connect was, and somebody had actually reached out to me and was like, you guys keep saying Connect, but I don't know what that is. Connect is a place where you can sort of push button deploy all of the things that you make, whether that is a Shiny app or a Quarto report or a Dash app or a Streamlet app or all kinds of other things to be securely viewed by everybody inside of your organization. And you can create vanity links and all of that stuff. So hopefully that helps somebody with a definition. And then Leslie mentioned Workbench, and Workbench is a place where you can go into a browser-based environment and have access to things like RStudio, Positron, and notebook editors and all kinds of things.

Live Q&A with Paul Lloyd

All right. Let's head over to Slido again. And one moment while I read things, because we had a little asterisk. Okay. So there was an anonymous question, but I see that Paul added that it was Paul who asked it. So Paul, if you would like to unmute and ask your question live, that would be great. And I can put it in the chat. Hopefully we know which Paul this is. Hmm. We have a few. Because we have four. We have four Pauls. I think this is Paul Lloyd. Okay. Okay. There we go.

Hey, sorry about that. I pressed anonymous before actually getting a chance to put my name in. No, it's okay. It's easy to do.

Yeah. Thanks for hanging out in this session, as I rejoined it today. Really interesting topic around harmonization. And I think you partially answered one of the questions I was asking in asking about how often do you just have to say no, like this shouldn't be harmonized, and really, this is not a compatible data set or data type that you want. I'm sure you have lots of really interesting things and lots of people that want their thing to be part of it all. But as the expert in the room, how often do you actually have to push back and say, yeah, no, we're not doing this because it's not good science?

That's such an interesting question, because I think it's a really good question. And I think that's such an interesting question, because the situation that I often end up in is the reverse of

that. So I'm often having to say, no, we can harmonize these. You'll have to consider that there are differences going in when you interpret it, but it is possible to harmonize these and analyze them side by side. Because often the stakeholders that I'm working with, they're so close to the data. They're so close to one specific study or one specific disease that they are thinking about all the time. It's hard for them to think that you could compare multiple myeloma and lymphoma, because they are really different diseases. I'm sure they are, but also a lot of the data that you put together for these two different diseases has valid comparisons that you can make. So I'm often the one arguing that we can harmonize it.

And I think the situations that I have run into where you just can't harmonize it at all, it's often more an issue of we didn't capture the right data to begin with. So you want to analyze some particular biomarker across two dozen studies that you know have been done and that you know it would have been valid. But in half of the studies, that biomarker that is so interesting was only captured at one time point or was only captured in a dozen patients, because it was something they were just trying out. Wonderful question, Paul. Thank you. And that was Paul Lloyd.

I'm dying because of the Spider-Man memes that somebody put in the chat. Thank you, Mariana. I really needed that today. Okay, we have a question from Wisdom. Wisdom, if you are able to unmute and ask your question live, I would love that, because I even have some follow-up on that as well. Yeah. Hello. Sorry for the background. I'm on my lunch walk. That's all right. I love it. I was thinking about, so I'm a scientific consultant, right? So I help with the data stuff, just like in life science, people who have big data problems is what our company works on. And we keep coming back to the harmonization and how everyone wants to approach it in different ways. And I was curious if in your current flow at all, have you figured out how to use the ontologies and data dictionaries, which are two different directions for addressing that? And if so, you need upstream buy-in from that, from the data producers. How are you getting them to agree and actually label their stuff? And if you're not, what have you tried so far? I'm just curious.

Yeah, such a great question. So about data dictionaries, a lot of the data that we start with from the clinical trials comes in SAS format, and the SAS format files have embedded data labels. So in our harmonized data sets, we'll pull those labels directly out of our starting data sets and maybe make some changes to make them compatible across studies. What I would like to do is to have an LLM help with combining the definitions that might be different across studies. I haven't gotten there yet.

And then as far as ontologies, so this goes back to the question that somebody was asking about OMOP. you could drive yourself crazy trying to pick like the one perfect ontology or controlled vocabulary to use or controlled vocabulary to use for any particular project. I've been stuck in that kind of question with multiple projects at this point in multiple different jobs and no one perfect ontology exists so you should also try to avoid the classic xkcd problem of coming up with another ontology to solve this. I think the solution that I'm happiest with at this point is trying to combine multiple ontologies so pick the first ontology that covers the the broadest the largest proportion of what you need to include and then if there's anything that's not covered by that pick selectively from other ontologies that you have available.

ontologies so pick the first ontology that covers the the broadest the largest proportion of what you need to include and then if there's anything that's not covered by that pick selectively from other ontologies that you have available.

I think there was also a part of the question about getting buy-in from data generators. That is so hard and especially at an enormous organization like BMS. The people putting the clinical data sets together to begin with are so far upstream of the analyses that my stakeholders are doing and the data sets, the specifications for these data sets are put together often years before the analyses are done.

I think it is easier to get buy-in from trying to collect data that's already there as far as trying to get it included from the beginning. I think the only thing that can really help is just building strong connections with those different parts of the organization. So I have tried to build connections to the biostatistics organization at BMS which is in charge of building these initial clinical trial data sets so that maybe when they're in a meeting next year for planning a new study they'll think, oh you know I had this conversation with Leslie last year about how they don't know what is in this data because we didn't capture all of the documentation on it. So maybe we need to include that from now on.

Thank you to the chat for finding us XKCD memes so that we can look at them. Also I just have to take a small break here. I know we have 17 minutes left but Leslie we love your cat so much. We have decided in the chat that he's perfect and he gets to do whatever he wants and that we want more Wallace. So let's introduce Wallace because Wallace is very special. He's an old kitty. Yeah absolutely. So I agree Wallace is the perfect cat and he does get to do whatever he wants. He's 17. He is named after Alfred Russell Wallace. Yeah and I'm sure he'll show up again. Yeah he's like tantalizingly just halfway out of frame but he's making himself known every once in a while and he has a lot of fans. I just needed you to know.

Okay Wisdom thank you so much for that question. We all wish that we were out on a walk with you Wisdom. I know that you probably can't see the chat but we were all going like wow Wisdom has it right. Like let's go outside.

Career advice: sitting at the intersection of technical and domain expertise

Okay I wanted to get to this anonymous question because I think that it touches a little bit on career advice and we love that. It's from an anonymous asker and it says can you share any advice for getting into the type of work you have been doing specifically sitting at the intersection of technical knowledge and data science and having that skill set valued by an organization? What a great question. Yeah do you want to tell talk a little bit about how you got here maybe? Yeah absolutely. It's interesting because I had no intention to work at a pharma company. How did this happen? So you know I did my PhD in genomics. I wanted to be a professor I thought but then it turned out I didn't want to leave Seattle. I did not want to move. I didn't want to have to give up everything to move wherever academic jobs were. So right out of grad school I took a job as a research scientist in the biostatistics department at the same university I'd gone to and worked there for five years and it was great.

I was in like sort of an analysis core team which is like you know our team did statistical analyses on genomics data and related clinical traits for the people that had genomic data and so I found this position that was like technical expertise but still working in science and that gave me some time to learn exactly like the kinds of tasks I wanted to work on. I was able to clarify that I really love being able to do this sort of thing where I'm getting a task done for a person in the scale of weeks or months and not years or a lifetime. So take you know taking that time to figure out what things were important to me in a job and in a career and what I wanted to spend my time doing and then when I made my next career jump I thought I wanted to work for a tech company.

I was like I really want to spend more time on building tools and working on data like that is what I find most gratifying and so I thought I wanted to work as a data scientist at a tech company and I kept looking at job listings and thinking this like I don't want to look at ad data, I don't want to work at look at sales data, I don't want to look at website traffic data and it turned out I still wanted to do something that was I felt having like an impact on people and for their health and so I found this job listing that even though it was at a pharma company that you know I wasn't particularly thinking I wanted to work at a pharma company the job listing itself was like oh that's exactly what I want to be spending my time on and it was about building Shiny apps, it was about making analysis ready data so I had a lot of those skills that I developed in academia already.

And in terms of like positioning yourself to be in these roles that are at the intersection of things I think the most useful skill to have is being able to talk to people in different specializations and different fields so a lot of what I do in my job is talking to somebody who is not a coder who doesn't know anything about R and trying to figure out like what is their goal and they often can't articulate it I need to be able to ask the right questions to figure out exactly what they need and then it's often like troubleshooting skills so being able to like you know jump on a screen share with somebody because they're having trouble with some Shiny app I built and they can't type to me accurately exactly what's going on so I just jump on a screen share with them and I can walk them through oh I see what's going on here's you know here's what you need to do or here's what I need to change about the app to accomplish what you're asking for.

I think the most useful skill to have is being able to talk to people in different specializations and different fields and trying to figure out like what is their goal and they often can't articulate it I need to be able to ask the right questions to figure out exactly what they need.

Amazing. Thank you so much for your question and I wanted to say I had said I had a follow-up for Wisdom's question and I forgot to let people know what that follow-up was which was it was on the topic of data dictionaries. I wanted to remind everybody that we're going to be joined by Hadley Wickham on Tuesday at the Data Science Lab and he's going to be talking about data dictionaries specifically the way that he incorporates clod code and using parquet files in order to create data dictionaries all in one go when exploring and getting to know a data set to make those data sets more readable and understandable by both humans and LLMs and agents so join us on Tuesday for that and Rachel put it in the chat pos.it slash dslab.

MCP and agentic workflows

Okay let's head back over to the Slido and let's hop over to I saw a question that was about MCP there it is so it was an anonymous question and it just says are you incorporating skills slash MCP for providing necessary context have you dove into using skills with agentic work or MCP? So I haven't yet I will say that on the the data harmonization LLM project I am working with an excellent data engineer Jason Moore who is taking like the the workflows and prompts I'm working on and he's building those into an agent and so he is he has also developed some MCP servers for some of our internal resources so I anticipate that being something that is incorporated into this LLM data harmonization workflow but it's not something I'm working on directly. All right cool valid question though and we also have a data science lab coming up on April 28th with Garrick Aiden-Bouie that's going to be all about agent skills so if you're curious about learning about agents that use skills or learning about skills with agentic work what is a skill what do they look like what makes a good skill when should I use them where can I find them join us on April 28th at the lab for that because Garrick is going to tell us all the all the things.

Okay we are going to head over to Slido again and we are going to talk to Russ. Russ would you be willing to ask your question live I don't see an asterisk there so hopefully you are. Oh no Russ I think I think I muted you I accidentally clicked your picture

Rachel's probably fixing it. At the same time maybe there we go. Hey we're still friends this is we did it sorry sorry Russ that's fine go right ahead. I'm a musician among a lot of other things or lots of other people and there are levels of harmonization in music and there's also levels of dissonance is there the same thing with data because I'm still kind of new to this is there some way to measure the level of like harmonization or dissonance like I was just thinking if it could if these things could harmonize well I'd call them Mozart if they were more dissonant I'd call them Stravinsky. I mean is there anything like that?

I love that as an analogy. I have I have kind of a personal annoyance at how like regular English words get co-opted for specific like tech or science things and I've been struggling with the word harmonization so I think you just helped me get over that a little bit I love that. I think there it would absolutely be valid to try to come up with a measure of like how harmonized is something. I haven't done any particular work in that I will say one of my colleagues back at University of Washington who I worked on my first big harmonization project with Adrienne Stilp she put together this workflow where we were harmonizing a data set and then after you so these these data sets were used for like the same linear regression model over and over again for large genomic analyses and she developed this pipeline where we would do a harmonization and then she would run like a null model on this data so basically you know not using the genomic outcomes and see like what were the residuals there were there any like patterns of variation that were showing up in the data that could be from like harmonization issues that we hadn't addressed yet so I think that's the really rigorous way to do that.

Transitioning from academia to industry

Thank you what an interesting question Russ. Okay Adam, Adam T which I think is Adam Taggart would you like to hop in and ask your question live? I will stick it in the chat for everybody so they can see it as well. Hi Lesley, thank you for all of your insights so far I'm I really appreciated the point you made quite quite early on about getting immediate feedback or payoff from making something for your colleagues making something which which someone will see value in straight away as opposed to academic which I've been in for the last eight years and you can spend up to I think five years was the longest project I worked on where you're wondering when you're going to see the payoff and whether anybody's going to care do you still have projects like that and if so how do you think you'd motivate yourself through them these days as opposed to just like pushing through the the pain as you might have used to certainly right I think a lot of us have been used to in our grad school experiences.

Yeah that's a great question so I I do have some really big long-term projects still one of them is this automating data harmonization you know it started out as just like the pipeline just the config files before llms were even in the picture I was putting my pipelines into config files so that I wasn't having to write like the same dplyr filter statements over and over again I could just say like what what parameters I wanted to use for that and so that is I think that's motivating in and of itself because it's continuing to help me to get my other projects done so it is like a big long-term project where putting effort in is also helping along all these smaller projects that do have timelines and deliverables that I'm handing off to other people.

I also have an example of a big long-term harmonization project at BMS that has not had the amount of use to justify the amount of effort and time I put into it and I think that academia mindset kept me from noticing that and realizing it as early as I should have so now I'm I'm working on like looking at metrics for example for the Shiny apps that I've put together so that was a big harmonized data set I put together and into this really highly customized Shiny app with all these detailed visualizations that I spent tons of time on and everybody said they wanted and then once it was built they weren't there they weren't looking at it so you also often run into this problem like if you say to someone hey like I could harmonize this data set for you do you want that like they're probably gonna say yes so you have to be really careful about the way that you frame things like trying to ask people what they need and then honing in on whether a giant harmonization project would help them with that or whether a really customized Shiny app would help them with that or whether they're just excited about it because you suggested it and they know it's possible now right.

just excited about it because you suggested it and they know it's possible now right.

Knitting data and personal projects

No I really appreciate that thank you thank you so much Adam we only have three minutes left so I'm going to ask the shortest question which is from Noor that says do you have a Ravelry do you want to talk just a little bit about knitting stuff yeah yeah I absolutely do have a Ravelry I'm 50,000 stitches on Ravelry and one of my favorite parts about knitting is keeping track of my knitting data so I have I have photos there I have project details and then every year I do an export of all of my project data and all of my my yarn stash data is tracked there as well so I do an export at the end of every year and then I have a bunch of visualizations that I put together to track like how long did I spend on each of these projects how long did I wait between buying some yarn for a project and actually knitting it up into that project I don't want to know that it can sometimes be embarrassing so I do this annual extract of that data and then I've got this Quarto report that I put together with a bunch of visualizations and I do it kind of in like January of every year it's super fun and then every year I add like a little bit more to it.

and you also track what you read and I'm going to put a link in the chat to library thing which is the app that I know or the tool that I know Leslie uses you also track all your reading.

okay and I will I will pick out like a few choice visualizations and post them in the discord later oh that would be so much fun because I know that the the report the Quarto report that you make it's not public it's okay it's private but I think it would be so much fun to see some of those Leslie thank you so much for joining us I had so much fun I hope that you did too I absolutely did thank you so much amazing thank you thank you for all your questions yes everybody had amazing questions thank you so much for asking them and I hope that you will hang out with us on Tuesday at the data science lab with Hadley Wickham where we're talking about data dictionaries and parquet files and using cloud code and then also next Thursday we're going to be joined by Hansel Palencia manager of data science at DaVita I cannot wait for that conversation it's going to be fantastic you are all going to love Hansel I hope you have a wonderful weekend and enjoy your Friday goodbye everybody if you'd like to save the chat top three or top right hand corner of your chat unless you are in the browser there's gonna be three dots that'll let you save the chat but of course come hang out on the discord server in the data science hangout channel we will make sure that we get Leslie there and also it's just a great place to continue conversations so we'll see you next week bye everybody thank you so much everyone I'm going to reshare the hangout site in the chat too because it looks a little bit different it might be fun to check out but also I've noticed a few people who have been trying to join the hangout which we're using our old link so if you could help us spread the word I know it's been a year since we've been using this one but I hate when I see messages and people are like I can't get into the hangout so I just want to make sure everybody knows that they can register there on on that site too it has been almost a year I think it was June of last year that we switched over.

Havala I see you asked about the workshop on Tuesday it's actually a weekly meeting on Tuesday I'm going to share pos.it slash dslab you can go check it out there that will let you register as well as see the stuff that's been planned and upcoming and then there's also a playlist on our YouTube channel for the data science lab where you can see previous episodes. all right everybody I think that we have got everything covered you're welcome Havala I will see everybody next week bye.