image: thumbnail.jpg
Supporting 100 Data Scientists with a Small Team | Mike Thomson | Data Science Hangout
We were recently joined by Mike Thomson, Data Science Manager at Flatiron Health, to chat about managing open source tools and maintaining R packages, creating reproducible reports for Word and Excel using Quarto, the “hub and spoke” support model for data scientists, and applying R and Posit tools in Real World Evidence (RWE) oncology space.
In this Hangout, we explore creating reproducible outputs using Quarto for formats like Word and Excel. Flatiron Health uses Quarto because it allows the reproducible publication of analyses to multiple formats simultaneously (like HTML and a downloadable Word document) from the same source code. A specific challenge discussed was outputting formatted analytic tables to Excel, as this is not natively supported by Quarto. Erica Yim, from Mike's team, detailed how they built an internal R function that uses the flexlsx package along with flextable to easily output pre-existing formatted tables from a Quarto document into an Excel template.
Resources mentioned in the video and zoom chat:
🔗 flexlsx R package GitHub repository → https://github.com/pteridin/flexlsx
🔗 DBPlier PR for Snowflake Translations (contributed to by Flat Iron Health) → https://github.com/tidyverse/dbplyr/pull/860
If you didn’t join live, one great discussion you missed from the zoom chat was about the pain points of exporting data from Quarto to Word or Excel, particularly concerning table formatting and styles. Attendees in the chat strongly highlighted the difficulty of managing table formatting, including issues with table cross-references, headers, and footers. They noted that dealing with styles often requires workarounds, such as creating flextables that match desired Word styles instead of relying on default table styles. Let us know below if you’d like to hear more about this topic!
► Subscribe to Our Channel Here: https://bit.ly/2TzgcOu
Follow Us Here:
Website: https://www.posit.co
Hangout: https://pos.it/dsh
LinkedIn: https://www.linkedin.com/company/posit-software
Bluesky: https://bsky.app/profile/posit.co
Thanks for hanging out with us! 💛
Timestamps:
00:00 Introduction
02:23 "Can you talk about what Flatiron does and what your teams do?"
03:29 "Could you give us a few examples of the data types or collections that you might be working with?"
05:00 "Do you have longitudinal data?"
07:46 "Are you aware of any computer vision applications in the health care industry from your perspective?"
09:38 "Do you use mixed models or Bayesian MCMC?"
10:56 "How does your team use Quarto?"
16:59 "How do you convince stakeholders of the value of going open source (and handle security concerns)?"
22:56 "Do you allow people to have a certain amount of time to contribute back to open source?"
26:03 "I just want to understand a little bit about your support model for that group."
29:57 "Do you have any tips for asynchronous working?"
31:02 "Are you like a Jira team or an Asana team for assigning tasks or tickets?"
32:10 "How many people on your platform team support Posit teams?"
34:24 "What does your team use for unstructured document analysis?"
36:24 "How important is domain knowledge in your recruitment?"
40:02 "Where do you store all of this stuff (data storage and databases)?"
42:04 "What is the approximate timeline from the time you do analysis to final deployment of results in the real world?"
44:31 "Is there a process for people getting things approved to use in your environment?"
47:39 "How do you handle the challenge of going back from Word to Quarto source code (after changes are tracked)?"
50:22 "What does a typical Workday look like for you?"
51:47 "Is there a piece of career advice that has either really helped you, that you've really liked, that you try to give to other people?"
Transcript#
This transcript was generated automatically and may contain errors.
Hey there, welcome to the Posit Data Science Hangout. I'm Libby Herron, and this is a recording of our weekly community call that happens every Thursday at 12pm US Eastern Time. If you are not joining us live, you miss out on the amazing chat that's going on. So find the link in the description where you can add our call to your calendar and come hang out with the most supportive, friendly, and funny data community you'll ever experience.
Can't wait to see you there. I am so excited to introduce our featured leader today, Mike Thomson, Data Science Manager at Flatiron Health. Mike, hi, how are you doing? Hey everyone, how are you doing? I'm doing well. This is my first time showing up in this big of a data science hangout. It's been a little bit since I was on the last one, but it's really good to be here and see you all. It is. Yes, we've grown. We keep creeping further and further over that 100 mark, 150 mark every time we meet.
Well, Mike, it would be great if you could tell us a little bit about what you do and something that you like to do for fun. Yeah, definitely. I am a Data Science Manager at Flatiron Health. We are in the real-world evidence space, and I support a team of roughly 100 data scientists across research science and data analytics who use a lot of our tooling internally in the Posit team platform. I just got a golden retriever puppy who is four months old, and that is my part-time job at this point of training and just really enjoying having a puppy. It's my childhood dream come true. Oh my goodness. Oh my gosh, I love it.
I'm looking for Mike in the chat window as well. I don't see him. I'm hoping for a teddy appearance. We have lots of puppies at the hangout, which makes me very happy.
About Flatiron Health
I wanted to ask a little bit more about Flatiron itself. I know that Flatiron is a subsidiary of Roche, but you operate independently. Because we're coming from all these different industries, who doesn't understand or know the healthcare space, can you talk a little bit more and explain it to me like I'm five details about what Flatiron does and what your teams do? Yeah, exactly. Flatiron has been around for over 10 years. The initial idea is it has an oncology-specific medical record software, very similar to Epic or Cerner, but it's targeted towards community oncology practices that are treating patients all across America and increasingly global in their community oncology setting.
We have real-time data being collected of a really diverse patient population. Correspondingly, we're able to do a lot of research after de-identifying those patient records to be able to understand what treatment patterns are we seeing real-time, real-world, separate from, for example, the big academic medical practices that may have different patient populations and different treatment norms accordingly. We're really trying to learn from the experience of every patient with cancer across the U.S., across the world, and glean a lot of different scientific insights from those really rich data sources and really rich patient experiences.
We're really trying to learn from the experience of every patient with cancer across the U.S., across the world, and glean a lot of different scientific insights from those really rich data sources and really rich patient experiences.
Could you give us a few examples, speaking of the data that you're working with, the data types that you're working with, or maybe the types of data sets or collections that you might be working with? Are they patient-level? One row is one patient. Do you have visual data? Yeah, we have mostly, because it's the EHR, electronic healthcare record, it's mostly structured patient records. You could have things like visits information, this patient was seen on this date and received this treatment. We could also have different clinical notes that we can extract and say, do they have this specific subtype of cancer, and correspondingly be able to analyze based on that.
We're actually very fortunate as scientists and analysts to have pre-cleaned data sets. I think one of the unique part of Flatiron's model is how can we transform and aggregate all of those source-level information from the EHR into analytic-ready data sets that allow scientists to take those and generate insights a lot faster. It would be anything from one row per patient demographics, descriptors of state, gender, that sort of thing, and then it can be really detailed. Here are the characteristics therein.
I broke up just a little bit there. I don't know if it was me or if it was anybody else. I wanted to ask as well, do you have longitudinal data? Do you follow patients across time ever for long-term studies, or is this just observational data? It is observational, but it's relatively longitudinal. Of course, you're subject to what patients are receiving. Obviously, a patient with cancer, we don't want them to have really long treatments if they're progressing well. Ultimately, we are subject to records that are being seen. It's not like a clinical trial registry where we're proactively collecting data, except for some parts of the business model. It's typically observational, and the idea is how can you complement the data that may be collected in a clinical trial with observational data in the real world. Okay, great. That makes perfect sense.
Topics and audience questions
I wanted to sort of do a little bit of context building for everybody so that we can ask Mike the best questions and talk about a couple of topics that could be good. I want to ask Mike about Quarto, and we have some team members for Mike's team to help chime in and talk about that, because I am really, really invested in Quarto having better outputs into Word and Excel. When I work in Word and Excel, I tend to actually go back and use our markdown, so I can use the officer package or the office verse to do stuff like that.
I would love to talk about that. If anybody out there in the Hangout world also still finds themselves really, really frequently needing to output things to Word and Excel, even though the rest of the developer world feels like they've moved on from that, I really would love to talk about that, so throw your support behind that in Slido.
Another thing is that you oversee people who make tools, and I love making tools. I'm very tool-oriented. I feel like I've been told my whole life, quit worrying about the tools, just worry about the outcome. I'm like, but I really love the tools, so that would be a great thing to ask Mike about, and then also advocating for using open source tools. If you are out there listening to the Hangout, and you're like, I would love to use open source tools instead of this proprietary point-and-click software that I have to use, maybe we could talk a little bit about that.
All right, so let's go ahead and take a look at Slido here, and I saw that Jacob had a question that came in early on. Jacob Lackey, if you are available to ask that question live, we can find you and help you unmute. Yeah, can you hear me? There he is. Hey, Jacob. Hey, Mike, I'm just wondering if you're aware of any computer vision applications in the healthcare industry from your perspective. Are you guys doing anything with that, like using OpenCV and Python or RoboFlow to any extent? It's definitely not within my wheelhouse and not something I'm familiar with. There may be other folks who are, but unfortunately I can't speak to them.
Okay, all right, thank you. That's a good question. I had the same question. That's why I asked, like, do you get any visual data? Especially with cancer, I feel like there's all kinds of scans and things that get, you know, we even get diagnostic, regular diagnostic work just as regular humans to look for cancer and stuff. So great, fantastic question. All right. I can add a little bit to you. There are other data types that we do have. They just aren't core to my day-to-day and what my team directly supports, but I agree. In the cancer space, imaging is super important. Genomics is super important. And I think there is a large appetite and interest in thinking about how can we add on those dimensions and multimodal data layers to have a more complete or a deeper picture within the journey of the patients with cancer. So definitely to say it exists and it's extremely important. I likely just don't have enough direct experience to articulate specifically the tools there.
Totally fair. We can't all have the direct experience and all the stuff. But if anybody wants to chime in from the chat, any question that gets posed to Mike is really, like, a question we can all talk about in the chat. If you have any experience with computer vision stuff with healthcare or diagnostics, I would love to see it. Share some info for us.
Okay. So there was a question in the chat that was anonymous or not in the chat, in the Slido, that sort of touched on my question about longitudinal data, which was, do you use, like, mixed models or Bayesian MCMC, which is Monte Carlo, in what you do? Or anybody in your team uses that? I don't, specifically. I think a lot of the baseline that we can do is mostly descriptive statistics in that a lot of the directional insights we can get are, for example, looking at treatment patterns over time, survival patterns, looking at follow-up and patient journeys. That's at least the core baseline of what we can do. That said, there are a lot of great scientists at Flatiron Health who do think about more advanced methods than I do, but I just haven't been hands-on in that space.
Using Quarto for reproducible outputs
I wanted to ask about Quarto before we got too much further, because I'm seeing some support for this in the chat. How does your team use Quarto? That would be a good way to get started. Yeah, we use Markdown for a very long time, and we're a relatively small team. We're roughly anywhere between two to four people who are supporting our tooling ecosystem for a team of, as I mentioned, a hundred data scientists, data analysts. And so we really have to be precise on where we focus our energy and what tools we want to build when. I think we had a narrative of our Markdown works really well. To your point, if we're looking to create word outputs in the officer ecosystem is extremely impressive and allows us to do that really well, at least for word outputs. At some point, Quarto kind of became the mainstream and we said, okay, should we switch to this? And one of the main drivers for doing that is that one, it's where a lot of support has been thrown in building up that ecosystem. And two, it relatively easily allows us to output multiple formats at the same time, which is honestly magical.
So for example, we could use Posit Connect as a way to publish reproducibly our analyses and generate an HTML that's visible by our clinicians that our scientists are working with and sharing insights with. But at the same time, there's also a Word doc that can be downloaded in the click of a button. And that Word doc is typically what is shared with our research partners or our customers with whom we're delivering analytic reports. And so it allows us flexibility to easily have the same source code and then leave kind of the rendering details and formatting details mostly to a Word template and Quarto on the backend. So it took us a bit to get there, but once we did and we worked with a wonderful team at Plymouth Analytics to kind of lay the foundation of how we would use Quarto in our day-to-day work.
Oh, that's such a great call out for like bringing in a third party that specializes in that type of stuff. I've worked with groups like that. They're amazing. If you don't know that that exists, there are groups that will help you or help your team figure out a workflow that works for you and sort of templatize things or even help you build themes, color schemes, all of that stuff. I know that one of your team members is here, Erica. I don't know if Erica wanted to hop in and add anything about the Quarto to Word and Quarto to Excel journey and what that looks like.
Sure. Yeah, I sit on Mike's team and one of the larger projects that I've been working on is creating a function in R to allow for to easily output analytic tables from Quarto to Excel and kind of getting the formatting of that as well. Most of our services teams are generating analytic outputs with like dozens to hundreds of tables and as those analyses get more complex in Word, it's really hard to maintain formatting, which is one of the reasons why we're pushing towards using Excel for our analytic outputs only. But unfortunately, there's not like a native way to output formatted tables from Quarto to Excel.
So we have been using the FlexLSX package along with FlexTable to easily output formatted tables from a Quarto doc to Excel and the function that we built pretty easily fits into our current workflow and some of the other functions that we've built to generate those initial tables in R. Yeah, and then using FlexLSX in our internal like output Excel function has allowed us to output to essentially like an Excel template similar to what a Word template would be.
Yeah, I know these struggles. I know these struggles so well. And Luke was saying there's also OpenXLSX2. There are a few different packages that have been helpful. I'm sure that you guys have gone through and kind of vetted and tested a lot of things, huh?
Yeah, we're also using OpenXLSX2. I think some of the formatting in OpenXLSX2 is a little manual. So we already had like pre-existing formatted FlexTables and the FlexLSX table allows us to like persist that formatting into the Excel output rather than having to manually do it, especially since our tables have varying numbers of columns and rows and headers. The FlexLSX package allows us to just take the existing formatting and basically paste it into Excel.
That's awesome. Yeah, that formatting is the big, is like the bugbear, that's like really, really hard to keep going. And also, if anyone's ever needed to put something into Word over and over again, Word does not allow you to change its default table template, which is really frustrating to me. You can change your styling on your table, but the default one that your table uses over and over again, you can't modify. I don't know if you guys have dug into like the OpenXML, all of that stuff that goes into like the back end of Word and Excel and stuff. But this is a really interesting topic. I'm seeing a lot of stuff in the chat.
There's more stuff. Javier added an example of formatting Excel to GT Summary plus OpenXLSX, which is really interesting. Go check that out. But I wanted to hop over back to Slido and take a look at the questions that we have rolling in.
Noor, you asked one about open source. Would you like to ask that one live?
If we can find Noor. There she is. Hello. Sorry, let me like dig up the question I asked it. I highlighted it in Slido and I'll stick it in the chat as well right now.
Okay. So basically, how do you convince stakeholders of the value of going open source instead of sticking with their continual contractual things with proprietary tools? And secondly, how do you handle security concerns with sensitive data and open source? Because I used to work as a contractor and that was often a concern of like, well, we know how this works. We really want to go into open source and so forth.
Open source adoption and security considerations
Yeah, really good question. I think I come from a place of privilege in that fire and in 2016 made a very intentional decision. And I think the people who made this decision that we wanted to be open source first and using R as a baseline, because we had the privilege of being a new company to define how we wanted to operate. And I think part of it too is we're a tech company in the healthcare space. And correspondingly, you have a little bit of different attitudes on where you want to start. That said, I think you do end up investing a lot more in risk control frameworks, security review processes, et cetera, in order to make those viable, safe and secure. Because at the end of the day, patient safety and scientific integrity is paramount to what we do.
Because at the end of the day, patient safety and scientific integrity is paramount to what we do.
And so one of the things we've invested in a fair amount is working with our quality team to define and security team to define processes and review necessary before integrating open source tools. So I think there's a whole area of conversation there and that don't necessarily need to get into here, but it just requires careful consideration. I'll give you an example for something like the Flex LSX and OpenXL. I always forget exactly what the package is named. But before in taking those into our system, we did a review of, for example, how many dependencies does this bring in? Is the maintainer relatively active? Are we introducing business risk by taking on a dependency from open source? Because sometimes you find a magical package in the wild that does exactly everything you need, but it hasn't been touched in a very long time. And you're introducing either security risk because there isn't active development or you're introducing technical risk over time and maintenance burden. So our team tries to be fairly intentional around what we decide to use when.
And one of the things we love about open source is you can see what's going on. And generally, if it's a very actively maintained area, then if issues do come up and it's well used, then a lot of folks will jump onto that and help support and iron out those wrinkles. And so that's why we try to really, again, anchor to those that are widely used and will have had a lot of those issues vetted already or will continue to be vetted as they come up going forward. So that's a little bit on the topic. And we just try to be intentional and grateful, especially for those in the open source community who have already paved that path because it's not always an easy journey.
Contributing back to open source
Yeah, we're all thankful for our open source heroes. That actually brings up a great topic because I think that your team and people in your team under you contribute to open source and the open source ecosystem. I know that you had mentioned a few packages. What were the packages? I know one was dbplyr. Yeah, I can give an opening to Anthony, who's on our team here.
One of the things we've all probably felt when trying to use an open source library is we don't exactly know which aspects of it are well-supported. And so you may think, oh, this should work off the shelf. And thankfully, someone has already solved that problem really well for you. And other times, you're the first person to think about that problem or use that backend that hasn't yet been paved. And so when we started a translation from, we went from using Redshift to Snowflake, we were using dbplyr really heavily on Redshift as a way to pre-process some of the data in SQL, and then we went to Snowflake. And I'll let Anthony describe the experience from there. I think this was at a time when I was relatively new to R as well. So it was a bit of a baptism by fire to learn R while learning the innards of the dbplyr package.
Definitely diving into the deep end, but effective, I think. Effective. So over the course of a few months, we were able to work with the tidyverse team to develop a set of translations from R to SQL, and we contributed those to the package. And they're in pretty prevalent use at this point. And I think we can be proud of that. So I'm excited. Yeah, no kidding. That's amazing. I cannot imagine learning R at the same time as learning the innards of a package.
What was your background before R, Anthony? Some Python. Some Python. Some Python. Well, that's really inspiring. Do you have any tips for anybody who's like, I really want to get started with contributing to open source? How can I do it? Do you recommend the deep end? Like, I have a problem that exists that this package could almost solve, but it doesn't, and trying to help add that? Yeah, for sure. I'm not sure if it's my personality type or just general guidance, but I find it most useful to have a problem that I really need to solve if I need to, when developing a tool. Yeah, same. Or at least something that you're really interested in doing. I see Marcos adding support to that with a reaction. Mike, do you have anything to add on to that, like getting started with contributing?
From a management standpoint, do you allow people to have a certain amount of time in their day or in their week or month to contribute back to open source? Or does it need to be really focused on problems that Flatiron is solving? Yeah, it's a tough one, right? We're all busy with our day jobs, and the question is, where do you focus your energy always? And we generally try to, I don't want to say limit, but focus on, would this be critical for us to be able to move forward again? Because we're a fairly lean team. We don't have a lot of latitude to carve out as much free capacity as we would likely like. And so we've tried to be intentional around, let's say we're doing a big migration, a snowflake, this isn't really helpful to do this, then it's also a great opportunity for someone like Anthony or me or Erica to jump in and try to solve that problem, not only on behalf of us, but also others who may encounter this problem going forward. So I think it's like, when we need to, we definitely are willing to, and make sure we work with our legal and privacy team to share any considerations for where we give back. But I think importantly, I learned too, even just opening an issue or sharing feedback or discussing with people is a form of contribution. I've had some amazing conversations, even at Posit Conference, where you ask a question of, I'm trying to think about this, and they say, please open an issue. And it's not so much that you're complaining, but you're just sharing ideas and sharing feedback that, for example, you may have a system internally that others don't have access to, and you can help cross those communication gaps via GitHub or via that open source community.
I learned too, even just opening an issue or sharing feedback or discussing with people is a form of contribution.
It is such a great point to make that opening an issue or just asking a question can be so super helpful, because you might be starting a conversation, and there could be people out there in the dev team that have been thinking about it too, and have just needed the push to sort of push it over the edge. That happened just this year with tidyverse, actually with dplyr. There were some kind of holes in dplyr where it was not as easy as it probably should have been to modify a column based on the values in another column or based on the values in a named list or a named vector. And people in the community started talking about it, posting about it, asking questions about it, and it turned out that the tidyverse dev team was like, we've already been thinking about this. Now that we have all of these issues popping up, let's do it and put it in there. And so those functions now have been put out and exist. So it really does matter.
Hub and spoke support model
All right. I wanted to call in somebody from Slido, because Nick, you fixed your question. It contains a question. Now, Nick Drew, if you are available to ask live, that'd be great. And if not, I can ask for you.
Hi, Libby. Can you hear me? Hey, I can.
Great. Yeah, I think parts of my question were answered a little bit before, but I'll just go ahead and ask it. Mike, thanks for joining us today. Really good information. It sounded like you have some folks that you oversee or supervise, and it wasn't clear to me whether they report directly to you or there's a matrix reporting relationship. But I just wanted to understand a little bit about your support model for that group. When people use desktop applications and open source software, oftentimes people come up, as we've been talking about, come up with their own ways of doing things. And so how does that support look like?
Yeah, it's really evolved over... I've been at Flatiron Health for six years, and we've really tried to iterate and learn from different ownership and support models. I'll share with where I am or where we started and then paint the picture to where we are now. As a team of scientists, we are not always software engineers at our core. And so correspondingly, figuring out who supports the R installation scripts that we all have, or who supports the libraries that we may or may not have created on a one-off.
We at some point said, okay, we're at a team where we want to be able to manage some things more centrally, and how do we do that? So we created what was called, and I would give a shout out to Nathaniel Phillips, who's not here, who created the R Enablement Initiative, to think about a dedicated core set of maintainers who can support that ecosystem. And so I think that's kind of a core principle where we've kept. We have a core set of tooling maintainers and a core set of tooling users. And I view that really as a gradient along a hub and spoke. The maintainers are really the hub where we can support and make sure we're prioritizing the things that can impact the most users across the most use cases. The spoke may be, I have a one-off analysis where I want to contribute a function, or I want to create something that's really not necessarily applicable to everyone, and we would push that support out to the spoke.
So currently, I manage two teams, a team of data analysts who are primary users, and a team of data scientists, including Erika, who are supporting our core tooling ecosystem. And we found that generally to be a really helpful model where our users are focused on either execution or science or analytics, and their goal is to be empowered with the tools they need to do their work really well. And our team of maintainers are aligned with the incentives to say we need to empower our users, our analysts, our scientists, to use those tools really well. And so we are always trying to build tools that empower our users hand-in-hand. The other thing I would say is we have an engineering platform team who supports our Posit team infrastructure and manages Workbench, Package Manager, and Connect alongside our core tooling team to make sure our platform supports our tooling maintainers who ultimately support our users using the platform and the tools.
This is such a big topic. And there was one more question, anonymous one on Slido that I saw that was along these lines, and it also had a reply that was like, hey, let me add to this. So let's add it on here. The anonymous question said, did you say you manage about 100 data scientists, or did I mishear that? If so, how does that look? I think you just talked about that. I do not manage 100 data scientists. I manage a subset of those. There were some wonderful managers and some great people across the organization. Our tooling team is roughly two to four people at any point, and our data analytics team is more in the range of 10, but our broader scientific org who is using the platforms is in the realm of 100. Okay. That makes total sense. The add-on question to that was, do you have any tips for asynchronous working? How do you communicate with your team? And I think that there was another question somewhere in here that was about
communication as well. So if I can find that, I might add it on.
In terms of communication, we generally use Slack a fair amount as our way to intake and triage, which maybe gets to the second part of that question. In interfacing with users, especially if they're less technical or not excited about the technical details, having a quick way to iterate and get down to the root issue to unblock them is so helpful, because ultimately they're not necessarily sure which question to ask, or they see an error message in the package and not necessarily clear where to go from there and what team to escalate to. So we really try to first triage, and then second, determine when we need to solve and what the urgency is back to that user, back to that use case. So we try to be fairly cooperative with the users and within our teams to make sure we're always communicating on what needs to happen when. Is there anything besides Slack? Are you like a Jira team or an Asana team for assigning tasks or tickets? We do use GitLab internally and use issues as a way to track across our internal libraries and development in the same way most folks would use GitHub. We also use Jira as a team to operationalize against some of those developments. So we're trying to say across these next two weeks in the spring, what do we want to focus on? What are the areas that we want to deliver as a team, in addition to balancing our inbound user support workflows as well that mostly come in through Slack and may get ticketed in Jira or GitHub or GitLab.
Okay, awesome. Yeah, Nora is saying GitHub projects is also super useful. I will second and third that. I love collaborating on GitHub. I've never used GitLab. I'm guessing it's very similar with the project structure. But yeah, I feel like some of my most successful projects have just been managed completely on GitHub and it was such a wonderful experience as long as you have somebody who's a great project manager working on it. Well, there's a question in Slido that I saw that we could probably knock out really quick, rapid fire style, because I think you answered it. It was how many people on your platform team to support Posit teams? So Workbench, Connect, Package Manager, is that that team of tooling people that's about four people? We support typically the user or application specific aspects of that. And our engineers would support more of the Docker Kubernetes configuration. There is a platform team who supports a bunch of different core development tools, and they are more in the range of four plus engineers, but they're not full time on Posit team. I think generally, as we stood up Posit team, we invested a lot more upfront to get something working and get our configurations where we wanted it to be. And over time, you can shift into more steady state maintenance, quarterly updates, that sort of thing.
Audience questions
Okay, great. Let me take a look at Slido here because we have so many questions that rolled in. We will definitely not be able to answer all of them in 20 minutes that we have left. But let's see if we can get a few of them in there. One of them that I thought was interesting, Hubert Hickman had asked. I will highlight that in Slido so that everybody can see it. And if he is available, you can ask. And Cecilia had asked a follow up question to this. I have the same question. Yes. Hello. Thank you. I work for the University of Chicago, Data for the Common Good. We're the home of the Pediatric Cancer Data Commons. And I've been in the clinical research informatics space since 1993. And so a long time warping around things. I've been using ours since 2017, but mostly are in Python, a mix between the two these days of what I end up doing.
But my question is, when you talked about analyzing the documents, is there a tool set that works for these things? You know, there's like C takes and deep fee and those sorts of things. Everybody's, of course, a little gaga about LLMs to do entity recognition and things in the documents. But, you know, that's mixed results and you've got to have one handy if you're sending an identified data. It has to be in a secure space and all that. So what does your team use for those sorts of situations that work? Yeah. And again, I'm probably the wrong person to speak to the actual extraction process from documents. We have a lot of really good engineers, both just in terms of data processing and in terms of extraction from those records. As data scientists, we're pretty privileged to be able to look at the transformed extractive loaded views of what those records are. So I don't think I'll be able to answer your question, unfortunately.
That's OK. And the follow up question from Cecilia was, Hubert, what do you mean by notes data? I think what you're meaning is just the unstructured text to data that accompanies medical records, right? Yeah. A lot of it is unstructured. Still, there's kind of the things like laboratory data that's structured. You know, here's the long, here's the coding, here's the range, here's this very sort of structured. There's some note data that comes in that has structured elements with it. Some things are available, sort of a structured XML as a side effect of how reports are generated in health care space. And then there's just text stuff that a lot of it's still that that comes in. And that can be procedure notes or encounter summaries, all those types of information. Well, this is a great question. If there's anybody in the chat who has information about this or experience with this, let us know in the chat so we can talk more about it. And then maybe in a future Hangout, we can even talk more about it.
Getting into the health industry
Alrighty, let's take a look back at Slido. How many more questions can we fit into our last 20 minutes here? I had a handful of anonymous questions. One of them I would love to ask because it was about getting into the industry. Here we go. It says, hey, Mike, it's anonymous. I'm a data analyst working in other industry, but interested in getting into the health industry. So how important is domain knowledge in your recruitment? Good question. I can just give a little bit of background on how I entered this space. My background is in science, math, chemistry, a little bit of economics. And I was always passionate about at one point, I thought I would be a doctor and or a dentist or veterinarian. But at some point, I realized my passion is doing that behind the scenes, doing the math, doing the modeling, analyzing the data and being able to tell that patient narrative in a different way. So for me, I was always curious on how I could apply that science and healthcare within the analytics base accordingly. So I went into economic consulting out of grad school and was doing a lot of health insurance litigation and decided rather than focusing on how much to pay claims, I wanted to dive deeper into what is the clinical patient journey that supports those. And so fortunately, I was able to shift into Flatiron Health and dive deeper into the space.
I think that sort of depends on the role that you're entering. Some roles can be really domain agnostic. And let's say it's more specialized in the example that Hubert gave, maybe you're an expert in extracting unstructured records. And that can be within finance reports, it can be within healthcare records. And you can still apply that internal to different industries. But there are other areas where let's say you're interested in epidemiology and modeling of scientific methods, then having that domain expertise and experience is pretty critical for you to be impactful in the role you enter. So I think something I always value is curiosity and willingness to learn. And for certain data analytics teams, you don't necessarily need to have that clinical experience to do the analytics aspects of it well that support the business and support your stakeholders. So I think it's just important in the interviewing conversation process of understand how much is needed for the role and what do you bring to the table. And if it's a role that you really want, figuring out if there are ways to close those gaps either on the job or in your own learning or in a different path towards that position.
So I'm always a learner and advocate for learning. But I think at the end of the day, I support people finding creative paths as well.
There are so many winding paths that lead to data science you would not believe. I think getting involved in the communities that are around healthcare and health and life sciences and pharma is also a really great way to learn more about the area. So the R in Medicine conference is so amazing. It's all virtual. It's full of workshops that you can take to learn from. Our pharma is an amazing conference. And then also going to PositConf and meeting all of the many people in the healthcare space that are going to be there. Many of them are in this hangout right now and they're going to be there in person or virtually. So that's my two cents that nobody asked for, but definitely go do that.
Electronic health records and data storage
And there was a question in Slido. Oops, Slido is moving around on me as you all add questions. That was about electronic health records. It says, as someone who is not familiar with U.S. electronic health records, how much data cleaning is involved in transferring medical notes into your database and are they relatively clean and easily ingested? I think that Mike covered this with that they get clean data, right? But I would like to spin this question and ask, where do you store all of this stuff? Are you using Snowflake? Are you using Databricks, like a catalog? I have worked at places who store enormous records in ancient IBM, like Netezza, which is very slow. I would love to hear about how your team handles things or how Flatiron does. Yeah, I think there's a whole area in data pipelining and ingestion here that happens before we get to the point of data science. I want to shout out to a lot of good engineers and a lot of good architects who have figured out how to do that securely, scalably, et cetera. And at the point of data science, we typically have in Snowflake, importantly, de-identified patient records that we're then able to use. So as an analyst, we're pretty unified into the tech stack that we can connect to Snowflake, have reasonably well-structured data and focus on doing the science at that point. So most of it would live in AWS S3 if it's source data and or Snowflake if it's analytic data.
And you said that you were previously on Redshift, right? Yeah, we were. Okay. I wish I knew more about all of this. I don't know if anybody else in the chat can identify with this, but like being a data scientist that feels like they don't know a ton about data storage and databases, I have lots of FOMO, like data engineering FOMO. If anybody, oh yeah, I was just going to say, if anybody has experienced Netezza, it looks like Felipe has. It says Netezza is a classic warehouse though. Yeah, classic is the word. All right. Thank you, Mike, for answering that. And for the anonymous question asker who asked an anonymous question for us, we have a whole bunch more and only 12 minutes to go. So let me look through Slido here.
Analysis timelines
There was one that was about timelines. There it is. I'm going to highlight it so it goes to the top. And it's, what is the approximate timeline from the time you do analysis to final deployment of results in the real world? I think that that's a really interesting question. For everybody in the chat to chime in, what do timelines look like for you? And Mike, I don't know if you're close enough to the analysis at this point to answer, but I will forward this to you and maybe you can call in Anthony or Kim as well. Yeah. I mean, it really, as a former economist, I just would say it depends on a lot of different things, the complexity, the depth, the amount of pre-specification you've done in the study. I think we can generate pretty quick insights. Let's say you're trying to get directional estimates of number of patients who received a drug with a given diagnosis. That's something we can turn around and share in hours at max. But let's say you're trying to do a really detailed time to event or covariate adjusted analyses or patient matching, then that's something we want to be really rigorous around pre-specifying that analysis, understanding what covariates you care about, understanding the data model and treatment setting in which you're studying before then executing, which is typically at that point, the shorter aspect of it. So I think one thing that we really try to bake into our timelines is quality control, scientific rigor, and that sort of thing. And correspondingly, it can increase your timelines, but for the right reasons. So those analyses can be, again, if it's really fast in the realm of maybe a day or maybe a week, or maybe it's actually a three-month study that requires a lot of stakeholder engagement and feedback with your partners.
quality control, scientific rigor, and that sort of thing. And correspondingly, it can increase your timelines, but for the right reasons.
So that's my best guess. All right. Thank you. I agree that sometimes increasing the timeline of something is not a bad thing because you have found cool new stuff.
Supporting new tools
I wanted to call Rachel in because Rachel had a question when we were chatting with you before. Rachel, would you like to ask that? Sure. I'm trying to remember which question. It might be the one I was just thinking about right now, which is, Mike, I think when we were chatting, we talked a little bit about Positron and that led us down a path of like, how do you support people in the cool new things that they always want to use? Do you have a lot of people who are always coming and asking you for the latest and greatest? Is there a process for people getting things approved to use in your environment? I think one of the most impactful questions we can ask as data scientists is, what's the impact or why are we doing something, right? Is there business value for doing this? Is it a quality of life improvement for our developers? Is it allowing us to answer more questions from the data that we have? Before we decide either what tool to use or whether to introduce a new tool, that's the question we always try to anchor to. Let's take Positron,
for example, where we haven't yet switched folks over in part because we used Posit Team and Posit Workbench as a way to, en masse, allow users to have their development environment pre-specified for them. That said, we try not to be overly prescriptive on what tools are supported. We want to work with users to hear their feedback on what we think is a good approach forward or what others have suggested to us to hopefully move in that hub and spoke so that most of the folks along the various spokes feel well supported as we decide on which tools to introduce and when.
So, that's a little bit of the concept. I'm happy to dig into anything deeper, too. Yeah. Is it the same way with individual packages, too?
Yes. We don't have a formal intake process for, let's say, someone wants to use a new package, but let's say we are adding a new dependency to our core sets of projects, then, yeah, we'll have very intentional decisions around whether we want to do something. Coming back to the example we talked about earlier, Excel templates, we are recognizing with Word in Quarto that's really well supported and that's not something we need to worry about as much investing and maintaining going forward. But if we are building out our own in-house R functions that wrap something like OpenXLSX or FlexLSX, then we need to be comfortable assuming some amount of maintenance cost going forward and we need to have the right expertise internally and we need to feel confident that is something we can sustain before introducing.
So, I think a high school guidance teacher once said we can take any course that you want, but you can't take them all and it's very much the same in the tooling space of you can use whatever tool you want, but you can't use them all and you can't always support them all well. So, it's a matter of making your trade-offs that allow you to maximize your impact, maximize your focus.
You can use whatever tool you want, but you can't use them all and you can't always support them all well. So, it's a matter of making your trade-offs that allow you to maximize your impact, maximize your focus.
Thank you. Thank you, Rachel. All right. I saw a question from Rodrigo in Slido. It has a little asterisk, though. That means I will read it for you, Rodrigo. And Rodrigo was asking about, like, he sees the value of reproducible Quarto slash R Markdown for creating a draft of a paper and exporting it to Word, but coming back from Word to Quarto is really challenging, especially if there are, like, comments or tracking changes if people want to edit things, and he was wondering if you had that particular challenge in your work and how you go about it. He also mentioned the Redoc package. Don't know if you've ever experimented with it.
I don't think we've asked the question of how to go back from Word to Quarto and the source code. Eric, I'm not sure if you've thought more about this. Sorry, I was trying to unmute. No, I don't think we have. I know that one of the discussions before was about, like, the lack of commenting in Quarto, because I know that feature is used in Google Docs and Word, but we haven't really explored going from Quarto to Word. But a commenting feature would be cool.
Yeah, if anybody has experience with that, thoughts about that, put them in the chat. That, as a person who used to do freelancing, that's a really big thing, right? Like, you have very frequently a non-technical audience who is wanting to give feedback on things and do them in their own time, and transferring that back is much harder to do.
Just in general, this is not a Quarto problem. This is, like, an in general, you are creating something with code and collaborating with people who do not code type of problem. Thank you, Rodrigo, for your question, and I put the link to Redoc, the Redoc package, in the chat if anybody wants to take a look at it.
Okay. Oh, go ahead. I would add a little bit to that. We have certainly thought about this and how do you create collaboration norms with folks, because I agree. Like, generating the code and generating the insights is one aspect, but where a lot of our scientific and clinical input happens is in the outputs thereafter, and that's typically in Word or Excel, and so we've thought a lot about commenting in Quarto, and is that possible and something we could do through Posit Connect, but I understand that that's an area of active development, and it's hard to combine the Google Docs aspects of live collaboration with the reproducible R aspects or Python aspects that happen in Quarto, but definitely an area relevant to data science collaboration.
One solution that I have seen happen frequently is just like, hey, non-technical person, I'm going to teach you how to use GitHub, because Quarto files are just text files, and maybe we can add stuff to them in GitHub and then submit those as PRs or issues or things like that, and that has ended up sometimes working okay. Again, GitHub is a great place to collaborate.
Career advice and typical workday
All right. We always try to ask a career advice question by the end of our session, and I saw one in Slido, so let's do that. There's an anonymous one that says, what does a typical work day look like for you? I'd love to hear that, and then I'll ask you a little career advice question.
Yeah. I am located in San Francisco, and I actually moved here in 20, fun fact, I moved here by meeting some people on a plane who introduced me to their son, whose company I then ended up working at, and so I never intended to be on the West Coast, but my team is all on the East Coast, and so my mornings are usually starting with coffee, taking a moment to warm up for the day, and then hopping into meetings, and then my afternoons are typically more deep-thinking time, able to dive into some coding or dive into some problem-solving and review.
That is a really fun origin story. I feel like I need a reiteration of that. You met somebody on a plane, and their son worked for a company, and you ended up working for that company? Yes. Yeah. I helped them get connected to airplane Wi-Fi, so then I ended up chatting with their son, who had a very similar background in economics and math.
This is amazing. This is just a great example of why talking with other human beings is so important, and that is hilarious and amazing. I love it so much. Okay, well, our career question is, is there a piece of career advice that has either really, really helped you, you've really liked, that you try to give to other people, that you mentor people on your team?
I think something I always anchor to is, think about what problems need to be solved, and then come up with solutions against those problems. Importantly, as you grow and as you stretch, think about what problems people aren't trying to solve yet, or what is a solution that someone hasn't yet been able to come up with to solve that problem that they care deeply about. Being able to connect with people on the problems they care about allows you to be a clear partner, be an innovative technologist as needed to match their solutions. For me, it's all about collaboration and aligning on the problem to solve before jumping into anything else.
Being able to connect with people on the problems they care about allows you to be a clear partner, be an innovative technologist as needed to match their solutions. For me, it's all about collaboration and aligning on the problem to solve before jumping into anything else.
Do you recommend getting to know your end users and your stakeholders and stuff for all of your people on your teams? Yeah, that's the most important thing. If you build a tool and people don't use it, then you haven't really been successful in building that tool. Being close to the people who need it is the most important thing, and they're ultimately the measure of success. If they're using it to deliver better science, then you've done your job well. Right. Yeah, then you contributed to delivering better science. That's the whole part of understanding the business problem, right? You've got to understand it from the perspective of the person who has the problem if you want to help solve it.
Closing remarks
All right. This was a wonderful chat. We have one minute left, so I will say we must move on and say goodbye. Just a reminder to everybody, there's no Hangout next week. We are at PositConf next week. Well, I'm not. I'm here in my house, but I will be doing all kinds of PositConf stuff. Mike, thank you so much for hanging out with us. This was wonderful. I hope you had a good time, too. Yeah, likewise. Thanks for having me. I'm looking forward to chatting more with you all.
Yeah. Erica and Anthony, who work with Mike, thank you for being here, both unmuting and in the chat. I saw Anthony in the chat answering questions. If there are some topics that we talked about today that you would like to see us talk about in future Hangouts, fill out that survey after we leave and let us know. We would love to take that into consideration. It was so nice seeing everybody's face today. I will see you in two weeks' time for another Hangout, but I will hopefully see you next week and this weekend on the Discord server for PositConf. Okay, so everybody go find me on the Discord server. Say I'm from the Hangout, and I will make your name yellow as part of the Data Science Hangout crew. Okay? All right. Goodbye. Thanks so much. Yeah, save the chat. Top of the chat, three dots.
I'll stay here for another second. Oh, Benjamin, you're asking where is the PositConf Discord? Log in to the event platform. There will be a little thing there that says join the Discord, and you can click that. I recommend logging into Discord first. Otherwise, you might create a temporary Discord account. Okay, everybody. And Libby, just so everybody knows, that's if you go to the conference page and go to... I just want to make sure everybody knows how to get into that portal. Yes. You've got to go to posit.co slash conference, and there will be a button that says register slash log in. You click that button, log in, or register for the conference virtually, and then you will be able to get in there. Yeah. All right, Benjamin, find me on the Discord server. Bye, everybody. Bye, everybody. See ya.