image: thumbnail.jpg
Accountability and transparency in AI systems | Sam Tyner-Monroe @ DLA Piper | Data Science Hangout
Speaker bio:
Sam Tyner-Monroe, Ph.D., is DLA Piper's Managing Director of Responsible AI. In this role, she leverages artificial intelligence and data analytics to ensure accountability and transparency in AI systems. With several years of experience working at the intersection of data, law, and policy, Dr. Tyner-Monroe is skilled at turning data into insights, using algorithms responsibly, and communicating complex topics to both technical and non-technical audiences. She focuses on ensuring AI and data-driven technologies are developed and applied ethically and for the benefit of society.
Links shared in the Hangout chat:
š Posit Conference: https://posit.co/conference/
š Is GPT Right for My Project? https://presentofcoding.substack.com/p/is-gpt-right-for-my-project-practical
________________________
āŗ Subscribe to Our Channel Here: https://bit.ly/2TzgcOu
Follow Us Here:
Website: https://www.posit.co
LinkedIn: https://www.linkedin.com/company/posit-software
The Hangout is a gathering place for the whole data science community to chat about data science leadership and questions you're all facing that happens every Thursday at 12 ET.
To join future data science hangouts, add to your calendar here: https://pos.it/dsh
We'd love to have you join us in the conversation live!
Thanks for hanging out with us! š
Transcript#
This transcript was generated automatically and may contain errors.
Hi, everybody. Welcome to the Data Science Hangout. I'm Rachel Dempsey, and I lead Customer Marketing at Posit. Posit, if you have never heard of us before, is the open-source data science company building tools for the individual, team, and enterprise. Thanks so much for choosing to hang out with us today. The Hangout is our open space to hear what's going on in the world of data across different industries and connect with others facing similar things as you.
We get together here every Thursday at the same time, same place, except next week is the 4th of July holiday here in the U.S., so there will be no Hangout next week, just a reminder. But if you're watching this as a recording in the future and you want to join us live, there will be details to add it to your calendar below. And just make sure it adds it for 12 Eastern Time. I know sometimes the time zones, it gets a little mixed up, but it's 12 to 1 Eastern Time every Thursday.
I've started adding this because I know people really love connecting with other attendees, even if you're not jumping in live. So if you are interested in connecting with others, I want to encourage you to say hi in the chat and introduce yourself, maybe share your LinkedIn if you want or your role, where you're based, something you do for fun. But at the Data Science Hangout, we're all dedicated to keeping this a friendly and welcoming space that you all have made it and love hearing from you no matter your years of experience, titles, industry, languages that you work in. You can definitely just listen in if you want, although I love getting to hear from you live.
So there's three ways that you can ask questions or provide your own perspective. So first, you could raise your hand on Zoom. If you're new to Zoom, there's a little reactions bar or button below in the bottom Zoom bar. That's how you can raise your hand. You can put questions in the Zoom chat and just put a little star next to it if you want me to read it or else I'll call on you to introduce yourself and jump in. And then lastly, we have a Slido link, which I'm sure Curtis has probably already shared here, where you can ask questions anonymously too.
With all that, thank you again for joining us today. I'm so excited to be joined by my co-host, Sam Tyner-Monroe, Managing Director of Responsible AI at DLA Piper. And Sam, I'd love to kick things off with having you introduce yourself and share a little bit about your role, but also something you like to do for fun.
Yeah, thanks, Rachel. And thanks for inviting me today. I'm really glad to be here. I'm a big fan of Posit. I've been to Posit Conf several times. Big fan. Yeah, so my role as the Managing Director of Responsible AI at DLA Piper is a very unique one. I don't think there's any other role like it in the world. So DLA Piper is the third largest law firm in the country by revenue. I believe that's correct, but don't quote me. That was correct last time I looked it up. So it's a very large law firm, global law firm. We have offices all over the world as well as in the United States.
And I was brought on in March of 2023 as a part of a larger effort within the firm to build up an AI and data analytics practice group. And so that practice group consists of lawyers and data scientists and lawyers slash data scientists and all sorts of other technical and legal folks. And so I'm not a lawyer. I have a PhD in statistics. That's my background. That's how I got involved with Posit in the first place. And so I was really interested in the intersection of law and data. And I sort of started at that intersection in graduate school, working at a place called the Center for Statistics. Oh my gosh, it's a really long name. I'm going to forget it. The Center for Statistics and Forensic Evidence or something like that. I forget the name.
I'm so sorry. My PI is going to come through the screen and slap me or something. But yeah, it's a forensic evidence. So we worked at Iowa State University, a bunch of other institutions, California Irvine, Carnegie Mellon, a bunch of other places. And so we were doing research on, yes, acronyms are hard. Even after working in the government for a year as well, it's still hard.
Yeah. And so, yeah, really interested in law and data. I got started there. I took a break from that for a little bit. I went into the Bureau of Labor Statistics. And then I sort of had this really cool job offer that I had accepted. I was going to be a baseball analyst, actually, for the Washington Nationals. But then the pandemic happened. And then they were like, oh, sorry, baseball isn't happening right now. So we can't afford to hire you anymore.
And so all good. No hard feelings there, right? I still get some tickets on occasion to Nats games, which is great. But yeah, so then I sort of started looking for a new job. And I found this job at a data science technology company that was a subsidiary of another law firm prior to the one where I'm at now. And so, yeah, that was kind of more background about me, Rachel. So I'm sorry for kind of going off on that tangent. And I'm happy.
No, that's great. About the actual what I do now.
Yeah, I just like telling a story. It's a fun story. It involves people DMing me on Twitter and saying, hey, I think this job would be good for you. And that's how I ended up at my prior company, right? So it's, yeah, it's a fun, fun story. It involves baseball, obviously, which I love talking about. So yeah, good stuff.
So what else do you like to do outside of work for fun? Well, I know go to baseball games too. Go to baseball games. I have two dogs, Archie and Zoe. If you're interested, definitely will drop their Instagram handle in the chat if that's something people are interested in. Um, yeah, I also crochet. I'm a big yarn enthusiast. I also started embroidery recently. So very, very crafty. And also just being involved in my neighborhood, hosting a garage sale this weekend in my neighborhood. So yeah, exciting stuff.
Working at the intersection of data science and law
That's great. Thank you so much for joining us, Sam. And so I know you shared a bit about your own journey, but that you weren't necessarily a lawyer going into this role. So what was that transition like starting to work at, uh, as a data scientist working with lawyers?
Oh, it's very, it's very difficult. It stays, um, you know, as with any sort of field where, and especially, you know, being in graduate school and doing the postdoc and then, you know, working for the Bureau of Labor Statistics, right? You're very much like talking to people who you are expecting to understand you and know everything that you're talking about, and you don't necessarily need to give them a lot of background. And so, and lawyers are the same way, right? They're used to working with lawyers. Everyone's gone to law school. Everyone's been a clerk. Everyone's, you know, done all of these things. Everyone's written a brief.
And so, you know, it's, it's very unique. I think these two particular communities of sort of like the data science, techie, statistics, data community, and then like the lawyer community are two very, uh, I mean, it's, I think it's, it's probably this way with every profession, but I think in particular, these two professions, for whatever reason, they're both very, like, I talk to my own people, um, and no one else, kind of. And so, having to make some of that crossover is, is really difficult, um, just from the perspective of both, one, understanding the legal things I need to understand to do my job, and also helping the lawyers understand the, the statistical things as well.
Uh, how did you go about, like, learning what you needed to know from the law side of things? You just, you just learn as you go. I mean, so, you know, my, um, I reported to a partner at the law firm, and so, and he was, um, you know, very skilled. He was also a data scientist, had a, a master's degree in, in business analytics. And so, you get somebody like that, who kind of crosses over a little bit into one of the areas, and then you can sort of start talking to each other. And then, from there, you get the, okay, well, this legal issue in this analysis, you have to consider, um, because of, you know, X, Y, Z reasons. So, you just sort of learn on the job. And also, there's a lot of, I mean, a lot of reading involved, right? Um, I keep joking, like, I'm going to go to law school, maybe, eventually. And then, everyone says, oh, no, no, you'll just, you'll learn enough here. You don't, you don't need to go to law school. Don't torture yourself with that. But.
What responsible AI means in practice
Yeah, absolutely. So, um, the responsible AI portion is, um, part of a larger effort under our practice, where we are advising clients on how to do AI responsibly. So, that involves implementing third-party tools, that involves developing their own tools. It also has involved, um, evaluating the trust and safety of a foundation model and how that is, um, deployed and how that is being monitored. Um, so, there's lots of different, um, responsible AI things.
And so, my role as more of the quantitative, uh, person with the data background is doing, um, the testing. So, any sort of quantitative testing. So, that could be, um, bias testing, for example. So, we'll look for, um, uh, for instance, we have a company who gives out loans, um, and the loans are, use all these different factors, such as, you know, what your income is, or what your credit score is, right? So, all of these different factors. And then, of course, there's, there's regulations that say you have to give these loans out fairly. You can't discriminate based on protected class. So, a protected class could be race, ethnicity, sex, age, um, gender, so forth. Um, there's some protect, there are some legal protections for even like your genetic information, right? So, there's lots of different issues, um, that are at hand in, in determining that sort of, that sort of testing.
We can write a plan for you. We can do the testing ourselves, you know, and all under attorney-client privilege, which is a big benefit to the client as well.
Thank you. I see we have a question in the chat already. Arcinus, do you want to jump in here?
Hey, good morning, y'all, or good afternoon to some. Um, thank you very much for joining us, uh, Sam, today. Um, my, my question is pretty simple. I'm actually just curious about whether you can describe a, you know, scenario when maybe your needs as a data scientist to perform an analysis, you know, for example, like data access or anything like that, might have conflicted with what, um, you know, the lawyers thought you should do or, or what, you know, might have been necessary for the case where there was, um, maybe some, yeah, some, some kind of conflict with that, if you could, and, and how you kind of dealt with it.
Yeah, that's a great question. Um, the, it's typically, so I, I am in a position where if I'm going to do an analysis, I do get to say, hey, here's the data we need. Um, and so typically, I get to say, this is the data that we need to do. Um, there's some cases where, um, we may not have that, um, but specifically in client work, typically we'll send a document that says, send us a data set with these fields, make sure you include a data dictionary, you know, all of those types of things.
Um, but, you know, there, there might be a case where, um, we frequently get some sort of ad hoc data analysis where it's an Excel spreadsheet, for example, that's too large to open on a computer. Um, and so, um, attorneys, even who are very technically savvy, you know, they're not able to open the file and they're like, oh my God, how can I handle all of this data? And so they send it to us and just sort of say, hey, what's in this file? Can you tell us about it? Um, but typically it's, it's more of a cooperative thing as opposed to like an adversarial thing. Um, and so we're not, we're always trying to be curious and ask lots of questions. And so it's, it's always a very collaborative endeavor.
Shiny apps and data tools in legal work
Sam, I was looking back and really, I couldn't believe it was two years ago that you and I were chatting about having a legal meetup and we were talking about like sharing some different use cases and one, um, being a Shiny app that the team was using. I know this was at a past company, but I was curious if you could share a little bit more about that use case or maybe another favorite one you have in mind.
Yeah, that use case. Yeah. So that was from a past, um, company that I worked at. So there was a, uh, and again, this is not a, I don't think it was a unique, um, I'm sorry, my AI camera follows me around. Let me turn it off. Um, okay. Um, so yes, I don't think it was a particularly unique, um, uh, like it, it didn't have to be Shiny, but it just happened to be the tool that I knew. Right. Um, and so in this case it was, there's, um, a, a lot of what happens at large law firms is, um, patent filings. And so you have to, with the United States patent and trademark office, there's a set set of forms that you have to fill out. Um, and you have to fill out, you know, depending on if the patent is international or, or based in the, uh, the developers are based in the U.S. or the inventors are based in the U.S., um, lots of different things. If it's a company versus an individual. So there's all these rules and all these rules require all these forms to be filled out.
Well, typically, um, what happens or what was happening was a paralegal, uh, was collect, was getting sent all of these documents from the client and from the partners involved in the matter and saying, okay, go fill out the forms. And so this involved a lot of repeat information.
You enter the company name on everything, you enter the law firm name on everything, all of the ID numbers, you know, the names of the inventors, everything like that. Um, and so what we did was we developed a tool whereby they could just input, you know, one, the employee name or the company name, the, you know, the inventor name, the inventors, all that. And it was for one particular company. So we were able to collect a lot of data on a company that filed a lot of patents in the United States. Um, and so we were able to just create a sort of form essentially where the paralegal would go and pick all the information she needed to fill the, the, the PDFs. And then the behind the scenes hour would be filling in the, the PDFs and then she'd get out. So it would take a task that normally would take her 60 to 90 minutes and it would take her 10, 15, 20 minutes. So it was saving time. And typically with, um, these types of matters, they were processed at a very low fixed fee. And so any time, so we wanted to eliminate time in that case, that's not always the case in a, in a law firm, right? The billable hour is a big thing. Um, but yeah, in this case it was a very low fixed fee, um, service that was, that the law firm was providing. And so we wanted to try and make that more efficient.
That's awesome. Thank you for sharing. That's an interesting concept to think about, like, maybe not wanting to reduce the time if it's per hour. Yeah, it's a, it's a struggle. It's something we've been navigating here. Um, you know, it's, it's a difficult because obviously coming from the technical side and if you're, you know, a product manager or you're developing software, right? So a lot of what our team does is develop new products for clients or for the firm internally. And so that's a very different work structure than the billable hour work structure, which is dominant in the law firm. And so you're billing your time at an hourly basis. And so if you bill 14 hours in a day, which is often what happens for, um, associates, that makes more money for the firm. But if you're able to do your job more efficiently with, you know, the use of AI or some sort of other, you know, electronic tool that then is costing money essentially is how some people see it. Um, because you're not able to spend your full time and you're not able to build that full time to the client. So it's, it's a difficult, it's difficult to navigate for sure.
Protected classes and AI discrimination
Um, I see somebody had asked a question in the chat was they were curious about the protected attributes and if you could say a bit more about that. Yeah, absolutely. So, um, and this is again in the United States. So in the United States there are what are called protected classes. So these are classes of, um, categorizations of people that are legally protected, meaning that you can't discriminate or you can't treat someone differently based on their membership in a particular protected class. And so the protected class refers to the general category. And so there's, um, federal categories, which are protected. And then there's also state level protections and it may also go down to city level. So like, for example, large cities, New York City, Chicago, LA, those types of things. Um, they may have also additional categories that because they're so large, they can sort of make their own rules. They don't need to necessarily rely on, on the state to do that.
Um, and so federally, I don't recall all of the protected classes off the top of my head, but it's things like race, national origin, color, gender, sex, um, age is one. And there's various laws that protect, that are enforced that are, are what are, what is protecting these categories. And so, um, like the Civil Rights Act, um, the Americans with Disabilities Act, um, I'm trying to think of some other ones off the top of my head. Um, they're escaping me right now, but there's various legal laws that say it's illegal in this country to discriminate based on this attribute. And so what's, but what's important to remember, and this comes up all the time for us, um, specifically, um, in like the employment context or like a financial services context, the category that is protected, it refers to all people. So all races are protected. So sort of regardless of historical status, or maybe the historical majority, or the historically overrepresented, or, um, overly advantaged, or, you know, on the other side, the historically disadvantaged groups, right, of, of any of those categories, they're all equally protected, sort of regardless of their historical status. So it could be the case that, you know, you are discriminating against a historically dominant class, right? So that legally is, you know, protected under the law as well.
The NIST AI Safety Institute Consortium
Thank you. I, um, I saw on LinkedIn, maybe this was a few weeks ago, but that your firm AI and Data Analytics Practice joined the U.S. AI Safety Consortium. And I was wondering if you could tell us a little bit about that too. Yeah, absolutely. So we're, uh, very involved. We're one of, I think there's only two law firms. We are one of them, and there's 200 plus organizations that are a member of the NIST AI Safety Institute Consortium. And so the, the consortium is made up of several working groups, and those working groups are focused on various aspects of, um, generative AI and safety around that. Um, and so you have a working group dedicated to, um, maybe, uh, CFAM materials, um, so that's child sexual abuse materials, or, you know, um, deep fakes and things like that. So it's, it's definitely, it's part of a, we're part of a big group that is, um, getting together and talking about, you know, how can NIST create standards that will advance AI safety?
So it's people in the working group from all different companies? Yeah, yeah, all sorts of different companies. There's, there's over 200. I can, I can find a list, um, somewhere. But yeah, the NIST has been very busy lately, so the, the consortium working groups are not super active at the moment. Um, you may have seen NIST recently put out, um, the, the gen AI, um, or generative AI risk management framework profile, um, so the, the risk management framework is something we work very closely with and it's put out a couple years ago, um, and then they recently added a piece for generative AI, um, so they're very busy, um, and so the, the consortium's not super active at the moment, not, not a lot to talk about.
Maturity of AI conversations with non-technical audiences
Abigail, I see you had a question in the chat. Jump in here next. Thanks. Yeah. So, like, I guess, how mature do you find the conversations you're having with folks who aren't technical? Like, are people going to and just being like, let's do AI or do they have, like, kind of clear use cases? Like, where, where kind of are, are they on that? And what are they, like, I know what I'm reading and what technical people are reading, but like, what are lawyers, how are they getting their information about this stuff? Yeah, um, lawyers get their information from the same places that, you know, we get our information from. And so typically, right, so if they're in their area of expertise, so they're looking at law review journals and they're looking at, you know, Westlaw and LexisNexis and all these legal databases and looking at court cases, right? So that's where they're getting their information from. But in terms of AI, they're getting it from where everybody else gets it from, right? You know, the news, um, Bloomberg Law, um, you know, various, you know, CNN. So just learning about it as everyone else does.
And there are, there are definitely some, in terms of your question about, you know, the level of sophistication, it really runs the gamut. Um, so we do have several attorneys on our team who are very sophisticated and who were data scientists in a past life. And so they are totally on board. Like they get it, you know, a hundred percent. And then, yeah, we do have some people who say, oh, we want to do AI. Um, and then you sort of need to be like, okay, well, what's your use case? What data do you have available? You know, things like that. So it really, really runs the gamut.
I think it'd be interesting to hear for some, just some of those conversations about like someone saying, hey, let's do AI. What are a few other examples of the questions or how you have to like shift the conversation with them? Yeah. In a lot of ways, it's really about mutual understanding. And so really when someone says let's do AI, you kind of have to figure out what it is that they're actually asking for. Right. So that puts a lot of, you know, onus on you to really be curious and really understand and sort of try to come at it from their perspective and really understand their issues and, and be empathetic to their needs and so forth. And then again, same thing. Sometimes it's, you know, and you need to be able to explain, you know, at various levels of sophistication, all of the tools that people are talking about. And so, yeah, it's a, it's a combination of your own deep understanding and ability to explain things as well as, you know, your empathy for the other person.
In a lot of ways, it's really about mutual understanding. And so, yeah, it's a, it's a combination of your own deep understanding and ability to explain things as well as, you know, your empathy for the other person.
Using Posit for bias testing tools
Thank you. Well, Sam, I know you're part of this unique group that has brought in Posit to multiple companies and have worked with Posit across multiple teams. So I was curious if you could share a little bit with us all about the ways that your team is using Posit today. Yeah, absolutely. So we are using it, we use it to host tools. So we do a lot of, you know, repeated computations. So typically let's use, for example, when a company comes to us and says, hey, I want to test my system for racial bias. They might say, oh, but by the way, we don't collect race information from our customers. So we need you to figure it out for us. And so in that case, we'll use inference methodologies, specifically something called the Bayesian improved surname geocoding methodology, which we also augment based on other research using first name as well. And so that we turn into a tool that draws on census data. And so we input, you know, first and last name, street address, city, state, and zip code. And that is able to give us some probabilities that according to census data and other data sources, these are probabilities that the person belongs to each of the different racial groups. And sort of on aggregate, we never use it on an individual, we always use it in the aggregate. And so we're comparing groups. So we're averaging these probabilities. And that methodology is highly accurate when you aggregate it and determining the racial makeup of a particular group. So we have a tool that computes it for us that we use that is hosted on Posit Connect.
That's great. Thank you. It's always, it's always fun to get to hear the like actual use cases of the tools, because sometimes if we're so early, like, in the conversations with the team, we don't know actually, like, what is that exact use case?
How AI systems are tested for discrimination
I know there was a question a little bit earlier, Jared, and I think this actually just touched on it. So I just want to make sure did that answer your question? Or do you want to jump in? I think it was a diving a little bit deeper into the features that you use to identify discrimination. It says, do you need instrument variables or something similar to avoid conditioning on confounders or colliders? I can say I have no idea what any of that means. So sorry if I mispronounced anything. Yeah. So it sounds like the question is, or if whoever asked the question wants to come off mute and maybe ask it again. But it sounds like the question is about how do you actually do a test for discrimination? And so the answer is, it depends. In a lot of cases, it depends on what the legal regulations are. And so in a lot of cases, that first involves a lot of research, just trying to figure out what exactly is illegal and what exactly we need to test for. I mean, typically, we have attorneys who are telling us that. So we don't need to do a whole lot of research. But it's nice to sort of get your bearings and really understand the core of the issues. But yeah, in terms of actual testing, we typically look at outcome testing first. And so we'll look at, OK, how are these groups of people, there's an outcome from some AI system or machine learning system. And we will aggregate that in some way according to group membership. So male versus female, white versus Black versus Hispanic, and so forth. So we aggregate those results, and we average them in some sort of way. So it could be in an employment context. Someone was automatically selected to move on to the next stage of a job application process. So we look at the racial makeup of, OK, there's x percent of people who are applying are white. But maybe y percent of them got through versus some other percentage of people. There's a much lower percentage of people who are Black who are applying who get through to the next stage. Then that's an example of illegal discrimination that needs to be remedied. So I hope that answers the question.
Yes, I think so. Jared said that's great. Let me see. It looks like there might be a hand raised. Yes, I didn't see your name, so I just wanted to make sure and double check before you unmute. Yeah, I have. Sorry, I have a weird name for this. I wanted to follow up on this discrimination questions we're raising, because I guess how can companies address the discrimination in their systems if they're if they are not able to? Because from what I understood, these systems are built like blind to race, so they don't collect race data. So how wouldn't it be? Wouldn't it make more sense to actually collect this data to then be able to distinguish if racial groups are being discriminated instead of having to go through a process of inference, which I guess is quite expensive and complicated to actually find out whether these groups are
being discriminated? Yeah, there's lots of considerations that a company who is using their data to generate some sort of AI powered decision in terms of whether and how to collect that information from people. And so there's a lot of considerations that go into place.
One is sort of the more traditional viewpoint or sort of old school viewpoint, which is I'm not going to collect the data period, then I don't have that data and no one can accuse me of any racial discrimination. That's old fashioned. That's not true anymore. We know that AI systems, if race is not an input or if gender is not an input, it can still make decisions that are highly correlated with race or with gender or with any other protected class. And that's just due to the level of sophistication and correlations within data that are just out there in the world.
And so it's no longer that's sort of the old school way of thinking. And so, yes, I agree, it is more effective and more efficient to just collect that information up front. So you can just do that testing on the back end. That's sort of a responsible AI way to do things.
But again, companies still are very averse to collecting that data. And again, it also matters to there's a there's a level of trust depending on sort of the product that you're that you're selling. So what your customer base looks like, you may not want to ask them what their race is that may engender or their sex or their gender identity or their sexual orientation. Right. There's various sensitivities that all of these and all these groups are protected legally, but it depends on sort of what the company, what their goals are in relating to their own customers. And so they may not want to collect that information either, just based on developing trust with their customers.
AI evolution and use cases
Sam, as someone who's spending most of their days thinking about AI and having these conversations, I'm curious to learn more from you about how you see AI evolving over the next five years. Yeah, definitely. It's so it's such a hard question. And, you know, there's the you guys all seen the well, of course, not everybody has seen it because not everybody has seen everything. But there's a hype cycle, the Gartner hype cycle graph of AI. And so we're very much in like we're coming down kind of on the on the peak of the hype cycle right now.
And so I think that there's a lot of things. And even within our own AI group, we're like, oh, yes, Jenny, I is totally going to be able to do this. And then we try it and we finishing the meeting. You know, we tried to do it. Sorry. One second. Let me make sure we get somebody. OK, here we go. Yep. So so we were like, oh, yeah, I can definitely do this. A large language model can definitely do this. No, we tried. We tried really hard. It couldn't do it. And so what we ended up using was a very traditional machine learning model that performed much more effectively for our purposes.
And so it just depends on the use case. And, you know, it's I like to say focus on the use case first. Like, what is it that you want to do? Don't just focus on, oh, my gosh, I want to do AI. So cool. Like, let's just throw it. I'll limit this. Right. That's how you get into trouble, too. That's actually something we advise our clients on all the time, because you've seen some several, several famous cases now where a company has put up a GPT powered chat bot on their website and then it's agreeing to sell a brand new twenty twenty four car for a dollar or it's telling you, yes, you can definitely get a refund for this thing. And actually the company is saying, no, you can't. And so it's very, very tricky. So it we like to I like to focus on what's the use case. Right. So focusing on the use case, I think, is going to be more useful in terms of figuring out where it's going.
And I think is is really going towards a place right now that I'm to be perfectly honest. And this is my own personal opinion. I'm not wild about it where we're trying to replace a lot of humans, human aspects of things, which I don't love. And there's a there's a famous quote. I'm sure I saw it on Instagram or Twitter or something, but I saw it making the rounds of, you know, I want to help me do my laundry and, you know, pay my bills. I don't want to help me create art and be a creative person. Right. So there's there's sort of a dichotomy there where it's like the stuff that I would actually be really good for. It's actually it actually can't do. Yeah. So. I'll stop there.
I want to help me do my laundry and, you know, pay my bills. I don't want to help me create art and be a creative person. Right. So there's there's sort of a dichotomy there where it's like the stuff that I would actually be really good for. It's actually it actually can't do.
Communities and partnerships
Oh, it's really interesting. Thank you for sharing that. I am I was wondering if you talked about the working groups, but are there other communities that you're a part of or maybe things like this to where people are talking about AI? Yeah, so I'll say that with that, well, within our group, we do a lot of thought leadership. So we're very involved with a variety of institutions. We have a partnership with the Duke Institute for Health Innovation where we're working on bias and health care data. And specifically, there's a nonprofit that we're a part of with them that is geared towards, you know, helping low resource communities implement AI tools and health care in a way that's actually going to help them as opposed to make their lives more more burdensome.
So that's one aspect of things. We work closely with an organization at Stanford University called the Center for Legal Informatics. So determining working with them on the legal and sort of technical implications of the use of AI. So how can AI be used in a legal context? Are there legal data sets that we can create with them? Various like what kind of guard rails do you need to put into place? So in order to protect yourself and your customers from, you know, an AI behaving badly, there are things you can do and we work with them on that. So lots of different institutions that we collaborate with. We just had a big feature at the UN AI Summit for Good in Geneva, Switzerland. So we were a big part of getting that off the ground where the AI legal and the exact name is escaping me right now, but we're like the AI legal leaders of that organization.
Protected classes and confounding variables
Awesome, thank you. I know there's a lot of questions from the group and like conversations in the chat on like protected classes and I think that's something maybe a lot of us are are focused on. And I see Scott, you had a follow-up question there too. You want to jump in?
Sure, there we go. We can hear you. Yeah, great. Basically, I know there's a lot of discussion about protected classes and disparate outcomes based on things other than the definition of the protected class, such as socioeconomics, different opportunities available for different groups. So I was wondering from an analysis perspective, do you identify potentially, you know, confounding or hidden variables that may change the interpretation of the analysis that you're doing?
Yeah, so I'll answer that question by talking about an example that we've done. There is a, and again, so everything that we do is driven by the legal sector. So if it's not illegal, we don't care. And so we're focusing on what is strictly legal that companies need to comply with. So a recent example is the legislation that was passed in Colorado a few years back, which said, you know, if you're using an external source of data and making any sort of decision about someone's insurance, whether that's health insurance, life insurance, if it's how the insurance product is being marketed, if it's how their claim is getting processed, if it's how they're being, like, if you're determining their premium price. So if you're using any sort of external data and any sort of AI or automated decision-making system in that insurance context, in any sort of business aspect of insurance, you must not discriminate. And so that was a law that was passed.
And so again, this is one of the instances where the companies don't have that data. And so something that we look at is what data is going into the model. So again, they're not saying, they're not putting in race into that model because they don't have it. But because of how the, because of the world that we live in, there are patterns that algorithms are able to pick up on that result in disparate outcomes. And so one example that I think is very widely known is an example of the credit scoring system. And so there's a specific score, maybe it's your FICO score, there's other credit bureaus, right? So those are not used directly in these systems, but they have specific, and insurance companies use credit-based insurance scores. So as a part of their business model, these credit broker companies, they sell data to companies that say, okay, based on, I have 200 million, I have data of 200 million Americans. And I think that based on all this data that I have, this person has a higher mortality risk. And so therefore you should not offer them a life insurance policy at this price. You need to offer it at a higher price.
And so those are the kinds of things that we look at. And if you sort of take a step back, well, why is, what's, what are the factors that are connected to somehow in this credit scoring system that are leading to that, right? So there's baseline levels of things like, because of just baseline levels of racial discrimination in this country, for example, certain groups have worse mortality outcomes, and it's directly because of discrimination. Now, the insurance company didn't do anything there. They're not causing that discrimination. They're not causing those health impacts, but those health impacts are still there. So from a business perspective, what they don't want to do, what we don't, they don't want to get blamed for that because they're not at fault for that. And so what we can do is we can say, okay, well, we know that if we're using this credit score, it's going to affect this particular population in a worse way at the same value than at this other population. So what we can do is we can say, okay, maybe we'll lower this threshold, or we maybe will use a different variable. There's a legal standard in the employment context that has to do with an alternative, and I'm going to not say the name correctly, but it's an alternative data source, or you have to prove it's like the worst, the best possible alternative. So like it's making the, it's able to make the same business decision while also not discriminating. So you have to prove the existence of some other factor that you could use in order to make the decision that would result in the same business outcome.
Career advice and getting started in data science
Thank you. So maybe shifting gears a little bit here, but if you think back, Sam, to when you were just getting started in data science, is there anything that you would do differently or advice for anyone just getting started? Yeah, it's an interesting question, right? Because data science, let's see, when I started my PhD program in 2012, I don't really know that data science was really like a big thing. It was sort of still developing, and it was not as big of a topic as it is now. And now you have people who have PhDs in data science or PhDs in AI, right? And those didn't necessarily exist just 12 short years ago when I started my PhD program.
And so one thing that I would recommend that I think I've highly benefited from sort of making the switch from statistics to data science, I think I have benefited from the statistical background, because that is hugely important to understanding the context and sort of what models you can use and what scenarios, and understanding how to do exploratory data analyses and things like that. So I would definitely recommend getting a good solid statistical grounding in so that you actually understand the mathematics behind some of these concepts and sort of why the assumptions that you make matter in an analysis.
I see Sunday just asked a question in the chat as well, which was, do you find any value in assigning everyone to the same race and operational models after first building models that control for all confounders?
So I don't actually do a whole lot of model building these days. So I'm going to go ahead and punt on that question. And if people have thoughts on that, feel free to just add them below in the thread there too.
I just lost my mind. Oh, what I wanted to ask you, following up on the last question, we've talked about it a few times here in the Hangout that some AI roles are asking for like seven years experience or like in some of the qualifications. And so I was just wondering to learn a little bit more from you on making the transition from your data science role to the AI role. What was helpful for you in the interviewing process for that? Yeah, I don't have a whole lot of advice there, unfortunately, because I sort of, this AI role that I have now was sort of a factor of a bunch of people moving over to a new company. And so I did not have an interview process. I got to make up my job title, right? I got to make up my own job description. So I don't really have a ton of advice there in terms of experience and whatnot. So I apologize, can't answer. Well, you might have some advice for us then in terms of networking at your current company then. Yeah, exactly. I do think that it's important to
work out loud. There's, you know, it's good to have an online presence, you know, make LinkedIn posts, sort of have a website, write blog posts, right? Those sorts of things. And that's always good. But then also just going to events and learning and networking, I feel like it's kind of a bad rap, but it's really just asking people about themselves. And everybody loves to talk about themselves. As you can see from me just talking during this little 45 minute time that we've had together, everybody loves talking about themselves. So if you think about it as just being curious and learning about people, I mean, you don't have to, you can talk to a hundred people and you find the perfect job opportunity, right? So it's not necessarily like, oh, I have to talk to this exact right person. You just talk to people. And then that is what leads you to the path that you are destined for, basically. I didn't mean to talk about destiny, but wow.
No, I love that. And Toyin also added in chat, love the phrasing work out loud.
Is there like a specific moment that comes to mind where you were like working out loud and it helped connect you to somebody?
Oh yeah, absolutely. And so I had a, I have a personal website that I host with, oh gosh, I haven't updated it in a while, but it's like, it's Hugo. I do it all in RStudio. And so I, you know, I'm using all of those packages to update it. And I do all the blogging on there and stuff. I haven't blogged in a while. I've been a little busy, but yeah. So I, when I had this job fall through, this really cool job that like, I was going to get paid to go to baseball games, right? Like that's like the dream, but then this fell through. And so what I did was I, you know, I beefed up my website. I have a pretty solid Twitter presence. I know Twitter is not that much of a thing anymore, but you know, at that time it was. And so I put out a tweet and said, Hey, here's my website. I'm interested in these things. I like, you know, data for good. I like data visualization. I like communicating data science concepts to broad audiences. So, you know, I like all these things. Here's what I'm good at. Here's my website. Take a look. And then I got a DM that said, Hey, I have this job for you at this company called Prytora. And I were hiring for this role. And I think you might be really good at it because we need somebody who's sort of in this area of like data for good, like really passionate about, you know, making social progress and also has that data, that data expertise. And so then that turned into a job which then turned into this job that I have now. And so that really was an example of me having that online presence, me having that vulnerability to say, Hey, I need a job guys. And putting that out there, it got me to where I am now.
me having that online presence, me having that vulnerability to say, Hey, I need a job guys. And putting that out there, it got me to where I am now.
I love that. That's great. Well, thank you. Thank you for sharing. I see, Javier, you've started an interesting thread in the chat and I can't say I've actually been able to read it in multitask, but I just wanted to see if you want to jump in Javier and share that thought.
Thanks. Yeah. And Sam, I loved your comments. I had another meeting, so I jumped in 30 minutes late, but, you know, really interesting like background. Yeah, I, I don't really know what I was making this comment as a response to. Sam said something and it kind of like, you know, reminded me of a trend that I feel like I'm seeing. I help with a lot of like local data science I help with a lot of like local data science events. I went to a, it was a master's of business analytics program that I did a few years back. And so I'm part of like the Southern California, our users group, and we'll collaborate with UCI occasionally for different events. That in conjunction with like the interns that we hire at my company, I was just commenting, I'm seeing this trend where it's like, these really talented CS students that are seemingly crushing how to make like foundation models or Gen AI tools or LLMs are struggling with like just reading a CSV file or, you know, processing tabular data. You know, everyone wants an LLM or Gen AI, like that's, you know, they want it implemented in production yesterday, but I still feel like 99% of the work that businesses really need is still like tabular in nature. So anyway, that was just kind of the, my stream of consciousness there.
Yeah, I love that. I would love to expand on that. So I agree with you a hundred percent. And I think too, there was a really interesting article written a few years back. I think it was in Wired maybe. And I had an interesting conversation on Twitter with one of my sort of stats friends about, who's a stats professor in Minnesota. And she was saying, the article is saying like, we've created exactly one generation that knows how to use computers. And that's sort of the millennial generation, right? Or some of the like Gen Xers, right? Younger people, everything is app based, right? So they don't really understand how the guts of a computer works. And so to your point of like not knowing how to read in a CSV, well, they don't understand how file storage works on a computer because they just search for stuff. And I have siblings who are much younger than me. I have siblings who are 12 and 14 years younger than I am. And so I'm able to go and ask them, you know, hey, Gen Z, how do you look at your files in your computer? Oh, I just search for them. Or, oh, I have a Google Drive where I put everything and I just start typing for the file name and then I find it. And so like they, and even though, you know, and it sounds like there's some, even some CS students who don't understand that.
But yeah, but to your point about- I mean, I'm sure they do understand it, but where their effort is, is not like, let me learn how to, you know, do basic analysis on a 2D file. It's like, oh, I want to join this hackathon because it's like some advanced astrophysics problem. It's a highly dimensional data set. Or it's like, oh, you know, it's this really unsupervised problem. Like I want to tackle it with some transformers. It's like, it's really advanced stuff that's impressive and there's a space for it. But yeah, I don't know. Yeah. Well, I think a lot of it too, right, is sort of like what's flashy, what's sexy in the moment, right? Like, you know, data science was like super sexy, you know, five years ago and now it's all gen AI and LLMs and stuff like that, right? So part of it is just trends. But I think too, like you said, I totally agree with what you said about companies just needing some like very basic, like tabular analysis. By far, that has been the thing that has surprised me the most about this job and about some other jobs I've had where just creating good summaries of data is incredibly useful for companies. We had an example recently where a client sent us, hey, the client sent us all this, or an attorney who's an attorney at the firm sent us, hey, our client sent us all this data. But we also found out that this same data might have been leaked. And so, hey, can you look at this data that was leaked and sort of see if it matches this client data? And it turns out we were able to just directly say, okay, match phone numbers. And we were just able to directly match the phone numbers. And even that, which is a very, very simple data science ask, but being able to find the 500 phone numbers that matched in a client data set of maybe 200,000 rows of data, that was very, very useful to the attorney. And so even something as simple as that is very, very useful in the business context. So yeah, agree with you 100% there.
AI tools, learning, and code literacy
And not to drag this side conversation out, but Amiko just touched on something. His comment says, I'm training some folks to process data in R. They just asked an example code. They then processed to ask, or they proceeded to ask chatGBT to explain the code.
Like I spent about two years doing nothing but Shiny apps. So I feel like I haven't done it in years. I'm behind, but I feel like I was at one point getting pretty advanced at Shiny. And I was trying to teach a coworker who specializes in Python, or I mean, that's most of what he does is in Python. And he's great. He's great. But he wanted to build a Shiny app in R. He starts building the Shiny app in R and about a month later, this was recent, this happened this first half of the year. He asks if I could review his code. And I start reviewing his code and it looks crazy. It is R code, but it is like R code that comes from an alien or something. And so I start telling him, I'm like, hey, your app is running really well, but like, how did you create this code? He used some LLM. So he was pretty much using an LLM to create this code over time. And it worked pretty well. But like, it's this weird situation where now you have R code that's borderline illegible to someone that knows R and Shiny really well. Anyway, sorry.
I'll make a personal, I'll add a personal story to that, which is the last time I made a Shiny app, I asked chat2pt to do it for me. And then I changed a couple of lines of code and then bam, I had a Shiny app. And so I think there's a difference between doing and learning. So if you're learning something, if you already have the knowledge, a tool like chat2pt is great. I also spent several years writing Shiny and learning R and all of those things. So then in that moment when I quickly needed a way to do whatever it was I was doing in Shiny, that was great. That saved me a ton of time. But as a learner, it's definitely something I'm concerned about. So like I will, I have been guilty on occasion of going to chat2pt and saying, hey, how do I do this in Python? Because I don't know how to do this in Python because I like R better. And so just sort of working through that. And one way I help myself with that is I have a colleague who does use Python over R. And so we look through the code together and we talk about, you know, what is this good code and sort of what is this trying to accomplish? And is this the best way to do it? So yeah, I love that example. Amiko, did you want to add anything there or anybody that's in the thread there?
And so I think there's a difference between doing and learning. So if you're learning something, if you already have the knowledge, a tool like chat2pt is great.
Javier, you were saying... That took way too much time. Sorry, Rachel. No, you were saying you apologize for this, but these are some of my favorite parts when we get to like deep dive into a thread there.
Lisa, I see you had a few thoughts there too. Anything you want to add? We're good. I'm good. Just fun conversation though.
I know. These are the points where I really am excited to go back and read through the chat too. But as we have two minutes left here, Sam, is there a piece of career advice that you would like to share with us that you've either given people on your team or you've received across your career? Always learn new things. Always be learning new things, for sure. That's the number one piece of career advice because the more you learn, the more you can grow and do new things.
I love it. Well, thank you all so much for spending time with us today, hanging out with us. Huge thank you to you, Sam, for sharing your insights and experience. Thanks for inviting me, Rachel. Have a great rest of the day, everybody, and a great 4th of July if you have it off. As a reminder, we won't be here next Thursday for the Data Science Hangout, but we'll see you the week after. Bye, everybody. Bye.