Resources
Jul 21, 2023 |
1h 1min
|
378 views

Matt Frazier @ Pie Insurance | Removing blockers for your team | Data Science Hangout

We were recently joined by Matt Frazier, Strategic Advisor and former Chief Analytics & Underwriting at Pie Insurance

At 35:50 - Matt shared that being an effective leader is about being the person who makes sure that all of the conditions are set up, so that each and every one of their employees can succeed and can fully explore the domain of their competencies.

Find smart people, and then set up the conditions, and get the heck out of the way.

If there's a blocker that is in their way, find a way to remove that blocker.

Matt adds, “Now, are there performance reviews and things like that? Yeah, every company has them. I am a strong proponent against them. I think that performance reviews should be done every single day between the individuals that are working together. I don't think it should be between a boss and that boss's employees – I think it should be the individuals and their colleagues that are working together with one another. That's where that feedback should be coming from.

…I think that if we can create an egalitarian society where everyone is responsible to everyone else and everyone is giving constant feedback to everyone else on how to improve with empathy and with honesty, that is where you can create an organization or a social structure that actually allows everyone to win, and allows everyone to succeed.”

On the topic of removing blockers, what should you do if you’ve identified certain roadblocks you need to get past?

38:12 - Whether it's an I or a we…sometimes there's a blocker for a whole team and sometimes there's a blocker for an individual.

I don't know if any of you have taken any courses in nonviolent communication. I don't believe all of it, but I think it is a toolset where you can sort of mix and match.

1. Listen for needs and express needs. Start with, “I'm trying to do this for the company. I believe that this is going to be very valuable, do you agree?” You've got to get buy-in that what you're trying to do is valuable for the company.
2. In order to do that, I have certain needs that are not being met. This is what those needs are– it could be an individual, could be a group of individuals, a dynamic. It could be technology that isn't there or budget.
3. And then you really express that as a need. Not as so-and-so is doing something to me, and I'm mad. That's a sort of selfish communication. What you want to do is you want to try to abstract that as much as you can.

Now, you can't always because sometimes there's just a person who isn't on board and so, you say, “Look, I really need this person to be on board, but I don't think they are. And I want to ask you what we can do together in order to try and identify that other person's needs and discover why it is that they appear to be a blocker. It may be in my own head, and I'm just in an echo chamber. But let's have that conversation.”

And if you don't feel confident having that conversation yourself, go to someone who you think can have that conversation.

It doesn't have to be a manager, It could be a colleague who knows that person really well, and can set up that meeting so that you can have that meeting and identify what those needs are.

There's another book that I would really strongly recommend called Never Split The Difference by Chris Voss: https://www.blackswanltd.com/never-split-the-difference

That book basically teaches you to talk to an individual, have a dialogue with them until you get to why they're saying no to whatever thing you want to get through.

Yeses that aren't really yeses, don't matter. Make sure you understand what the other person or organization's needs are and then make sure that you're constructing the right conversation, and having those conversations through an expression of needs on both sides. Usually, that is the Black Swan that will bust right through that blocker.

________________

► Subscribe to Our Channel Here: https://bit.ly/2TzgcOu

Follow Us Here:
Website: https://www.posit.co
LinkedIn: https://www.linkedin.com/company/posit-software
Twitter: https://twitter.com/posit_pbc

To join future data science hangouts, add to your calendar here: pos.it/dsh (All are welcome! We'd love to see you!)

Come hangout with us! 💛

image: thumbnail.jpg

Transcript#

This transcript was generated automatically and may contain errors.

Welcome to the Data Science Hangout, hope everyone's having a great week. If we haven't met yet, I'm Rachel Dempsey, I lead our Pro Community here at Posit. This is our open space to chat about data science leadership, questions you're facing and getting to hear what's going on in the world of data across many different industries.

And we're here the same time, same place every Thursday. So if you are watching this as a recording sometime in the future, there'll be a link where you can add it to your calendar below as well. Together, we're all dedicated to making this a welcoming environment for everybody. And so we love to hear from everybody, no matter your level of experience, or area of work or industry.

I like to say that it's totally okay to just listen in, though, if you want. But there's also three ways you can jump in and ask questions, or provide your own perspective. So you can raise your hand here on Zoom, and I'll be on the lookout here. You can put questions in the Zoom chat. And if you're maybe in a coffee shop or your dog's barking or something, just put a little star next to your question in the Zoom chat, and I can read it instead. Otherwise, I'll just call on you to jump in and add some context. And then lastly, we also have a Slido link where you can ask questions anonymously. And I see Hannah just shared that in the chat here.

But thank you so much, Matt, for joining us here today as our featured leader. My pleasure. Matt Fraser is former Chief Analytics and Underwriting Officer, currently a Strategic Advisor at Pie Insurance. And Matt, I'd love to have you maybe kick us off by introducing yourself and sharing a little bit about your role and something you like to do outside of work, too.

Sure. So I'm Matt Fraser. I actually started in data science in 2008, right after the financial crisis. Prior to that, I was more of an insurance guy. I actually have a background in philosophy, specifically metaphysics. And so I'm sort of a Karl Popperian at heart, if anybody knows who Karl Popper is, who was the first empirical skeptic. And so I sort of transitioned my empirical skepticism over into data science and started to learn it in 2008.

Before that, I carried a number of executive positions at different insurance companies in the underwriting space. At that point in time, right, from let's say 1998 through 2008, I did not know of an insurance company that was actually doing any predictive modeling at all. And it is still the case that really the national carriers are using AI, they're using data science, they're using predictive modeling. But most of the smaller mutuals, right, the ones that are only in a couple of states, are really using other predictive modeling software as a service companies to do all of their predictive modeling.

So data science was a sort of an obvious transition after I made it, but I had no idea what the heck it was before I got into it. I was just a philosophy guy. I was a language guy. I love language. And I love understanding what is knowable and what isn't knowable, right. But I didn't really realize that you could translate that into mathematics until I got into a company called Valen Technologies.

So I sort of got out of the insurance industry and went into a company that was adjacent to the insurance industry, actually selling to insurance companies, predictive analytic and data science solutions, right. But before that, I was pretty good on the actuarial side. I'm not a classically trained actuary at heart, but I was the director of pricing at a number of different insurance companies as I sort of progressed through my career. And so I got pretty good at using Excel, okay. I got pretty good at grinding through data. I get pretty good at philosophically understanding data and understanding, you know, what it meant if certain data had low integrity versus high integrity and things like that.

So I was really good at data wrangling, but I didn't have any idea what a GLM was. And don't ask me to write any formulas down because, you know, I let you guys, I let the data scientists do that, right. But I'm pretty good at picking models apart because of my empirical skepticism that I learned at a very early sort of age through college and high school. My dad was a physician. He always used to kind of hammer me on, hey, are you sure you know that? Do you know that or do you think you know that, right? Do you believe you know that? Because beliefs can be wrong.

And so when I was at Valen, I was able to get trained by a former chief modeling officer at Capital One. Capital One Technologies, one of the major investors was Nigel Morris, who was one of the co-founders of Capital One. And Capital One was sort of one of the first banks that got into machine learning, got into predictive modeling on the credit card side. And so they had a very robust understanding. They were one of the early adopters, let's say, in the banking space and the fintech space of actually using predictive models in order to create a competitive advantage or arbitrage opportunity within the credit card space.

And so we were able to really capitalize on that connection with Capital One. We were able to take some really smart people from Capital One, bring them into Valen. And I really learned on the job from one of the former chief modeling officers on the commercial auto and auto finance side from Capital One. And the second I got into it, I absolutely loved it. I mean, Valen was my favorite job. The second I got into data science, I was like, oh, God, this is what I've been missing my whole life. Right. Like, I love the language. But to be able to blend the language and the math together, the language and the data together and actually produce something that gives another company a moat or a competitive advantage in a marketplace, man, that's really cool.

Like I can actually take things from the philosophy side where all you're really doing is creating more questions for yourself and actually translating that into real, durable value in the world, like something that's tangible, that can be seen through dollars and cents. I was super duper excited, you know, I got really excited about that. And then I translated that into Pie Insurance. I was one of the what I would call the silent co-founder at Pie. We took all of the learnings that we were able to extract from Valen Technologies, working with about 56 different insurance companies in the property and casualty space. And we translated that into building durable models that created a competitive advantage and an arbitrage opportunity for Pie Insurance.

Pie started in, oh, gosh, I want to say it was 2018 now. So we're now writing about 300 million. We're on the path to about 350, maybe 400 million this year. And I believe we're the fastest growing and quickest, most capitalized in insure tech that has ever existed. So somehow I was able to take that philosophy degree, translate it into data science, create a competitive advantage, and raise a whole bunch of money and create a company that now employs. Unfortunately, we just went through what we call a reduction force. But we had about 460 employees, we now have about 400 employees now. And I think that's the end of the cutting so far.

But I've been very pleased with what I've been able to do and what I've been able to learn in data science. And I'm just looking forward to a lot more learning opportunities as we move forward. You asked, what do I like to do? Yeah. Skiing, fly fishing, hiking, I like to be outside. So as long as it's outside and it's sort of a more of a single sport, I really like that kind of a thing. I'm not a big team sports guy. It's not that I'm not big on teams, because I'm huge on teams, I just don't play a lot of team sports. I play tennis and things like that, that are more double or single kind of thing.

Well, where are you outside right now in your background, your virtual background? This is Wadi Rum. So this is right near Petra in Jordan. And I try to keep it fresh with all my backgrounds, so I have goofy backgrounds like Animal House in Casablanca and other things like that. This one just happened to be up on my Zoom today. Well, thank you so much for joining us today, Matt. I feel like I can feel the energy from you and excitement coming through the computer. So thank you for bringing the energy here today.

AI and its impact on the insurance industry

Something I really wanted to ask you about initially, because it's something that comes up quite a bit in the data science Hangouts. So I know you wrote your bio for the Hangout with ChatGPT. So I feel like this is a good place to start off. But I'd love to hear your thoughts on ChatGPT, but also AI in general, and how you think it may impact the insurance industry.

Okay. I'll start with in general, and then we'll go into insurance. There's a lot of people on both sides of one particular argument, and that is whether ChatGPT has the latest sort of tangible instantiation of AI, right? Is AI going to take over the world? Are we going to live in a terminator universe where AI eventually is significantly smarter than us and somehow grows in tension? I'm in the camp that I think that artificial intelligence and large language models and the like are going to be a benefit to society. They are not going to be a detractor to society. Now, do we have to be careful? Of course, right?

Human beings are never at a space morally where we are technologically. And so the technology is going to force us to re-evaluate or trans-evaluate our scale of values, right? We're going to have to understand what can be done, and we're going to have to understand how we can make sure that we are distributing things like AI in some equitable manner.

In insurance, this is a real big problem, especially in the industry that I'm in. We have a lot of small carriers out there, but they don't have any of the data. There are about five or six or seven very large carriers that beat all of the small carriers up with data, okay? Now, we're not talking about large language models. We're literally just talking about simple GLMs. We're talking about any of the normal machine learning algorithms, whether they be gradient boosting, whether they be random forest, whether they be other elastic nets or tree models, it doesn't really matter. Whoever has the most data wins, usually in insurance, right?

In insurance, it's a little bit weird. We have a zero point inflation on all of the distributions that we're looking at. So in most cases, when you look at an insurance policy, about 75% to 97% of the time, the right answer is zero, right? People just don't have losses. But when they have losses, some of them can be very significant, and they can really hurt businesses, they can hurt individuals. And that's what insurance is for, right? Insurance is for that highly unlikely event that is catastrophic, right? That's where insurance really helps society. And at the end of the day, that's a really interesting problem to solve for a lot of data scientists, right?

The issue is that most of the carriers that are out there that aren't the big national guys don't have enough data in order to do that, right? And that's a big problem because you can use that to your advantage to become a monopoly. And if you see Hartford, if you see Travelers, if you see Liberty Mutual, if you see Swiss Re or some of the large reinsurers, they have all the data. So they can almost always build better models, because at this point, the algorithm doesn't matter as much as how much data you have and the integrity of that data, right? It's really important to understand your problem space and to think very deeply about that problem space and where you can go get data in order to solve that problem space.

And so at Valen Analytics, when I was there, we actually created a consortium data set, and that was specifically for small carriers. It was to allow small carriers and new entrants to the insurance market to be able to take advantage of large data sets with people who had subject matter expertise and data science expertise to be able to deploy models for those small carriers and democratize the insurance base to really allow new entrants and innovation into that space.

If anybody knows anything about the insurance space now, the one thing that is very, very clear is that it is probably one of the slowest moving industries on the planet. And the reason that it's one of the slowest moving industries on the planet is because there are five carriers that hold all the data and they don't have to move. They already have a durable advantage over everybody else. And so they're not really interested in innovating.

Now, that's changed in the last decade quite a bit because of people like Valen and other software as a service companies that are coming out or data science as a service company that are building consortium data sets or finding novel ways to use public source data in order to create large, either empirical or synthetic data sets that can bring value to bear for some of these smaller carriers and allow new entrants into the industry. And what I think that's going to do is, first of all, it's going to disrupt the industry. That's why the insurance industry has coined a new term called insure tech, right?

Most of those insure tech companies are not just tech. They should call it like insure data or insure machine learning or insure AI, because that's really what all of the insure techs are doing. They're using AI. They're using machine learning. It's really not AI. It's machine learning, right? It's building models on large consortium or synthetic data sets in order to extract or provide that value to smaller carriers. So they can compete with the big guys, because most of the small guys either just sort of lagger along, like barely soldier on through the snow in Siberia, right? Or they just get subsumed or bought or acquired by some of the large guys. And so they just get bigger and they continue their monopoly.

What I'm really interested in is making sure that data science and large available data sets can actually democratize the industry that I'm in, because I believe it's an unfair industry. And I believe it's a quasi cabal of monopolists that are actually controlling this industry. And that is true in many, many industries. And I think there's going to be a huge opportunity, especially with the advent of ChatGPT and these LLMs, for software as a service companies to go out and very easily get customers in order to optimize those standardized LLMs and do other data science and machine learning

projects for these companies, deploy those models for them, and allow them to hit those models with APIs and bring that value to bear.

Now, there's a lot more insurance companies that are looking to create data science practices inside of their own carriers. The biggest issue is you have to get a critical mass of data in order to do that. I mean, Pi is running $350 million in premium a year, and we don't have enough data. We actually had to go to Valen to purchase additional consortium data in order to build a robust model, right, because of that zero point mass. So you have to be not just a slightly big insurance company, you have to be about a six, $700 million insurance company that's been writing for three to five years at that level in order to have enough data to build a really robust model.

And so that's where I think using consortium datasets or novel datasets or even some of these new algorithms like adversarial algorithms to create synthetic datasets that are very closely mirrored to the underlying empirical data to be able to bootstrap and expand that data to be able to make more robust models. There's a lot of companies that I've talked to that are coming out with things like that, and Pi is actively doing a lot of POCs with those types of companies to find out whether we can take our smaller dataset and we can exponentiate that dataset so that we have a much larger dataset to be able to build models on.

Really interesting. Thank you. I see Liz had a question, a follow-up question in the chat. Liz, do you want to jump in here?

Sure. So I worked in the casualty space as an analyst for five years. I'm actually about to have an interview next week for a junior data scientist role trying to get back in the space.

But I was curious, regarding CHAT-GPT, I think, you know, there's a lot of people doing like paraprogramming and stuff with that, and they maybe don't realize that when they submit a question or a prompt to CHAT-GPT, if it's not a locally hosted LLM, that the host is owning that data, whether they're allowed to or not, and that creates a huge exposure. I'm curious if you have any thoughts or comments about that liability and that exposure.

Yeah, that's actually a massive liability. And I don't think that the insurance space is going to capitalize on CHAT-GPT, especially in the pricing and underwriting domain, until there is assurances that the data that they put in to train specific instances of the LLM are going to be protected in some way and cordoned off from the larger language model.

So right now, I know that there are several companies that I'm working with are like, well, we don't want to do anything until we have absolute assurance that our data is safe and that no one else can gain any advantage through the use of us putting that data into, right, some container or some space that allows us to train those LLMs. And I think that is where a lot of new companies are going to be extremely successful, is going to be prompt engineering and specialization of those larger language models, sort of the base model, and making sure that they're doing that in such a way that they can make sure that they have an understanding of the custody of the data and that they can either tokenize that data or make sure that that data doesn't get into sort of the larger pool of data that's being used in order to optimize the underlying base LLM.

It's really important. I think until that happens, Chad GPT is going to be used more for, in insurance, it's going to be used for automating processes around changing policies, right, making sure that the language and the chat functionality goes back and forth very smoothly without any human intervention. But I don't think that, you know, it's really going to, the insurance industry is really going to capitalize on Chad GPT as an underwriting agent, right, until all of that data can, you know, is safe and is completely separated and mutually exclusive from the larger pool of data. That's going to be a really important piece of the puzzle that some of these software as a service companies are going to have to figure out.

I mean, I know that I've already told my team at Pi, and I've worked with several other carriers, and I've told them all, don't put your data into Chad GPT right now because then everybody gets to use it, right? Like this, this thing learns. And it's like, you know, it's also about like work product and stuff, you know. Oh, totally. They, if you, if you say, oh, hey, I need help fixing this, you know, chain ladder model or whatever, but you have a little bit of secret sauce in there and then someone else asks and then boom, your secrets out and you lose an edge.

That's exactly right. I mean, even in our business, we have a very specialized transformation of the target space that we use for our line of business, workers' compensation. And we don't want anybody to know that. That's a specialized transformation. I haven't seen anybody else in the industry use. It works really, really well. I don't want to give that information to anybody, right? And the second I pop it in there, it's possible for someone to extract that.

Unless new technology and updated knowledge is actually given to the insurance industry by some of, you know, a new company, a new individual that's working with these companies that actually understands how to specialize these LLMs without delivering the data to the larger baseline model. I personally don't know how to do that yet, but that, I know that that's something that's being talked about very actively and heavily within the insurance space. It's, well, how do we take advantage of this? And if we're going to take advantage of this, how do we not give away our secret sauce and take advantage of it at the same time?

Like from my perspective, it seems like the best use would be like having a privately hosted LLM that serves as like a communication agent for people filing claims or whatever and their apps, you know, where it's kind of low stakes. You can segment the data, make sure it doesn't get out, but you also have like clear, knowledgeable communication happening that's automated and understandable. And the higher end stuff, like, you know, maybe we'll get there, but that's like pretty not there right now.

LLMs, underwriting automation, and bias

I think that's exactly right. The other big piece of the puzzle that is going to be there is right now, almost all insurance companies are, all insurance companies that are using machine learning are using it in order to straight through process some accounts, but there's always a human being overlay, right? So you allow the model to make about 30 to 50% of your decisions. And then the other 50% of the decisions, 50 to, let's say 30% of the decisions are sent to an individual underwriter in the world in order to make a human decision, right?

With the advent of LLMs, a lot of that need to have a human individual in the mix goes away, right? Or gets pushed back in the funnel. So you can allow a machine learning model, right? Which is, which is highly calibrated and constrained, right? And then you, here's the trick. You get to blend that together with an LLM that is highly specified and constrained and optimized so that those two pieces of the puzzle work together, right? And then you can remove the underwriter and use them after the decision is made as a check, right? So you've got a human in the loop who is essentially making sure that the decision, you know, didn't have any gaps in it, right? Or a human that can identify the gaps that exist in the system so that you can improve the system. Eventually, what you get to is that maybe you've got a human that needs to be involved, you know, 5% of the time instead of a human that needs to be involved 50% of the time. And the decisions are more consistent and create more separation between the good risk and the bad risk, right?

Now, the other big piece of that puzzle is, is equitability. So, you know, are you treating everyone the same across the parameters that matter, right? Whether it's demographic, whether it's race, whether it's something else, you've got to make sure that you've got a human in there that is making sure that the AI, the LLM and, you know, the GLM or the machine learning models are actually not introducing some specific bias into the overall solution.

That was actually one of the anonymous Slido questions was, how do you ensure that bias is not introduced when generating synthetic data? So it's, it's an awesome question and it depends on what you mean by bias, right? Because there's bias, whether it's statistical bias or some other bias inherent in every single model that's built, right? What we try and do is we, we look very closely at the variables that are going into all of our models and that's what's really hard to do with an LLM. I mean, you know, somebody else almost has to make sure that there's no bias in the, in the baseline LLM, but it's all about the covariates. So when we're building a machine learning model, we're taking a look at the covariates and we're making sure that those covariates are highly empirical and that they're based on, you know, insurance data for the most part.

Now, there's a lot of demographic data that can come in, especially in the line of business that I'm in, which is workers' compensation. If anybody doesn't know what workers' compensation is, it's essentially a coverage that employers buy so that their employees are covered if they have an accident and it pays for all of the medical bills plus their, their weekly wage, as long as they cannot get back to work at the, at the same role, right?

And when, when you think about that, there's a lot of demographics that come in. Things like average household size are actually important. Oddly, average hail size is important and you can't imagine what kind of biases that actually introduces. So if you're a homeowner's insurer, average hail size really matters, right? Like the people in Colorado, right? They have higher premiums than the people in North Dakota because North Dakota has less hail than Colorado does, right? Washington has less hail than Colorado does, whereas Washington has different problems, right? They have floods and things like that. California is highly biased because of other things.

But what, what you really care about is, okay, is there some group of people that already have it tough? And are we making it more tough because of something that's inherent in a covariate that we're using that just makes it more difficult for those people to purchase workers' compensation insurance? We try as best as we can to remove those variables, whether it's average wage, average household, any race demographics, anything like that is completely just dumped out of our data set right at the beginning, even though those exist.

And so we do our best to try and minimize those biases. But again, there's always going to be inherent and potentially unknown or unknowable biases that actually exist within an overall algorithm estimation. So we do the best that we can. Nobody's perfect at this.

There's a lot of new companies that are coming out with novel data like behavioral data. And that's something that the industry is talking about very, very closely, right? It's like, hey, if we can capture what your media content consumption is, right, and that has a significant risk separation between the good risk and the bad risk, is it highly correlated with things like race? Is it highly correlated with other things? Because it may be the case that people of different backgrounds consume different media. And that may inherently have some sort of correlation to things that kind of matter with regard to the current Zeitgeist in terms of which groups are having trouble and which groups aren't. Right? And so there's a lot of conversation there.

Definitely. I see a lot of great questions coming into the chat. So I want to make sure I get to some of these, too. I'll be more concise. No, no, no. These are great. This is a great discussion. I just want to make sure I get some of these, too.

But Anders, I see you had asked a question a bit earlier in the chat. Do you want to jump in here?

I think you largely answered it, Matt. But I was just really interested when I went shopping for life insurance recently that there were some companies that were not the biggest companies that were able to do everything instantly online and didn't require things like blood draws or physicals or any of that kind of like physical evidence. And I was assuming that that was because they are using their models. Or is it just because they are big enough that they can kind of like marginalize and don't need to be that specific? I'm just curious what sets up one company to be able to do that and one not to be able to do that.

There's a lot of moving pieces there. So generally speaking, what I would say is those people who are doing it online, their overall rate level is a little bit higher. Right? Because they don't have all of the data that some of the larger people who are not doing it online actually have. And so let's hear an extremely healthy individual. OK? You have no medical biomarkers. You have gone to some company and gotten a very extensive blood draw done, and you have a really good understanding of how healthy you are. Right? You're very likely going to get a better rate with a large company who's not online than a smaller company who's online.

Having said that, smaller companies are using more novel data than the older companies are in order to be able to allow consumers to purchase those types of products online. And the rate level isn't that different. Right? You are going to get a discount if you get your blood drawn and do the whole thing if you have extremely good biomarkers. Right? But if you don't have extremely good biomarkers, it's probably the best choice for you to go online and just get the policy because they're going to look at completely different stuff.

Now, you have to understand that a lot of those life insurance companies are also grabbing a lot of that data that many of us don't even know they're grabbing. Right? So purchasing patterns, media content, consumption trends, behavioral trends, these kinds of things that they can actually grab. So they're using your digital fingerprint a heck of a lot more than they're using your biomarkers. Right?

And so you're going to get a higher rate, but they can still do a really good job of separating risk, finding out whether you're really interested in drinking a lot of Jack Daniels or whether you're really interested in drinking a lot of mushroom tea. Right? So those are, you know, those are going to be very, very different. And companies now can use a lot of these third party API aggregators in order to get all of that information and consume that information through deep learning models that are actually

quite good at separating good life insurance risks from bad life insurance risk.

Thank you. Does that answer your question?

I see a thumbs up. So Arafat had asked a question, you put the star, so I'll read it. But it's, the question was, love seeing the energy in your communication. Do you have any tips on how we can improve our communication?

Tips on communication

So I have been talking to large groups of people for a very long time. When I started, I pictured everybody in their underwear. I'm sure you've heard that. It's an old trope. Right? But I think one of the most important things to understand about people in general is we all put our underwear on the same way. We all take a, you know, we all take a number two and a number one in exactly the same way. Okay.

I very early in my career was able to essentially come right out of college. And I got an underwriting position at an insurance company that was very small. I was employee number five. And so I had direct access to the CEO since I was right out of college. So my working environment did not include any social hierarchy. Like everything was flat.

And so that's what I would encourage all of you to do is if you're really going to speak your truth, you have to be able to punch through the imposter syndrome that you might have, the impression of a social hierarchy that you might have. You have to be able to talk to a board member or a CEO in the same way that you would talk to a colleague because we're all the same. We're all humans. We all have the same needs.

And, you know, maybe some people are smarter than others. And okay, that's fine. Right? But if you really want to do a good job of communicating, you have to communicate on the same level as another individual, which means you have to assume that you are on the same level as the other individual. I would just encourage everybody to understand that we're all human beings. We all have our foibles.

You know, there's something that's running around on LinkedIn right now, and it is you have to make sure that you are really open and that you're listening to other human beings, not just what they're saying, but how they're saying it. And the important reason for that is that in anyone's life, any other human being only knows about one to two percent of what's going on in their life. Right?

And so I would just really encourage you to empathize and consider yourself on exactly the same level as the people that you're talking to. And it doesn't matter whether they're a manager, a senior manager, a director, a VP, or a C-level employee, or even a board member or an investor. If you talk to them as if you are on their same level and they're on your same level, you will get better responses. I promise you. You'll get better responses.

And I know it's easy to say, and it's a lot harder to do because that anxiety comes up. If that anxiety comes up, picture them in their underwear. Understand that they take a crap just like the rest of us, right? And they've got their own problems.

I got where I got because I was lucky. I made a couple of the right choices. And I think probably 90 percent of the people that are actually in positions like mine, they got there through dedication. They got there through making sure that they were very consistent at a couple of things that made sure that they were a person that could be trusted and that they were a person that could be counted on, right? So doing those small things every single day is going to do wonders for you.

Now, it's great if you come up with a Svengali move and you come up with the greatest thing that the company has ever seen, right? That's sort of an added bonus. But where you win is doing small, consistent things every single day and talking to everyone like they're a brother or like they're a sister and making sure that you are really empathizing with them and that they know that you have the company's interest at heart, that you have their interest at heart, and that you're listening to what their needs are. If you can do that, then you will become much more confident in how you speak very, very quickly.

Now, I will say I've got a lot of bombast, too. My wife tells me again and again that I'm the only person she's ever met that doesn't give a crap what anybody else says about them. And that's completely true. I just don't worry about my image that much. I worry about being a good human. I don't worry about being a pretty human or professional human or the smartest human on the planet. I just worry about being a good human that people can trust and that people can count on. And that's it.

Effective leadership and servant leadership

What you just said about like doing those small, consistent things was reminding me of that video I saw you just posted on LinkedIn about like being an effective leader isn't just like going to the leadership workshop one time a year and saying, OK, well, I'm trained on being a leader. And so I was also wondering, what are other things that you think make someone an effective leader?

So there's a couple of things that I post on LinkedIn. Some of them are quite old. There was one that I posted that, and I'm sure everybody's seen this, the difference between a boss and a leader, right? A boss sits behind you with a whip and yells at you to get stuff done. And a leader is the one who's pulling you along. He's or she is the one who is removing the blocks, who is actually serving the employees. And I consider myself and I'm a strong believer in servant leadership, right? I'm not anybody's boss.

I am the person who is responsible for making sure that all of the conditions are set up so that each and every one of my employees can succeed and can fully explore the domain of their competencies. That's my job, right? My job is to find smart people and then set up the conditions and get the heck out of the way. And if there's a blocker that is in their way, I need to go find a way to remove that blocker, right?

I am the person who is responsible for making sure that all of the conditions are set up so that each and every one of my employees can succeed and can fully explore the domain of their competencies. That's my job, right? My job is to find smart people and then set up the conditions and get the heck out of the way.

Now, are there performance reviews and things like that? Yeah, every company has them. I am a strong proponent against them. So I hate performance reviews. I think that performance reviews should be done every single day between the individuals that are working together. And I don't think it should be between a boss and that boss's employees. I think it should be the individuals on, that they're colleagues on the team that are working together with one another. That's where that feedback should be coming from.

It should be a 360 degree feedback because performance reviews are a thing that can be weaponized so easily. And they say more about the person who's giving the review than they do about the person who's being reviewed, period. That is my position. Now, I'm happy to argue with anyone about that particular position because of how strongly I actually believe in that position.

I think that if we can create an egalitarian society where everyone is responsible to everyone else and everyone is giving constant feedback to everyone else on how to improve with empathy, and with honesty, that is where you can create an organization or a social structure and a social hierarchy that actually allows everyone to win and allows everyone to succeed and allows everyone to explore all of the levels of Maslow's hierarchy, right? That's how you become fully human and fully yourself.

Removing blockers — advice for individuals

Thank you, Matt. I promise I will get to some of the other insurance questions. And I'm going to be selfish for just a second here and ask a question. You said part of the role in a leader is removing some of those blockers for your team. And I'm wondering for some of us here who maybe are in a role where we've identified certain blockers for us, how would you recommend you go about like sharing that with your leadership to kind of get past those roadblocks?

Whether it's an I or a we, right? Sometimes there's a blocker for a whole team and sometimes there's a blocker for an individual, right? I think what you need to do is, I don't know if any of you have taken any courses in nonviolent communication. It's kind of a big thing now. We've taken a lot in nonviolent communication. I don't believe all of it, but I think it is a tool set where you can sort of mix and match some of the tools that come out of nonviolent communication. One of those things is to listen for needs and to express needs, right?

So you start with, I'm trying to do this for the company. I believe that this is going to be very valuable. Do you agree? Right? So first, you've got to get buy-in on what you're trying to do is valuable for the organization and valuable for the company. Okay? The second thing is, in order to do that, I have certain needs that are not being met. Okay? This is what those needs are. It could be an individual. It could be a group of individuals. It could be a dynamic. It could be technology that isn't there. It could be budget. It could be whatever, right?

And then you really express that as a need, not as a so-and-so is doing something to me and I'm mad, right? That's a very sort of a selfish communication. What you want to do is you want to try to abstract that as much as you can. Now, you can't always because sometimes it's just a person who isn't on board, right? And so you say, look, I really need this person to be on board, but I don't think this person is on board. And I want to ask you what we can do together in order to try and identify that other person's needs and discover why it is that they appear to be a blocker.

It may be in my own head and I'm just, you know, I'm just in an echo chamber, but let's have that conversation. And if you don't feel confident having that conversation yourself, go to someone who you think can have that conversation. And it doesn't have to be a manager. It could be a colleague who knows that person really well and can set up that meeting so that you can have that meeting and identify what those needs are.

There's another book that I would really strongly recommend, and unfortunately it's escaping me right now. The guy's name is Chris Voss. It's called Never Split the Difference. I don't know if any of you have read that. Strongly encourage you to read that book. Some of it's throwaway, right? But, you know, I follow the Pareto principle, so there's always 20% of, you know, absolutely fantastic nuggets in every book.

So that book basically teaches you to get to know. You talk to an individual and you have a dialogue with them. You have a dialectic with them until you get to why they're saying no to whatever thing you want to get through, right? Because yeses that aren't really yeses don't matter. You have to get to the no, right? And so that I think making sure that you understand what the other person's needs are or the organization's needs, if that's a blocker. And then making sure that you're constructing the right conversation and you're having those conversations through an expression of needs on both sides, a reciprocal expression of needs. Usually that is the black swan that will bust right through that blocker.

Amazing. Thank you so much, Matt. That's incredibly helpful.

Okay. Jumping back over to some of the questions in the chat. Alan, I see you had a question around some of the regulatory requirements. Do you want to jump in here?

Yeah, sure. Thanks. This is really, really interesting today. Matt, kind of following on the thread that you started with about these opportunities coming from data to democratize this industry. I wonder if there's a counterpart regulatory piece that you see. Are there regulatory gaps in that being able to be successful in making sure that you're successful in this industry that you described as being a monopolistic cabal? Are there regulatory things that you think are really important either for consumer protection or for pushing back against anti-competitive kind of practices that would make a difference?

Insurance regulation and information asymmetry

Yes. And this is where my opinions are probably the most provocative. I believe that, by and large, insurance is massively overregulated. Massively. To the point that it's actually been regulated in such a way that it's extremely difficult for new entrants to get in, right?

I think we've democratized some of it with a lot of the things that companies like Valen have done and other companies constructing consortium data sets. But the regulatory world in insurance is extremely antiquated. They're basically operating on 1970s principles, by and large, okay? And those principles were set up by the insurance industry at that time, which was populated by a very few, extremely large carriers, specifically for the purposes of creating barriers to entry for new entrants, okay? So, I believe all of the regulations need to be completely rewritten, and they need to be completely rewritten in view of the latest technologies.

There is something that one has to understand about insurance, and that is that it is an asymmetric information game, okay? The insurance carrier gets to ask questions, and they have to ask questions mostly through a third party, through the insurance agent, right? And so, not only do you have information asymmetry, but you have information distortion as well.

And so, what the insurance companies are trying to do is put themselves on a level playing field, because the insured has all of the knowledge about what their exposures and what the risks are. The insurance agent has probably 20% of that information, and the insurance carrier probably has about 10% of that information. A lot of what's coming available now is actually increasing that 10% to maybe 30%, 35% of the information, right?

There's still unconscientious, disagreeable people out there that are lying about all of their information. They're a roofer, and they're saying that they're a computer programming company. They're a trucker, and they're saying that they're a retail shop, right? And that's where models are actually quite good at determining whether someone is actually lying to you or not, right? And identifying and shaving down the moral hazard.

Now, having said that, there are still ways now, and will be ways in the future, that insurance companies can take advantage of information that the insured doesn't even know they have. And so, I think that information transparency is a huge piece of the puzzle. I think that if an insurance company is making a decision, letting the insured know what information they're using in order to render that decision is probably going to be very important in the future.

But I think that the regulations around machine learning and filing models with the Department of Insurance and the scrutiny that they have to go through is massively, like exponentially slowing down the innovation that could occur in the insurance industry.

It's also why most insurance companies that I've worked with are choosing not to file their models. There are very few people that are actually filing models in the commercial insurance space. In the personal auto space or the personal line space, you have to file all of your models. So, the Department of Insurance gets to look at it. They get to scrutinize it. They can say no on just about anything, and they do.

And there are some states that don't even allow for trade secret. So, once you have to file your model with the Department of Insurance, your competitor can actually get all of that information and rebuild your model, which I've done several times. So, I think there needs to be massive changes in regulation around data science, machine learning, and algorithms and how they're used in determining pricing and availability of insurance coverage.

But I think it also needs to be in light of the fact that insurance carriers are playing an information asymmetric game. We do not have as much information as we need in order to render the best possible coverage and pricing decision on an individual business. And really, that's what we're trying to do. We're essentially buying used cars, and we have to figure out whether that used car has had something bad that the seller of that used car knows exactly what's going on with that car. We don't know exactly what's going on with that car. And that's what I think most insurance carriers are trying to solve.

Thanks. There's a bunch there that's super interesting and a lot of perspective, especially about asymmetric information. It's new for me to think about. Appreciate hearing it. Thank you. Absolutely. AJ, I see you had a question in the chat as well. Do you want to jump in here?

Sure, yeah. I wanted to get your opinion on see how far is some of these job functions being automated, such as underwriting or app input, because AI is so much advanced, and it's reading so much data from your database, vast databases. So your thoughts on that?

Automation and the future of underwriting

Yeah, so I don't believe that all risks. And now with the latest technology, my opinion is changing sort of actively as I start to understand a little bit more about these LLMs and transformers and the like.

I believe that probably 85% to 90%, maybe even 95% of the underwriting decisions that are currently being made by human beings in the world will be eliminated through the use of machine learning and AI. I mean, that's just a simple fact.

I believe that probably 85% to 90%, maybe even 95% of the underwriting decisions that are currently being made by human beings in the world will be eliminated through the use of machine learning and AI. I mean, that's just a simple fact.

It is already the case that machine learning models can make a significantly better and more consistent decision than humans. Now, it is game theoretic, right? So what the models are not quite as good yet at doing, and obviously there's ways to solve this using models, they're not quite as good at understanding the market conditions and the changing market conditions with regard to what the best price might be, right?

Insurance carriers have to be able to understand good risk from bad risk, but they also have to write policies, right? And that means that sometimes you have to make trade-offs, and that's where human beings come in. They're making those trade-offs. They're saying, well, I might write this at a slightly higher loss ratio, but if I don't write this packet of business at a slightly higher loss ratio, I'm not going to have enough revenue, right? So the game theoretic piece of the puzzle, sort of price optimization, which is a really bad word in insurance, but that's how we think, right? We're actually doing price optimization every single day. It's just that we're doing it through human underwriters.

I think where the world's going to go in insurance is that a lot of the customer service personnel are going to be eliminated because LLMs are going to take on that, you know, chatbots and RPA and automation are going to be able to do probably 95 to 100 percent of policy changes, endorsements, things like that. But on the underwriting side, I think that it will probably eliminate 85 to 90 percent of the decisions. That doesn't mean that the underwriter's role gets eliminated. It means that the underwriter's role changes significantly. It changes to a portfolio management role. It changes to a gap identification role. And it changes to a role where an underwriter is now the subject matter expert that works directly with the data science team in order to identify and explore the gaps that may exist in the model solutions and helps to fill those with novel data and perspective that sometimes a lot of the data science teams don't have because they're not on the front lines.

And that's sort of a surrogate is it is really important in insurance that the data science team actually has a working knowledge of the line of business that they're actually looking at. So if you are building a model for commercial auto, you have to understand commercial auto. You have to understand how it's rated. You have to understand the coverages, et cetera.

I have a lot of PhD data scientists that I've worked with in the past that felt like, hey, I'm a data scientist. I don't need to know the underlying environment. I'm just going to grab the data and I'm going to build a model. I can tell you that in my experience, 90% of the models that are built in that fashion fail because they cannot be productionalized. The data scientists didn't find out whether the information was available at the time that the decision needed to be made or the target was not transformed in a way that was critically important because the data was centered or there was a treatment effect that they did not identify until after they built the model. They have to rebuild the whole model over again.

Finding valuable questions and building models

Thank you. This is kind of a follow-up question here about talking about building this model. There was an anonymous question. Can you walk through how your team approaches finding a valuable question, exploring that question, building and testing a predictive model, and then deploying that, the technical aspect of that?

I'm going to try and be concise there. I'm not sure how concise I can be.

Generally speaking, we will, on a regular basis, go out to each one of the department heads and we will ask them, what are your problems today? What is creating too much friction? What is something that you think could be a better decision that's being poorly decision today? Then we'll get all of that information back.

Once we construct all of the potential opportunities for data science, then we start to look around at the available data sources that we have and we start to do some discovery around whether or not we can actually solve that particular problem with data. Do we have the data available to be able to solve that problem? Is that problem a big enough problem to solve?

Usually, once we go to the department heads, we'll then go to the business or department analysts within that particular department and say, hey, how big of a problem is this? Can we quantify the friction? Can we quantify how much this is costing us? How many minutes? How many hours? How many days? How many employee days does this take?

Theoretically, if we were able to remove that and automate that, what would the savings be? Then, of course, there's other things in the industry. How much better can we make around these decisions? Can I separate good risk from bad risk? Can I identify the probability that someone's actually going to pay their bills? Can I identify the cost if they don't pay their bills? Can I identify how many times they're not going to pay their bills and we've got to do something with that policy, whether it be to cancel it and reinstate it and all that stuff because that costs insurance company money. All of those different things, even on the marketing side.

What's our current customer acquisition cost and where is it relatively high and is there a way for us to build a model in order to lower that? There's a wide variety of different things that you can do. Once you sit on a problem and you know that there's enough value there and you know that you have the data to be able to solve that problem, then it's a pretty standard process. You have to wrangle the data. You have to normalize the data. You have to work together with the department experts in that department to identify what they think might be interesting or valuable covariates.

So usually we will do some unsupervised modeling just to figure out what the data looks like. Where might there be blooms of data that could be interesting? And then we'll do a correlation study, which is with as many covariates that we can actually put together and then bring that back to the department and then ask the subject matter experts what's not in here that you think should be in here. And then they'll start to think about and construct some additional features. Then you start the feature engineering phase. You do that with the subject matter experts in that department and you try and get as many features that you think might be useful as possible.

And then, of course, there's some target transformation sometimes. Are you going to build a logistic model? Are you going to build a linear model? Are you going to build something that you're going to use for triage? Is it going to be sort of a categorical model where you've got five or six different decisions that need to be made and you're going to split those up? You kind of figure out what the best sort of opportunity is there.

And then we usually go through a sample and partitioning. How are you going to sample this? What's your strategy going to be? What is your validation strategy? So are you going to do cross-fold validation? What's your holdout going to look like? What's your train set going to look like? What's your testing set going to look like? Are those stratified correctly? Is anything time series based? Because that has a significant impact on how you're thinking about your sampling strategy.

Usually in insurance, depending on data size and the composition of the target, we use anywhere between three partitions and 10 partitions. If it's a more complicated problem and we've got a lot of data, we'll use up to 10 partitions and we'll hold three or four of them out. And then we'll use incrementally four or five or six or seven of those in order to increment through the overall model building process. We'll use the first partition, maybe even split that up in order to do feature engineering and basic variable selection and reduction. Then we will build our initial model and then we'll test it on the next partition. And then normally what we do is we'll build on, let's say you have five different partitions. We build on one and two, test on three. Build on one and three, test on two. Build on two and three, test on one, right? And then once you're really confident that you've got the right model using that data, you add the next partition. Now you've got one, two, and three built on four, four, three, and two built on, you know, tested on one and so on and so forth, right?

And we're constantly looking at coefficients, whether they're shifting, whether they're going up and down. If it's a tree model, we're looking at what the splits look like. We're sort of doing hyperparameter selection usually on those early partitions until you get to the final model, usually with two or three validations that's already held out. And in insurance, we're usually using between five and eight cross-fold validation at each one of those stages, just to make sure that we're getting, there's a lot of volatility because of the zero point math in insurance. So you really need to make sure that you've got enough cross-fold validation there because you can have a very large loss that makes things look really weird in any one partition of the data that you're looking at.

Hopefully that helps because I could go on for like an hour and a half. Thank you. No, thank you very much. We threw a lot at you with that one question.

Skills for new data science hires

I think we have time for one more question. Kalau, I see you had asked something a bit earlier about training. Do you want to jump in with your question?

Yeah, sure. I'm happy to jump in. Hi, Matt. My name is Kalau. I'm a customer success rep here at Posit. She's actually my customer success rep. Yes, for Pie Insurance and a few others on the call actually. But yeah, there seems to be so much to know in this space, not just for insurance, but like data science, programming in general. And as somebody who might be just starting out in industry or coming out of college, how would they get a job? I guess, what skills or knowledge do you think they should have or what are teams like yours looking for in new hires?

It's a really good question. I mean, so I think a working knowledge of both Python, and I'm going to say it, R, right? R and Python. A working knowledge of both of those is really important.

The biggest piece of the puzzle for me is an understanding of what I would call pragmatic, productionalizable model building, right? Do not narrow your scope or your interest down to research alone. You need to know how to get models into production so that they can be used by actors in the real world, right? If you're doing descriptive statistics, that's great. I mean, and if that's what you want to do, great. But you're going to cut yourself short if you don't know how to deploy a model. You're going to cut yourself short if you don't know what the process looks like in order to deploy a model. You don't necessarily need to know how to do the engineering, right? You can get an MLOps team for that. But you need to know as a data scientist, how to do things like parity testing, predictions between the production environment and the model build environment, right? You need to understand what that looks like, what the process looks like, right? And you need to understand how to actually go the last mile, if you will, right?

If you're just building general linear models and you're doing that descriptively and you're saying, look, I built a model and it looks great, right? The next question that any executive is going to ask you is, okay, how do we use this? How are we going to deploy this? Are we going to deploy it in SQL? Are we going to containerize it? Are we going to use Scala? Are we going to do something else in order to get that model into PMML, whatever the case may be? How do we get that model into production? And how do we know that that production model is going to score the same way as it did in the model build environment?

I think the other one is having a really good understanding and an ability to articulate what the model building process looks like all the way from finding the data sources all the way through to validating that model and making sure that you don't have an overflip model and that that model is really going to perform as you say it's going to perform in the real world on data that it hasn't seen before. Those are all really, really important. The last point that I'll make is, it is critically important for you to show in any interview or to any business manager that you have a high level of curiosity in the business that you're in.

Don't sell yourself short and just be a data scientist. You need to be a curious data scientist. If it's an insurance, you need to want to learn insurance. If it's in aeronautics, you need to want to learn aeronautics. If it's in process optimization for, I don't know, making candy bars, you need to really be curious and interested in what the process looks like to make a candy bar, right?

I think that that's really, really important is that data scientists can't just be data scientists. They have to be Renaissance people and constantly curious and autodidactic. That's the best advice that I can give is don't narrow yourself. Don't be an inch wide and a mile deep. Be a mile wide and more than a few inches deep.

Don't be an inch wide and a mile deep. Be a mile wide and more than a few inches deep.

Great, thanks, Matt. Thank you so much, Matt. I know we quickly came to the top of the hour here, but I really appreciated this conversation and you jumping on here and joining us. Absolutely my pleasure. I had a fun time. Thank you all for all the great questions too.

I'm going to put Matt's LinkedIn in the chat here if anybody wants to connect, but also I will share the recording, of course, to YouTube and to the Posit Data Science Hangout site. Have a great rest of the day, everybody.