image: thumbnail.jpg
Data Science Hangout | Tori Oblad, WaFd Bank | Getting Executives to Support Data Science
The Data Science Hangout is a weekly, free-to-join open conversation for current and aspiring data science leaders.
An accomplished leader in the space will join us each week and answer whatever questions the audience may have.
We were recently joined by Tori Oblad, Enterprise Data & Analytics Officer at WaFd Bank.
Here are a few snippets from our conversation:
1:14 - Start of session
3:00 - How to build an internal data science community
11:40 - Showing the art of the possible
14:00 - How do you get others to lead topics and foster engagement?
26:17 - Writing starter scripts for new users
35:55 - When to use R or Python versus BI
36:38 - Building toy models in Excel to explain it to people / to build relationships with business
38:33 - Avoiding vendor lock-in, being technology agnostic
43:35 - How to build confidence with IT and compliance
49:15 - Working with business users and creating business value
53:21 - Getting business and executive support
1:22:30 - What data scientists should focus on when communicating with stakeholders: value
► Subscribe to Our Channel Here: https://bit.ly/2TzgcOu
Follow Us Here:
Website: https://www.rstudio.com
LinkedIn: https://www.linkedin.com/company/rstudio-pbc
Twitter: https://twitter.com/rstudio
Transcript#
This transcript was generated automatically and may contain errors.
Welcome everyone to the Data Science Hangout. Welcome back to all the familiar faces. I think most everybody's been on here at least once, but also welcome to those joining for the first time.
Thank you, Rob, for covering for me last week while I was out on vacation. But for today's episode, or just to kind of level set on the Data Science Hangout as well, level set on the Data Science Hangout as well for people who are new to this. These discussions on data science leadership in the enterprise range from very human-oriented topics to a little bit of technical talk, but really focusing on questions that are most important to you all. So there's no agenda. Please join in live or put any questions that you have in the chat.
With that said, I'd like to just jump in here and introduce Tori Oblad, Enterprise Data and Analytics Officer at WaFed Bank. Tori is extremely passionate about training and building community, so I'm super excited to talk a bit more about that today, too.
Tori, would you be able to kick things off by introducing yourself and maybe sharing a bit about the work you do on your team? Sure. So for background, I have degrees in econ and math. I've been doing data science professionally for about 20 years in health care and in banking. I'm currently working at WaFed Bank where I'm actually starting the data science program there.
So I know we have these kind of standard questions that we kick things off with and then people start jumping in with questions. So one that I'd love to ask you is what are you really excited about right now with data science?
This isn't totally new, but when I first started, I was in SAS, and that's not very accessible. So over the years, it's been fun to see the accessibility, not just of tools, but really of data science being able to be shared with people. I'm now having conversations with people where they're starting to want to learn, and you couldn't take SAS to your house and learn it. But right now, currently at work, one of the things I'm instituting is having R as a standard. So anybody in the company that actually wants to do it can start learning. I'm not quite there, but we're close.
Building an internal data science community
That's something I'm super interested in because a lot of people will ask me about how they start their own internal user group or build their data science community. And I'm curious, what is starting to or what has worked well for you, or what are you doing to try and start to build this?
What I've found the most useful is it's really time expensive, but starting with a few key people and areas that want to learn something, working with them one-on-one on something very if I can fix or help them figure out how to fix their problem. So it has to be something that affects them. Otherwise, it goes away. We forget what we've learned. Starting that community where it's one-on-one, hands-on training. What are the issues? How do you think you should go about it? When you can find those key people that are excited, they will actually start learning. And they become these internal, wow, this is really exciting. And it's more credible to hear from somebody that is not a data scientist or super programmer that's scarier, that you can do it.
I've found that is really helpful to have residents whether they are Excel users or whatever type of more beginner analyst to help internally a community. Data science community is a little different. I can talk about that too because I think that's fun. So I don't know if people are more in companies where you have a central data science or you're more distributed.
What do people live in? I don't know. Feel free to weigh in in the chat or speak live too. But I know what I've heard from some people lately is how do we actually find those people that would want to learn or the super Excel power users that could maybe benefit from using a tool like R or Python? Like you said, they may not have that data science title. How do you go about finding all of those people?
One thing I've found very helpful is just for data science in general to be successful is to partner with the business. Generally, your business users are not programmers. A lot of that communication where we have an objective that we're trying to meet through those discussions, generally you'll find small bits of how are you going about that? Maybe we can tackle that in a different way. It turns into a lot of small side projects, but it's those relationships that you're making for the bigger data science issues that you find. The small guys that need help.
I'd love to chime in here. At my company, what I found to be very helpful is leading little learning cohorts. I really focus on EDA because RStudio is awesome for exploratory data analysis. I can create these little cohorts, some business analysts in there who work in, let's say, inventory management or transportation, pair them up with people working who you'd say are more technical data analysis roles. If you can get them learning together on a cadence, they form the relationships, they understand each other's problems, and they learn R at the same time. Then once you write any Excel user who discovers R, it's unlikely that they're going to say, wait, I'm going to only use Excel from now on.
Frank, if you feel comfortable doing so, it'd be great to introduce yourself as well to the group. Yeah, I'd love to. I'm really glad to be here. This is my first session. I work at Target. I live out in Minnesota. My whole life, I've been on the coast either up in New York, Boston, New Jersey, or in San Francisco. For the past three years, I came to work with Target in there. I came to build, let's say, a hybrid business intelligence team. When I came here, we weren't necessarily calling ourselves business intelligence or analytics or any of that. We called ourselves transportation performance to distinguish ourselves. Really, I built teams of pretty good Python and R users. Anyone who didn't know that, I taught them R first because I think R is a great introduction. Then if they want to do more machine learning, NLP stuff, they get into Python. For the past three years, I've been building teams that really sit in the middle of the business and then being able to do data analysis at a pretty rapid rate.
I lead a team now that I call decision intelligence. It's a little bit of a branding strategy on my part, but I need my stakeholders to be thinking about the decisions. When they say, hey, what do you do? What is your team doing? I say, well, we start with the decisions you're making and the actions you're taking. Then we can find the analytics to support that. That's what I do now. I lead a team, mostly support transportation within the supply chain of Target. I'd say my passion in Target is teaching people, especially heavy Excel users, how to get to the next step of what I call higher-powered data analytics.
I lead a team now that I call decision intelligence. When they say, hey, what do you do? What is your team doing? I say, well, we start with the decisions you're making and the actions you're taking. Then we can find the analytics to support that.
Thanks for joining, Frank. Frank, you brought up a really good point. Something I found very useful when you start having a community, even if it's just a data scientist and you don't know who is interested in learning R or even what data science is, is creating more of a, it has to be organic or it doesn't really work, but having those more of lunch and learns where it's, hey, invite your friends because I think people forget how powerful their networks are. You invite this friend who has another friend.
I'm new to this position. I'm still building that internal community. From where I came from, it was interesting because we had a few data scientist groups throughout the bank. Over just a few weeks where people would come and say, hey, this is the cool thing I'm learning, we started getting in compliance humans, which do not speak computer. It was exciting where the understanding of even what is possible out there really started to spread organically with people you wouldn't even expect.
Don't forget your friends and your friends' friends. It is a very powerful and good way to create that organic growth of information and knowledge. For the lunch and learns that you do, is that something that's weekly or monthly? We started out, I think, quarterly, but then people were really interested and they wanted to actually show, oh, I'm actually working on this thing and it relates to this other person's thing. We started doing it not quite every other week. That's a lot. I think we're doing it about every three weeks.
That consistency probably really helps too. It's interesting too. Something that started to happen, it started out more data science heavy, but what I thought was useful is a lot of data scientists really love to talk about the nerdy geeky stuff. It turned into not just algorithms, but more context where people are getting a good audience of practicing for who you really need to be talking to to get buy-in. Your executives, they're not going to totally care about your algorithms, unfortunately. Understanding how to speak to that broader audience. It became fun for people though because it was, this is this neat thing that I'm doing, why I'm doing it, and how it works. Let me explain it in layman's terms.
Then what was also nice is people from other parts of the bank that weren't in data science, but the girl that was running up ads, she had no background in it. She explained how her job worked, how optimization of Google works. She started working with one of the data scientists to make it better, which I don't think would have happened without that internal community, which is awesome.
I don't want to leave this topic yet, if that's okay with everyone because I see there's a lot of great comments in the chat window too. I'm just wondering if maybe Sep, if you wanted to weigh in with your experience here too.
Yeah, sure. Hi, everybody. My name is Sep. I'm sure we'll get to know each other more over the weeks. I find this a very interesting forum just to see how people in different industries even are impacting their organizations. I think for me, being in healthcare and healthcare tangent organizations, which in general, stereotypically are not very technical forward sometimes, I find explaining a lot of or showing a lot of the art of the possibility to be a fun exercise. I see Rick's comment and I totally agree. I don't like to just do necessarily flashy, but just to say, hey, look, there's efficiency gains that could be made if we do things this way. That's just the comment I wanted to make there.
I think too, the flashy, even the simple, what generally we think is really simple, most people don't realize how easy it is to get a data set and play with it and make it easy. The horrible manual processes that people go through, none of us would do that anymore, but it's everywhere. Fixing that, they go into Excel and manually changing things every single month, just revealing that that is not something you have to do ever again is mind blowing.
The funny example that I always give is because my name is Seth, if you put Seth in Excel, it renders in September. I find it annoying and it's another reason I just hate Excel. I always show that example is that if you're going to be dealing with names and you get a name like mine and Excel renders it in September and then you go export that and someone's dealing with it, it ends up causing these issues because it's trying to make it easy for you. I use that example all the time. It's a pretty funny one.
Fostering engagement in an internal R community
Eric, I see you had a question around fostering that internal community. Do you want to ask that one live? Sure. Great to be here. For those who don't know, my name is Eric. I'm in life sciences industry and quite a passionate R user in my various endeavors. My question is a bit loaded. I don't know if others have experience with this, but I lead an internal R forum at my company and it's very well attended. Everybody seems to enjoy it. I try my best to let others feel like be comfortable sharing the cool things they're doing with R because we know R is being more prevalent in our company and across life sciences in general. I don't know if it's just me. I have a hard time getting others to lead topics. It tends to fall on me or somebody over what I'll call the honest power users, if you will. I don't know if you had advice on fostering that engagement of everybody feeling comfortable to share their wins
with R, cool things they're doing, even if they just have questions. It's a bit loaded. I don't know if you had any advice on making people feel comfortable to present their work at a forum like that. It's hard.
Yes. It's really hard. I found out the hard way. I've had to spend a lot of time one-on-one with the person and just be like, we're planning to present this. Would you be part of that if you need any help prepping? I would love that, but it'd be so great to debut your work because X, Y, Z.
It's relevant in this way, which people haven't seen or they could relate to you, but it's time expensive. Training and getting people on board anyway is super time expensive unless somebody's found a magic pill. I've been looking for a long time. I have not found it yet. The training piece is even harder than just the community piece of it, in my humble opinion.
I do have some people helping me with the forum I lead, which is very helpful. We try to reach out to certain people, but I just was hoping I could get more people coming to us and saying, hey, I had this great idea, or I just don't get that as much. I'm trying to figure out ways of incentivizing that a bit. I'll keep searching, if you will. The only thing that I've found is helpful is I haven't, what I've done so far hasn't been restricted to a specific topic at all.
You're not required to do R or something amazing. It's more of, we just want to share something interesting with each other that might benefit. That's helpful because it's less stress. That's a great point.
Something that's helped me with the Boston user group is if I can't find a speaker sometimes, I'll make it a series of lightning talks and so have very short presentations that people will give. People seem to feel a lot more comfortable with that, especially if it's their first one. I think that's a much easier ask to someone than saying, leave this 45-minute session. That's a great point. I'll keep that in mind. Thank you.
Hi, my name is Edgar. I work with Daimler Trucks. I'm happy to be here and join the forum too. I was going to share, Eric, what we've done in our company is kind of phrase it as, what gap have you mastered lately? Then that gets people talking from an I learned standpoint versus I don't know how to do this standpoint. That's one approach.
Then the other approach is we just rotate who facilitates the meeting and that tends to take different interests. Sometimes people have a very concise problem they want to fix or a way to go about something that's not available, whether it is IT is not permitting it or the software doesn't do it. Then it becomes that kind of self-guided type of conversation. Definitely something along the lines of what gap have you mastered lately. That's cool. Thank you for sharing, Edgar. Happy to do so.
Getting people comfortable sharing their work
Maybe I can chime in on how to get people to show their stuff for Eric. I'm Rick. I also work in the life sciences. I work a lot with researchers that are doing and do a lot of training in R. 90% of my work is doing training and teaching and mentoring and all variety of different capacities. One thing that I've done a lot is to get them to be more comfortable discussing and showing their work because there's a lot of hurdles for new starters in any topic to get over is that it's not perfect and it's not the right way. They're a little bit embarrassed to show their stuff.
To help them get over that, what I'll do is I've done this in person and now it's easier. Now I do things all online. I get them to share their screen or I get them to explain what they're doing. If I give them an exercise I would just ask them how they solved the exercise. Maybe something that a lot of pedagogical people disagree with, I don't give them really simple questions that have a really nice, easy, clear answer. I give them an example that has the answer and I say, okay, do it. It's amazing how many times people just don't do exactly what's in the example.
They're trying all kinds of weird and wacky things. I'm like, okay, fine, just do it. I don't say go and look at that example and do it. They start doing weird things and I say, okay, how did you do this? Then they describe it to me and then I'll type it out or I'll get them to share their screen if it's possible and then other people see what they're doing.
That also helps them work. What I did in the workshops I would usually do with researchers is they would have time to do their own analysis and at the end of the workshop I would be looking at what they're doing and if I saw somebody that had a really cool example or really nice and nice progress, I would say, hey, do you want to take five minutes and show everybody the stuff I did in the class? Then I would help them and you can be like co-presenters and we would present it together. Then I would say, oh, look at this person over here and then I can go into the computer and I can type and things like that. I was kind of their co-pilot in doing that. That seemed to help them a lot and then they would be a little bit less self-conscious about it.
The third thing that I do is I would have them, they would just ask me questions. I would go around and say, okay, what is your project? What are you working on? Then they would say things like, is it possible to read in several files? I'm like, yeah, it's not a problem. You can read in thousands of files, no problem. They're like, wow, okay, how? Then I would say, okay, well, you know, let's try to figure it out and then I would present it to the class and then that would give me a chance to explain deeper all the problems with that topic and things you need to watch out for and then they would be very much engaged because they're like, oh, I really see the usefulness of this activity inside of here. All those things I think help to get them a little bit easier to present. Oh, that's very nice, Rick. Lots of good nuggets there.
We definitely have tried having certain themes throughout the year of like, maybe we're going to talk more about our markdown or we're going to talk more about interacting with databases and stuff like that, but I kind of see your point. If you get them actively working on something and don't hold their hand, but let them be a little more innovative and a little more creative, then you can glean some nuggets from that too. So, yeah, that's very cool.
Helping new R users through the learning curve
Thank you, Rick. I feel like that kind of leads into one of the questions that Hugh asked as well around helping new R users power through the initial learning curve. Hugh, do you want to provide any more context on that question?
Sure. My name's Hugh Welch. This is my third time on these calls and my first time interacting. I'm a management analyst with the Veteran Health Administration. I've been using R for the last couple of years to build informatics products, and we've been getting a lot of interest in those and people around different facilities wanting to kind of replicate some of that. Like, that's cool. How do I get into R?
And I kind of feel like, you know, okay, well, here's two books, a couple data camp courses, vignettes on all the tidyverse packages, read through these, and then we can start talking about what to do. It's been kind of difficult getting people who, some of whom have some programming background, maybe they've done some visual basic or they've done familiar with SQL or familiar with some stuff, but a lot of them are just Power Excel users, and kind of getting them through that mental shift into there's a different way of thinking about this has been an area that's been pretty difficult.
I've noticed remote is harder for me to get people into it, because I used to just go to somebody's desk and sit down next to them and talk through even getting R on your computer, setting it up and doing simple things initially, because then the person is set up, they know these simple things they can get into. But without the — what I've now started doing is doing a lot of screen sharing. I can't remember who brought that up, but screen sharing where you can interact at least if you can't type on each other's, you can either have the person watch you for a while and then have them do something so that after you hang up, they can at least refer back to the ones that you went over.
But if I don't do that, if I'm just like, hey, go watch this tutorial, it doesn't go anywhere. I've noticed there has to be some initial, I can do this on my own when somebody's not holding my hand. And then people are comfortable going to different resources. And it has to be absolutely applicable to their daily life or they're not going to do it.
Definitely. Finding that practical application is sometimes challenging, especially with new people who want to do something really complicated. Let's start with this very simple problem and do some EDA on it, do some basic aggregation, pull in some files. We're not going to build a model in the first 10 minutes that tells you what you need to know. But thank you.
I was thinking actually about putting together maybe some company R Markdowns with some very simple applicable to broad areas. We're pulling in data. Well, it's generally Excel. Make that simple for them to swap out what data. And we want to graph it. Make it very simple, just so they have little snippets of code that is super simple if you've played in R. But if when you're new, you're like, what is GG? What are my inputs? And it's overwhelming, so people just stop. So just making it, I'm going to make you feel powerful and make something beautiful from a very simple script that you can run on your own. I should probably actually do that.
Think about it. Yeah. And I just finished a master's in analytics, and some of the professors had great primers on where to start because the backgrounds vary. So establishing difference between R and RStudio and RStudio libraries and how powerful RStudio is, I mean, you immediately fall in love with it. But the cool thing is, you know, the primer includes things to the effect of create a folder, put this file in Excel in it. You'll read it with this script, but then go open it and check out what you did because you're going to edit the RStudio. And that just not only builds kind of a mental model of how things work, but also the trust in your new interface. So that really worked well for us. Well, for me as a student, that is.
It's funny trying to imagine if we were all in person right now. And so I take the signal of unmuting as thinking maybe somebody's about to speak. So I was wondering if, Frank, if you wanted to add anything too. Yeah. Following off what Tori was saying, I feel as though I've had good success writing starter scripts for people just taking a bit. So we are lucky at Target to have both an awesome package called Bullseye Connect written by this guy, Alex Meyer, that connects to all our databases. So you install the package and then you connect it to any database basically in the company. So that's one.
And then the second thing that's been huge is we have hosted RStudio and hosted Jupyter Notebooks. So all you do is go to a link and then say, like, start a new instance of RStudio. Just like if you were doing it on Kaggle, but we have it for Target so you're on our VPN and you can access the data. So that has been huge and it makes it very easy for me as someone who's interested in R and say, hey, what data do you use most often? OK, let's find your data source and then basically just build a script that's, what, 20, 30 lines of code and go through, read in the data, maybe do a tiny bit of cleaning, aggregate it, build a cool visualization with Plotly and immediately they're like, oh, this is cool. And then you can change one or two fields and say, like, you have an awesome facet plot here, change one field, and now you have something completely different and you're seeing your data in a way that you've never seen it before.
You can get people to that really fast and some people will stick with it. Some people will be like, this is great, I'm going to use this insight and I'm never coming back to this again. Some people are going to go with it, but then the cost of doing that and building those starter scripts individually is really low because we have the right tools so you can find out who is actually, like, the level of engagement for different users it doesn't really matter the portion of them that fall off versus the portion that adopts it because the portion that adopts it will then, I think, go on to become evangelizers.
Don't discount the, oh, sorry, don't discount the people that don't use it, though, because at least they know what's possible and those people can become very good evangelizers. Like, knowing the possible, yeah, but they're like, oh, wait, I know if you talk to so-and-so you can actually solve this problem. That is a big cohort because a lot of people are not going to keep going. It's just how it is.
don't discount the people that don't use it, though, because at least they know what's possible and those people can become very good evangelizers.
It's still not a waste of your time, so don't get too discouraged. Sorry, Hugh, I interrupted you.
No, you're good. I was just going to say thank you to Frank. I really like that idea of doing some of the grunt work for them to make it really easy to grab some stuff and get some immediate results so that they can kind of engage quickly and then, if they're interested, figure out how it actually works so they can do it themselves and break it apart and make it better. And they will. There will be plenty of those people that run with it and think this is amazing and, yeah, then they start doing it every day. Some of that setup stuff scares a lot of people away because they see, oh, there's code and connections and I don't know what this means. I'm just going to, you know, I'm going to move on.
What was that called? Bullseye Connect. That's actually the first package I wrote here because I was like, people get stuck on connecting to the data and you can't do anything without that connection. Others don't have that internal thing. That, I think, is absolutely key. You can't get to the data. They're kind of stuck and they're not going to get out Excel because they don't have access to anything else. That's a great point.
Oh, it sounded like someone was about to say something. If I can jump in again, I completely agree with the
starter script package or the idea of having the starter script. And I think one thing that I add on top of that is that it's helping them to, a lot of the, really, people that never did anything programmatic or command line tools or any typing in the terminal or anything, they just don't get the, they don't have the mental model. Somebody mentioned this idea of the mental model. I think that's really important. Not just in terms of the algorithms, but also like how this thing fits together, right? So they don't appreciate the self-referential functions that change the input, that you can't run them twice, right? Like if I do a log10 of a variable, I can't run that function again because I'm going to get error messages.
And then they freak out. It's like, well, why'd I get a message? Well, because you did the log10 of a log10. Like that, they don't get that. They don't see why that's a weird thing. They don't see that they can replace a variable and they don't get a warning message if they replace it. So what I do is, I think one next thing is to try and help them to build up the whole workflow.
So if you have a solution in Excel that they've already done, and this comes back to the other idea, right? Like how do you make it relevant for them? That script is going to be based on something that's really relevant in the project that they're working on or some idea. And you have this solution in Excel. And the one I like to do is QPCR because scientists love this technique and there's no clear standardized way to analyze the results because every different system is a complete mess. And everybody's spending a lot of time doing their analysis on QPCR. So they have their terrible solution in Excel and SPSS and graph type prism and things like this.
And then I say, okay, so let's write basically just like pseudocode. Like what do we need to do? We need to calculate this and we need to calculate this and we need to calculate this and we need to make this plot. And then they have a hard time sometimes telling me like what they want to do. And then there's communications issues. And I get to talk to them about how they need to describe what they're talking about and how when they say something like, I need to take the mean of the weight. And then they'll take the mean of I want the mean of each group.
So they'll take the mean and then inside the brackets they'll put the grouping variable. And I'm like colloquially when we say take the mean of each group, we know what we mean. We mean to take the weight according to each of the group variables. But that's not how we have to type it. So they type literally how they think. And then all these kinds of discussions help them to kind of see in their head this conversion from what they do in Excel into R. So then we build like the pseudocode piece by piece by piece. And then they can basically fill in the blanks of all the commands and they can see all the different steps.
And then in doing that, I also tell them if you can't get one part, don't worry about it because the analysis part here is separate from the visualization part. And it's separate from this part down here. So if you get stuck on that part, just do what you can and just leave it and then go to the visualization. And if you get stuck there, then go back to the analysis and then go to this part here. So you don't have to do one piece to get to the next part. And you can kind of go back and see. I had no idea that I could then fill it out in the end. And I think those kind of things help.
Finding the right level of data literacy
Thank you, Rick. I think Stephen asked a question that kind of revolves around this conversation as well. Stephen, would you want to introduce yourself and ask that question?
Yeah, absolutely. This is the first Hangout I've joined, so this is kind of cool. I started in SAS and I moved to R and I learned that it is not as simple as like, oh, just check out this vignette and go do that. And I really resonate with what he was saying. It's like, go take this specialization in Coursera and come back and we'll talk like six months later or something. I'm trying to find the right level of data literacy to build in our company. I'm comfortable with R and R Markdown. I have a senior statistician who is, but for those junior members, I want to enable them to get out of Excel.
I even want to enable them to just do simple things like pivot tables. So I'm trying to figure out like, does it make sense to really invest in having them learn R over six months to a year or, you know, we're talking about these dashboards and interactive things. There's middle grounds, I think, to take. I guess my comment, or maybe I'm curious what you guys have done in finding like which people are the right ones to just make them like pivot table masters versus those who should really be spending time to learn R and know how to code things ground up.
It really depends on the person's interest, right? I've noticed some people just are not interested at all. They see that Excel is their thing. And so there are a couple people I've been working with lately that fall into this category. And so I have gone through Excel with them to try to at least make it a little more efficient, but it still wasn't doing what we were doing. So on the side, I did something in R for them and have like this little dashboard that they can go interact with the data. And one of the guys is like, oh, this is way better. You know, I actually, I can actually learn some things and it was interesting after spending a couple months with him in Excel and then just being like, can you play with this? Because I can't get it to work exactly in Excel the way I want. Does this work for you? And he at that point, after we've struggled together forever, he's like wow, okay, maybe I will try. So it seems really person to person and interest level.
I don't know, like you mentioned learning Tableau, Power BI. A lot of particularly management doesn't seem to, well actually a lot of people don't understand the difference between why you'd want a full blown programming language versus say a Power BI or Tableau. And if the data is great, sure they can use Power BI, but if it's not good, they're going to struggle to death if they want to do anything that's not out of the box.
I went down that path, I don't know, maybe five years ago and I was like, nope, I'm not going to train people on Tableau and Power BI. It's too much to take on in addition. Plus I think most of the problems have to do, they run into data issues.
Excel as a prototyping tool
Anybody else have thoughts? Well, I was going to say, do one of these. When I started introducing R to a lot of the management at Target inspection, they were like, what do you mean it's free? I'm like, it's a free source. Just use it. But going back to Excel in particular, even my team of analysts, I encourage them to build toy models in Excel. If you can think through the idea in Excel, you're going to be able to easily explain it to people. So I'm a huge proponent of starting that way for both business analysts, operators, data analysts, because it's so fast.
And if you can build a toy model in Excel, then you can say, OK, we want to do even quicker or we want to do this at a higher scale. Then you move over to the R world. Really quick example. Again, I work in supply chain. We are trying to figure out how do trucks move around our yards and when are they going to get too full? I built the first thing in Excel because it was super easy to explain to everybody how I was thinking about it, the information that I didn't have. And once everyone said, yeah, it looks like it gets pretty close to the answer we're looking for. It helps us make decisions. And I said, OK, great. This is for 1 BC. I need to do it for 25, not creating 25 spreadsheets. Then I just put the logic in R and I was able to do it pretty darn quickly for all of our distribution centers.
I think people knock Excel. It is a great prototyping tool and it allows for that communication with the business. A lot of people aren't going to learn your code, but they will go through an Excel workbook. It is a useful tool that helps with communication and building that relationship with the business. I know we don't want to use it, but what? Meet them where they are.
I think people knock Excel. It is a great prototyping tool and it allows for that communication with the business. A lot of people aren't going to learn your code, but they will go through an Excel workbook.
Platform agnosticism and vendor lock-in
I was just going to be quiet for the rest of the time, but I've actually been down this exact road with Tableau and Power BI for the last few years. VHA had an enterprise contract with Tableau a few years ago. We developed significantly there, thinking that was going to be the future. Then two years ago, they decided they're going to cut the contract and go to all Microsoft. They said, we're going to migrate to Power BI. It's not a migration if I can't take my work with me.
We found that we were suddenly tasked with redeveloping so many things that we built over the last several years. We started looking around to see what were some other options. That's why we started to go down the R path. The ability to be platform agnostic from my perspective over the last couple of years has been invaluable to know that there's talk of contract shifts again here coming for the federal government. Knowing that no matter what happens, I can take my work with me and I don't have to keep rebuilding things. I can give somebody the code to reproduce and repeat it without having to explain all the data transformations and how the calculations interact has been just invaluable.
It's not a migration if I can't take my work with me.
Good point. That's another thing I keep evangelizing specifically to. A lot of banking heavily relies on vendors and often I know other industries do too, but it feels worse in banking than it even was in healthcare where proprietary software comes with support but you're putting yourself in vendor lock-in and you end up having to try to figure out what's going on and you have no idea. I agree with you 100% here. To be technology agnostic is what you want.
Compliance, explainability, and change control
With both the industries that you work in, I'd love to cover David's question around compliance. David, would you be able to introduce yourself and your role and ask that question? Sure, I'll be happy to. My name is David Dreyer. I'm a technical expert at Syngenta Crop Protection. We work in a pretty regulated environment in the pesticide industry. As a result, we have compliance with what are called good regulatory standards. We have to understand the data from cradle to decision essentially.
With a lot of these guideline studies, they rely on black box methods for data analysis and system statistics. There's a lot of opportunity here to potentially use some of the open source software to improve essentially the analysis, transparency, etc. of our studies. I'm just curious. Obviously, you work in a very different industry, but still compliance is common. I'm interested in how you manage those relationships. Could you tell us a little bit more about that?
In my experience, if I cannot explain my models, compliance isn't going to go for it. Unless it's from marketing, you have a lot more leeway. I'm not sure people call what they're doing self-modeling, but within modeling and particularly in banking, if you do not have good explainability, at least none of my compliance people or auditors are okay with it. Sorry, I wish I had a better answer.
I was going to say, working in healthcare, you're right, explainability, the ability to backtrace whatever you've done and show your work is a requirement. There's regulations out there that you have to be able to show somebody why the algorithm picked whatever it picked for them. My use here has been pretty niche. It's mostly operations, and it's not very widespread, which is kind of why I asked about getting new users on board, because they're starting to see the value in it. But we have to be incredibly careful that everything we do is approved by our IT and that we're incredibly careful with what we do with our data, because it's not our data.
We are very thoughtful at every step of the process to make sure that we're in compliance with all the rules and regulations and bureaucracy. Most of what we do is, we're not building black box models that I can't tell you why this neural network spit out this answer. What I'm more trying to do is tell you how many patients are going to be here tomorrow, and where are they going to be, and what's their likely cancellation rate, so you can maybe overbook or things like that.
Echoing David's question too, it'd be great to also hear how you build confidence in the tools as well, especially in these regulated environments where you need everything from documented verification of infrastructure to change control and validation quality systems.
That has been pretty good, actually, because Excel, what everybody has been doing for years, it is not easy to tell when people make a change or why they did the change. And so showing change control and how you can revert in the past and exactly what the change was, why, when, it makes people go, oh yeah, why haven't we always done that? So there's something I'm working on right now specifically because the auditability wasn't in the process.
They're like, we actually need to understand what's happening when. I was like, oh, well, we got a change out of Excel, guys, and let me show you why. And they're like, oh, yes, this is exactly what we need. And it made compliance super duper happy. With your relationship with compliance. So I have a lot of talks regularly with compliance people. I've built some good relationships. So we have open conversations where we're actually teaching each other some things so we can have more common of the language because if you don't have that relationship, compliance and audit can be
horrible. But if you can find people that you start having more of a let's understand each other's vernacular and where we're coming from, it goes a lot easier. It is a lot of work. The way a lawyer type thinks is very different from a data scientist. Very different. But the constant communication and camaraderie is useful in the long term.
I agree. Being now in a data governance role, it's just painful. They talk assets and we talk data objects and they're like, what? But yeah, once you bring them along, it's really pleasant.
It also builds the confidence in what you're doing. If they feel like they can see transparency and ask you questions without feeling stupid. Yes. And I've noticed that in a room with more than three people, they will not speak. I don't know if there's a ratio to courage, but it's definitely interesting.
I think most people are scared of looking stupid. The title data scientist is scary to most people. Unless you can destroy that bridge, that wall, people aren't going to talk to you.
Accuracy versus explainability
I was just going to say, in balancing accuracy versus explainability, depending on the methodology you choose, sometimes we may have to take a hit in accuracy because we can produce something that shows this is why it came up, it followed this path. We use a tree model over something else because I can give you a visual to say this is why the computer did this in this way versus something that's not. I can't give them a handout or a PowerPoint to explain the work. You just have to trust me.
Sorry, Alan, I realized that was your question. I just couldn't see your name on where I'm keeping track of all the questions. If you want to provide any other context or feedback on that question, feel free to jump in too.
I was trying to think if there was anything that I wanted to add there. Our environment has been changing such that there is more room and more opportunity for these kinds of things, but the industry and in particular the folks who drive the quality system and change control at that broad organizational level, it's pretty hard to bring in new things and have confidence. I've been lucky enough for a few years to be focused mostly in an area that didn't need to be compliant after having come from a space where I had to be super compliant all the time. I was like, oh, great, freedom. Now we're pivoting back and becoming a divisionally oriented function that is probably going to have to figure out some of these things again.
That's an interesting place to be thinking about where we have enjoyed being really nimble and flexible in terms of architecture and infrastructure and how we approach things. We need to start wrapping some process around it again. It's just on my mind a lot about how do we retain a lot of that flexibility and I think the power that comes with it while being able to demonstrate that our environment does what we think it does, that we could demonstrate that if need be. It's going to be a negotiation that we'll have to figure out, but it's encouraging to hear that folks have been successful in those efforts, even though I know we'll have lots of details that are going to be up in the air for us for a little while.
Tying projects to business value
I'm trying to scroll through to see if there's any questions that I missed so far. Hi, I'm Colin. I have a question. I'm just going to jump in here. Perfect. Love it.
I'm curious if you could speak to how you think about the different data analytics projects that you tackle and how it ties to business value. I say that because I find often I can go off on sidetracks of what I think is pretty interesting and kind of go down rapid holes, but then I have to step back and be like is this actually creating any business value? I'd just like to get your opinion on that. How do you think about the project that you're tackling and how it relates to creating business value?
It's really easy to go down those rabbit holes, but I've found works best for me and my team members that I've had do things is make sure that you are constantly working with business so that you're not going too far down a rabbit hole that they're like I don't care about that. That doesn't affect me at all. Where if I can get the business partners feeling like it's their project and they're right beside you helping through it, that decreases the rabbit holes. It's still easy, but I think we all have a tendency to be like that's an interesting problem. Let's go look at it, but if you have that consistent relationship with the business person where you are truly working together and it's not just okay data scientists go do your thing and then come back when you're done, that's when I think the rabbit hole problem gets bad.
There's a tendency for I think both sides to be like well I'm busy in the business so you just come back to me later. As the data scientist, I think you just cannot you can't go for that. I require if I'm doing work with you or for you, you have to be my partner. Otherwise it's not going to succeed.
As the data scientist, I think you just cannot you can't go for that. I require if I'm doing work with you or for you, you have to be my partner. Otherwise it's not going to succeed.
I don't know if that helps you. I have noticed people don't spend enough time creating that relationship and enforcing that relationship. What was that, sorry? That makes total sense.
I've definitely had experience of putting in hours and hours on analysis and then the business is like cool, but I don't know if they're using it or what exactly is happening with it. It's hard to say if that was even worthwhile.
Sorry, maybe I can jump in. Sorry to interrupt you. What you said before about talking with the people in compliance, I think was really relevant here also because I think oftentimes it's a communications issue. One thing that I found is that people will ask how to do something and I will show them and maybe I'll spend half an hour or even more doing literally word for word what they ask for and then I'll give them this beautiful result. I'll be so proud of myself that we came up with this beautiful solution and they'll be like, no, that's not what it was. That's literally word for word what you asked for. They'll be like, yeah, but no, what I meant was this. Then I realized that I didn't really get what they really wanted.
I think oftentimes in that relevance, what isn't really useful is if you can help the business is really asking them what you really want to do. Why do you need this number? What is really the goal? Maybe you can better help them really understand, okay, what are we going to do to have the really maximum ROI? That's something else that I really talk to my students about. What is the minimal thing, the easiest, smallest thing you can do that can have the greatest impact on your work and on your life and make things easier? Then focus on that, the 80-20 rule basically. Then they kind of start seeing things about it in a different way. Am I really doing something that's valuable? Is it really something that's going to have a huge ROI? Then they're a little bit more convinced to go about that. That's a lot of communication I find.
I think that's a common theme that keeps coming up too is communication with the business. Understanding how to communicate so that it's relevant to them.
I find that, at least for me, the objective is to make them feel ownership. If I can get that ownership where it's their baby and they care about the success or failure, that's the key. They're like, cool, it's your idea, then we're good.
That's a great point. I feel like that came up on a few of the past conversations as well. How do you make sure that you're working closely enough with the business so that they feel that it's their project too?
Engaging the business tactically
Tori, if I could follow up on that. What does that look like tactically for you? Working with the business? Yeah, like communicating, getting them on board.
I've spent so much time. For this current position, I actually created a whole vision which then I had to go evangelize. I had to get top level support where it was the executives, all the executives were like, yes, we agree. I said, okay, if these are the projects you need, who is responsible? Who is my point person that has the flexibility to take the time to be with me? You have to tell them. This is important, not just to me, but to you as the boss and the executive leadership.
Then it's been a lot of one-on-one conversations. I could try to make it not just about the data science part. I think a lot of the communication to build that relationship to get the ownership is really understanding what are the real underlying business problems. A lot of people will be like, they think problem A is their problem, but it's not. Only during a lot of conversation and really understanding and feeling their pain do you understand what the real problem is. That is what you got to suss out.
It makes the relationship good. People feel heard if you actually are understanding the real problem. Does that answer your question, Raul? Thumbs up.
One last thing before I need to drop off. The way I operationalized this was during the engagement process, we were doing elicitation. I would ask the leaders that were asking for the work for a co-pilot. This person was going to share and learn. They share business, we share tech. Eventually, they take over the analytics so it can keep surviving. They didn't upfront commit to a co-pilot to do the switch when the project was almost done. That told me that they were really not that upset. They were just kicking tires. We never really engaged with them.
Something that I've also found that helped build the relationships and helped the business take ownership is when you're presenting, have them do it. That really makes people feel ownership and proud. They care, particularly if they're presenting. Then they feel cool, which is great.
Documenting the problem discovery process
Tori, I see Rick has a follow-up question to what you just shared. Rick, do you want to jump in and share that? Sure. I put it in the chat. The question was, when you mentioned this thing about getting them to come up with the real problem, I'm also doing that. The question was, do you document that somehow? Do you have a protocol where you state what the problem is and then you take the steps? Then we can see the progression of how we went from the original problem to the so-called real problem, which is interesting in itself to understand how that process worked. You can show people how that worked. Also, the other one is it covers yourself because you can show whoever, like, well, that is on paper. That's what we said we were going to do and we did it so that people don't come back afterwards and say, hmm, it wasn't. Do you have some kind of strategy for doing that?
Because where I am now, it's so new, I've not been as good at documenting that whole process flow. Previously, yes.
For each project or working with somebody, we'd start out with saying, this is the objective and where we're going. We'd basically have minutes and notes that would just tack on. Not that fancy. Just note meetings, the notes from the meeting. Basically.
Sometimes there's more of a structured document. In some protocols, if I'm going to do an experiment, there's a clear, this is the input, this is what the reagent was, this is the incubation, there's all the little pieces inside and then you have the results. Then you can make a clear direction between the results and the condition of the experiment. I would think of something like that for, instead of just unstructured meeting notes, which leave a lot to subjective people's writing of what they want, and also interpretation, more of a clear like, okay, this is the goal of the project, this is the variables, these are the data sets, this is the main question, and then this is what each of the variables mean, this is the problems with each of the variables, these are the techniques that are recommended, the pros and the cons, and then a little bit more of a cleared outline in that way. Do you think that's worthwhile?
Yes, so by the point at you're actually producing some output, yes. But like, to get to that output, there's so much iteration beforehand that a lot of the findings, they're documented, but not as strictly as final outputs. Like here are the iterations, this is what we've tried, this is why we didn't use this variable. Sorry, I thought you were thinking more of in the discussions of trying to assess out what the real problem is. I found that's a lot messier. At least for me, I wish I could come up with a stricter, but I think I'm not as clear cut as I probably should be in that part.
It gets easier, I think, once you start swapping out variables and being like, oh, no, this didn't work, let's document it. Does that kind of make sense? Yeah, okay.
Clearly, I should work on that a bit.
I know we're getting to the top of the hour here and a few people have to jump off, but I'm happy to stay on for any other questions that we haven't had a chance to answer yet. I know there's one question from earlier that actually kind of touches on a few of the earlier points around building the community and training, but touches upon, like, why now? Like, why is your team starting to make this change, whether it's from SAS to Excel and moving over to R and Python? What has been the key driver there?
Can I also jump in for about one minute? Actually, from the other side of the table, the last few years I've been teaching R for students, and I think the main challenge is how to get students on board. I also teach Tableau, and I think in comparison, I think R is more user-friendly, and then Tableau is like a different Excel. For me, still most students prefer to use Tableau because they thought maybe it's more popular. I tell them the truth is that if you learn R or Python, there are more jobs available.
I think Tableau probably did something which could help them grow their popularity in the business community and also in academia. Primarily, I think they introduced university ambassadors to university students. Let's say if you are good at using Tableau, if you are proficient, maybe they will give you a title because this student is a Tableau ambassador, and then they send their portfolio to the company, and then they apply to be ambassadors, and then they can organize meetups. I mean, let's say R has been such a great language in which I benefit personally and professionally.
I would say maybe in the future, if you can organize more data analytics competitions for students or probably meetups or probably these kind of recognitions. I mean, the next generation of students that could benefit from these skills. Otherwise, the graduates, they only have Excel or SPSS. I mean, SPSS probably nobody wants to use it from a company, but the majority of professors still teach SPSS. I understand maybe a few years ago when I started teaching R, I had maybe less than five students who picked up the R programming language, and at the end of the semester, he told me that he was working for a banking company, and he said he was using sentiment analysis to analyze customer complaints for the first time ever.
He has an engineering background, so the majority of students do not have that background to understand how R works.
Yes, thank you. That's a good point, Jimmy. Tori, is that something that WaFd Bank does for internships or maybe data competitions? I know in the sports analytics meetup, we talked a little bit about the big data bull, and that's how they find a lot of people for new positions.
We are in our absolute infancy, so no. Previously, one of the things that I found useful, not where I am, but my prior bank, we did a lot with the local university, which was awesome. It started building a relationship between the data scientists at the bank and the university, and it helped drive actually what the university was teaching and found some hires there. That was awesome.
The issue with banking is so many of the universities are like, can we get an actual data set? Real data versus the stuff we have, they're not comparable. Real data is so much messier and harder than the lovely Iris data set. People come out of college, and they don't actually know how to really deal with problems in the data set.
I've yet to figure out how to get something like that going. I know there are a few banks that have been able to get their compliance okay to let people have some competitions with their data, which I think would be awesome. I have not yet figured out how to do that.
Advice for aspiring data science leaders
I don't think I see any other questions in the chat, but one other question that I'd love to ask Tori is, do you have any advice for people who are on or maybe aspiring data science leaders for getting into leadership roles? Into a leadership role, I think it has to do with partnerships with the businesses and communication. I've noticed the people that are able to speak the language of the business people are the ones that do the best. I know it's fun to talk about the statistics and the fun modeling that you go into, but that does not generally make an executive go, wow, they care about the dollars. If you're able to relate things back to how it affects the business in the bottom line and speak that language, those are the people that generally get picked to start presenting and to talking with other leaders because they can actually communicate and understand what you're saying.
It's a lot of work to understand the business and to be able to speak both like mathy nerd and business, but that's really the key.
It's a lot of work to understand the business and to be able to speak both like mathy nerd and business, but that's really the key.
Thank you, Tori. That seems like so much of it really goes back to the communication piece.
I know we're a little bit over on our original time and I don't see any other questions. I just want to say thank you so much, Tori, for sharing your insights and your experience. Thank you all for joining as well. Tori, quick question. If people want to get in contact with you, what's the best way?
Probably my Gmail. Here, I'll type it. You can try LinkedIn. I'm just not good in social media. I will check it eventually. There we go.
Awesome. Thank you, Tori. And thank you so much everyone for joining. Hopefully, we'll see you next week, same time.
The case for a data science professional certification
Can I ask a quick question for people who are still here? Something I was thinking about that has been driving me crazy just about data science is I wish there were frustrated that there is a lack of a professional certification organization. This runs into a couple of different things. One of the issues I see where data science fails is because there is bad analysis. People aren't actually able to produce something because they don't have the basic skills that they need. But then people that want to actually understand that they have the skills and can tout it have a harder time doing that. I was like, man, these societies exist for every other profession.
You've got accountants becoming CPAs. You have financial analysts like the good ones are CFAs. Sorry, I'm saying all acronyms. Chartered financial analysts, lawyers, they have the bar. Everybody in all these other professions have some kind of professional organization, but we don't as data scientists.
A lot of people trying to become a data scientist, they don't have an understanding of what the minimum requirements are, which I think would be useful for them. I'm curious if this is a sentiment that other people have or if I'm just going crazy. Maybe JJ can fix this one, Rachel.
One of the things that all those professions or most of them have in common is their protected names. You can't call yourself a doctor. You can't call yourself a lawyer or an accountant unless you have that. Those are names that are regulated and protected. If you call yourself a doctor and you don't have that, then you're screwed.
I don't think data scientists would be ever a protected name because scientists are not. I'm a biologist. I have a doctor title, but a lot of people that don't have a doctor title call themselves a biologist. I disagree with that, but I can't say anything to them. I think science in general is not really a protected name that people can have.
I also wanted to mention, based on that, there is this data management organization. I'm going to send you. I'll put it in the chat. This is the only one that I ever found because I got a task to do a data management proposal that I disappeared into the ether. I don't know what happened with it, but because of that, I did a lot of research into these guys. This is the only one. There are schools that they pan out. To me, it was the only one. Maybe other people in the group know other ones. It seems to be the only quasi widely recognized certification in the field, although everybody and their brothers got a different certification program.
You can do the certification for us. I have seen people ask for it. As data management becomes popular, this is being asked for on job profiles, but it's not specific for actually doing data science itself, but it's something related.
One thing that has been frustrating for me is going through. I feel like a lot of people will take one course on Coursera, and they're like, I'm a data scientist. I'm like, hold up. What are some fundamentals you need to understand? P-hacking is a problem. Do you actually understand basic statistics? I get really frustrated as well.
I'll check that out. Thank you. It's a great point, especially as people talk about reproducibility crisis and all that. The fact that there is no real certification.
Balancing technical and business skills on a team
I was going to ask, actually, you were talking, this is sort of the technical skills. Do they have the stats and the understanding and certification on that? You were talking earlier about the business understanding and communication. I'm just wondering, how do you, I don't know, is that a hiring thing? What is that? How do you make sure you get people that not only do they know how to do the data science piece, but they have the business piece as well?
When I'm hiring? You're building a team.
When I've inherited teams, it becomes pretty clear who is interested in owning and really thinking. Unfortunately, a lot of people don't like to think. They generally leave if the expectations are high and their peers don't feel like they're pursuing it.
Are you asking how do I make sure a new hire has those softer skills? I guess what I'm wondering is if that's a critical component of the success of the team, how do you make sure that the team has those skills? Hiring seems to be an obvious place, but if you already have a team, I guess what you're doing is you're setting expectations and they weed themselves out.
That does work, actually. For the statistics and understanding what they're doing, I feel like that is an absolute fundamental. If you can't talk to somebody, that's a different problem. If you're making just bad analysis, it's going to destroy the team. The credibility is gone.
My previous team, there was a range of who liked to do what. Because the technical skills were all solid, not everybody wants to talk to the business. The people that actually were good at it could take that role.
In hiring, every time we're up to get a new person, the team would actually get together being like, what qualities are we lacking as a team? What do we need to really focus on to make sure this person is adding that value to our team? I don't think everybody is the same. I think you rely on each other. That's when it becomes powerful.
Have you ever seen a project manager taking that kind of role, the communication with the business? Or is it usually someone from the team that wants to do that? My experience of project managers is not awesome. I've had one that I really liked.
I'm sure there are some great ones. It's not that big, but we would just take on project management ourselves. Because generally, the project managers we had would turn more into task managers and not understand really the intricacies.
I think not that long ago, somebody brought up their frustration with Scrum. Now we're doing stand-up meetings and it doesn't seem to be working for data science. I get that completely. Because data science and building a software product are so different, but everybody assumes the same. It feels to me like project management is generally more on a, this is the dedicated timeline and you're going to hit that.
That's not really data science. I've not found project managers that useful. The one exception is they often will know more people initially at a company so they can help connect you with the person.
Can I ask on the product managers and Scrum, because this was a big conversation for a while, that data science cannot be done in an agile way. It just doesn't work that way. There's a lot of people who argue that it can work in that way. Also, it comes back to the topic we mentioned before. I'm on the fence. It doesn't matter either way. What we mentioned before, reiteration and communication coming back and back. That would fit well with these kind of sprints and having a clear result and checking in, are we going in the right direction?
Some people argue that actually it does work quite well. I think it can work well. I think people try to shove data science into how they create a software product. Shoving it into the box, it doesn't work. Data science, if we aren't actually agile, if we're not failing fast and making sure what we're doing still makes sense, will fail. What frustrates me is I'm like, hold on, the idea of agility isn't Scrum. It seems like people cross those wires.
Yeah, you want to fail fast. Let's make sure we're doing things that are reasonable and not going down a rabbit hole and not talking to somebody for a month. The timeline thing can be tricky. I'm not sure about all that either. I still kind of prefer working on boards to the current Scrum. I don't know. I'm on the fence too.
Rob B., I see you unmuted yourself. Did you have a final question? Thanks again for doing this, Tori.
Hiring for logical thinking over industry experience
There's business skills. Then I guess one level deeper is there's industry level skills. You're a data scientist at a financial services company. You understand finance. Is that on the critical path for a hire on your team or in your organization?
No. When I look at hires, I care if they can think logically. I do not care if they even have programming or if they understand the industry. I care about the way they think because I'm like, okay, if you think logically, hopefully they have programming. Most people now that are applying have programming. I can teach banking. I cannot teach logic. If you don't think that way, that's not going to work.
I can teach banking. I cannot teach logic.
Banking, I can teach. Discount cash flow, why bankers do things the way they are, that's simple compared to what I expect from a budding data scientist. There was one person I hired internally. Everybody else had it came from oceanography, physics, math, and none of them were banking. I actually found that really helpful because people see problems differently when they come from different industries. It's more of how do you tackle a problem?
Testing candidates for logical thinking
When you interview, how do you test for that skill? Can they think logically? I do actually use tests. We do have the regular talking to people. I usually give a couple different tests. One is a take home, like you have X amount of hours and here's some data. Do something with it. Give me a report. It's really open-ended. I found there's not so many ways you can solve it correctly. The wrong answers are just blatant.
It's just obvious. The other thing that I don't know how to, we've been doing it online and it's weird to do it online, but it's nice to have somebody actually come in and try to do something live without internet. You're like, okay, here's a computer if you want. Here's your task you have to solve. Go for it. I don't really care if they solve it. I want to see how they go about it. I had one guy that could not for the life of him remember the package. I think he chose to use Python. He was like, I don't care what programming language you do. You have this task. He could not remember how to run a regression. On pen and paper, he figured it all out and then used his own algorithm and implemented it. He was one of the best hires I've ever hired. It was beautiful.
Anyway, I try to pinpoint what are the skills and the thinking that I want to see. Can you demonstrate that? What I really want to avoid is the people that will be like, I'm going to throw all this data and use a convolutional neural network to solve something that really could be done with regular expressions. I've seen this. That's what I don't want. It's really obvious. It's a simple task.
That's great. Do you ever ask them, because you touched on it before, asking them to explain to someone what they just did? If you then get people talking about here's the model and that piece as opposed to the value. Do you test for that as well?
I played around with a couple things when somebody is doing a live coding or a live... Can you explain your thought process as you're going through this, which is really enlightening? I find it useful to see what type of questions people ask or how much will they persist? How much understanding are they trying to dig? That sounds subjective, but it becomes very apparent.
Bridging data science and business value
Going off of Robert's question, do you find a lot of hires that do think logically but really only when it comes to the data science aspect? In the process of teaching them the banking and the business side, I guess a lot of these questions around what is the value of these things connecting with the business partners. I think I was thinking what Robert was is it might be helpful to find people who already have that sense of why this is valuable to the business. It sounds like you're more confident in your ability to just train them to do that and you want them to have those core skills instead. What is my question here? What is training them on the business look like to you?
I always force people to actually have serious partnerships, but I think I understand your question because I've had a few employees where they get so excited about the modeling that they forget that actually we have to make value or there is no reason that we exist. We're just becoming an expense. I've had to have those conversations before. I know this is exciting, but there is a balance between figuring out a really cool way of doing this and giving real value without taking too much time. Those are conversations and depending on the person there are just people that would love to be in academia but are not.
Some of the PhDs I have had more of that with. You've been in school forever. That is what you do.
I had one other thought because we're still talking. Getting people to present their results is useful, but I've had to coach multiple people being like, remember who's in the audience. You're talking to the CEO. Is he going to care about this? No. He cares about his financials showing this, this, and this. How can you relate this back? Instead of spoon feeding it though, you've got to get them to actually understand how to do that, which takes time.
I do have a meeting in five minutes, but I have three. I was going to say I'm waiting to see if someone else jumped in. Thank you so much, Tori. Thank you, guys. This was fun. This was a great session. I hope to see you next week, too.