Resources
Apr 17, 2024 |
1h 0min
|
501 views

Greg Shick @ Charles Schwab | Data Science Hangout

We were recently joined by Greg Shick, Director of Analytics at Charles Schwab to chat about turning data into actions – taking large volumes and a variety of data, producing previously unknown insights that lead to a data-driven strategy.

Speaker bio: Greg Shick leads the centralized data and analytics functions for several internal organizations at Charles Schwab including asset & risk management and capital markets. His career has spanned a variety of industries, including finance, telecom and consulting, with expertise in data and analytics for finance, operations, customer care, HR and actuarial science. He earned his M.S. from the University of Colorado at Denver on Finance and Economics and graduated from the University of Denver with a BSBA in Statistics and Mathematics. He sits on the advisory board for the University of Denver BIA department as well as the KOSON Schools board of directors. In his free time, he enjoys spending family time with his wife and two boys, hiking, traveling and 3D printing.

__________

ā–ŗ Subscribe to Our Channel Here: https://bit.ly/2TzgcOu

Follow Us Here:
Website: https://www.posit.co
LinkedIn: https://www.linkedin.com/company/posit-software

To join future data science hangouts, add to your calendar here: https://pos.it/dsh

We'd love to have you join us in the conversation live!

Thanks for hanging out with us! šŸ’›

image: thumbnail.jpg

Transcript#

This transcript was generated automatically and may contain errors.

Hi everybody, welcome back to the Data Science Hangout. I'm Rachel. If we haven't met yet, I lead customer marketing at Posit. So I'm excited to have you joining us here today. If it is your first time, let us know in the chat. We'd love to say hello and welcome you in. The Hangout is our open space to hear what's going on in the world of data across different industries, chat about data science leadership, and connect with others facing similar things as you. And we get together here every Thursday, unless it's a holiday, at the same time, same place. So if you're watching this as a recording on YouTube in the future and want to join us live, there'll be details to add it to your calendar below.

Just make sure it adds it for 12 Eastern Time. But at the Hangout, we're all dedicated to keeping this a friendly and welcoming space for everyone. And we love to hear from you, no matter your years of experience, titles, industry, or languages that you work in. You can choose to use the Hangout Hover. It suits you. If you just want to listen in, you can do so or be a part of the party happening in the Zoom chat, share resources and links with each other. I love to see when people get to make friends in the chat as well. There's also three ways you can jump in here and ask questions or provide your own perspective. So you could raise your hand on Zoom, and I will call on you to jump in. You could also put questions in the Zoom chat and just put a little asterisk or star next to it if you want me to read it instead. Maybe you're in a coffee shop or something. And then third, we have a Slido link where you can ask questions anonymously. If Curtis is here now, he'll share that. But if it doesn't get shared, I'll copy it over in a few minutes.

A few notes I like to just share because we have everybody here. I want to make sure everybody knows POSIT Conference is coming up in August in Seattle. We'd love to see you all there in person or virtually. And then this is something new, but I was just curious if anybody joining the Hangouts is in London, because our team will actually be there next week for a Databricks event. And there's some fun events happening. So I'd love to make sure that I send the invite to you if you're in the area. So just comment, and I'll go look at the recording in the chat later and send it over to you.

Or you can always just email me, too. Perfect. Well, I am so excited to be joined by my co-host today, Greg Schick, Director of Analytics at Charles Schwab. And Greg will say he's a long-time listener, first-time caller here. So really excited to have you joining us. And Greg, I'd love to have you introduce yourself and share a little bit about your role and get us started here.

Yeah, absolutely. Thank you, Rachel. Yeah, a long-time listener, first-time caller, been joining the Hangouts for quite a while. And it's kind of surreal now to be the featured speaker. So I appreciate the opportunity. Just before I go into my background, I would love to just highlight a couple of things that I love about the Data Hangout. Number one is the chat. So my whole goal for today is to really get the chat going and riled up. And I'll probably drop some references in there that'll get everybody talking. And two is the walking treadmills that we see all the time, too, the walking desks. So I don't know, Rachel, if you keep stats on how many people use those, but I'd like to make it my goal to set the record for that today. So if anybody's got a walking desk, we'd love to see that.

Greg's background and career path

So a little bit about me. I guess I'll probably start furthest back and work my way forward. So I graduated from the University of Denver. I won't say how long ago, because now it's starting to feel like a long time ago, with a degree in statistics and mathematics. And while I was at the University of Denver, kind of typical college student, I got to my senior year and thought, oh, I've actually got to go get a job. I've got to do something with this. I don't know what I want to do. So talking to my advisor, I kind of narrowed in on actuarial science and got an internship and got a job straight out of college working as an actuary and consulting. My advisor had told me that actuaries were high paid and low stress, and I still always tease her to this day that I think she got those backwards.

So it was a good intro career for me. I learned a lot about programming, learned a lot about presentations, business. We were in the consulting space. We were not in the insurance space. So working with customers and trying to understand their needs and things like that, spent about three years doing that. And after that, I went to HR analytics for just about a year, which was really interesting to look at data that you could actually tie to specific people. Prior to our consulting, it was more broad types of data. Spent a little over a year doing HR analytics, and then I moved to DirecTV, where I did field operations analytics. And that was a really great career move. For me, I was about five years out of school at that point and really hadn't found what I wanted to do yet. I knew I didn't want to keep being an actuary. I didn't want to continue doing HR analytics. And DirecTV, I had a great boss who hired me and really helped me find some of my passion about data and analytics, and worked there for just about six years, was part of the AT&T merger, too. We were acquired by AT&T, and then ended up coming to Schwab back in 2017.

So Charles Schwab, for those who aren't familiar with us, we're a full-service financial firm. So we offer investing, banking, advice for those who are interested in those types of things, all kinds of financial products. And so the last six and a half years or so, I've been at Schwab. And one of the cool things about my role has been, since I've been here for six and a half years, I've probably had five or six different jobs. I actually started in the marketing analytics space and working on new clients that were new to the firm and also referrals, and moved naturally into what we call journey analytics, so how customers go from point A to point B. And that can be everything from a micro-journey of you're trying to move money from one account to another account, to how you get onboarded, how we find you as a prospect and turn you into a client and manage the lifetime of your relationship. So I spent about a year on the data science team doing that. And after that, moved into more of a role focusing on prospect conversion. This was on the call center team specifically, so supporting call centers and trying to use natural language processing and large amounts of text to try to determine why customers were calling us, what were their needs, how we could help them, those types of things. And then most recently, this is probably the longest tenured role I've had at Schwab. I've been in more of the product analytics and risk space, and just recently picked up support for our treasury team that manages cash and margin balance and things like that for us. So it's a large kind of cross-section of different responsibilities, some interaction between them, but it's been a really interesting role.

I have one more thing to add to that too, but I already saw a question about why was that boss a great boss? And so I want to tackle that because he was a great boss. He actually now works for Toyota. He hasn't been my boss in like 10 years, and we still keep in contact, still good friends. And why was the boss who hired me at DirecTV a great boss? Purely because he got me, he had a really great way of getting us excited about the impact that we can make, right? At DirecTV, when we were helping with field, so we would do things where we would launch new products, and then our team would be responsible for not truly A-B testing, but like measuring the effectiveness of the products and trying to compare them to similar products and things like that. And he did such a great job of connecting the work we were doing to the value that we were creating for clients and we were creating for the firm. And so it really made me excited to come to work every day. I think that was probably one of the first times in my career that I was excited to come to work. And after I would go home, I would still be thinking about problems at work. So it was kind of different.

Purely because he got me, he had a really great way of getting us excited about the impact that we can make, right? And he did such a great job of connecting the work we were doing to the value that we were creating for clients and we were creating for the firm.

And then the one last thing I'll mention about Schwab is I also currently, I still work with the University of Denver, with my old department, which is now rebranded as the Business Information and Analytics Departments. I'm hoping I may have some students from DU on with us today, but we do regular capstone projects with them. So that's really cool to be able to connect real world data to problems or projects at the undergrad and graduate level. And I also help run the data science at Schwab user group. We actually have a meeting here in about an hour and a half. I don't know how I scheduled the data science hangout and the data science user group on the same day, but we've got a meeting in about an hour and a half where we're going to be talking about a Python app that some folks have built to do NLP, interactive NLP analysis. So it's really cool, really exciting. So I think I'll pause there. That's probably enough talking about myself. See if there's any questions.

Oh, that's awesome. Big community day for you. One thing about yourself I forgot to ask you, what about something you like to do in your free time too? I always love getting to hear these.

Oh yeah, that's a great question. So I think as Rachel knows, I'm based in Colorado. I should be a skier or a snowboarder. I'm actually a fifth generation native. And my great, great grandfather came to Colorado specifically to compete in ski jump championships. And I have never skied or snowboarded in my life. And so I feel, I can already feel the disappointment from Rachel and probably from others that I've never skied or snowboarded. You're asking about what I like to do though. So that's not, that's kind of the opposite. I'm a big fan of just being outside. Anything, anything hiking, paddle boarding was been something I picked up recently. And then building too. I've been really into woodworking and 3D printing probably most recently during the pandemic. My two sons and I, we got a 3D printer and built it ourselves and then are now printing things from the printer we built. So that's pretty awesome.

Mentors and career trajectory

I know we kind of covered a little bit of this in Russell's question about why that boss was a great boss. But I see Jared had a question as well. I know you said you, I'll read it, Jared. Can you discuss any mentors or people that you have come across that helped shape your career journey? Maybe it was this boss, but also any specific stories of events that might have changed your trajectory?

Yeah, absolutely. I've been really fortunate that I've had a mentor at probably each role that I've been at. And even early on, this is again, is going to date myself, but one of the mentors I had when I worked for Mercer in actual consulting, this was fresh out of school. I was just kind of bright eyed and knew a lot of the, maybe the technical things I needed to know, but not of the business side, the relationship building and networking and offering to help people when they didn't necessarily, I didn't necessarily get anything back from that side, just volunteering and things like that. And so I had a great mentor there, who's also still a friend 15 years later. And he suggested, he gave me really tangible suggestions of things to build my career. At that time, one of the basic things was like learning, what is it called? Mail merge in Word, between Word and Excel. That sounds like so outdated now, but the idea that you could like, not really program something, but you could create something that you can scale up to one iteration or 10 or a thousand or a hundred thousand, right? It was really interesting. Also building some of my technical coding skills. We had a platform, like a proprietary platform that was SQL based. So I learned a lot of SQL-like terms there. And then even more recently at Schwab, we've had, I've had some great mentors who've talked to me about like thinking about personal brand and think about, when someone, when I'm in a room and I'm, or when people are in a room and I'm not there and they say things about me, like, what are they talking? Is it positive? Is it negative? Do they even know who I am? That's probably the first battle. So trying to get that personal brand out there and being kind of aware of what your reputation is at a firm.

That's great. I mean, it's very clear you have a focus on relationships as well, with both of these examples, saying you're still in touch with people 10 or 15 years later too.

Veronica, I see you had a question a little bit earlier. Do you want to jump in here and ask that?

I'm a walking desk. I wasn't walking. I was just curious how some of the approach or methods you, especially from the boss you learned, that you took to major success with your customers?

Yeah, absolutely. I can probably talk about that most recently from Schwab because we've taken

quite a few, in the past two or three years, we've taken a lot of learnings from Gartner and from other folks in industry who do research about these things, because measuring data science projects or BI or analytics projects, anything really is tough, right? A lot of them, we have this idea maybe after the fact of what it might be worth, but it's hard to get that up front.

about two or three years ago, we went through a pretty robust process at Schwab to try to understand what the expected value of a project could be. And so we got really down into the details and the weeds from this and thought about, is this satisfying a regulatory need? Is it satisfying a business need? Is it an efficiency play? Are we just making life easier for people? And how can we quantify those things, right? Efficiency, generally, I think was, from my experience, was one of the more easier ones, right? If I can save somebody 15 minutes a day by automating this process or creating this dashboard that they can go to, we can turn that into dollars pretty easily. Where it gets tough is the research and development side of like, what would be the value of this model if we built it and it was accurate and could we use it to leverage that? So that's the side that's a little bit more, at least up front, it's tough to assess.

One thing that we've had a lot of success with was providing kind of ranges and estimates too. Like if this is fully successful, it might be X. And if it's partially successful, it might be Y. So it gives you kind of a threshold. And really, at the end of the day, I don't think the dollars or the values matter as much as they do for helping us prioritize, right? So if we have this set of expectations, we think about which of these projects is our actual top priority, and then we can stack rank them a little bit easier. So I don't know if that answers your question.

No, this is helpful. Thank you so much. Absolutely.

I see Arsena's had a question for me to read. But the question was, what methods did you use to learn what you needed to in order to perform analytical work in such different domains as people analytics, then customer analytics, and now product?

Yeah, absolutely. I think, you know, my move to Schwab was probably the only time in my career that I made a purposeful move for a domain, right? So up to that point in my career, six years ago, I had seen lots of different areas and ways to use analytics, to use data science. And the two that stuck out to me the most were definitely marketing analytics and biostats. That seemed like both of them, you needed to have, you know, you needed to be able to have like a PhD in biology, plus like a PhD in data science, if you wanted to go into biostats, or in marketing, you needed 20 years of marketing experience, plus 20 years of data science, you know, it was really tough to get into. And so that's, you know, one of the things I targeted.

Prior to that, I don't know that I was as purposeful. So when I was in Actuarial Sciences, I was learning the coding that I needed to know to support my clients. So, you know, for that part, it was mostly SQL, some BBA as well. In HR analytics, you know, again, SQL and some other tools at DirecTV, it was Oracle Business Intelligence, and it was, you know, SQL, that's when I started using R and Python as well to, you know, kind of expand those skill sets, SAS. So the approach that I started taking at Schwab is that I really want to get the widest breadth of experience that I can, right? I think as data practitioners, if we have the skill sets, we can apply them to any data problem. And I kind of use the analogy of like building a house, right? If you're hiring somebody to build your house, you don't care which tools they're going to use. You just care if they're going to do a good job, are they going to meet your expectations, those types of things. And so the more experience they get building different types of things and approaching unusual situations, especially the ones where when you start out, you have no idea what you're doing. You have no idea how to solve it. Those are probably the most fun ones. So, you know, just continuing to tackle new types of data and not necessarily coming in and being the expert. And that's kind of how I feel today with my new role supporting a new group within Schwab. I'm coming in and I know very little about the data behind it, but I'm learning every day, which is the best part of that role.

just continuing to tackle new types of data and not necessarily coming in and being the expert. And that's kind of how I feel today with my new role supporting a new group within Schwab. I'm coming in and I know very little about the data behind it, but I'm learning every day, which is the best part of that role.

I see everybody wants me to read the questions today. I don't know if everybody's coffee shops or what. I feel like that. Did I? I think I scared everybody off.

Explaining ML and AI vs. getting stakeholder buy-in

But Jared asked the question, what is harder explaining ML, AI to decision holders or getting buy in from stakeholders?

That's a great question. If I can caveat the question to explaining ML and AI, maybe realistically, outside of the hype, I might say that. So we've seen a lot of those terms. And, you know, frequently when I hear some of the projects, the way that I would think about it is not, you know, ML or AI, I would think of it as automation or maybe regex, right? Like this can be solved through that. This can be, this is a report, you know, things like that. You know, we recently, you know, had a conversation with someone who was looking to like automate forms and, and like have a template that everyone could fill out. And like, they were talking about, you know, and, and like doing it all in like Microsoft products. And, and maybe I'm, you know, just, just have this history, but I was sitting there thinking like, it sounds like you want Microsoft clippy, like somebody that would like just recommend things to you. I see somebody has that in their background too, but, you know, things like that. So, so kind of trying to separate that.

And then your, your question about like buy-in, you know, is a really good one too, because we, we try to have a balance of like a reactive and a proactive approach, right? Where we're listening and hearing from each business and we're reacting to what they're telling us and to their, what they're saying are their top priorities and their needs, but also having a proactive approach where we're, we're closely partnered enough with each organization at Schwab that we can go and say, you know, Hey, you haven't asked us for this, but we were thinking about your business and we think this could be important. And so those are the ones where we're buying is super important because they may have like operational or very tactical things they need to get done. And we're pitching kind of a, you know, art of the possible or, or kind of a a esoteric type project where we want to go above and beyond. With those, I think demos can always really help, like having something tangible, talking about some of these terms before you've built anything or before you've, you know, can have anything to show concretely is really difficult for folks who aren't practitioners and who aren't in data, you know, every day.

Hearing that example or like where you're pitching this idea to somebody, I was, I was curious, how does that normal like interaction go with the business teams? Like, do they come to you with projects as well? Or how are you working together? Yeah. So for each team that we support and, you know, right now I'm supporting three different, you know, three separate organizations within Schwab. And each one, we have a data and analytics roadmap that we, we map out. We try to keep it somewhere between six months and 18 months out where we're saying, these are the things that are most important to you. Some of those projects are, you know, the very logistical, tactical, like we don't have this data and we need this data or I'm sure, I'm sure I'm the only one that's to deal with bad data, but, you know, we have this data, but it's wrong, or we need an improvement to it or enhancement, those types of things. So it can be kind of that foundational layer of, we just need data quality fixes or data enhancements. Then we also talk about what is in their business plan, right? So what are their objectives for the year and how does data impact that? You know, if we're trying to launch a new product or a new program, you know, page on the website, it could be anything. What, what, how does data play a role in that and how can we support that? And then finally, is that kind of that last layer that I talked about of like, what are we not doing that we should be doing? What are we not really talked about? And how can we add value from trying something we've never really tried before?

Are you able to give us an example of one of those use cases or, you know, it doesn't have to be like super specific, but just so we can kind of understand the problems. Yeah. I'd say generically, and I can, I can talk maybe a little bit of specifics, but generically, a lot of it is, is automation type things. Like how can we reduce the manual creation of data, the manual input of data? Can we use any, any types of models that may help us, you know, automate that, that portion of it. Right. And we're in a, you know, very highly regulated industry within finance. I really appreciated, I think it was Michael's talk a few weeks ago from Wells Fargo, because I, you know, had a lot of synergy or connectedness to that, you know, approach that we can't do anything where we just let, you know, let models make all of the determinations. So a lot of it is, you know, assisted, you know, to make life easier for, for the humans who are doing this. But big picture, I'd say a lot of automation type work like that. Can we move data from one system to another? Can we, you know, populate these fields? Can we, can we create a report that satisfies all of these, you know, requirements?

Hiring and human qualities

Yeah. Sean, I see you had a question in the chat a little bit earlier on hiring. Do you want to jump in?

Hey, Greg. Just a quick question. When you're, when you're looking to collaborate or hire folks in the data science space, what type of human qualities, so not like practical skills, right, or technical skills, but what type of human qualities do you look for and has those qualities evolved over time? Yeah, that's a great question. Sean and I get the opportunity to work together. Sean's been one of my, you know, most recent or recent favorite ads to the firm, even though I think you've been here about three years now or four years, you know, fantastic human being.

I can actually even use an interaction that Sean and I had as a great opportunity when Sean joined the firm and helped, we started having conversations about combining some of our user groups. You know, he made sure to make a comment about ensuring that we had adequate representation and diversity and, and that we were, we were representing all of Schwab, not just like a subset. And I'm butchering the way he said it. He said it much more eloquently. But it was such a great first interaction to see somebody who was, you know, very comfortable saying that. And, and, and, you know, it's a very important topic, right, that we always try to stress with our data science user group. So in addition to your question, Sean, I'll tack on, you know, the probably the biggest thing is just kindness is just somebody I'd want to work with.

Right. I think that's a very underrated skill set. And actually, when I think about my first mentor, I mentioned at Mercer, you know, that's something coming out of school, I had zero focus on being a good person to work with. And, and, you know, I really had this like statistician type mindset of like, I'm just gonna go and crunch numbers. And, you know, I'll provide the results. And I'll just sit in this, you know, this far off corner and do those types of things. So now I really try to, you know, probe and ask questions about like Rachel asked, you know, what do you do in your free time? And, and what types of projects do you like working on? What is what are the worst things that you you know, you hate doing? Right? Trying to understand more about, you know, about that person. And would they be somebody that I would, you know, enjoy coming to work with every day?

the probably the biggest thing is just kindness is just somebody I'd want to work with. Right. I think that's a very underrated skill set.

Jobs and programs at Charles Schwab

Well, while we're talking about hiring, I feel like I should mention, if anybody is hiring, feel free to share roles in the chat here. And Charles Schwab, especially.

Yeah, and I'll give a I'll give a usually post a link in the chat. But I'll just give a quick plug for you know, I'm not hiring directly as of today. But we do have, we still do have roles posted, and you can find them on about Schwab.com backslash jobs. So we pretty much constantly have jobs, you know, data analytics ones, you can, there's actually a category for those. So if you're ever interested in those. Plus, I would plug a couple of programs that we have that are really great, such as our Intern Academy. Each year, we hire probably around 300 to 400 interns nationally. And we, we do that pretty early. So for this summer, I think we're already we've already got all of our interns placed. But we probably start that that that hiring process in September, October, and have a lot of our interns placed by the end of the calendar year for the following summer. So I always when I when I work with students from the University of Denver, you know, always telling them, you know, apply early. And then we have a couple of great, if you're looking to break into finance, or tech in finance, there's a couple of great programs. One is called our NERD program, which is actually actually an acronym, we're not calling the NERDs, New Employee Resource Development, I believe, that helps place folks in our technology services organization. And then our Financial Academy as well, too, if you want to go down the road of of an FC, financial consultant.

Thinking about internships and applying so early just brought back like stressful feelings from college. Mark, I see your hands raised, want to jump in?

We can't hear you yet, but you might be muted still.

I've tried to unmute. Yeah, I was curious, how do you, how does your team and how do you think about the distinction between the data scientist versus the product manager versus the, you know, maybe consultant, or, you know, sometimes it's referred to as like the storyteller, the explainer, like, do you take on all those roles within your team? Is there any kind of formal separation between those functions?

Yeah, that's a great question. And frequently we do ask, I hate this cliche term, but we ask folks to wear lots of hats. So we do have, we do have a data scientist title. My title would be like, it's analytics consultants. And then we have analytics developers. All three of those, I would say, have some kind of role in that, though, right? So we might see analytics developers or consultants doing data science type work. We see data scientists doing, you know, data work to get the data they need for a model or working directly with business partners.

I think for the, for the data storytelling portion of it, we always try to get whoever is most involved in the work in front of the business partner, right? So if you're the model developer, you know, we like to give you the opportunity to, you know, to have those connections. Some folks, you know, we need a wide variety of roles here at Schwab. So some folks maybe don't like, don't enjoy that as much. They'd rather focus on the model development, which is totally fine. But I think the role of, you know, my role analytics consultant is really supposed to be tying with the business and understanding what they need and how that could connect to data. So frequently in meetings, I feel like somebody says something and I just start thinking about tables and joins and, you know, writing SQL or something like that, that could solve their problem. I probably look like I'm staring off into space, but I'm really thinking about how to solve their problem with data. So probably every role I'd say, you know, has some degree of that where they're thinking about that.

Helping stakeholders understand data projects

I see there was an anonymous question that came in on Slido and somebody said, not being a people person myself, how can I help stakeholders understand and support data analysis projects? Sorry, can you say that one more time, Rachel?

So it's not being a people person myself, how can I help stakeholders understand and support data analysis projects?

Yeah, no, that's a great question. Actually, a long time ago, when I started at Schwab, somebody, when I was on the data science team, somebody said, somebody told me you're very outgoing for a data scientist. And I didn't know whether to interpret that as like an insult, like a backhanded insult or a compliment. I think it was meant as a compliment. And so I kind of took it in the way it was intended. But, you know, that's a real thing. I think everybody, we have a large centralized organization. We probably have around 225, 230 people in the centralized data analytics organization here at Schwab doing every flavor of anything data you can think of, data governance, BI, analytics, data science, model development, you know, all kinds of different things. There's varying degrees of comfort with that, right, with being the presenter, with being the people person, so to speak, right, the soft skills side.

I think, you know, convincing, or maybe I should say influencing folks to sponsor data analytics projects is always a challenge. Schwab, and I think a lot of financial firms are very PowerPoint or slide deck based, where we do a lot of our persuasion through designing slide decks that can explain the problem statements, the potential solution and the value. But I say that's really the key, regardless of how you do it, if you do it in PowerPoint, if you do it in a verbal presentation, if you do it in, you know, a model demo, connecting your work to business value is probably the single most important way you can, you know, get buy-in. If you can say, you know, that this is going to save us money, or that this is going to help us avoid risk, or this is going to automate things so that people can, you know, be freed up to do other things, you know, I think that's one of the lowest hanging fruit ways to go, you know, approach that.

connecting your work to business value is probably the single most important way you can, you know, get buy-in.

So we, I think this is something that comes up a lot, like trying to connect what we're doing to business value. So we could we maybe double down on that topic? And just like, do you have an example of? Yeah. Yeah, absolutely. And actually, one of the one of my former managers, you know, just posted this on LinkedIn a few weeks back, and it really connected with me that, you know, the single, you know, the single most important thing a data scientist can do is understand how their business makes money, or how their company makes money. And so start with that, right? So Schwab is a financial services firm. Big picture, we make money a few different ways, right? We can, we can earn money on interest, on interest revenue from assets that we hold, we can earn money when you make certain types of trades, most of our trades are now $0, they're free, you know, we were the one of the we were the first firm to go to $0 trades, we can make money when you pay us for advice, right? So if you're paying us for financial advice, that's another way to, I'm probably oversimplifying for this call, but they're, you know, understanding those types of things, and then connecting that to how, you know, how we could approach the data problem.

So if we just, you know, focus on the advice side, how many clients are we giving advice to? Like, how does that? What does that advice turn into? Right? You know, what does it look like? How can we quantify that? How can we measure that? A lot of those are in person conversations between our clients and between their financial consultants. So it's hard to kind of quantify the value of a conversation or the value of having a physical presence or those types of things. But those are the types of problems that we try, we try to tackle of understanding how our advice turns into value for an individual client, right? Are you better off long term with with the advice that we give you, you know, versus otherwise?

Call center NLP and topic discovery

Hey, Rick, I see your hands raised want to jump in next.

Well, thank you, Rachel. And good morning, colleagues. Greetings from Southern California. Liz, I'm especially amazed that you have Clippy in your background. I wonder how you did that. Hey, Greg, thanks so much for your great comments. I do have one about a very specific technical approach that I personally have struggled with and hope to learn from your wonderful experiences. You mentioned you did a project in the call center, where you were trying to gain and help your client or your customer gain an understanding of why folks are calling in. Did you use an approach like LDA, latent, gosh, I cannot pronounce it, LDA, Latency Allocation for Topic Discovery? Or did you use some other topic discovery approach? How well did it work out? Were you able to 100% automate topic discovery? Or do you still have to have human editors? And not including my question that I posted was something else I thought about. Did you also do sentiment analysis? And how did that work out?

Yeah, absolutely. So this, so that, you know, that question, or that project was, was now a couple years ago. So I'm trying to remember the exact methods, but it was definitely an NLP project. And actually, it's probably another good highlight of the reason that any anybody in data should be, have an understanding of the full pipeline of your data, right? How it gets created, how it gets manipulated before it's stored, how it's stored, and the output that it turns into. So we did use LDA, and a couple of other, other methods, I think one of the biggest ones for that one that we ended up using was, was actually TF IDF, and trying to compare subsets of calls to broad topics of calls.

That one was a really interesting one, because it opened my eyes to a lot about how often people call us and some of the, you know, the topics of conversation that I, you know, I would have been surprised to learn that I, you know, didn't know beforehand. So the reason I mentioned bringing up the pipeline was because one thing we found out after we had already started the analysis of this specific team, or, you know, subset of calls at Schwab, we found out that not all calls were being transcribed by our platform. So only a subset of agents were getting this transcription. And at the time, you know, that was because, yeah, so this is, you know, this, if you're only getting a subset of your data, you can ask some really interesting questions, like, is it representative? Is it, you know, why is that happening? It turned into be like a licensing thing, right? It wasn't any, like, you know, anything nefarious. It was just, you know, there was a licensing, you know, challenge behind it. So now, as we've moved to, you know, be able to transcribe, you know, more calls than we were, you know, say, four years ago, when I was doing this, I think that's still one of the biggest things that we try to understand is the sentiment.

There was actually a really interesting presentation in our data science at Schwab that I know Amazon's done something similar where they look at, like, the volume of the voice, not just the topic of the call. You know, is it like a, if you're talking louder, you know, you might be more upset if you're talking quieter. So there's all kinds of interesting things you can do with those types of things. Yeah, I read a similar article, Greg, about stress analysis. And vocal intonations, which I thought was fascinating, and how they were able to merge that with the topic analysis to say, oh, you know, in real time, they were advising the call center people or whomever, you know, have a lighter touch or give them a special offer or something like that. Really brilliant. Yeah, absolutely. Yeah, and I think as George just said, some people just talk loud, though. Yeah, I totally agree. That's me. So it's not a perfect one.

Probably one of my favorite. As we were building some of those models, we were obviously very in the weeds with the data. And I kept seeing this term in the text, the transcribed text that said, this is a man from Dallas. And I was like, that's really weird. Like, I don't understand why this keeps coming up. And then we went and listened to the actual recording of the call, and it was saying, this is Amanda in Dallas. So you're talking to him. So even, you know, things like that, like, obviously, that wasn't going to fit, you know, that individual record wasn't going to affect our sentiment analysis very much. But, you know, getting into the weeds and really doing descriptive analysis on your data before you jump into any modeling is super important.

Greg, if I add one thing to this, could you elaborate a little bit on the way you were surprised in topic discovery, the automated topics. You mentioned that some things popped out. I've had a similar experience. I just wondered what yours were like, where you're going, wow, I did not realize that's an important issue. How did that affect your business clients, your customer?

So that's probably crossing into the realm of the, I can't talk to you specifically about some of the findings. But there was definitely, you know, we have call categorization too, where we just, without text analysis, we say what types of calls we're having. And we had, you know, some pretty decent findings of things that fell into buckets that didn't seem to match, you know, challenges that clients were having. You know, probably, I mean, this is what I could probably talk about, because it's probably consistent across any industry, right, is clients calling in because they were having trouble doing something on a website, right, or trying to find something on a website. And those calls getting miscategorized. So that was a big, you know, big finding that we had for some of our partners. It's really interesting to think about all these examples. Like when I was just calling to like switch a credit card, like I probably could have said something to get extra points, like thrown in there or something. Exactly. Exactly.

Measuring customer journey touch points

Mark, I see you had a question a little bit earlier. Want to jump in?

Sure. This is like, I guess, like a little bit of a tangent in the sense that we talked about it way earlier. But you mentioned like doing work along like the customer journey and whatnot. I guess what sort of like measurement methods are you using in the case of like a journey having multiple touch points and then like attributing, I guess, if you're looking at like one outcome at the end of the journey, like attributing, like how do you determine which touch point along the journey is kind of like the one to focus on or where you should spend resources?

Yeah, absolutely. And this is the work that I started three years ago when I was on the data science team. And that team has since taken it to like entirely new frontiers and just expanded it past my wildest imagination. I'd say one of the hardest things to start with is defining what is a journey, right? Like what is a client journey? Differentiating when somebody goes on the website and it's just kind of maybe poking around or maybe they logged in because they were bored and they're just reading articles. Are they really trying to do something purposeful and definitive or are they just logging in? Once we decide you are on a purposeful journey, say again, I use that example at the beginning of moving money from one place to another, right?

What is the start of that journey? Is it when you initiated that transfer? Is it when you logged in? Is it when, how is it completed? Those types of things. So we actually, we try to measure every client interaction we can, whether that be on the web, that be on a call, on in-person branch visits, can even be one. Those are a little bit different to measure, I think, than call, a little less automated, I should say, than calls and those types of things.

But trying to programmatically or systematically assign a journey to a category, like this is what the client was trying to do and this is what they did while along that journey. Because sometimes we've also seen that they can change their mind. They log in and they were going to make a transfer and then they saw an interesting article. So then they go read an article and then they're like, oh, maybe I should open an IRA. So they start exploring that page and then they go back to, oh yeah, I was here to make a journey. I know I do this not on Schwab.com, but probably on Amazon all the time, right? I just kind of fumble around looking at different things.

To your question about measurement, a lot of it can be related to the things that we have tangible data on. So like how long maybe you spent on a specific page or were you calling at the same time you were on the websites, those types of things. Where we can measure that, we'd like to gain understanding of that simply for the purpose of making that journey easier, right? We want to make it easy to do business with. This is probably getting into UX and UI design, right? But if you're in an e-commerce company and you want people to buy stuff from you, you should have that button very visible all the time, right? You shouldn't make it hard for people to figure out how to buy your products.

Tech stack and infrastructure

Yeah, I see Bill had a question a little bit earlier and it was, can you describe your infrastructure and how people work together on their PCs, which I'm not sure if that what the acronym is for, if that's actually a computer, but also role of cloud and prevalence of Microsoft products. Are you seeing a bias towards R, Python, or other languages? And Bill, I appreciate you adding, he said it's the chance to plug any Posit products you may be using.

Yeah, so that's a couple different questions, right? So I'll probably tackle those one at a time. We all try not to work together ever. We try not to see each other. No, I'm just, I'm totally joking. It's a very collaborative space. Actually, one of my favorite places, you're probably not talking about where we physically work together, but my favorite places on the Schwab campus here in Denver is the cafe. I love to go down there in the morning and get a coffee and sit at one of the tables, you know, watch, you know, Bloomberg feeds or MSNBC or something like that. And, you know, get some of my catch up on emails or coding or whatever I need to do that morning.

In terms of tech stack and capabilities, I'd say for better or for worse, we have a little bit of everything, right? So we have, when I started at Schwab six years ago, I was pretty much primarily a SAS programmer and heard immediate, you know, pretty, pretty quickly after I joined Schwab that SAS was going away. And so I better learn R or Python. So I took that very seriously and really focused my personal development on that. Six and a half years later, we still have SAS and we still use it for a lot of different things. R and Python, I would say Python might have a little bit of an edge on the data science team and, you know, purely for the model production standpoints, but we still do have quite a few R developers.

We even, one of the new teams that I support, we, you know, we have a lot of like R-Python conversion back and forth. So that's been interesting using things like the reticulate package has been super helpful to, you know, hopefully avoid a full conversion and just, you know, code in both languages. Let's see, other than that, you know, Tableau, Alteryx, like pretty much every, you know, everything that you can think of. And so a lot of the challenge comes from trying to determine what our kind of platform strategy is. Like where is the right use case? I would think we have a consensus that if you're building a dashboard, it should probably be Tableau. But for other things, like if you're building a model, if you're doing an analysis, I don't think there is that consensus. It's really whatever gets the right answer.

And then I think the final question was about Posit products. Okay. Yeah. So another great question. And to be quite honest, I've been trying to figure that out myself. I know that one of our teams has Workbench for model developments, and they're actually, it's a Schwab organization, but it's actually a wholly owned subsidiary. So it's kind of a different, different for legalese for contract negotiation, right? It's a different contract. And then another one of our teams that does model development has both Workbench and Connect. And it's pretty limited to that team, from my understanding today. So I'm trying to bribe the leader of that team with lots of coffee and chocolate to let us expand that.

The Schwab data science community

Yeah, absolutely. It's been really interesting, the growth of our data science at Schwab user group. It started out originally as a R user group, because we wanted, a few of us were new to Schwab at the same time, and we all wanted to learn R and we're just doing it on our own. And so it started out with a group of about probably four or five of us, literally every month, just presenting back to each other. Like it was the same four people over and over and over. And now our group has probably about 600 members across Schwab with about, our monthly attendance is somewhere around a third of that, about 200. So we'll see how many we get today. And we've expanded to all types of programming languages, all types of skillsets. We'd love to have people who come in and have just written their very first, hello world scripts. And we also have the people doing very advanced neural networks and topic modeling and all kinds of different things.

And now our group has probably about 600 members across Schwab with about, our monthly attendance is somewhere around a third of that, about 200. So we'll see how many we get today. And we've expanded to all types of programming languages, all types of skillsets.

How do you share the data science community across the company or how do people find out about it? Yeah, we've moved over the years through a couple of different iterations. Back when we started, it was all on WebEx and it was just like an Outlook invite. Now we have better things like Teams, Microsoft Teams and Confluence from Atlassian. So we can actually post invites, we do trivia each month, those kinds of things. Since I'm leading the user group meeting today, I get to pick the trivia questions. So I picked an R-based question. So I think probably everyone on here would know it, but a lot of them go into like data science or AI or machine learning type trivia questions. So the Teams space is our main channel and it's also kind of de facto been our main kind of support system for Python R installation. There's just even basic things like downloading packages because we're highly regulated and because we have some specific, you know, situations that we have to work around. Getting help from the user group has been really beneficial. And for me personally, moving away from that small group of the four or five of us answering all the questions to the community answering their own questions has been probably the most rewarding part of it, right? There's people that I don't even know that are answering questions and it's awesome.

It looks like people want to hear the trivia question and maybe your answer to it too. Oh, I think it's too easy for this group, but I'll throw it out there, see if anyone can, I won't even give you the multiple choice that I'm going to give the user group, but all R versions package updates are named after a famous comic strip. What is the comic strip? Anybody know that? We'd be giving an answer for later. Oh no. I see a lot of right answers. And somebody said the answer to every R trivia question is Hadley Wickham, which is awesome.

Audience Q&A: feature transformations and customer journeys

Well, I know we have a few minutes left here, so I want to make sure if there's questions we didn't answer, feel free to just raise your hand here so that I can see. But I know there was a question a little bit earlier that was going back to the customer journey and it was J U. Do you want to jump in? Hey, this is James. I had a couple of questions kind of new to the data science world, but I understand that there's a lot of feature transformations that occur. And when you're dealing with a data set that is not as robust, I mean, typically a person that is modeling will try to transform it to a level where it's usable. How do you do over there? I mean, there's so many power transformations that are available and so many iterations that you can do. How do you come to a level of understanding that this is going to be fitting for a business use case? The touch points are very, very convoluted as you can see, right? There's so many different permutations and ways of segmenting the data that it makes it almost unusual at one point. So the question then becomes, what do you do? And then at that point, how do you even make practicality of the data that you're consuming?

Yeah, no, that's a fantastic question. I'm a big, probably my favorite type of chart is if I had to pick a single one as a Sankey chart. And when we first started building user journeys, we had a bunch that looked like an infinite loop, like a Sankey chart that was just an infinite loop because the customer, the data would show us that they were going, you know, A to B to C back to A and then, you know, just keep doing that. And I'm like, well, that's not very useful. That's not really a journey. So in terms of the feature transformations, one thing that we did hone in on is trying to create this new kind of identifier in a way where we could, we could have an identifier for the journey, right? Because we started from like the customer perspective, right? So this is what customer A did, and this is the order that they did it in. And instead, you know, transformed our thinking into thinking about journeys and how many customers did X, Y, Z, right? And even if it's not in that order.

So trying to transform the data that way, and then also just trying to not limit our thinking in terms of what touch points we should be measuring, right? It may not all be calls or web data that we can use. There might be other data, you know, systems outside of that, that would help us enhance that feature set. So a lot of it was almost kind of detective work of finding what data was available and how we could use that to enhance our understanding of what a client was trying to do.

Gotcha. Yeah. I mean, for me, it's, there's no, there's no quite one fit all solution. So I understand that there's, there's just so many different ways of interpreting. Yeah. If it's in consequence of time, it's just, it's very difficult. I mean, we're looking at different distributions and, you know, there's, there's so many complexities that are involved. And then the sequencing of time of, like I said, you know, certain events that are conditional on the basis of a user's criteria. And sometimes you don't get that data. So yeah, that's, that's where are these truly signals, or is this something that is just happening randomly? So those are things that are hard to be subjective for, in my opinion.

Yeah, totally agree. And the comment about signal is really key there, right? There's a lot of, a lot of journeys that don't have the necessary signal to really make any kind of meaningful determination. And so then we have to decide, you know, how do we handle that case, right? Is it worth trying to measure? Can we, is it physically not possible to do, or are we just thinking about it? Like, do we need to change our mindset a little bit? Yeah. I mean, going from deterministic and possibly, I mean, this, this sounds terrible, but I mean, using probabilistic modeling to then even make action off of, is that a consequence of using things that are, I mean, that's something that a business user has to make an action off of. Is that something that is recommended or perhaps something that needs to be evaluated further? Is the question then becoming, do we need more time to have more data to reach significance? What do you do in those circumstances?

So for the time question, I'll say too, we also have an approach where we really try to time box things. And so sometimes like a common one would might, might be 90 days or, you know, giving it a full quarter. It is to your point, you know, dependent on the project. So sometimes, you know, sometimes 90 days is not enough. Sometimes that's way too much, right? But I think I don't have a specific answer other than I think time boxing is really important because you can, you can waste a lot of time spinning your wheels, trying to find an answer that is not possible to get to, or, or not possible to derive value from. Maybe you can get the answer, but you can't actually get value from that answer. Thank you so much.

I think time boxing is really important because you can, you can waste a lot of time spinning your wheels, trying to find an answer that is not possible to get to, or, or not possible to derive value from. Maybe you can get the answer, but you can't actually get value from that answer.

Time here, but thank you all so much for the great questions and huge. Thank you to you, Greg, for joining us today. I, I did just want to let people know we've been experimenting with these kind of like spinoff hangout topics for things that come up quite a few times and data stewardship was something that we talked about with Jamie Warner and Dan at Biogen. And so we have an event coming up on May 15th, and I just put a link to that in the chat. If you want to add it to your calendar, but they'll be sharing a little bit about their experience and at data stewardship at the individual level, and then opening it up to a group discussion to just talk about like, is managing data part of your job as a data scientist? And what are the challenges that you're facing? So I just wanted to put a call out to that. It's not until next month, but thank you all so much for taking the time to spend the day with us. Hope you all have a great rest of the day.