image: thumbnail.jpg
Oscar Baruffa @ IDH | Hiring for skills you don’t have | Data Science Hangout
We discussed the "soft skills" side of data science and how we manage ourselves, our teams and our projects - along with hiring.
Specifically at 34:14, Oscar talked about the difference in hiring to replace you versus hiring for skills you don't have.
"When you're getting into a management position, at some point you need to start hiring people and a team around you. Hiring is actually very different depending on different points in your career.
I thought that it had been the same, because I had done hiring before, even when I wasn't a manager.
There comes this split that I think takes some people by surprise the first time. So before you're, let's say, a “manager manager”, if you are growing in your career and you do happen to be hiring someone, you are hiring someone most likely to replace you - like a more junior version of yourself.
You're going to train them and you are an expert of some level, and you're going to bring them up to your level or something like that.
You feel, I suppose, in a way, quite secure in what it is you're doing because you've already got much more knowledge than them and you are bringing them up to your level. You feel very confident in answering the questions and mentoring them and all of that.
But it becomes very different if you're starting to hire for skills you don't have.
For example, I don't have a stats background. I don't have a data engineering background. I don't have even that much, actually, of an analyst background. I was a market analyst, not a data analyst, per se.
When you start hiring these team members into your team, these are for skills I don't have. It becomes much more difficult for me to also assess - what are they good at or not?
I first started - like I think most people start - trying to learn as much data engineering and data science as I could to try and just stay at least up with them. But there's no way that you can do that.
It clicks and you're like, OK, wait, my job is not to be the expert of all of the people in my team. I am the facilitator of their work. If I know more than them, that's probably actually a problem.
We can't have me spend 10% of time on all of these different vast fields while still being the expert in everything. That's not going to be great and that's quite a different switch.
I think it takes a bit of getting used to - to start not understanding bits of what it is. Then the tables turn a little bit. I'm in some ways the non-technical manager of technical people, although I'm kind of technical myself.
They'll talk about things, and I also have to be like, whoa, whoa. Then I start also clicking a lot of what my directors and execs feel like sometimes. I'm like, yeah, you're getting into way too much detail. I can't do anything with what you're telling me.
My role mostly is to try and not just prioritize work but try and help them uncover where there's
work that needs to be done. Or if there's a dilemma, try and break down, well, what if we go this direction? How bad will it be if this happens? Or how good will it be if this happens? Can we weigh risk like this?
So I can convey to them as well where things just aren't a priority. Because no matter what they do or stress about here, it isn’t going to matter later.
That's quite a different ballpark. I think it probably happens the first time you hire someone that's not a replacement for you, it's an addition to your team. It can catch you by surprise."
► Subscribe to Our Channel Here: https://bit.ly/2TzgcOu
Follow Us Here:
Website: https://www.posit.co
LinkedIn: https://www.linkedin.com/company/posit-software
Twitter: https://twitter.com/posit_pbc
To join future data science hangouts, add to your calendar here: pos.it/dsh (All are welcome! We'd love to see you!)
Transcript#
This transcript was generated automatically and may contain errors.
Welcome to the Data Science Hangout, hope everybody is having a great week. I'm having an extra great week because I'm calling in from Puerto Rico today. I'm Rachel Dempsey and I lead our pro community at Posit. I always want to put this plug out there that we have the Posit conference coming up in September. And we just extended the call for talks to April 7. So I just want to make sure everybody knew that that call for talks got extended.
But back to regular scheduled programming. The Data Science Hangout is our open space to chat about data science leadership, questions you're facing and getting to hear about what's going on in the world of data across different industries. And it happens every Thursday at the same time, same place. So if you're watching this recording on YouTube later in the future, the link to add these sessions to your calendar will be in the details below.
But every week we get to feature a different data science leader as my co host to help lead our discussion and answer questions from you all. So together we're all dedicated to making this a welcoming environment for everybody that joins. So we love to hear from everyone no matter your level of experience or industry or area of work. It is totally okay to just listen in as well. But there's also three ways you can jump in to either ask questions or provide your own perspective too. So you can raise your hand on Zoom and I'll be on the lookout here. You can put questions into the Zoom chat and feel free to put a little star next to it if it's maybe loud where you are and you want me to read it. And then third we also have a Slido link where you can ask questions anonymously. But thank you so much Oscar for joining us today as our co host. Oscar Barufa is joining us as our featured leader. And Oscar, I would love to have you maybe jump in and introduce yourself and share a little bit about your role. And I guess also something you like to do outside of work too.
Oh, yeah, thanks. Yeah, so it's really nice to meet everyone. I'm a senior analytics manager at a company called IDH. IDH is a international development organization that works on improving the sustainability of global supply chains, mostly in the agricultural and manufacturing sectors. And the team, now these, you know, development organizations are usually organized into sort of specific programs that you work on. And the program that I'm working on specifically works with businesses that work with smallholder farmers across the world.
So you can think, you know, like maybe there's a company that's, you know, maybe a local company in Africa that's sourcing potatoes for the local market. Maybe there's international coffee companies sourcing coffee from all over the world, basically. And we work with those companies to improve the business model that's interaction, let's say, between these companies that source produce and the farmers that produce it, right? Because often many of these farmers are, you know, living quite far below the poverty line, they need a lot of services delivered to them and various things like that. So we work a lot with that.
And my team very specifically analyzes data from lots of different business models. And we're trying to understand which characteristics of these models improve outcomes for the profitability and sustainability, like economic sustainability of the company working with them, and the livelihoods of the farmers that are part of that interaction as well. That's a lot what we do. It's quite a small team as well. And so we're basically, there's myself as an analytics manager, then there's two data analysts and a data engineer. And, you know, I mean, a bunch of other people in the team, let's say data people, that's the most direct.
A lot of work, I'd say my time is split quite evenly between, I still do more data stuff, like I blog quite a lot about data related, career related topics. And whenever I get the time to do so. I also, one of my pride and joys is I created and I maintain this repository of R programming books, it's called the Big Book of R. And yeah, so just recommend anyone to go have a look at it. It's got over 350 titles of just about any topic focused on R programming. And the remainder of my time, I've got some young children who are just at the age now, which is fantastic that they can start playing Mario Kart, the ones getting into Raspberry Pi. So I spent most of my last weekend trying to get a VJ screen to work with a Raspberry Pi and I was not successful. And so this weekend, I think we're going to try to find an old HDMI screen. But yes, that's all my free time, basically.
That's awesome. Thank you, Oskar. And I was going to say, you are like a celebrity to us here. I feel like we've often shared the Big Book of R in the chat and a bunch of resources. So it's so nice to have you here with us. Thanks. Yeah, that's good.
Theory of constraints and data work
And Oskar, I know we chatted a little bit about some discussion points that we wanted to bring up in the beginning as we wait for questions to come in. But one thing I wanted to ask you about, as you mentioned, like theory of constraints and visible work, along with the negative feedback loop and data work. And I was curious to have you kind of expand on that and share a little bit more about what that means to you.
Yeah, so, you know, I've always managed like a bit of the background here. I've always thought of myself as a competent project manager of projects. So it doesn't matter. I mean, I wouldn't say I'm the best ever, but I'm pretty OK. I'd say I get things delivered on time and usually within budget and under sometimes very difficult conditions. And so it really interests me a lot about the management of projects. And I even wrote a very short book that's free to newsletter subscribers, but it can be free to anyone here as well, about project management fundamentals, specifically for data analysts. So I think about this type of thing a lot.
And, you know, I'm aware of, you know, you get agile practices and a few others for managing projects. But I only very recently picked up this book, The Phoenix Project, which I'm sure probably many people here have read or some have heard. I'd highly recommend reading it. It's really, it's interesting from a few ways, one in its delivery of teaching you theoretical material through the use of a novel. I'd never experienced something like that before. So that was just interesting on its own. But the second was that it introduced, so it's teaching of this concept, the theory of constraints and how it relates to management of delivery of projects. That book itself is based on another book called The Goal. And The Goal was written in the 80s by, I can't remember his name exactly, but it's, his surname is Goldblatt, I think.
And this, so yeah, it's talking about the theory of constraints and what its implication is for how you manage work and how you assign resources and how you, yeah, prioritize things basically. And why it's also interesting is because out of it, and out of all this, you know, this idea of visible work also emerges. So these things are all like quite tightly interwoven, right? And I've developed this, I had a bit of an epiphany while reading The Goal, which is, and I think I need to read both again to like really dig in. But I had a bit of an epiphany that I'm hoping others recognize or they've seen the solution to or people just like, oh yeah, you know, that is a thing. Or, you know, am I just being a bit crazier?
So if you're not familiar with the theory of constraints, the very, very basics of it is, is that in any system of production, and that includes knowledge work, where the work is being handed off between people, or like in a manufacturing setting, you know, like your car parts need to go through some machine at some point, somewhere in this line of like the series of work events, one or more of the items or one or more of the work sensors, as they call it, is your bottleneck. Your bottleneck is your constraints. And you need to, you'll get optimum performance out of the whole system if you design around the constraints, like most people design around the full capacity of maximizing every single work sensors capacity. And that causes huge all sorts of issues. I'm not going to recount the whole book, but the idea is that you, you basically scale everything down and keep the constraints fully, let's say fully capacitated.
And, and when I was reading it, I thought, okay, this is, yeah, it makes a lot of sense. And a lot of what they were saying, I recognize in my own kind of stuff I've learned in a reverse engineered kind of way, like, oh, yeah, you know, you don't want to maximize everyone's capacity and queues and workloads. And I thought, okay, this, so it was really like quite mind blowing, lots of those readings. But there's one thing that I know, here's my epiphany, that the idea is that your constraints needs to be able to keep pulling work as it's available. And that's the whole idea of having visible work, you keep pulling work as it's available. And that's work going through the constraints goes to other work sensors. If there's any delay at the constraints or any lowering of capacity at the constraints, the problem is that you keep building like each of those delays becomes cumulative. And that's why small delays, especially if your constraints somewhere near the front or on remember the exact positioning, but you get very large delays later on.
If there's any delay at the constraints or any lowering of capacity at the constraints, the problem is that you keep building like each of those delays becomes cumulative. And that's why small delays, especially if your constraints somewhere near the front or on remember the exact positioning, but you get very large delays later on.
And that's why you'll see I mean, you can see it when when people are working like small issues up front become someone else's massive nightmare down the chain, and they can just never catch up with work that was overwhelming, because they're dealing with all this upstream cumulative delays. And the cumulative delays is super important to be aware of.
But now the solution, I think, doesn't quite work for much of data work that just in maximizing the capacity of your, of your bottleneck. And I think the reason is, now I'm not talking, okay, so most of my data work has been around, you could say, like, data analytical work, not productionizing, machine learning systems, I think productionizing that is more akin to the DevOps environment, like an IT, someone designing and developing something and then handing over to someone else to deploy. That's more like how everything else usually works. But in a lot of data work, that's very iterative. There's this huge problem in that most likely your bottleneck becomes a victim of its own delays.
Right, because work often is not done, it's not just handed over, by necessity, you've got to hand over an analysis to someone. But that's not the end of it, we all know it comes back again. And a lot of the techniques that otherwise you might build in to reduce that rework. You know, in a software setting, you know, the whole theory of constraints talks about having feedback of information, but not of the work itself, the work itself isn't supposed to come back. We deal with the work coming back again, because you can't, a lot of the work you can't, especially like if it's an exploratory, any kind of exploratory data analysis, you're doing kind of research, and you're not sure what the answer is going to be like, there's no way you can specify upfront, what the outputs should be. And no matter what you did at the first, like how good your specification was, you will still get a reaction from whoever's consuming it, that they want more information or a feedback that, you know, once your bottleneck has freed themselves, usually it's your analyst and that of the work, it comes back to, that's just the nature, the reason it comes back is because you were successful in the first place, I think, right.
So my, now the only the only further bits I've gotten of how you might be able to deal with this is when you, you need to tweak the whole visible work and all of the system a little bit is not to maximize your constraints. Your constraints also need slack to take up the rework that's coming back. And the, you know, the reworking of work that comes back, you need to probably have some variable system that you'll, I mean, it's going to be very hard to get this to an exact science, but it'll be a function of how much have you already pushed out. So the more that you pushed out, the more variability you need to leave in your capacity until you show that this is not going to come back again. You'll need some way to know that this work ticket item is done, the project's done, the investment's been made, the whatever, and you're not going to get more coming back. So that was, you know, quite a big aha moment that I had a little while ago, like, because I was thinking, how could we implement this and I just saw it just wouldn't work. And it's like, let's say true form.
It's interesting you just said that exact part there, because I was just reading that part in the Making Work Visible book. It's taking me a little bit to get through it, but we just talked about that part. I see, Marlene, you had a follow up question to what Oscar is talking about right now. Do you want to jump in?
Hi, Oscar. Thanks for the talk this morning. Well, it's morning where I'm at. Yeah, I'm just curious, you're talking about decisions that get compounded down the line, right? Especially in production systems. I'm just wondering, in your experience, how much of those constraints have happened from managers, supervisors, C-suite, anybody kind of above you on the totem pole versus maybe, you know, lateral to you or below? Like what, roughly
what percentage do you think you've seen that?
Good question. I don't know, it would be hard. You know, I think, I think it's maybe more those who are dependent on the output to move forward at all are the ones who, I would say maybe more excluding the C-suite or the exact decision maker. Because my experience is, in a way, they don't seem to get impacted at all other than still just moving forward without your information. Right. So there's usually maybe like a lot of the work we do is often in as well in like publishing some kind of documents or a report of some kind. It's that person, that last one, maybe just before it gets to the C-suite, like in the Theory of Constraints Language, and that's where it leaves the workshop. Right. At that technical person's point of view. So I'd say it's, for me, it's, I would maybe classify this 100% laterally. Above that from the decision, like they'll, they need to go forward anyway. Right. And that's also where it starts becoming important to manage these things better, because you don't want to become a like irrelevant work centre service provider to the organisation. If your stuff is just never ready or good enough because of your delays or whatever, people are going to go ahead. Right. They were going ahead before your data team was around and they're going to keep going ahead without you. You know, maybe not for the better of the organisation. But yeah, I'd say I'd say it's 100% laterally. That's for me, at least where my focus has been. Yeah. Great. Thank you. Yeah.
The Big Book of R
I see there was a question in Slido that was, what compelled you to start the Big Book of R?
Ah, yeah, that's a good one. So I love talking about the Big Book of R. So that was, it was about, yeah, so it was August 2020 that I launched it. And a little bit before that, I'd just been bookmarking lots and lots of books, right, especially then. Well, I mean, until just recently, I was on Twitter a lot. And then I was, you know, all the time on Twitter. And people were sharing their books and that. And I just started bookmarking them. And I remember at once seeing my bookmarks folder had 80 of these R books. And I'd not seen a collection of such. And I just put out a call basically. I said, hey, anyone else is collecting? Like, how many do you have? Like, I was just trying to compare, like, and most people coming back with 5, 10, 25. I thought, wow, okay, actually, I've somehow stumbled upon this massive repository. And, and then also, it was one of the things I thought, maybe I could just quickly, you know, put this together, because I'd also just actually written a book called Twitter for R Programmers, which was a collaborative effort with Feriluf and Son. And so I just had this was my first time using R Markdown and write. So I quite liked this idea of being able to make a book really easily. So then these things just came together. So I've got this massive thing, you know, this great repository and, and this great collection. So that was the start of it, 80 books. And yeah, it's been really fun project.
Thank you for that contribution to the community, too. And I know there's so many people who have used that resource.
There is actually a way you can you can help me actually, everyone, if you want, there's a there's a I'm applying for a R Consortium grant to basically upgrade the big book of R. And so I'm collecting basically, like, I know, like, it's a two minute survey, we can just like add your name. And if you think this is a good idea or not, and it's part of the submission. So I think I've got about 110 responses. So if I can get like 150, then it can be like, really, wow, everyone really thinks this is a great resource. So I'll share it in the chat just now.
You're ahead of me. Thanks, Rachel. Yeah, fantastic. That would be great to get a few more.
Benefits of open source contributions
Ben, I see you had asked a question in the chat a bit earlier. Do you want to jump in?
Yeah, Oscar. First of all, thanks for all the contributions to open source, especially in the R environment. But what benefits have you seen from those contributions to your career overall?
Oh, good question. Um, yeah, I'd say quite a few. One is, is just that it's, yeah, how do I put up with this? It's, there's a certain, this isn't the right word, but a certain like level of confidence that you get that it's like, wow, you know, I'm doing something that gets validated quite quickly. Like a lot of our work, a lot of my work anyway, like there's so much stuff that I really like doing and it's really good, but who's going to know that it's good or bad other than me? It's like really difficult to know. My managers are like non-technical. If I make something that's like a really kind of terrible backend and everything's held jankily together, or it's like a super, super great system, it's hard for them to do that, right? So this is one way that's at least, it's great to get that kind of validation externally that, yeah, you know, apply the same kind of, let's say, level of efforts and enjoyment in both elements. So at least this is one, a sample of my work that gets, you know, a lot of thumbs up. So that's really great.
I'm doing something that gets validated quite quickly. Like a lot of our work, a lot of my work anyway, like there's so much stuff that I really like doing and it's really good, but who's going to know that it's good or bad other than me? It's like really difficult to know.
It's also been quite useful in like getting a good overview of the R ecosystem and what people are using it for. And for example, like, you know, I noticed, you know, if you want to do stats or something, there's tons of books, like lots of them. If you want to do geographic information stuff, there's quite a few books as well. I work a lot with Excel in my daily work as well. Lots and lots and lots. And there's Excel packages. No books about working with Excel, right? And so when I was doing quite a, quite a big project and we do interact quite a lot with it, but I always knew that at some point we would be limited with what some of what R could do with Excel. And we did hit indeed those limits, but I was not surprised by them at all. That's a small thing, right? But had I not already had this big overview of everything that goes on in R, I may have walked a bit more blindly into that. And also meeting people has been really fantastic. So often people tag me on stuff and then I start recognizing authors or I've been asked to review some book, book proposals as well. So, you know, you'll get, especially from CRC press, they do quite a lot of R books publications. So I've been asked to, you know, look at some of those. It's quite nice to see what some people are working on way before it's released. Yeah. So it's been, it's been really fun all around. And I must say that it wasn't, I didn't think that would happen at the start. So this would be really cool, but it's been great fun.
Awesome. I see there was an anonymous question that I just want to make sure I covered. Somebody asked if there was a grant program to assist students who wish to attend POSIT conference. And I just wanted to let everybody know that Hannah shared in the chat that we do have an Opportunity Scholars program as well. And she put the link in there. But Jonathan, I see you have your hand raised. Do you want to jump in next?
Context switching and tickets
Yeah, absolutely. And Oscar, thank you very much for co-hosting today. Rachel, thanks for putting this together as always. So one thing I felt that I could add to the conversation, I work in risk underwriting for the subprime, been doing it for about six years now, and it's a very flat organization, FlexShopper. And one thing I've learned in the, in theory of constraints applying it here is that priorities tend to shift very rapidly and it really causes a lot of context switching amongst the team. And one way that I've introduced to help to control that is even though it's not ideal, and hopefully maybe somebody else might be able to chime in on this later, is just having them create ticket stops for everything they get. So in that effect, if they get a new priority saying, hey, get on this now, just put a ticket stop in, have it there. So that way when we're done, we can go back and figure out, all right, what were we doing over here? Oh, we were doing this on the other thing. Here we go. And it doesn't eliminate the context switching, but it definitely helps to reduce it to the effect of you can almost pick up right where you left off. And it's helped to bring up the efficiency by around, I would say 20% terms of turning tasks around.
Okay. Yeah. No, great. Yeah. And the context switching as well, you know, I felt it myself and I've seen it with others as well. If you can reduce context switching output is massively, it's much better, but also people's job satisfaction is like way higher and including my own, right. It's getting harder and harder to not context switch, but I have noticed even for myself that that's switching, even if it doesn't get you work up, let's put it this way, even if it didn't improve productivity, it improves work satisfaction. It's a huge retention gainer, but yeah, that's, that's great. And especially on the tickets, that's a good one.
Thanks Sean. Yeah. Thank you. Anas, I see you have your hand raised too. Yes. Thank you. Thank you, Rachel. So I'm going to piggyback off Jonathan's question, Oscar. So how, what have, I like how you described the difference between how the theory of constraints affects broadly and then how it affects us specifically in data science. So what are other methods that you, that you've, that you've tried to help reduce that piece where, you know, we're repeating analysis and like, you kind of left me on a cliffhanger there. I'm like, what are, what are some things that you've experienced and things that you'd recommend? And then I have a second question, if I can, going back to the big book of art. I sometimes feel like I, there's things that I want to share, but I feel like because of the nature of my work, I have to kind of like hide what I'm doing, not hide, but you know, there's company confidentiality and I want to kind of like obscure that and then share. And that's just a tricky piece. And I'm, have you had to deal with that before? Has anyone here had to deal with that before of wanting to share things they've done in R, but at the same time, they're concerned about their company's data, not the company's data, but just that process.
All right. So let me start with the, on the repeat analysis. I think for me, it's not, not to eliminate the repeat analysis. I think that repeat analysis is, is precisely the valuable bits of the work. If you're with me, like, like getting that, especially when there's some kind of domain expert that back and forth to get something out there is really valuable. And it's actually that back and forth that I want to enable and protect that both parties end up quite satisfied, not to, I mean, there's a difference between, let's say, asking me, like asking someone to redo work because the communication was faulty in the first place. That's, that's a different problem. That's, you know, you know, just getting the two working together well, understanding definitions or being clear on the output, but when things are going very well, that back and forth and repeat, I quite like to keep going because then, you know, that for every iteration, there's more value being extracted out of the data we have. So I go, yeah, cool. Now can we look at this? Now can we look at that? What about this? What about that? I really, yeah. So I'll try and encourage that, but, but it's like how to still let that happen is by protecting the time of the person doing that work, that they don't feel like they have to juggle and do the switching, because then they don't want to also, you know, get into this back and forth and can we look at this? Can we look at that? So it's more about, I sort of see it as, it's like the stream of time that you're trying to protect. And as long as it's needed, it's got to be insulated until that's closed off. At some point it will come to an end. So, yeah, so it's, it's flipping it on its head a little bit there.
Sharing work while protecting confidentiality
On sharing work, yeah. So I can tell you what I do. I just don't share the stuff that I, even I really want to, I just don't. I haven't found a very good way for me to be able to show a lot of what we're doing, but then I zoom out a little bit, like into other things that I find interesting. Like, for example, how, like I want to talk about some of the projects we do, but it's a bit difficult. I mean, I think at some points I might, maybe I'll do it more when we've got time, more time to do it, but, but I'll zoom out a little bit. And what else do I think is interesting? Like project management, how do you manage your team? What's, you know, I'm trying to think of some other things. Like I was quite interested in, in, you know, some of these certifications that you see out there that, you know, I've seen many, many new data analysts are getting. And I wanted to see, okay, like the Google data analytics one, for example, I was super interested in assessing, like, what do I think about the content as a hiring manager? Now that's not NBA stuff for us. That's nothing. So I went and did it and it's still interesting. And then that I wrote about, so it still scratches that itch of being able to share information, not always the exact information I want, but I have seen someone else do is they took for something they worked on with us. They did a demo on their own, like toy projects to test it. And that's how they learned before they came and did it. And then that's how they, they pitched us like, no, we'd really like to do this type of thing on your project or on this project we're working on. But they had just done that, the demo of it publicly on a toy data set. So they still also got that kind of scratching that itch of being able to share a little bit, but yeah, but I'm keen on if anyone else has been able to, you know, come up with some cool NDA protection back in whatever, you know, yeah. What would you, what would you say? Like, I can't, I can't think of the word now.
I see Marlene, you had some ideas of what you're working on.
So what happened? I saw, I saw you, you said you had some ideas of, of how you were sharing this, like public facing portfolio. So I was curious if you wanted to share maybe how you were doing that.
Oh, sure. So for those of you who don't know, I work in agriculture, right? And some of the projects we work on here, for example, are food safety related, where we're showing lab tests of products. So like a piece of lettuce that gets sent to a lab, and then it gets homogenized, tested for foodborne pathogens, like E. coli.
And then we'll show those on a map, positive or negative. By the way, when you see a positive on a map, it doesn't mean anything terrible. It just means that something lived there, right? An organism. But anyway, my point is that maps can very easily be misconstrued. So I, I did a mock up for this. It's like right now, nationally blowing up this food safety data sharing project that my trade organization put together. But we went from the prototype that I built in R to having a subcontractor redoing it in basically like a click dashboard, which by the way, I feel is not as clean and as beautiful as some of the like leaflet packages. That's just my personal opinion. But it gets the job done.
But anyway, I really want to take that prototype of the map that I built and make it public facing. But there is no way like people will come slamming on my door if I have positives and negatives on a map. So my thought was, instead to do something like here where commodities are located, because that's not necessarily sensitive. And I would have to take a public data set. Those of you who are asking about public data sets, there are a ton out there, by the way. It's just a matter of finding a good one for your use case. Like you can literally just Google public data sets. Academia has their hands on tons of them. So they're definitely out there.
But I've taken things like, feel free to laugh at this. I know that because I used to work with the Water Board of California, that they have testing for stream, like certain fish or bio essays that are done there. And I've taken those and just, you know, jig them, like move them a fraction of a GPS point up or down. And just been like, here's my new point. And we're going to fake that here's where lettuce is grown or radishes are grown. So you might see like a radish grown in the middle of a stream. But hey, I just put a giant caveat along the top of this is meant for demonstration purposes. It's fake data, et cetera. But I'm basically going through that whole process for everything that I've ever built, which is not a huge portfolio, because I feel like the most of the things I've built are pretty action packed in like a single tool. So there's a lot of hours packed into a single tool.
But I hope that kind of makes sense on like some of the ways you can de-identify some of your data. I think it's more important to show the end visualizations and to also have a page that kind of chronicles the process a bit. So they understand all the work that goes in rather than just here's your snapshot. Because we do live in an age where people are a little underwhelmed now, thank you CGI and great movies, by beautiful visualizations. So you have to let them know like there are a lot of blood, sweat, and tears that went into this, right? So anyway, there's my two second spiel on how to kind of maybe de-identify some of your work.
Thank you, Marlene. That's such a great point too, because I was even just talking about that like internally at Posit that if we show an example that's using like empty cars, it's not going to look the same to like a finance company. Like they want to, even if it's play data, they want to see something that actually is relevant to their industry too.
Hiring to replace you versus hiring for skills you don't have
Another question for you, Oscar. We talk a lot about this like shift from being an individual contributor to moving into management. And something I've seen you write about before is hiring to replace you versus hiring for the skills you don't have. And just wanted to open up the discussion about that too.
Yeah, so this was, I'll give credit to, I think his name is Michael Kaminski from the locally optimistic Slack group. It's also highly recommended for anyone who's on this call to join that as well. But yeah, so when you're getting into a management position, at some point you need to start hiring people and a team around you. And hiring is actually very different depending at different points in your career. Like I thought that it had been the same because I had done hiring before, even when I wasn't a manager.
And there does actually come this split that I think takes some people by surprise the first time it starts happening because it certainly took me by surprise. So before you're like, let's say a manager manager, if you are growing in your career and you do happen to be hiring someone, you are hiring someone most likely to replace you like a more junior version of yourself. And you're going to train them and you're going to, you know, you are the experts of some level and you're going to bring them up to your level or something like that. And also you feel, I suppose, in a way quite secure in what it is you're doing because you've already got much more knowledge than them and you are bringing them up to your level. And so you feel very confident in answering the questions and mentoring them and all of that.
But it becomes very different if you're starting to hire for skills you don't have. So for example, you know, I don't have a stats background. I don't have a data engineering background. I don't have even that much actually of an analyst background. I was sort of like a market analyst, not a data analyst per se. And when you start hiring these team members into your team, these are for skills I don't have. It becomes much more difficult for me to also assess, you know, what's, are they good or not?
And also very quickly, like it's kind of weird, like I first started, like I think most people start, I was trying to learn as much data engineering and data science as I could to try and just stay at least up with them. But there's no ways that you can do that. And actually it's, then the sort of click happens and you're like, okay, wait, actually my job is not to be the experts of all of the people in my team. I am just the, mostly really a glorified facilitator of their work. And, you know, if they, if I know more than them, that's probably actually a problem. Like we can't have me be, you know, with my 10% of time on all of these different vast fields, be still the experts and everything. That's not going to be great.
I am just the, mostly really a glorified facilitator of their work. And, you know, if they, if I know more than them, that's probably actually a problem.
And that's quite a difference, which, and when it's, it's, I think it takes a bit of getting used to, to start not understanding bits of what it is, you know, then, then the tables turn a little bit, right. I'm in some ways the non-technical manager of technical people, although I'm kind of technical myself, but they'll talk about things. And I also have to be like, well, well, then I start also clicking a lot of like what my directors and execs feel like sometimes I'm like, yeah, you're getting into way too much detail. Like I can't do anything with, you know, what you're telling me, but then also it gets much better. Right. In, in trying to assess like, okay, what's the options that like my role mostly is to try and not just prioritize work, but try and help, help them uncover like where it is work needs to be done. Or if there's a dilemma, try and break down. Okay. Well, you know, what's, if we go this direction, how bad will it be if this happens or how good will it be? If this happens, can we weigh risks like this? So I can convert, convey to them as well where things just aren't a priority because no matter what they do or stress about here, it ain't going to matter later. Right. So it's quite a different ballpark and yeah, I think it probably happens the first time you hire someone that's not a replacement for you. It's an addition to your team. So yeah, it can catch you by surprise.
Any, any other tips for us? It's like we're navigating like that first hire where you don't have the same skillset as them, like how you communicate with them or in your one-on-ones with them.
I'm very clear that upfront that I am not the skilled person that they will be able to come to for, you know, you know, X, Y, Z technical thing. I just don't have that. And they can't expect that of me, but I am quite clear about what I can provide. And what my role is and that I am interested still in their technical developments and in, you know, but they need to guide me about what they think, where they think they should go, right. They're going to tell me some gobbledygook new thing, like, okay. And then I'm like, okay, I just need to assess, you know, this is what I'll need to assess if this is going to be good or not. So I suppose being much more, like, I'm very clear about my, like my thought processes and my decision-making. So I do that very vocally. Like if I'm weighing up things, I'm talking out loud, like, okay, if we do this and this, so that it's not this black box for them about, you know, they gave me some information and then I just gave a decision back. I think that's also been quite helpful in getting our communication to work quite well.
Yeah, especially with people of different backgrounds.
Career development for your team
I feel like career development is such a big part of being a manager, but I don't know if it's necessarily something that managers are always taught. And I'm wondering, like, how do you learn about growing your team's career and having those conversations with them?
Yeah. I think in one way, now for many of this might sound blasphemous, but in one way, I try and take the company out of this equation. Totally, you know, not just our immediate needs. And I think, okay, what's going to be really great for this person's career in general? Like if I bump into you five years or 10 years down the line, in a selfish way, it would be fantastic if your career was just going swimmingly, right? It's good to have contacts whose careers are just really flying. And then I sort of work backwards from that. And I say, okay, you know, I think, you know, would this be better for you? Would that be better for you? Or, you know, don't worry too much. Like if you could do anything, what would you do? And then, of course, you need to try and whittle it down and say, could we apply some of that now in what we're doing? Or do you see an opportunity? Is it something that really interests you?
And that really helps a lot. But actually, I found another way as well is to let people go with their curiosity as well. And I think that's one thing that at least I can appreciate coming from some kind of technical background is actually most of my career developments has been from just the bits that I was curious about that I couldn't explain to someone exactly like what was the ROI on this thing going to be? I don't know. It's just sort of you get a sense that some things may pay off, even if it doesn't, you're going to learn quite a lot. So I try and give a lot of room for, you know, if you want to go try that, just, you know, go do it. I sort of trust that some way your benefits and will benefits, whether it's successful or not, try and make a lot of room for that.
Project management applied to data
Thank you. Yeah, I want to make sure I go over to some of these anonymous questions too. And somebody had asked, I have acquired skills in project management and program evaluation. Could you elaborate more on the application of project management to data analytics?
Well, yeah, my as data is becoming, you know, used in organizations a lot more. And, you know, if you come from a digitally native organization, you may not realize this, right? But coming from any kind of traditional organization, that's everyone that's not a digital producer of some kind. Data is becoming more and more infused daily into work, right? Sometimes not always at the pace we like, but it is. And data as products are becoming more ubiquitous. So someone wants to develop a tool that assesses something or a shiny dashboard or any other dashboard or data starts moving back and forth between places. And these things need to be managed whether in whole or just in chunks. And I think that's where the let's say project management application from a data perspective comes and will keep growing as companies start adopting that kind of data as a product mindset.
Like, oh, this is a, it's not just this like inert data. It's not just like, yeah, like this inert pile of stuff that we occasionally will pick from when we need it. It becomes just like more living integral part of work that someone needs to maintain. Someone needs to have a vision for, and then naturally projects around that grow because you want to upgrade things, you want to add new capabilities. So that's where I see it's hitting lots. And that's partly also why I thought, you know, this data project management for data and this book is useful because a lot of data people find themselves suddenly managing a project like, oh yeah, I've got to get the comms people involved for the, you know, I don't know the funds on the front end of my thing that I'm about to publish, or I need to, you know, talk to, you know, just more and more people as data starts touching more bits of the organization. You gotta, you know, align a lot of people around some deliverable.
Definitely. Thank you. I love all these packages and resources that are being shared in the chat right now too. It looks like there's a lot of great resources on making dummy data sets and synthetic data. Another anonymous question. Actually, first,
let's jump to your question, Sal. I see you asked the question in the chat. Do you want to jump in?
And I can ask it if you want to jump in with more context. But the question was, at a certain level, I feel that managers are told, let your employees be autonomous, but simultaneously managers and chain of command have very specific expectations. Do you have experiencing balancing autonomy and directing towards a desired result?
Um, yeah, I don't think those things are mutually exclusive. Um, I mean, I suppose maybe just personally, I've been quite lucky. I've always had a lot of autonomy. Um, but I think autonomy now, okay, I'm not, I'm not talking about like, let's say toxic workplaces, right? Where people are micromanaging and there's power plays and all that. Let's just say there's at least some level of maybe autonomy being earned, perhaps. Um, let's say, let's say in those situations where you really could be given a lot of autonomy to achieve your goals, but just it's still not happening. Um, it's earned in the, in the sense that,
that it's going to sound so cliche, right? But really understanding like what the business is trying to achieve really helps because then most people, my experience, most people don't want to micromanage. They want to just let people run with things, right. And, uh, and deliver results. And I think that kind of autonomy not just comes from, um, understanding the business, but I found a lot of it's about being very empathetic, especially about those higher up who are accountable for the outputs of whatever it is. So, and what that means, you know, what I mean by this, I often put myself in the shoes of, of, you know, those who are saying they want autonomy or they need these results, whatever. And I really think, okay, what are they, if I was them, what am I trying to deal with right? I've also got, I didn't sleep the night before my kids were up and I've got all these emails that are in here and da da da da da. And now I'm about to send an email that says, you know, I can't do this without these resources. You know, I'm going to be like, you know, that's not going to land or, um, and that's how to sort of, um, it's maybe one small way in doing it is, is when you sort of practice that kind of empathetic aspect, then it's helped me a lot phrase things. Okay. Well, what's the most helpful I can still be in order to move things along, even if it like something went wrong or, you know, and then that's really helped me a lot because then I can say, right, well, I've also done the work of really trying to understand what it is we're trying to achieve and, um, and what the trade-offs are at the higher, you know, the higher up we go. And then I'll try and frame things in the same, um, like the alternatives that, that they are available. Right. And then I also show that, you know, I've thought about if we do X, we're not going to reach this, but it'll maybe do Y. But if we do this, it's going to do that. And I recommend also always say what you recommend, you know, the higher up you go, there's decision fatigue, like crazy. Um, it really helps to say, you know, I've thought about all these trade-offs and this is what I recommend. Um, you know, so I, I mean, I can't say if this will definitely always help, but, uh, it always feels like the times that I haven't really put in the effort to, to do that kind of work, uh, then things don't often go as smoothly as I'd like. Right. Then I get the answers that I've done with, I would prefer maybe not to get all the response lands badly.
And I recommend also always say what you recommend, you know, the higher up you go, there's decision fatigue, like crazy. Um, it really helps to say, you know, I've thought about all these trade-offs and this is what I recommend.
Dealing with difficult stakeholders
Yeah. Yeah. I wanted to put a plug out here for your, your website as well. Um, cause I came across a, an article that you wrote that I found very interesting on dealing with difficult stakeholders. Um, and I would love to give some time to that, that topic and for people who might be on the call who are hearing, like, we're too busy to go forward with this project, or this is way down on our priority list. Like what are some things they can do to get past that?
Yeah. Yeah. So, so, uh, that it's, um, that post is like a collection of every, um, let's say approach that I could think of that I've employed to make working with stakeholders easier in some way. Um, and they'll, they'll work in different contexts, but I think that for me, the most effective has always been canvassing. I don't know, but I think I put eight there or nine, I can't remember, but canvassing and, and most of you probably already do this, or you're not sure it's called canvassing or whatever, but, um, you know, when you're like getting the buy-in, especially when there's a group or any kind of committee or any joint decision, getting their buy-in before the actual public facing meeting is really the only way to get really good progress.
And what's difficult about it is it's extremely time consuming because of course, the bigger the committee or whatever it is, the bigger you've got to go. Um, and the key to that is that, um, you know, trying to understand people's incentives, like politically in the organization, whatever the incentives for accepting things or not. And, um, you know, so when you're trying to set up these one-on-one meetings and that those you want to keep quite small because you, you, the best thing I've found is people need to be able to back out of things. Like if you put people in a corner and you want the, you know, they're, uh, they're, um, that's what doing publicly in the group is not a good idea, right? If you're trying to get people's, uh, you want to give them wiggle room and doing the one-on-ones beforehand, um, and, uh, not following those up in writing, right? So, I mean, unless you're, you're already in some kind of combative situation and you need to do it, you need to do it, but this is ideally all the legwork you do way ahead of time that, uh, you understand where they're coming from, what they can agree to, what they can't agree to. And, you know, it's great if you can, um, go around and you understand what's their main concerns or what it is that they're looking for from this project or from this, whatever. And you build that all in beforehand that ideally you've got the agreements, then by the time it comes to the group agreement, everyone's already agreed.
And you will have to sometimes live with this uncertainty of some people not agreeing, but they will at least not object to the work or object to what it is you're doing. And, um, you know, sometimes I find that's, that was also for me in the first few times I had to do this kind of thing. I wanted the certainty, right? Of everyone giving me the yes before I go and spend all this time. But the reality is you won't get it from everyone. But for me, you know, not no objection is as good. And usually to then after the thing's got a bit of momentum, the objections don't, uh, they're too late then right later. But a lot of people, they want the ability to back out. And sometimes I think if that's what they need to be able to let this thing go forward, then you just got to be pragmatic about it. Right. And then just say, great. That's, that's as good as it's going to get.
Keeping up with the Big Book of R
Thank you. I wanted to, as we come to the top of the hour here, I just want to make sure if there are questions that you didn't ask yet that people have an opportunity here to raise your hand or repost something in the zoom chat. Um, one that was over on Slido is going back to the big book and, um, somebody had asked, I'm not finding the question here, but essentially like, how do you keep up with all of the links and make sure that they're not dead because so many of those books or websites are changed over time.
Yeah. I'd say actually, um, uh, I think the, the magic of version control and book down actually means that links don't break that often for a start people republish back to the same point. Um, so yeah, that's, it's actually not been too difficult to keep that kind of thing going, you know, occasionally what's also quite, I suppose, because I'm also fairly like reachable. Um, if someone finds a dead link, they just tag me and say, Hey, this link's dead. Right. So I'm not the one actually doing the maintenance on that per se. And very many people are very kind. They'll also say this isn't working. This is a new link. So my work's super easy. Right. So, um, I'd say that's, that's kind of where, you know, people just being really helpful helps. Um, and, uh, let's say keeping up, uh, just in more general of adding books. So that is a bit more of a challenge because there's always quite a lot being added. Um, but I have, uh, very recently also drafted in some help, uh, from Neil Olson. You'll, you'll see one of my updates. He's a, he's a fellow data nerd in Germany. Yeah. Kind of very kindly volunteered to help. So the last couple of batches he's been doing, it takes, it takes actually quite a lot of time to look at the entries. Cause I, you know, I want to make sure that it is an R book. It does the kind of relevance that, you know, where does it go? What should the introduction be? That takes a bit of time to, to do all of that. So I've drafted in a bit of help because it was getting a bit much.
Resources for R beginners
Yeah. Great. Um, another question I realized I had missed from earlier. Somebody asked, do you have recommendations for useful resources for beginners in R even beyond books? They said, I'm overwhelmed with the available resources and unsure where to start. Yeah. Um, I point to where probably most people would is R4DS book by, uh, Hadley and, uh, I always forget the second one, but that's how it goes. Right. Um, and, uh, then there's also a video, if you're wanting a video course. So actually on the big book of R, there's a big book of R slash start here, I think, which is indeed, if you are very new, there's two resources. One is a book and one's a series of, uh, free videos from R for the rest of us that, um, uh, you can check it out.
So, I mean, my, I mean, I I'd say I'm a, uh, a, uh, not a fantastic R programmer at all. Right. I'm, I'm not, I'm just not that fantastic, but what's great is you don't even need to be very fantastic to get a lot of use and power out of it. And I'm, uh, I'm 20 times more productive. So I think if you're a beginner from R, like, I mean, I did half of the R4DS book and it's been insanely useful, just the first 15 chapters or so. I've, uh, I've, uh, done basically everything I do now is just from that. So, so you don't need to learn everything, just learn a couple of things, uh, and get going.
Yeah. Trying to copy and paste the big book of R start here. So there it goes. For some reason, Zoom wasn't letting me paste for a second. Um, and Andy, yeah, thanks for the recommendation on the Posit cloud primers are a great resource too, if you wanted to share that. We had a team go through, um, Posit Academy this last winter, and we had two folks that had never programmed in R before when they signed up for, for Academy. Um, so those, those cloud primers were super helpful for, for folks to come up to speed. Oh, cool. Okay. Nice.
Thoughts on ChatGPT and AI tools
Trevor, I see you just put a question in the chat. You want to jump in? Hi everyone. I'm in sunny Jamaica, just to let you know, um, I do work for a multinational firm in Jamaica. Um, IGT, if you look up us, you can look on a stock exchange and you'll find us. Right. All right. So I've been watching the conversations around the very new shiny toy, chat, GPT, and even Bard from Google, right. You know, I've been watching the conversations about what we think and it's still evolving about what this will mean for us. Will we be writing code or not? You know, so I just wanted to get your thoughts on how do you see those applications affecting what we do positively or negatively.
Yeah. Thanks. Thanks. Uh, so yeah, we'll see if, uh, uh, what the test of time will tell us. Um, I think it'll be a useful tool and, uh, but we will soon find out what its limitations are, you know, uh, I'm a bit, I'm mostly worried that it's not going to be as cool as we think that, uh, it's not going to be as useful and revolutionary mostly. I'm not worried that it's going to be so, you know, going to take over everything. And that, I suppose, cause I saw, uh, AR and VR was going to change everything. And then it sort of fizzled out and blockchain was going to change everything. And that kind of, you know, there's a couple of useful things for it. And then I saw NFTs were going to change everything and then that's kind of died, my opinion, thank goodness. So then crypto was going to change everything. And then, uh, I mean, the difference is chat, GTP, I've used it a few times and I've found it useful. At least the version I have, we've got a, um, a, uh, well, it's on my wishlist in a way to use it for, uh, some elements of our work, especially we've got a lot of contextual information. So I liked the idea of being able to retrain it on our own, uh, on our own content and then use it in some very specific applications of trying to summarize some stuff. Um, and I think, yeah, I think if it does very, very well, it'll be useful.
History shows it's going to fizzle out to, you know, a couple of, a couple of other use cases or, you know, I really hope it does fantastically well, because I think what my, my feeling is it's going to, if it does fantastically well, it's going to, yeah, it's, it's going to ease a lot of the low level work that is not of enough value to hire someone for, or to farm out to someone. We just live with the mess of not doing the summary or not, uh, you know, drafting this thing or whatever. We just kind of deal with it because it's not, you know, it's not enough value and then it will be fantastic. Right. And, and I think on balance it's, uh, will be very useful.
if it's very good. So we'll see, but we'll see what, uh, what happens. Okay. Thank you. I'm hopeful that it's going to be good. Yeah. And Trevor, it looks like a lot of people want to come work with you. They want to come to Jamaica. Yes. You're invited.
Well, thank you so much, Oscar, for, for joining and sharing your insights with us. This has been so fun. Um, if people want to stay in touch with you is the best way through LinkedIn or Twitter?
Uh, yeah, I'd say probably LinkedIn is the one I really, I would check the most these days. Um, I'm just trying to find my LinkedIn. Um, yeah, if I'm like checking messages, um, thanks. Yeah, please. Everyone do, do, um, next with me. I've seen some connection requests come through. I'll also post it. Yeah. And, um, yeah, I'd say there, and then also if you, my newsletter as well, I mean, if you kind of interested in that, uh, always read any email replies to that, but yeah, probably LinkedIn's, uh, the best these days. So thanks. And this hour went by so fast.
It goes by very quickly, right? I wanted to put one more plug at the end for the big book of R and the form that you mentioned to get, um, support. And I am trying to grab that real quickly. Here it is. Uh, so I just put that form there. If anybody is interested in, in helping out there too, but thank you so much, Oscar. That was great. Yeah. Thanks for the invite. I really appreciate it. Thanks. Have a great rest of the day, everybody.