Resources
Mar 20, 2025 |
57 min
|
1k views
|

Data Science at the Command Line and Polars | Jeroen Janssens | Data Science Hangout

To join future data science hangouts, add it to your calendar here: https://pos.it/dsh - All are welcome! We'd love to see you!

We were recently joined by Jeroen Janssens, Senior Developer Relations Engineer at Posit, to chat about his career journey from machine learning to developer relations, the advantages of using the command line for data science, his books "Data Science at the Command Line" and "Python Polars", and advice for aspiring DevRel professionals.

In this Hangout, we explore the benefits of working on the command line versus not. Jeroen explained that while the initial command line interface might seem stark, it offers a very different and powerful way to interact with your computer. The Unix command line is ubiquitous across various systems, from Raspberry Pis to supercomputers. Its strength lies in the ability to connect tools together through standard output and input, allowing for quick and iterative solutions by combining specialized tools. This fosters an interactive nature with a short feedback loop and provides closer interaction with the file system, making ad hoc data exploration efficient.

Resources mentioned in the video and zoom chat:
🔗 Jeroen's LinkedIn → https://www.linkedin.com/in/jeroenjanssens/
🔗 Data Science at the Command Line → https://jeroenjanssens.com/dsatcl/
🔗 Python Polars: The Definitive Guide → https://polarsguide.com/
🔗 Plotnine → https://plotnine.org/
🔗 Winner of the 2024 plotnine Plotting Contest → https://posit.co/blog/winner-of-the-2024-plotnine-plotting-contest/
🔗 Talk about plotnine → https://www.youtube.com/watch?v=xdD8r84sqYY
🔗 R for Data Science → https://r4ds.had.co.nz/
🔗 Jeroen's plotnine translation of R for Data Science → https://jeroenjanssens.com/plotnine/ 🔗 froggeR package → https://azimuth-project.tech/froggeR/
🔗 Reticulate → https://rstudio.github.io/reticulate/
🔗 Install Windows Subsystem for Linux (WSL) → https://learn.microsoft.com/en-us/windows/wsl/install
🔗 UTM for macOS (Virtualization) → https://mac.getutm.app
🔗 fish shell → https://fishshell.com/
🔗 Quartodoc → https://github.com/machow/quartodoc
🔗 Focusmate (Accountability Partner Tool) → https://www.focusmate.com/
🔗 Surface Area of Luck → https://modelthinkers.com/mental-model/surface-area-of-luck
🔗 CRAN R Extensions Manual → https://cran.r-project.org/doc/manuals/r-release/R-exts.html

If you didn’t join live, one great thing you missed from the zoom chat was people sharing their varied experiences with the command line, with many admitting they primarily use it for basic navigation or only when necessary, and some sharing helpful tools and tips for those less familiar. Let us know below if you’d like to hear more about this topic!

â–º Subscribe to Our Channel Here: https://bit.ly/2TzgcOu
Follow Us Here:
Website: https://www.posit.co
Hangout: https://pos.it/dsh
LinkedIn: https://www.linkedin.com/company/posit-software
Bluesky: https://bsky.app/profile/posit.co

Thanks for hanging out with us! 💛

image: thumbnail.jpg

Transcript#

This transcript was generated automatically and may contain errors.

Welcome back to the Data Science Hangout, everybody. If we have not met, I'm Libby. I'm a Data Science Community Manager here at Posit. And I'm also a really passionate data science educator. I have a background in business statistics and experience teaching both Python and R. And I'm based in San Antonio, Texas, where right now it is warming up, turning into spring, flowers are blooming. So I'd love to know in the chat where you are. And if it's spring yet, are you on the other side of the globe? And maybe it's definitely not spring, please let me know.

If you're not familiar with Posit, Posit builds enterprise solutions and open source tools for folks who do data science and R and Python like me and probably maybe like you. We're also the company formerly called RStudio. So if you've used RStudio, then you've definitely used Posit. And if you've used something like Shiny or the Tidyverse, then you have used a Posit ecosystem package. The Hangout is our open space to hear what's going on in the world of data across different industries, because we are all from different places. It's kind of the magic of the Hangout. And we get to chat about data science practices, data science leadership, and connect with other people who are sort of in the same spaces we are sharing the same experiences that we are. We get together every Thursday, same time, same place here on Zoom. And we hope that you keep coming and hanging out with us.

If you are watching this on YouTube, you want to join us in the future, down in the description box, there's going to be details on how to add this to your calendar. If you are adding it to your calendar, please make sure that it's adding for 12 p.m. Eastern time, which is a different time all over the world right now, because we just hit daylight saving time. So for a couple of weeks, we are out of sync with the rest of the world.

I want to say thank you to everybody who's made this the friendly and welcoming place that it is today. And we're all really dedicated to keeping it that way. So if you have feedback about your experience today at the Hangout, there's going to be a survey that pops up very handily in Zoom in your browser when you leave this meeting. And we'd love to hear what you think. That survey is anonymous. So if you have nice stuff to say and you don't mind me knowing who you are, please add your email address so I can email you back and say thank you. Or if you have feedback that you need to address and you just want to hear from me or Rachel here at Posit, please add your email. You can always find me on LinkedIn or Blue Sky. Say hi there. I think I'm the only Libby Heron on Blue Sky and LinkedIn.

And at the Hangout, I want to remind you that we love hearing from you. And it doesn't matter what your years of experience are, your title, your industry, what languages you work in or don't work in. I really encourage you to say hi to everybody in the chat. The chat's wild. If you haven't been here before, it's the place to be. And introduce yourself, where you are, what you do. Fun fact about yourself. And also, if you are hiring for any roles, let's connect everybody in our community with roles. If you're looking for a role, please put that in the chat. Let people know.

We are going to be doing questions on Slido again today. We really like that. It's been going really, really well. So Slido link is going to be in the chat. We will continue to add it. I know people hop in and out of our call. On Slido, you can put your name on your question. You can also leave it anonymous. There's a little space to put your name. And then also in Slido, you can upvote questions. So if you see one that someone else asked, and you're like, oh, that's a really great one, you can vote for it there. And it will pop to the top, bubble to the top of our little screen here in Slido.

Introducing Jeroen Janssens

All righty. I am so excited to introduce our co-host today, our Feature Leader, Jeroen Janssens. He is a Senior Developer Relations Engineer. Did I get that right? At Posit, here at Posit. If you would introduce yourself, tell us a little bit about what you do, your background, and something you like to do for fun. Hey, thanks for having me.

And yes, my official title is indeed a Senior Developer Relations Engineer. And that's quite a mouthful. I've been practicing these past few weeks getting it right. So I am based out of Rotterdam, the Netherlands, where I live with my wife and two kids. My background is in machine learning and artificial intelligence. I studied that a long time ago. It was nothing like the AI that we see today. But as I was finishing up my PhD, and I think we're now talking about the end of 2011, I got the opportunity to work at a startup in New York City as a data scientist. This was a term back in the day, it was brand new, wasn't even known on our side of the pond in the Netherlands. But hey, it seemed to align well with what I did during my PhD. So I moved there and eventually worked there at three different startups over the course of two and a half years.

And during that time, I also got the opportunity to write a book, my first book, Data Science at the Command Line. And when I came back, I was invited to give a workshop about this wonderful topic, how to process data using the Unix command line. And I noticed that I enjoyed this a lot. So I started doing this more and more often, also about related topics related to R and Python and things like machine learning, data visualization, until I decided like, hey, let's start a company around this. So that's when I founded Data Science Workshops, which I've had for seven years.

Now at a certain time, the pandemic hit. And, you know, teaching was no longer as fun as it used to be. I struggled a lot with getting new clients. So I decided to change things up a little bit and become an employee again. This is when I joined Xomnia, a small data consultancy based out of Amsterdam. This is where I got to know Polars, which we can talk more about later.

And yeah, and a month and a half ago, I joined, I joined you, I joined Posit. So I believe we joined around the same time, right? I think we might have joined on the same day, actually, as part-time employees. Jeroen and I are Posit twins. Yeah. Yeah. We're actually buddies. We are buddies. We're onboarding buddies.

We need to know something you do for fun, though, too. Oh, yeah. No, definitely. Fun, fun. Yes. I remember. Because, of course, the past year and a half, I've been spending almost all my free time on my new book, Python Polars. But I really enjoy woodworking and doing things around the house. Every now and then I play some Zelda. Those are things that I, yeah. So I just started Tears of the Kingdom. And wow. Awesome. Yeah. Yeah. Any Tears of the Kingdom players in the chat right now? I'm still playing it. I'm just drawing it out and side questing the heck out of it. And I'm never gonna never gonna do the last battle just so I can play it forever.

Writing Data Science at the Command Line

Well, I wanted to ask sort of a general question. I also I put your books in the chat. So anybody who is curious, it's probably gone in the chat already. I posted both of Jeroen's books. There they are. And I wanted to clarify Data Science of the Command Line. That one's free to read, right? Yes, it is. At a certain point, I asked O'Reilly whether I could put it online. And I was actually inspired by R for Data Science by Hadley Wickham and Garrett Grohman. They had done this. And I felt like, wow, isn't that great? And at a certain point, so the first edition I was allowed to put online. And then the second edition was written out in the open from the start, actually. And there's a previous chapter out right now, right? I was just sleuthing around on O'Reilly. The previous chapter of the Python Polars book. That's right. Yeah. Nikita says O'Reilly is almost the only publisher that lets you put your book online. Well, that's awesome.

If anybody else has any experience writing books or is writing one right now, feel free to share with us in the chat so that we can take a look at what you're doing. Good luck. Good luck. Yeah, it's a big endeavor. Well, what motivated you to write your first book? I'm curious because you said that you got to give a talk about it. But then what's the process of actually pitching a book and showing the sort of social proof that you have enough people looking at you to publish a book? Yeah. So for Data Science at the Command Line, this is how it started. I was invited to give a talk at the New York Open Statistical Programming Meetup hosted by Jared Lander. He's been doing this meetup for the past, what, 15 years.

And he said, like, hey, would you like to give a talk? Like, what am I going to talk about? And I noticed that I, in contrast to my colleagues, I was using the Unix Command Line a lot in order to do various tasks for getting data, cleaning data, and whatnot. And I thought, like, hey, I can give a talk about this. And as I was preparing for this talk, I was like, hey, why don't I go ahead and write a blog post about this, right? Two birds, one stone. And this blog post, this is where it all started. This came out in September 2013. It got to the top position at Hacker News, stayed there for at least 24 hours, hundreds of comments. I'm like, okay, this means something, right? This is definitely, there's definitely more here.

It got to the top position at Hacker News, stayed there for at least 24 hours, hundreds of comments. I'm like, okay, this means something, right? This is definitely, there's definitely more here.

So that's when I was put in touch with someone at O'Reilly, expanded, well, thought a little bit more, okay, if we're going to expand that blog post into an entire book, what would that look like? Wrote a proposal, and yeah, that got accepted. And then 10 months later, the book came out. 10 months, oh my gosh.

Yeah, yeah, it was quite fast. I was, I just finished writing my PhD thesis. So I was still having this momentum, waking up at 5am, writing for two hours and then go to my day job, which of course my wife didn't like pretty much. She thought I was done after writing the thesis, but no, here I am, I have to write another book. So yeah, that was quite quick. And I unfortunately, the second edition took longer, took a year. And well, and now we just finished Python Polars. That took over a year and a half. So it's not always that fast. You wrote that one with somebody though, too, right?

Writing Python Polars and discovering the library

Oh, yeah, that's right. With Thijs Nieuwdorp. And so if I, no, it's safe to say that if I were to start this endeavor on my own, I would never have completed it. It's just, it's just way too much. So I'm really happy to have Thijs write this book together with me. But for the first book, you could say that the motivation was really, I started out with this, well, minimum viable product, you could say, like it was an experiment. And only when I saw a lot of people reacting on this with feedback, with tools, Unix command line tools that they had been using themselves, only then did it occur to me like, hey, could there actually be a book here?

Yeah, actually, as you were- I'm sorry, go ahead. No, no, you go ahead. I'll take a sip of water. I was going to say, whenever I see Joe Chang coding, he's always in the terminal, and I kind of like don't know what he's doing. And he goes really, really quickly. And so when I saw your book, I was like, I could be Joe Chang. I could figure out how to do data science from the command line. If only, if only we could be more like Joe Chang.

But no, no, no. I mean, being at the command line is always a great conversation starter. Either they throw you out of the coffee shop you're working in because they think you're hacking them, or you meet someone like mine, and they're like, hey, you use the Unix command line?

But then, yeah, we can talk about the command line all day. I can. But as for the second book, so Python Polars the Definitive Guide, yet again, quite a mouthful. The motivation for that book was a little bit different. When I joined Xomnia a little over two years ago, I saw this guy working behind his laptop. He was intensely focused, while all the others- this was a Friday, and that's when everybody at Xomnia comes together. Everybody was already having fun, but he was laser focused on his screen. Turned out that this was Richie Fink, the person who created Polars. So he used to be my colleague, and also Thijs, he used to be my colleague at Xomnia.

So this is where I first learned about Polars. And Thijs and I, we were actually working at the same client, in the same team, and there we had the task to speed up their pipeline. So it was a very big code base, there was R involved, and lots of Pandas code, and we had to speed that up. And then I felt like, hey, why don't we use Polars for this? Because I already noticed that Polars was pretty fast. You could say blazingly fast, as they often say. Yeah, I've heard that rumor. Yeah, yeah, it's sort of becoming an in-joke. Because, yeah, well,

so as we were getting to know Polars better, Thijs and I were like, hey, wait a minute, this is so cool, this package is so not only fast, but also a joy to work with, thanks to its API, this deserves a book. And I was having a meeting with someone at O'Reilly anyway, so then I thought like, hey, let's just test the waters, see if there's any interest in a book about Polars.

So this was over a year and a half ago, and O'Reilly hadn't yet heard about Polars. Well, they had already received four proposals for a Polars book, which all got rejected. But they weren't, you know, they weren't really convinced that Polars was going to be a big thing and deserved their attention.

They were wrong, I think. Now they realize that they're wrong. And, well, they're very happy that we're finally finished with the book. So, but yeah, Thijs and I, we wrote a 20-page proposal in order to convince O'Reilly, and at that point, they could only say yes. And that's when this endeavor began.

That's awesome. Well, we look forward to it coming out, and we'll have to go check out the preview chapter. And I would love to hop in here and ask questions, because we asked for questions in Slido, and they are flowing in finally. So let's get started. Yeah, we have an anonymous one that has popped to the top here and gotten lots of votes, which is, what is the benefit of working on the command line versus not working in the command line? So, like, pitch us on that.

The case for the command line

Yeah, so the Unix command line, when you first open it, when you first see it, you're greeted with this, well, usually a black box, and this prompt and the cursor that's blinking at you. And it's not very inviting, right? It's a stark and harsh environment.

But once you get to know it a little bit, right, once you get to know a command or two, you'll realize that this is a very different way of interacting with your computer, right? You don't have to use the Unix command line. I keep saying Unix command line, but I've been on macOS for the past 15 years. Now, macOS is, of course, derived from Unix, or you could say it's still a Unix. But, and even on Windows, there are various ways where you can still get the same command line. So it's important to make that distinction. I'm not talking about the DOS command prompt or the Windows PowerShell. So, but even on Windows, you can have the same Unix command line environment, right? And then you're usually working in a, well, you're always working in a shell, and sometimes that's Bash, sometimes that's the Z shell.

Anyway, the Unix command line, it's, well, first of all, it's everywhere. It doesn't matter whether you're logging into your Raspberry Pi, your router, or some supercomputer. Chances are is that it's running Linux, and that you then have a command line available to you. So if you know how to work with the command line, that opens up, yeah, a lot of different ways, or a lot of possibilities, I should say. From that command line, you can invoke other tools, right? And it doesn't matter in which language that tool is written in, whether it's Python or C or JavaScript or R or Go or Rust, the command line doesn't care.

But then here's what really makes the command line shine, is that it has this mechanism of connecting these tools together. So whenever a command line tool produces output, right, it writes to standard output, then you can pipe that to another tool, yeah, which then, assuming it would then read from standard input. And so you can imagine that if you have a collection of command line tools that each do one thing and do it really well, right, that's also part of the Unix philosophy, then you can stitch those together and very quickly and iteratively come to a solution.

So it really stimulates this interactive nature. You have a very short feedback loop, right, which is kind of similar if you're working from the Python console or the R console. But then now the command line is very, very, and as you can notice, I can talk about this all day, it's so much closer to the file system. So interacting with files and directories becomes a lot quicker. And that's, so when you're in this ad hoc mode, you're just starting, you know, with this new data set that you want to get to know better. That's where I often use the command line. And you know, the book, I guess I could also refer you to the book, because in that first chapter, there's probably some more reasons. So let's leave it here with this.

But then here's what really makes the command line shine, is that it has this mechanism of connecting these tools together.

Yeah, I think the thing about the command line for me is that I don't always know what I'm doing. I always feel a little bit worried about the code that I am copying and pasting into it. And sometimes when I go to install something, software, website, that's the instructions. It's like, use the command line to install it. And I'm always just slightly worried. I'm always like, Googling, what is this command going to do in the background? So I think it would be really great to actually practice and have somebody explaining to me what's going on.

Yeah. Can I add two things to this? Sure. Can I add two things to this? If you're really worried of ruining your system, right? If you're, if you're worried that you're going to break anything from your laptop, there is the possibility of doing this in a virtual environment or a Docker container. So that might help, right? Mentally speaking, right? And I've also noticed since the command line has been around for so long, over 50 years, your favorite large language model is probably pretty good at helping you either producing a command, or if you have a command that you're not exactly sure what it does, interpret it for you. So yeah, that might be helpful too.

Yeah, I have been actually like, since I switched back to Mac, and Mac is no longer bash in the terminal, it's now zsh or zshell. I've been doing a lot of like, is this bash command going to work in zshell? And translating back and forth. All right, well, let's hop back into some more questions and see how many we can get through because they're great. Don't forget, you can go in Slido and ask questions and also vote for them.

Focus time and productivity tips

So a popular question is Rachel's question, actually. Rachel, did you want to ask that? Sure. Thank you so much for joining us today. When you were talking about how quickly you were writing your books, I was thinking like, there's probably so many things that a lot of us have in the back of our mind of things we want to do, or maybe books we'd want to write. And I was wondering, like, what kind of focus time rules do you need for yourself? Or do you have any focus time tips for us?

Right, yeah, that's a tricky one. Because what may work for me may not work for someone else. And I also know that some things that have worked for me only worked for a short amount of time. So I had this period, I believe it was half a year, where I woke up every morning at 5am. I had someone else, an accountability buddy, he was working on something completely unrelated. But we were both in a Zoom call, sharing our screens, and then just working. And having a fixed period really worked well for me, especially with having two kids. And especially when no one is yet awake.

So when it comes to time that has worked for me, I was lucky enough that I, you know, had the occasional weekend day to work on a chapter, especially when I was behind some deadline. Deadlines definitely are your friend. If you are taking on writing a book or a similar endeavor, one thing that I can recommend is that you set deadlines for yourself.

Yeah, I will echo the necessity of an accountability buddy, somebody who can just get on Zoom with you and not say anything, just be muted, but be there present with you to get work done. And I actually do that with coworkers sometimes. I'm like, hey, can I put an hour on our calendars tomorrow morning so we can just get stuff done together? And it's always super productive, highly recommend. And if you are Googling about the sort of psychology behind why that's helpful, you can Google body doubling, a weird term for something that is very normal.

What is DevRel and how to get into it

Okay, next question. This is one I'm curious about as well. We have lots of questions about DevRel. What is DevRel? How did you get here? How did you decide that you wanted to be in DevRel and sort of like your journey from the machine learning engineering space into developer relations? Wow. Okay, so that's like four questions in one. Let's start with what is DevRel?

What is DevRel? I think DevRel means something different at every company. I think if you, similar to data science, right? If you ask 12 people, you get 13 definitions back. And so DevRel for me at Posit, it means a couple of things. It means that I get to advocate Posit's polyglot data science mission. I very often create documentation or videos or other resources. I contribute to open source, whether that's to packages being developed within Posit, like plotnine, or the broader PyData ecosystem. I get to interact with the community and, you know, every now and then give a talk or host a workshop. So that's how I view developer relations. You're sort of in between users and developers. But now, since I also want to interact with the larger PyData ecosystem, I'm also in between Posit's developers and other developers, package maintainers. So, yeah. I think the blessing and the curse of this role is that there's just so much to do.

Yeah, I think that DevRel can mean different things in different companies based on what that company provides. So is it software? Is it open source? There's all kinds of different types of DevRel. I wanted to sort of tag on to that question, the rest of the anonymous question, was what are some of your biggest tips for applying and getting noticed in this space or just establishing yourself in this space? And Jeroen and I have talked about this a lot, actually. So let's dive into that.

Yeah. So getting noticed. That's already the advice. You need to allow others to notice you. And one way you can do that is to work out in the open, work out loud, as we said, Libby, right? So related to this, right? And you can do this in various ways. You can post regularly on your favorite social media platform, or you can keep a personal blog, right? You can broadcast it, or you can talk to individual people. Say like, hey, I've been working on this. What do you think? Or I found this. So related to this is try and see if you can already work with the people that you want as colleagues.

So to join Posit, and I'll share my secret, this took close to a year of getting this job. And the reason why it took so long is there was never a formal vacancy, and I never formally applied to this. What I did instead here is at a certain point there was a contest for plotnine, right? A data visualization package being developed at Posit, and I've been a fan of plotnine for a long time, and I knew like, okay, I've got to do my best here. I became a runner-up, which was great, and somehow I got the opportunity to talk about this at PyData Amsterdam, and what I learned later is that it's these little things that get noticed, and people start hearing your name more and more often.

Now another thing that I did is I gave a workshop about plotnine at PyData New York, and what I did is I said to Michael Chow, he is the whole reason why I became interested in Posit about a year ago, he had sent me a message about something. I asked Michael, like, hey, would you like to join me on this workshop? So now it's no longer just me representing Xomnia, it's also at this workshop there is Michael representing Posit, and it's these little things that definitely, well,

didn't hurt, and so when the time came that there was actually a spot at the developer relations team, at that point, well, as I've been told, the team, well, my manager, he was actually, yeah, he knew that it had to be me. I mean, it sounds weird saying this, and believe me, I am incredibly lucky and happy the way it all turned out, and just to give you some counterexamples, I have applied many times at other companies that didn't work out. Haven't we all? Yes, yes. So, I mean, that is something that you should realize as well, is that it can be very difficult. The hiring market right now is crazy.

Yeah, yeah. So, which is perhaps even more of an argument of taking a different route, yeah, instead of looking for vacancies, applying over and over again, driving yourself nuts, and there are more examples of this, right, and Libby, you know even more examples, and yeah. Yeah, I tend to collect people in my life who have done similar things that Jeroen's talking about, which is like just doing what you really enjoy doing and what you're really good at out loud, right, so that people can see them, and when I say out loud, I mean on social media, on the internet, on Zoom, organizing things, or just sharing your work, and that can be really nerve-wracking. It's not an easy thing for me to do. Every single time I put myself out there, I'm very nervous. I wanted to shout out Michael Chowk because he's here on the call today. Hey, Michael. Hey, Michael. Wave at your camera. It's nice to see you. All right, let's ask another question from the Slido. Let me take a look. What do we have here? Let's see.

I have a question that is very, very long, and I don't know if I'm going to be able to read it and get it all correct. I think it's a little too technical for us at the moment. I am going to skip down here and ask you something that is absolutely coming up a lot in the chat and everywhere, which is just, can you talk about your video call setup because you look amazing? I told Jeroen this when we were getting ready for the Hangout, that his camera setup and lighting is so good. Thank you. Thank you. Yeah, part of it is, of course, genetics. No, I'm joking.

Video call setup

So this is because I had my training company, Data Science Workshops, and the pandemic hit us. So all of a sudden, I had to give all my trainings online, and that's when I decided to invest in some serious hardware. So I have, as a camera, I have a Sony Alpha 6600 with a 16mm lens. I have this, what do you call it again? This setup, a prompt. I have a, yeah, right? You call it a prompt? I don't know. A teleprompter. You call it a teleprompter. Oh, yeah, yeah. Yeah, where you have a screen laying flat, and then you have a mirror that's diagonal, which reflects the screen partially and also lets in light. So I can look straight at you.

I have a Rode VideoMic NTG, which is not the fanciest microphone out there. We talked about this, Libby, as well. You have a very fancy mic as well, which makes you sound really good. Do I sound great, everybody? I have a shore mic. Yes. Yes, you do. And then I have three key lights from Elgato. Yeah, that's about it. That's about it. Yeah. And I use, on the software side, I use OBS. Ah, I have heard lots of people playing around with OBS. I think it's really, really cool. I use my regular Mac camera because I'm afraid if I use anything else, it's not going to work. And I just do plug and play for my micro, so I'm afraid it's not going to work.

O'Reilly book covers

All right. I have a great question, and I've always wanted to ask this one to whoever asked this anonymous question. I'd love to know who you are, which is, does O'Reilly require authors to have an animal on the cover of their books? And if they do require it, do you get to pick which animal it is?

The answer to the first question is yes. They do, okay. That's part of their series, their animal series. O'Reilly has other series as well. The Head First collection is different from this, but a lot of their technical books are in the animal collection. And no, you do not get to pick your own animal. Maybe, I think maybe Hallie Wickham had a say in this animal. It's a particular bird from New Zealand, but everybody else, every other author is, as far as I know, not allowed to pick one.

Many people ask me like, hey, why is there no polar bear on the cover of this Polar's book? And that's because, I think it was already in 2009 where there was a book, Information Architecture, came out with a polar bear. And once you have a polar bear on a book, no other book can use that. But that's okay. Thijs and I, we are very, very happy with our Iberian lynx. So. It's very cute. Yeah. I really like it. You guys got lucky.

I feel like I would want to suggest at least a few options. It's always so interesting to me seeing what pops up. I have a question from Kylie, and I wonder if Kylie is around so that there she is, and she can ask.

Command line skills and polyglot data science

There we go. Okay. Sorry. It was giving me messages that I couldn't unmute. Hi, Jeroen. I am based in Amsterdam, so not too far away. And I was just wondering if, you know, you've talked about using the command line, and a lot of people are a bit afraid of it, but I wonder if the skills or the features of using the command line can be ported to other coding infrastructure or settings. So, like, if you use the command line a lot but then actually are more comfortable using RStudio or something or VS Code, are there sort of skills that the command line gives you that you can take to those other IDEs? Yeah.

Good question. So, the way you work with the command line and its many tools is that most of those tools do only a very small thing. Think of them like functions. And so, you start to become, I've noticed that I've become better at decomposing the problem, this big problem, into smaller problems. And that is always helpful to be able to do no matter what language you use. And it's important to know that it's not a binary thing to use the command line. I very often use a number of tools at the same time and only use the command line for a very small portion of the problem to solve it. So, you'll become better at recognizing what kind of tools in my tool belt can I use in order to solve this particular subproblem well. And then move on to the next one. Instead of forcing yourself to do everything in a single language, which sounds great, but it's not always feasible, you'll be able to use the tool that gets the job done. So, composability, right? Or actually decomposability, if that's a word. Splitting a big problem into smaller ones. And then also identifying which tool you can use for that. I think that's, those are two skills that you might, where you might benefit from when using, say, R or Python. Yeah. I feel like it's sort of a gateway into polyglot data science, which is, I know, something that you're really passionate about. Did you want to talk a little bit about why that's important to you?

Yeah. So, again, it's the idea that we have all these tools out there, right? Whether they are small command line tools, or big software packages, or programming languages. I think it's important to not constrain yourself to a single tool, because every tool has its strengths, right? And being able to harness those strengths, right? To borrow some functionality from another language into the main language where you're working in, that's a superpower, right? Imagine that you are working in R, you have an entire analysis in R, but then there's this specific algorithm which is only implemented in Python. What do you do? Do you need to rewrite your entire analysis to Python? No. There are various ways in which you can, you know, combine the two. reticulate comes into play, for example. That's a package that you can use to incorporate Python into your R script. And there are lots of other ways in which you combine these things. Of course, also using the Unix command line. But I think it makes you, on a whole, more flexible. I think so, too. Yeah, if you're comfortable using multiple tools.

To borrow some functionality from another language into the main language where you're working in, that's a superpower, right?

I think so, too. And I think that there are a lot of things to be said about just expanding your horizons and realizing what's possible, right? So, the things that were possible to you before get wider and wider the more tools you sort of put inside of your toolbox. And I was kind of wondering if there was any advice you would give any people who are looking into expanding their language tool set. Would you start with another language or would you start with incorporating another tool set into what you're doing already? Like adding Python into R or adding a little bit of R into Python. Or would you recommend taking a Python course if they're an R user or vice versa?

Yeah. Oh, wow. That's a very good question. And not an easy one. It depends a little bit on what you want to accomplish. If it is just about using this one algorithm that's implemented as a package in that other language, then, you know, go for it. I think having a clear problem to solve is always better than just say, you know what, I'm going to learn Python, start to finish, and then read a whole bunch of books, follow courses, but then never actually implement it. I think it's really important that you have an actual problem to work on. I totally agree. I'm so glad that's your answer. Because I have done the same thing, right? Like I would never have gone towards JavaScript to try to learn JavaScript. If I did not have a problem to solve, if I didn't have to have a problem to solve, if I didn't have to figure out how to use jQuery when I was working in Shiny, and if I didn't have to figure out how to use Google Apps Scripts, which is server side JavaScript, but I had specific problems to solve and I don't think I would have gone end to end to try to add that to my toolkit otherwise. So, that is a fantastic answer. There's a great anonymous question that's in the Slido right now. And don't forget, everybody, you can go in the Slido and vote for questions. And it's looking back, are there any sort of pivotal moments or influential people who really significantly shaped your path in data? Wow, that's a heavy question.

Pivotal people and moments

So, where do you start with this? I owe a lot to my thesis supervisor, Erik Posma, who was my supervisor for not only my PhD thesis, but also my master thesis and before that my bachelor's thesis. We worked together for eight years. So, of course, I owe a lot to him. But then there are also people that I have known for a very short amount of time, or interacted with for only a short period, that just did this one thing. For example, I was a visiting researcher at Cambridge University in the UK. And there I became friends with Ferenc Huzar. And he is the one who introduced me to this startup in New York City. And if he hadn't done that, I wouldn't have gone to New York City. I would have never met Jared Lander, who invited me to speak at his meetup. I would have never even dreamt of the idea of writing a book. And there's this. So, it's very hard to point this to one person. But over the years, there have been many interactions that I can, you know, that really

steered my trajectory of some sorts. Michael Chow is yet another one. If he hadn't sent me that message a year ago to discuss a blog post about plotnine, we probably wouldn't be colleagues at this moment. So, yeah, yeah, a lot of different people.

I love that. Michael, do you want to weigh in?

I feel like, too, I mean, I think this goes back to what you said earlier about putting your work, like, working in the public, that, like, on the one hand, it's kind of serendipity that I reached out. On the other hand, it's not because you put out so much public work. Like, I found your room translated part of R for DS, R for Data Science, to plotnine. So, all the visualization to use plotnine and Python. And that's, like, an incredible bit of public work that just, like, I had to reach out. So, I think serendipity on both ends, like, you're putting out public work, was just really inspirational.

Thanks, Michael. And I think there's a term for this. It's your luck surface area that you have to increase. I love it, yes.

Yeah. It reminds me of a saying I've heard, which is, luck needs to find you working, which is, like, you better be, like, already doing the stuff, so that when the opportunities do pop up, you're in a place to take them.

Open source beginnings

All right. We have about 10 minutes left. I wanted to sneak in a question that was a personal question, if you wanted to answer it really quickly, which is, how did you get into open source package development and maintaining? Because you're a maintainer, right? An open source maintainer, or you have been? I wouldn't call myself a maintainer.

Okay. I might be wrong. I've built my own projects, my own open source projects, a couple of R packages, for example, and Python packages. But I wouldn't call them being maintained. I was kind of wondering, like, did you have a background in computer science, or did you come at this all from, like, just learning as you go, being a staff person? Yeah, no. So, during my PhD, it was all MATLAB, and so very much closed source. You just went like this.

Well, being a Dutch person, right, I didn't want to pay for this license myself, when after four years of university, of doing a PhD, you have to pay for this license yourself. I'm like, no, no, thank you. I went out to look for open source alternatives, and that's when I discovered Jupyter, Jupyter Notebook, or actually IPython Notebook, back in the day, and Pandas version 0.4. So, we're talking ages ago. And that's also around the time where GitHub became a thing. And I think my very first contribution was to a tool called JSON to CSV, which, as the name implies, it's a command line tool, which converts JSON into CSV.

It was written in Go, and what I needed, it needed to handle nested fields or something. So, I didn't know anything about Go, but I just, the source was available. I went in, I learned about pull requests, and, you know, eventually made a contribution. So, yeah, that's how you, you know, you learn about these things. You try, and then you stumble, and then you learn new things. But again, you had a reason to do it, and you had a goal, right, that you needed to achieve.

Writing the Polars book

I think that's great. I wanted to ask another Slido question, which is, what unexpected thing did you learn about Polars through writing your book? Did you already know everything there was to know about Polars before you started? No, but that's, on a meta level, that was one of my biggest realizations of writing the first book. You see, I thought I knew a little bit about the Unix command line. I'll get to the Polars book in a moment, but I thought I knew about this. But as I wrote this, I was like, oh, wow, there is so much to learn here. And of course, feeling like an imposter all the while, you know, after a while, I started to realize, like, hey, you don't have to know everything in order to write a book. In fact, I would argue it's even better not to learn, not to know a whole lot about the thing you're writing about.

So I discussed this with Richie Fink, the creator of Polars, and we both agree that he would probably not be the best person to write a book because he's so deep into it, right? And of course, he wants to just focus on building Polars itself. So going to the Polars book, I knew, I didn't know that much about Polars, but we were able to use it on the job, right? We had to convert this big code base from Pandas to Polars and R to Python. And I just knew that as long as we were one step ahead, we could figure things out as we go and we could write this book.

So about Polars, a couple of things, well, what I didn't expect, the little things, the little things. When you are writing a book, you take a somewhat of a higher level view and then you start to notice missing things, right? Polars is back, especially a year ago, very much under development. So you start to notice missing functions or inconsistencies or why does it work this way?

So, yeah, it's the little things. So there's so much to say here. Also from, what I also noticed is that the developers behind Polars, so that's both the Polars team, but also the core contributors and all the volunteers around it, is that they're a friendly bunch and they're very responsive. So they're mostly hanging out on the Polars Discord server, which if you're interested in Polars, I can recommend you join. And you'll find that they're, yeah, yeah. There's a link on, if you go to polarsguide.com on the homepage, there's a link to the Discord server. Okay, great. They're always very quick to help and to come up with answers to your questions.

Awesome. Well, I think that our last question that we had on Slido here probably got answered. Zach had asked, how did you decide which subject to write a book on? I think we accidentally covered that, Zach, without pulling you in. But there is a funny anonymous question that is, did you actually get kicked out of a cafe for people thinking you were hacking, or was that just a joke? No, that's just a fantasy. It's a...

Career advice

Okay. Well, I had a little bird message me in the chat and suggest to share the Frogger package to help store YAML info. So if anybody can find the Frogger package and share the link in the chat, that would be great. Thank you. Thank you, Kyle.

And I wanted to ask you, Jeroen, something that we ask everybody, which is about career advice. Is there a piece of career advice that you got that was really impactful for you, or that you would like to impart on all of us that you think will be helpful?

I think we already covered some of that. Probably, yeah. Which is work out loud and work with the people or interact with the people you want to be colleagues with. And then my third advice would be, well, my meta advice would be to take all this advice with a grain of salt. But then coming back, my advice would be to ask for things. Just ask. And you'll be surprised how often people are willing to help you.

I agree.

If you go on to... So the forward of data science command line was written by Tim O'Reilly. That's right. The Tim O'Reilly. Yeah. Do you think he came up to me and suggested he write the forward? No. I'm guessing not. I asked him. I asked him. And I was comfortable, kind of a little bit comfortable, not so much actually, doing so because I had interacted with him before about the command line. And he actually is one of four co-authors of Unix Power Tools. There's this thousand page book, which is way too heavy. And so I knew he really had an interest still in the command line.

The point is, is that I asked him. And if you go look on, again, Polarsky.com, there's a page with praise. All those people, all right, don't tell this to anyone else, but all these people I reached out to and asked if they wanted to say something. Well, of course, first read the book. And then if they liked the book, if they wanted to say something nice, right. And those are the two examples that I can think of right now. But there are so many more that if I didn't ask, then something also wouldn't happen. So that is my third and final advice when it comes to your career.

Just ask. And you'll be surprised how often people are willing to help you.

I love it, Jeroen. Thank you so much. And I will echo that. You don't get what you don't ask for. I think a lot of times in my past lives, I have assumed that my work would speak for itself, which is absolutely not true. I see a lot of nodding heads. Rohan is nodding. Abigail is nodding. Andrew's nodding. Connor's nodding. Yep. There's a lot of people here who probably at one time thought my work will speak for itself. You speak for your work.

Think about, have you ever seen the sort of meme that's like, I'll let the data speak for itself when it cleans itself? Think about that and apply it to your career. Your work is not going to speak for you. You've got to do it. And you've probably got to do some controlling of that narrative while you're at it. Well, this was fantastic, Jeroen. I had such a good time. Thank you everybody for asking questions in Slido. I wanted to let everybody know who's coming up next week. So next week we have Jay Timmerman, Head of Data Science and AI Platforms at Biogen. So if you are up for a insightful conversation about actual real-life practical use cases of AI in a big company, come hang out with us next week. I can't wait to see you. Same time, same place. Tell your friends. Thanks for hanging out. Thank you, Jeroen. Thank you. Thank you, Libby. Thank you to everybody else. And I just wanted to say that if someone still has a question that we didn't get to answer in this show, if you Google for my name, you'll be able to find me. And feel free to reach out to me and I'll do my best to respond. Sweet. I just put his LinkedIn in the chat, everybody, so go click that link. And I'll see you next week. Bye-bye.