Resources
Nov 11, 2024 |
1h 14min
|
34 views

IIIRqueR Plenaria Hannah Frick

image: thumbnail.jpg

Transcript#

This transcript was generated automatically and may contain errors.

Sir, do you understand this translation? This means move to a new window?

Yes, probably.

Okay. Sir, where is the, is it an extended left?

Okay, we're going to the right. Yes.

I'm going to share my I'm going to go to the scenes, sorry, to share the screen.

Where is the second one?

At the window, sorry.

This one?

this one.

And you can maybe just click this one maybe. No, no.

Ah, this one.

Now, close this window because in the Plenaria This one?

Where is my image?

And the Plenaria thing was the whole whole sounding by when I receive an image.

Do we need to leave?

Yeah, yeah, yeah, I need to get back to this.

I think the click is sort of massively delayed right now. I promise I can fix it.

Oh, we're not sharing it?

Okay, so we're sharing the desktop?

This is yours.

For our English speakers. The translation.

It's late at night. So maybe transport is a bit busy.

Okay, so good afternoon everyone. It's a pleasure for me to present our last at not least Plenaria speaker of the Congress.

The policy in the United Kingdom. She is PSD in statistical psychometrics in University of Austria. In the University of London, I guess. She's also a founder of the She's going to present us What's new with tinymobiles?

Publicatory mic check. Does that sound good?

Excellent. Well, thank you very much for the introduction and thanks to the whole organizing team for having me. I'm delighted to be here.

Yes, I do work on the tinymobiles team and this is what I'm going to be talking about. I am a statistician by training. Now my title is software engineer and I'm going to be walking down the line between math and machine learning and software engineering in this talk as well.

And once you leave academia you get the joy of presenting work that you haven't done. I've done some of the work that you saw, but this is work by the tinymobiles team and these are the organisms. So a lot of the work has been done by these three guys alongside me.

This is Anil. This is Max, our available speaker. This is Simon. You will see their names scattered throughout the talk.

What is tinymobiles

Before we jump into what is new with tinymobiles, I'm going to give us at least one slide to say what is tinymobiles. So it is the framework for statistical modeling and machine learning in R.

And it is a framework that uses tinyverse principles. So the same ideas that have been used in tinyverse are now being learned and used in design ideas of how functions should work, how you should be able to interact with them. All of these things that have made the tinyverse popular are the same design ideas that we try to apply to what I consider the heart of our modeling statistics.

It's still sort of like the crafting base. And essentially if you feel at home with the tinyverse, you should feel at home with tinymobiles as well.

What's new and released

And when I got asked to talk about what is new, the usual interpretation of what is new is what is new and has been released. And I'm going to be talking about this, but I'm going to be talking about this more briefly than the second part here, which is what is new and in progress. So you're getting a little bit of work in progress.

And part of the reason why I've decided to keep new and released a little bit more contained here is that we most of these big releases of the year happened in the first half of the year, and then we talked about them in various places. Amongst them, PositCons in August, and the talk videos from PositCons are now online since last week. So they're all on YouTube, including ours. We had a session of tinymodels talks with plenty of others, and I do believe that the quality of these talks at PositCons was really good. So if you're curious, I encourage you to go check out those videos, and I'll give you sort of like a pull-out, a tinymodels session. In a minute, you can decide whether that's of your interest or if you want to try some other ones.

So the thing that probably took the longest out of these releases is adding support for timed event data to the entire framework. So timed event data, as the name suggests, has like two aspects of it. The timed event and the event itself. And that event may or may not have happened yet when you're like, but I don't want to analyze stuff. If it hasn't happened, that's a censored observation. It's not a missing observation. It's incomplete. But you still want to make use of that data.

So for the stats people in the room, it's pretty obvious that the methodology to deal with both an outcome that has two aspects, and there's something involved. This methodology is survival analysis. For people who are not so familiar with this, the name survival analysis usually does something that's a circle like medical research and that's not applicable to me. And I've been trying to tell people this year in particular that it's timed event data and then methodology for that. And that event can be really a lot of things. Yes, it can be data in a medical context, but it can also be like time until your pet gets adopted in the shelter. So pretty broad applications and we've wired our support for it throughout the framework with its own mode, regression, and I got to talk about the general why and what in more of an introductory talk on data and models. And then that's it. A follow up talk right after me talking about how to measure predictive performance for these types of models. So if that is data you may encounter and you want to use tiny models, these are the two talks to check out.

My colleague Simon talked about a very different topic. He talked about fair machine learning. And the tiny models does include several metrics for fair machine learning. And there isn't like one silver bullet, one metric that answers all your questions because there just isn't one. So we also did not try to implement one. But rather Simon and other colleagues at Posit spent almost a year, I think, sort of catching up on literature on other implementations and then deciding on what to implement for tiny models. And they've chosen it to have metrics that support you when you're thinking about a problem rather than presenting you a too easy and not real solution. And he does a really nice job in situating that research and questions in his talk on fair machine learning.

And there isn't like one silver bullet, one metric that answers all your questions because there just isn't one.

And last but not least is Emile's work on making predictions happen in databases. So if you're working in a context where you make AI a model and you have loads of data in your database, it's nice to not have to pull that data into R to make a prediction and then write it in R, but that you can do it in your database first. And that is

what the tiny predict package was doing, but it was doing it for a personal model. So in the easy case, if you have a linear model, you know your coefficients, you can write a SQL or other expression that multiplies your vector coefficients with your predictor values, and you get your prediction.

And that can happen on a database. But that wasn't including any of the preprocessing. And Emile has gone through the work of trying to make this work for tiny models of workflows so the objects can include preprocessing and the fitted model. The package is called Orbital if you want to check it out, and the talk is also linked in the slides. My slides are online. I'm not sure if you can read it, but they're on my GitHub. You know, get my GitHub profile. It's just a trick. I'm sure you'll find it, but it's sort of the retro 2024 dash 3 RPR.

So that is what we've been working on earlier in the year. That is our desktop videos and blog posts and whatnot. If any of this catches your fancy, go check it out. And I'm going to very smoothly segue into the new vision in progress. Some things get a lot more... less polished. Let's put it that way.

Post-processing

And the first thing I want to talk about in that moment where we're actively working on things is the topic of post-processing. And I've decided to slap the subtitle on. That's what you do after you've done fitting your model. And you do it to your model predictions, not to your data. The reason usually being you're trying to squeeze a little bit more performance out of it, trying to make the predictions better after you figure out how to make the predictions.

But let's look at an example. So I brought some delivery data. That's just how long it takes to get something delivered. Since I talked about censored data earlier, this is all uncensored. This is the actual time until the event. Just looking at the time now, we're keeping things easy in this example. So time to delivery is our response. We've got a couple of predictors. We're doing the usual tiny models thing where we split it and try to do testing. And we're also preparing something for 10-fold cross-validation.

And then we're going to pick a model, and I'm going to deliberately make the model really bad to show you why you might want to test your predictions. So this is a boosted tree with a glorious number of three trees. So you're not going to get something very performant, but you get a model. We're not doing any pre-processing on the predictors. We are just showing it straight.

We're going to resample that model, put it on the 10-fold cross-validation. We're going to get some metrics out of that, and we are going to save those predictions.

We have metrics, so we might as well look at them. And we get an R-squared of like 0.8 something, which is not too terrible. But as always, you've got to remember what your metric is actually measuring. And R-squared makes the correlation. It doesn't tell you how close your prediction is to your observed value. That is a different topic. It's calibration.

Why not look at it as a thought? This is done with the probably package. You can see that for higher actual observed values, the model quite severely under-predicts them. This is not a new model. I mean, deliberately so, but if you're trying your hardest to get a good model and that's your result, things are challenging.

And R-squared makes the correlation. It doesn't tell you how close your prediction is to your observed value. That is a different topic. It's calibration.

And one of the things that you can do with this is calibration, and that's what we're going to be doing. Generally, why would you want to do any of the post-processing? Usually the motivation is to get better predictions out of it with sort of adding a hit-stack, and calibration is another model on top of that. Or sometimes you simply want to better satisfy distributional limitations. You might have a range that your predictions can't be outside of, something similar.

And currently, you can do that, and things keep moving from it. It is not necessarily the immediate way. You can just do straight-up decline manipulations on it, if you're just thresholding predictive probabilities into hard classes with something other than 0.5. Or you could use the PROBABLY package which has calibration compliments in it, from America.

That is all okay, but what we are now doing is integrating it into the workflows objects. So a workflow in kaiju models is the thing that contains, currently, your pre-processing instructions and your model instructions. The idea being that putting them together makes it easy to cross-validate them together, and you're not creating data that you can pre-process. Now we are adding post-processing to go into that object, so that you can fit the whole thing. Do fit-me samples on the whole thing.

Introducing the Tailor package

We have a new package on the block for that kind of stuff. Meet Taylor. So, the Taylor package is the thing that connects stuff in PROBABLY and DeFi, so to say, with the workflow.

If you think of recipes for pre-processing, think of Taylor for post-processing. That's the mental model that we're borrowing from. If you have one for recipes, you're a good bit of the way there for Taylor.

Similarities and differences in that mental model. Recipes works on the training data because we're doing pre-processing. Taylor works on the model predictions because we're doing post-processing. But you both initialize an object for recipes and services, and then you add more specifications to it. Steps to do. Recipe steps. And in Taylor, those steps are adjustments to do.

And then you sort of have not only the nature's fault. There's nothing more, just dead space. And then you have a part where you estimate something. And then the part where you execute some things, often false predict. And there are false fit and predict for Taylor. I assume you're somewhat familiar with recipes, so that is the corresponding parts to practice it. I think those are harder to learn than fit and predict. But that is a very brief intro to Taylor.

And let's use it. Let's use it in the workbook. We need to call library Taylor. Nothing happens with that. But then we can make a Taylor object. We initialize it with call to Taylor like we would with recipe. Recipe. Brackets up and close. And then we're adding the additional instructions of what to do with our model predictions. In this case here, I'm adding an adjustment that is a numeric calibration. So that's off the wall predictions. You're going to get calibrated.

And we have this Taylor object that I knew would have a recipe object. And like we would add a recipe to a workflow with the add recipe function, there is now also a add Taylor function. So hopefully all of your intimidations that you've built up with tiny models already are working and you can be like, yeah, I'm mapping this knowledge to a new part.

So we can add the Taylor here. And now we have a workflow that has instructions for pre-processing, for the model fit, and for post-processing. And then we can send that thing through fit resamples, like we did before. The only thing that's changed here is the object that goes into it. We're still saving the predictions so that I can do the original plot. And you can see, still not perfect, but we're getting closer to that diagonal line. Still struggling with predictions for large observed values, but we're getting it. So success.

What Tailor can and can't do yet

I think this is work in progress, so let's talk about what it can do and what it can't do yet. So what is implemented in Taylor as options for post-processing is what we expect to be the most useful and commonly requested ones. So if you have something that you need that's not on this list, I will show the site where to get the feedback. Let us know. But we're hoping we're covering a lot of bases with this already.

You can calibrate probabilities. You can either pick a different threshold or use equivocal signs if you're turning your probabilities into hard class predictions. And for numeric outcomes, you can calibrate them or tweak the range.

And we have implemented both Taylor itself. There's support for it in workflows, so that you can say add Taylor. And as you've seen, fit with samples and tune works with this already. And then there's like this unassuming line of support for our sample for sundry samples. That concerns calibration. So calibration essentially fits another model on your model predictions as an input and your true outcome as an outcome.

And if you fit yet another model, you want fresh data, unseen data. You don't want to risk overfitting by just adding a model on top of a model on top of a model and fine tuning too much on each range of data. So what happens underneath the hood is that we hold back some of your training data for calibration if you have told us in your specification that that is what you want to be doing. And how we do that is implemented in our sample and we have that for cross-validation and booster samples and there are some more to go including temporal datasets.

And yeah, sort of that is part of what's yet to come. And the other thing that we still need to wire up in this is that you can tune with all of TUNE and not just say fit pre-samples but you could say TUNE on a workflow that has pre-processing on specification and pre-processing info. And if you care about this topic and want to give us feedback that is very welcome. The best place to do this is by issues on GitHub on the Taylor repository. So in the tidy models or on GitHub is the Taylor repository if you've got issues leave us your feedback there.

Um yeah so that is one thing that is work in progress that is in the pipeline. It's not too far off I would say and we're just going to move a little further. The other topic that's sort of hooking in the tidy models section is support for sparsity in a better way than currently. What I call sparse tables and um sparse data in the wild would be something you encounter if you have categorical predictors with a lot of categories and you turn those into indicators or you tokenize text or you have a graph data set. You can easily end up with a lot of variables and then a lot of zeros in them. So like the information in your data is pretty sparse in that context. And people often shorten it down to like sparse data. So that's where the slogan or the name comes from. And having a lot of data with a lot of zeros instead of like going from one column of a categorical variable to like multiple, multiple columns can sometimes be a challenge in two terms. One, memory

allocation. Your data set is already big and then you can like turn one column into many, many. That's not always easy. And the other thing is speed. Because you need to work with each other.

And the way that this is typically addressed is a different data representation. Different to what? Is a solid question. So the fault is called a dense representation. We have a vector like this and we store all of the 25 values. That's a dense representation.

I don't need you to count the zeros, but you probably have spotted the non-zero values because they're only two. So if we don't want to store all 25 of them, because these two are the things that really interest us, we can try to switch to a sparse representation. And we only need five of these for that.

So we need to remember how long that vector is. So that's one for the 25. We need to save the locations for the non-zero values. That's another two here. And we need to save those non-zero values themselves. So another two. But I'm five versus the 25. And whether that's an interesting loop depends on how sparse your data is. There isn't like, this is always better. I think we all know that's no free lunch, right?

There is a little bit of a trade-off, but if you are in a situation where you do encounter a lot of zeros, it's interesting. In this complete layer, in R, we're implementing stuff. This is called matrix packet, with a capital M, that implements sparse matrices and sparse vectors in efficient matrix operations for this. So we're addressing both the memory return and the speed.

And you can make a sparse vector. I made the one that I just showed you, with the 25 values and two non-zeros. Even a great method looks pretty sparse, right? And matrices are good when you're really close to knowing if it's a model, when you actually have a model matrix. And the efficient matrix operations are for that.

But that's pretty late in the game. Usually when you have a data, you have it in a data frame for a good reason, because you have often different types of data, right?

Integrating sparsity into the tidymodels framework

So there's a little bit of a hiccup in using the matrix package directly in the high-resonance framework. So if you give us such a matrix to go fit the model, we can take it and pass it on to the modeling function. We're not crunching it, it's fine. That is what is there already.

But we have a slight problem trying to stick sparsity into tuples.

I feel like in these sort of things it's always good to remember, what are you trying to do, what are you trying not to do, and why? And the best place to start is why. So let's pull back for a moment and talk about what we would really like to have here.

Simple one, if you have something that isn't a sparse data format, we would like to preserve that throughout the framework. If you run into something that's concave or sparse data, they will materialize and fall back to the bench representation.

We want to preserve it and we want to preserve it across the framework. And the thing that carries data across IE models is a tuple. That's where my interest for putting a sparse vector into a tuple comes from.

The other thing we would like to have is to be able to make use of sparsity where it makes sense. Maybe if you're turning your categorical variable into a lot of indicator variables, you want to make use of them and you want to make sparse indicator variables. That's where we would like to be able to add sparsity.

But note that this is sparsity on a prediction of or in terms of objects, we want actually sparse vectors and we want them in a tuple.

And the last one is kind of casual, like we want to make things easy. Sounds great, but it really is like a fundamental idea of all of models. The idea of always being not easy for easy, say, but rather if you're dealing with compressed problems, we don't want you to spend all your energy and making capability on the technical bits of handling sparsity. We want you to be able to rely on value models for that and keep your focus on the actual modeling. So that's why this matters.

We want you to be able to rely on value models for that and keep your focus on the actual modeling.

And we want you to not sit like, as a small example, we don't want you to be like, oh, but I actually don't need all of the rows and the fills of this. Will it break sparsity if I take out some rows because it's not going to change anything? You just want to be able to say dplyr or mutator or dplyr or filter. You do probably. We certainly do.

That is a long-winded way of saying why do you really want sparse vectors in a tuple? Not all of them have to be sparse, but we want to be able to have some vectors to be sparse.

And as you saw, unfortunately we can't just straight up stick sparse vectors from the matrix, capital M, package into a tuple. For technical reasons, I'm not going to dive into, but you're going to be like, welcome sparse vectors.

The sparse vectors package

So that is another moment of what are we doing, what are we not doing? So sparse vectors is not intended as a replacement for the matrix package. It is our way to have sparse vectors that we can stick into tuples and we're not aiming to implement any of the efficient matrix operations that the matrix package has. So if that's what you're interested in, then go to the matrix package for matrix operations.

That said, sparse vectors is its own package, and if that sounds interesting to you, you can go use it. We won't use it, or we are using it in tidymodels, but it's its own thing and you can use it in other contexts if that fits to you.

So it allows us to do that thing that I was after, put sparse vectors into tuples, and because we want to be able to talk with the rest of the world, also it has fast conversion methods to go from a tuple with a sparse vector in it to formats of the capital and the matrix package so that you can then send that to a model fitting function, for example.

What's implemented and what's not yet

Okay, I said work in progress, I said some stuff's implemented, some stuff's not, so let's get to that part. Implemented already in recipes is that the server of everything in recipes, the recipe function itself, knows what to do with this and doesn't just smash it first and then materialize it. So that will pass through. That will pass through. It will also pass through prep and bake.

Parsing's fitXY function also does not store its sparsity, neither does predict. And in workflows, fit and predict also leave us alone. Which is really nice.

But obviously, what's not happening? So there's currently well, I could say a caveat, but really it's like the area where the work is at right now are the recipe steps.

So, whether a step touches a sparse predictor, it may or may not be sparse at least. So for some steps, that's just inherent to what you do. Like if you're sticking a vector into a PCA, you're not having something sparse there. And you will not but if you're doing sort of like the indicator variables, or other things where it might make sense to keep sparsity, the goal is to preserve that. And some of them work already and some of them don't.

So whether or not the data set that you get found at the end of your recipe preprocessing is still sparse has the answer. It depends. And it depends on what kind of steps you're doing.

And then, there is one thing that some people want to do, and some people are skeptical about, which is why I put a big baby on this. So if you want to use a formula interface, which is lovely, which is a great thing about R, and in tidymodels, that is sort of called fit on a participant model, or if you invoke that by app, using app formulas as the preprocessor in your workflow, then you're down on code paths that will eventually lead you to model.matrix from base R. And that one uses dense data.

And unless we write something that replaces model.matrix, that will be the case. So maybe we will have this in special cases. It will, I'm pretty sure it will not be a it will always work kind of situation because of the nature of things. But I don't want to send you off to the side like, it's not going to happen. This is very much a work in progress.

But we are pretty excited about this because of the possibilities that this opens up for sparsity in preprocessing maybe. So far, sparsity is mainly relevant for model fitting, and now there's more happening on the preprocessing side. So we don't quite know yet where that will lead us, but we're pretty excited to find out.

This is the feedback side. If you have feedback on sparsity in general, and sparse vectors in particular, put the issue with the sparse vectors repository. That is not under tidymodels, that is under Erlang. Presumably if you search it, you'll find it. If you have feedback on sparsity in context, go to the corresponding repository, so if it's sparsity in preprocessing, go to this. Leave us a note with a key in here.

Better errors

And now we have post-processing sparsity tables. And I brought you a third one because fleeting is fun, right? And that third one is better errors. Not quite as big and flashy as the other topics, but more like the bread and butter of things.

And sometimes you notice. Usually when you run into an error, it's not the best moment in your life. And recipes used to be pretty tricky.

So you write your beautiful recipe. You try to fit your workflow. You get an error. You're like, okay, fine. I know my debugging strategies. I'm going to bake that data set. First, I'm going to bake it. Remove the complexity of the work. And then you try to prep it and you still get an error. Hopefully the error tells you something useful. So here it says a column already compounds new level. So new level is like, oh, yeah, I did something and you can look for that step.

And I know that people have been devising a debugging strategy where they'll comment in and out of steps to narrow it down. That is a good strategy, but wouldn't it be lovely if this thing just told you which step the problem is at? And that is what it does now.

And, you know, it's not a new feature, although that one is so helpful. You could somewhat argue it's a new feature, but often the error messages are not the thing that you can write a lovely blog post about. We do write about them, but they're sort of slotted into things.

And I've decided to put a little bit of a variation on the keynote topics then, usually. So I have given you an exciting work in progress. But I also want to talk about the other

work that we do. And it doesn't always result in extremely helpful error changes that you can change.

So this is a little bit of a how we work kind of section of this talk. And we typically call this type of work upkeep. It is largely the same as maintenance work, but not in the sense of somebody else changes their package and breaks your manual update stuff. That falls into upkeep as well.

But upkeep also is the idea that we make an effort to regularly touch our packages, because we have quite a few of them, and sometimes you don't do new feature work on it for two years, and then you get a bug report, and you're like, why did it fix that? You open up the code base, and you're like, ooh, we are doing things differently now. And it's just, it's not because you wrote bad code in the past. It's just because the world around it has moved on.

And so we try to touch things regularly. And if you're writing your own packages, you don't have to look after the amount of packages a week. You just have to be busy and do a lot of different things. You also might not get to touch your package every week.

So the idea here is to show you what kind of tools we use to both provide users with helpful errors, and to do that in a way that lets us still stay on top of things, to not accumulate the biggest maintenance burden in all of this.

So I hope that's an interesting variation on the new inflection and work in progress.

What makes an error helpful

So what makes an error helpful? And I think you do need to tell your users what happened. That's like the basic. It'd be good to know where it happened. And the hard question is like, why did it happen? And you can't always use it as an error message, but you can sometimes be pointing them in the right direction. And sometimes you can also point them even further in the right direction by giving them like hints to further places to look at things, or if it's like a clearly typical mistake, you can be like, did you mean to do this instead?

With moderation. Don't be quick. But sometimes that's helpful. So how do we do that kind of stuff? And I do hope you can read some of it. Otherwise, I'm going to be beating out a lot of holes. How's that working? Sort of? Okay, cool. Excellent.

So I brought in a small function. I'm going to use speed here so I can see this a bit better. It has nothing to do with timing models. This is just not overload your brain with statistics. This is just about when it's time to water your plant. And that plant should be straight. Otherwise, it's going to error. Because all you do later is paste the string into the text and be like, I'll get it. That plant doesn't need to be watered yet. But you don't want to see that plant. You want to see the specific one.

But if you use that with a number, it's going to error. Plant must be a string. Okay, fair. The thing we have done so far to try to be helpful is put backticks on this here. Just to indicate that that's the argument. And that's the thing where you might want to pull up this documentation and be like, what was that? Why does this not work? So that's very minimal stuff, but it helps.

But as a maintainer, you have to remember to put those backticks there. And you have to remember that backticks are the thing how to format arguments. Or... Oh, I highlighted this. Okay. Or... you switch from an R-line abort statement, which is our version of, like, baseline stop. That's the thing that throws a turn. You can switch to CLI abort.

And CLI does a lot of things. Here, it just does the styling. Not that exciting, especially if it's just a few backticks. But the point being, these are all small changes and they add up.

Using CLI for better errors

I'm going to show you a few more things about what CLI can do. But amongst other things, it can do a lot of different styling parts. And you just have to remember to put the right tag, which is often easier than you'll remember to consistently apply format yourself. So the user list doesn't change. And that's a good thing. This just makes your own life easier.

And I said I'd show some more things that CLI can do. I'm going to change that message that you get if you succeed. What we do in here is apply CLI functions across the board, including CLI abort. So we have, like, we're basically doing string interpolation. We have our string and then we inject the value of class in there. And then we print it out and we are really good citizens and we're putting a new line at the end and we have to remember to write that new line.

If you want to do the same with CLI, you just use CLI text and do string interpolation by putting curly brackets in the value itself. So that's kind of like a matchup of the glue package and the Arlang abort. And then it does an additional thing because nothing keeps you from putting several plots in.

So it would be nice if that sentence worked for all of them. So what this does is, like, it looks clever enough to see that plants may have multiple values and if it has only one, it uses the first option here. It says that plant doesn't have to be watered like here. I'll get it. The monstera doesn't have to be watered yet. But if you put in a vector of plants, it says, I'll get it. The monstera and that other plant don't need to be watered yet.

This is not ground breaking. It's not playing with math. But it's pretty darn convenient. You get to offer your users nice English sentences without doing all the figuring out whether it's plural or not and how that pluralization should go and where it pastes in all of the objects. That's pretty nice if you do this a lot of times or if you only do it very sporadically. Because if you're not touching that often, you might go through the trouble. I feel like that is a nice thing for someone who maintains a package. The result is a nice thank you for someone who uses a package.

But it's pretty darn convenient. You get to offer your users nice English sentences without doing all the figuring out whether it's plural or not and how that pluralization should go and where it pastes in all of the objects.

I said there's a bunch more formatting options. The ones I use most regularly are the catch-all. Format this as code. Format this as a class. Format this one as a function. If you just give it the function name, it will add records and pull out semantics. What's really cool is what happens if you give it package name, colon, colon, function name. It creates depending on which IDE you're in, hyperlinks to the documentation pages. You get an error message that might say look at this function. You click to get there.

Then you can do more jazz with links. In Parsnip, we sometimes want people to look up the model specification and the object is called model for spec. That's okay, but sometimes we just want to say model specification. The error message makes it so that if you click on model specification, you land there.

If you're in Quarto slides, you're not in an interactive IDE. You don't get hyperlinks there, but you get to see how it looks if you're doing it say on a console because that's where you run your R. Then it will add the code that you could run to get to the help page. It doesn't leave you in a bug if you don't have hyperlinks in your thing, but if you can have them, it will give you hyperlinks.

Surfacing the right function in error calls

That's a little bit of CLI niceness for the error message. The other thing that we're changing throughout the framework is to pay more attention to the kind of call info that you get when you use an error because that tells you where.

We have a bunch of functions that are internal. They're not actually meant to be user tracing. We have others, and then they call them, but what happens if you call plant hair, which then in turns calls time to water, is if it throws an error, it will show error in time to water. Here that sort of means something, but not all internal functions are named with the care that you do for user-facing features. Fact of life, I would say.

To then expose your user to your internals in an environment where they are under great stress because they produced an error is maybe not the moment you want to do this. What we would like here is for that thing, if you call plant care and it throws an error, for it to say plant care and not be like, tell your user to be like, sorry, where's that coming from? If they want to go jump into a debugger, they still can if you're not keeping them on it, but a casual user might get confused with this and you can hide it pretty easily.

CLI abort has a call argument. The nifty thing that happens here is that in the function definition, we say capture the calling environment and then pass that to CLI abort. When you execute plant care and then you execute time to water, time to water is where am I being called? Oh, I'm being called in the context of plant care. Capture that and then when you error spit that one out. So now you have the user facing function as the function that throws the error.

Also not the flashiest, do you think? But helpful. You will see these changes getting rolled out across time. We sure do have a lot of error times so it is taking us a while. We've been at it for a good bit but I think we've halfway there. Once all of these changes have passed, we will get errors in this regard as well.

I am like at time, am I not? Should we think? I'm... Well, okay. I have two more things to show you. Let's do them, right? You still awake? You're doing better with this time than me, actually. Okay.

Why did it happen

So we were like what happened, where did it happen, and now we get into the meaty question of why. And this is sort of like back to the start where we're moving the plant care where we're just like bare bones time to water. It tells us it must be a string. And it's good it tells us what it must be but sometimes they're like okay, fine, I get that part but what was it if it wasn't a string? I thought it would be a string. And that's a pretty easy way to add that information.

It doesn't always save a user from having to go debug stuff more severely but it's a nice starting point to get to something like plant must be a string not a number. Where did the number come from? Well in this case you can see that but sometimes you're like hold on, why is it that? And the way to do this in CLI is to stick fonts in curly brackets and then have it in the tag of object type friendly. And what that thing does is evaluate what object type plant is and then spit out a friendly English language description of that. So 5 turns into a number and it works for a multitude of objects pretty broadly and my test to see how far I can stretch it is to take our basically hello world example of a linear model, so yes I put a linear model in the time to water function and it does tell me it must be a string not an LM object.

Good. So that's pretty helpful and it doesn't take much effort to add that. So that's a little bit of nudging you giving your user a first step on the path of like why did it happen? That doesn't always solve your problems but it's a relatively cheap and easy way to get people in the right direction.

Sometimes you want to be more explicit and have more stuff to say about how to fix a particular problem and you want to put that hell into the error message but then if you have a lot to say you run the risk of producing an error message and then nobody will read your helpful text. It's just a long text. But if you break it up a little bit and give people three bullet points your chance is much better at reaching them.

So what CLI and RLA errors can do is give you bulletisms and that happens when you not just give it a string but rather a vector. And then I say I equals and that's sort of like an informational bullet point down here. It still says error in that function and then the error message, which I have changed to switch things up at the last minute and then it has like I and it gives another hint.

And that is usually helpful if you want to point people to other functions. You use the linking of our shoulder. So where we do this for real and not just like picture parts, you will see when you need to audit them, a situation where this is helpful is if you're expecting data in a particular format, say spatial data, and you can leave a hint of saying, right, look at the help page of this particular function to convert your object from a regular data frame to an SF object or something else.

We sometimes point to other functions in the time models ecosystem for this.

Wrapping up

Time to wrap it up. I think errors are most helpful if they tell you what happened. That's the error message itself and CLI has a bunch of options for you to add, style in, string interpolation, pluralization, links and all of that jazz. It can help you be more direct where an error happened.

You don't always have to make it just a user-facing one. If your function does call out different things and you think it is helpful to know where, we're not going to say, oh, it's error in tune rig, but that's a non-server function and sometimes it might be helpful to know if that was a pre-processing error or a model fitting error or something else. But it gives you the control to say the level of abstraction that you're going to give here and which call is going to show up.

The driving question of why does this happen? You can help people by not only telling them what was supposed to happen, but also what actually did happen. So that was impartial, object-friendly and if you have any helpful hints to tell people, you can add that on informational messages or other types. There's a bit more styling in what the bullet points look like. But essentially, break up your text, add some more context if you think it will be helpful.

But essentially, break up your text, add some more context if you think it will be helpful.

And to my last slide, what if you want to know the news and I'm not there to talk to you for an hour. Announcements about new things go on to the tidyverbs blog at tidyverbs.org slash blogs. And then our column on the web is tidybubbles.org where we have a collection of articles that are more in-depth texts about various topics of statistics and modeling and what you might want to know about that in the context of tidybubbles. And with that, thank you for your attention.

Q&A

Thank you very much. How is combining languages and translations in your CLI messages? Combining translations? What happens if you want to have your message in both English and Spanish depending on the language?

That is an excellent question. And I know that there's a working group that tries to translate base R errors for new installed R in a different language. I can blink on how that translates to CLI. I'm not sure where it does. But there's work being done on this very interesting area. Maybe they haven't converged yet, but yeah.

So, should I call another question? Okay. It's more a personal question than other things that Intel has featured in this area. It's more like how your team in general, people that work in this kind of huge package that there are thousands of people using for enterprise or for academy or whatever, how do you manage to help or take decisions or what are you going to do now? What are you going to do now? Or how do you deal with it? How do you work?

I'm in a lovely situation. Well, lovely or not, but our work is public. You're not sitting in our team. But the vast majority of what we do is visible on the web in the GitLab repositories. And there will be explanations about why we made a certain decision. And the web also has documentations for us that you can take and apply to your own workings. So we have documentation online about how we write code.

There is a document of how we do code review, how we discuss changes in the code. There is a repository that has documentation on the bigger decisions, why we're going one direction with an implementation on another. That was called a tidy-up. It's sort of like a similar idea to like a pet for Quicken, where you say this is our problem. These are potential solutions. We've gone with this one for this reason. This other idea that sounds really good did not work because of.

Sometimes these things are written up after the fact, but usually those are written up before you make a decision to gather feedback. So if you're interested in how we work, look for this kind of documentation. It is really like the guideline of how we do our work. And we make it public in case it is helpful for other people doing their own work on similar or a different scale. But yes, I know that part.

Okay. There is now a question. Sorry, I didn't see you. I have a questionable or difficult question for you. Do you want to start? Okay. What do you think about the VPN and the chain or the way of your company? GPL licenses? VPN. Yes.

So I know that some packages will license GPL. By now, Tinyverse packages have been relicensed to the more permissive MIT license with the consent of all the people who contributed and had a say in the whole story for the reason of them being more permissive. Posit does not make any direct money off of our open source packages that we provide. And I am not a lawyer, disclaimer that goes with it. The conscious choice was to go with the MIT license. Like a GPL license in my understanding is that you can use it, but if you use it, you have to keep it under that license. And I think that makes sense in many situations. It wasn't the choice for this. Being more permissive with the MIT license for the Tinyverse packages. Does that help?

Okay. I have a question. I'm not completely familiar with MIT planning and the planning. It's been a while since I've been here. I've been here for about 10 years. I do this every day. I don't know what else I can tell you. As a performer, you know, there is not a full feature parity. You can do certain things in Python, you can't do in R. There are certain things in R you can't do in Python.

Coming sort of from the background of more of a computer science approach to modeling and more of a statistic approach to modeling. Like something that is bread and butter, just statisticians being able to specify interactions in Python, whereas other things like text processing has a little bit more functionality in Python. What is interesting for your problem really depends on your problem.

I very much think you can put R into production. People do it. But that also depends a little bit on your definition of production. It depends on like what your context is. If all the people around you are computer scientists or general software engineers and they don't know what to do with R, that is hard but not because of how R is, but because of the context. And if they are like, no, please use a general purpose language like Python, then that's a learning center. Do that. But it's nothing about R per se that keeps you from it.

And in the spirit of sharing work in progress, the topic of R in production is sort of one of Hadley Wickham's focus of that one moment. He's given some keynotes on it and he's working on a book called R in production and several of us are working on some more technical bits along the way because everything is there and sometimes it might not be as smooth as it could be, so we're trying to make that experience other than lesser a little easier. But it's by no means impossible. So there will be like technical infrastructure and social infrastructure and how to talk about this with like improved packages in a book that Hadley is writing. So if you look for Hadley's work, R in production is the key word. You can follow along on that project too.

Well, in this case, so say thanks to see you

... ...