Transcript#

This transcript was generated automatically and may contain errors.

Well hello and welcome back to another awesome edition of the Shiny Developer Series. My name is Eric Nance and as always thank you so much for tuning in. And if you like what you're going to be hearing in this episode, which I know you will, you'll also want to check out the back catalog that we have on the Shiny Dev Series YouTube channel. And also I've been having a little fun doing some live streaming lately and you'll see the recordings of those as well.

So as you might know I like to do a little continuity here on the Dev Series and last episode we had a great conversation with a member of the Shiny team who recently joined Nick Strayer. He showed some of the awesome things he's been doing in particular with the new Shiny App Stories section. And if you saw in the example application he shared, it had some great showcases of some new functionality that's been added in Shiny as of this year. And one of those features that really got a lot of attention we're actually going to be talking about in-depth with the architect of much of that work.

So it is my pleasure to welcome to the Shiny Dev Series for the very first time RStudio software engineer Carson Sievert. So Carson, thank you so much for joining me today.

Yeah, great to be here. Thanks for the invite. Just want to say that I've really enjoyed all the service you've been providing with these videos. I think it's great that you're really putting a highlight on Shiny developer experience. So it's great to have this as an outlet to share our work.

Oh, absolutely. This is one of the funnest things I do these days. And you and I have had a lot of conversations in previous conferences and I never was able to snag you on the podcast. But hey, we're here now. So I'm going to take full advantage of this. And for our listeners who may not be as familiar, maybe you could start off with sharing a bit about how you actually ended up joining RStudio and a little bit about the journey to arrive on the Shiny team.

Carson's background and path to the Shiny team

Yeah, so some viewers might know my work on the Plotly R package, which I started all the way back as a PhD student at Iowa State. I did a PhD in statistics there and had the great fortune of working with Heidi Hoffman and Dai Cook on my dissertation, who also advised Hadley Wickham. So I had already had an interest in interactive graphics, but being able to work with them was a great experience and was very fortunate to kind of almost have it fall into my lap that it just kind of made sense for me to start working on the Plotly R package. And I was fortunate enough to get paid for some of my time working on that, as well as writing about it a little bit for my PhD dissertation.

So stars aligned for me pretty well in grad school to be able to work on that project and then continue working on that project after graduating. And so I did that pretty close to full time for a couple years after graduating and, you know, really enjoyed that project. And then it just kind of, you know, worked into a place where, you know, there was kind of mutual interest in me joining RStudio and RStudio having me join and potentially, you know, keep the Plotly project going. And, you know, also had the opportunity to learn more about the whole, like, Shiny ecosystem, the R Markdown ecosystem, and sort of grow my skills as an R developer.

So yeah, that's been a really sort of seamless transition from those experiences in grad school and working on the Plotly project. And now I'm becoming more and more focused on sort of the whole front-end experience of developing Shiny apps. So yeah, since joining, I've sort of, you know, broadened my scope from just focusing on, like, highly interactive data visualizations to, you know, how do we bring more modern front-end web development to Shiny?

Yeah, it's like a natural coupling, in my opinion. And I can speak, I'm sure for many in the community, Plotly has absolutely been a huge transformative kind of package, so to speak, in giving my outputs, whether they're in a Shiny app or even an interactive report, that extra bit that just takes it to such greater heights with the stakeholders I share these with. And I even have a little fun learning a little bit under the hood. I actually remember attending one of your workshops at a previous R Pharma, where you dived into a lot of the Plotly details. And I've been taking those learnings to this day, along with the awesome book that you authored, which I'll have a link to in the resources below.

Highly recommend to check that out if you want to get into the hood of Plotly. And we'll even touch on that a little bit, I think, towards the end as well. So, of course, thank you for making that. Like I said, it's been transformative to a lot of the work I do in my day job and even outside of that.

Introduction to bslib and thematic

But that's really not all you do, for sure, these days. And I'm thinking back to RStudio Global earlier this year. And as kind of being involved in some of the community stuff with Shiny in the conference, and I started networking with others attending, there was a very common theme of a new development that got a lot of praise, a lot of excitement from myself included. And that is the awesome work that you've been doing with bslib and thematic. And I'd love for you to take our listeners through kind of the journey to build those. And also, it looks like you have a great demo you want to share with us on how you can get started with bslib effectively and really customize your applications in a seamless way. So, let's dive into that.

All right. Yeah. So, really, the more ambitious of the two projects, you know, you mentioned bslib and thematic. bslib, I guess, is one of the more ambitious of the two. Both of them are sort of designed to give you the tools to make custom theming easier than it has been in the past.

So, bslib is a pretty ambitious project in the sense that it completely rethinks the model for how we've traditionally implemented custom theming in the past, where if you've ever tried to change some of the default styles of Shiny, you might have written like custom CSS and HTML by hand. Right. Where bslib is trying to do as much work as possible to make custom theming as easy as possible. So, the sort of technology that makes this possible is something called SASS. It's a programming language that is designed to make writing CSS easier. And in particular, you know, really provides a nice infrastructure for sort of scaffolding your CSS in a way that you can allow somebody else to easily change like a main or primary color on a page and have that influence like hundreds of CSS rules on the page.

really provides a nice infrastructure for sort of scaffolding your CSS in a way that you can allow somebody else to easily change like a main or primary color on a page and have that influence like hundreds of CSS rules on the page.

So, we'll see some examples of that in a second, but that's kind of like the primary thing that has motivated all of this work, not only bslib, but thematic as well. So, bslib is really all about customizing the bootstrap CSS that goes on the page to really any Shiny app or markdown document. It's sort of providing that foundation to customize that CSS. Whereas thematic is more aimed specifically at static R plots generated by an R graphics device, which, you know, uses a different styling API than CSS. So, it's fundamentally kind of a different problem, which is why these are two separate packages. You can find both of them at rstudio.github.io slash bslib or slash thematic.

Yeah. Well, definitely have links to all that in the show notes. Yeah. So, you'll even see in the sort of hello world example for thematic with Shiny, you'll notice that there's some bslib code in this example Shiny application that gives some like custom main colors to bslib to influence bootstrap CSS. It also brings in some custom fonts. And that will end up say, you know, customizing the look of a tab set panel in a Shiny application. And influence like pretty much anything that's styled through CSS on a Shiny application.

But without thematic, you'll get a result like this where you'll see, you know, kind of the usual default styling that you would get with ggplot2 unless you're doing something custom. But the main idea of thematic package is with essentially one line of code, you can say, you know, throughout the life cycle of this application, all of the plot outputs in this Shiny application should use styling defaults that derive from the CSS on the page.

Interesting. So, essentially, just by enabling thematic, now your static R plots will derive their global styling defaults from CSS. That right there is just amazing in and of itself. Because there are so many times that I did do some custom building of my CSS in a few major applications at the day job. But then the plots would stick out like a sore thumb. Because I wasn't an expert in styling them individually. But now you ease that transition into making the whole app have this kind of cohesive UX and UI experience.

Yeah. And I would say that that's particularly relevant when you're going for sort of a dark mode kind of look. Thematic, you know, goes through a lot of trouble to make sure that, you know, all of the colors are kind of inverted in a sensible way. But then it also does a lot of hard work to make it so that you can do something like leverage bslib's ability to automatically import font files for any Google font that you want. So, with bslib, you can say, you know, font Google and then whatever font that you find on the Google font API. And it will go out and download those files the first time you run this application. Cache those font files locally so that it doesn't have to do it again in the future. And that essentially allows you to like when you deploy an application, the first time you visit the application on the hosted instance, it will go out and download those font files and cache it locally on the system. But then when the next viewer comes to the Shiny application or any other future visit to the application, those font files will be cached on the system so that it doesn't like it already has it locally on the file system.

Oh, very nice. That definitely saves a lot of time. Yeah. So, this allows you to essentially just have this one function call and you could then distribute the Shiny application to somebody else. And they, you know, have basically Shiny will serve those font files to the browser so that even if, you know, you're in a shutdown system where you don't have access to the internet or you're, you know, you're in a firewalled environment, you can still render those fonts without an internet connection because those files are, you know, available locally and served by the Shiny application.

Excellent. Yeah, that gets hugely important for some things I have to do where I have to share an app in somewhat more confidential way to have it be kind of self-contained like that. That's excellent. Right. So, this is functionality that's, you know, not only can you take advantage of this by providing a BS theme object to any sort of page layout in Shiny like FluidPage or NavBarPage or sort of the more fundamental BootstrapPage. You can also use a similar API for R Markdown as well. And there's support with the HTML document format at the moment, which you can see on bslib's package down site. There's a getting started example towards the bottom where it kind of shows you the basic, most basic example of using bslib's BS theme with Shiny. And then the usage with R Markdown is that if you just provide the theme parameter, any arguments to the BS theme function, it will use those values essentially in the same way that it would for the Shiny use case where all of the sort of customization options are documented on this BS theme reference page.

where you'll see there's named arguments for choosing a major Bootstrap version. So, this has been another big aspect of this project. Oh, yes.

A lot of breaking changes happen in it going from Bootstrap 3 to Bootstrap 4. And this has vast enough consequences that we, at least for the foreseeable future, will probably be defaulting to Bootstrap 3 in Shiny. Possibly in R Markdown, there's talks about changing the defaults. But probably in Shiny, at least for the moment, we'll be defaulting to Bootstrap 3. But bslib is kind of the recommended way if you want a more modern version of Bootstrap is to choose the version that you want to use in this BS theme.

The default is actually subject to change. So, currently, the default is Bootstrap 4. But in the development version of bslib, actually, we've just added support for Bootstrap 5. And in the future, the defaults will likely become Bootstrap 5.

So, we do recommend when you actually use this for a real project that you specify the version of Bootstrap that you want to use when using bslib. And that will sort of guarantee that if you come back to this project in the future and run it again, there's more of a guarantee that things are going to be reproducible if you pin a specific version of Bootstrap. Yeah, that's a huge detail that I think, yeah, definitely needs to be underscored as well. Because for me, reproducibility is not just about kind of the code itself, so to speak. It's also the dependencies that an app has. And yeah, bslib is a front end to Bootstrap itself, which, of course, Shiny is heavily wrapping. So, even for someone like me who does package dependency management with Rm, for example, that's not quite enough. It's also, for the UI toolkit, if we're using bslib, yes, you want to make sure you have that fully reproducible. And these are the kind of things that I've had to learn from experience, but it's great that you call out these examples of specifying it here directly. And hopefully that is something that if you're in a similar situation I am, and reproducibility is just hugely important on a lot of levels, that you take advantage of that parameter.

Yeah. And especially if you're a sophisticated Shiny app user that's using pretty popular packages from the ecosystem that RStudio itself doesn't maintain, say Shiny widgets or some other sort of UI creation framework, you might be relying on Bootstrap 3 functionality that might break with a more modern version of Bootstrap.

So, that is why we're kind of, we'll still continue to default on Bootstrap 3 because we take backwards compatibility very seriously with Shiny. We always want you to be able to feel like you can upgrade to a modern version of Shiny without breaking your existing application. Right. So, and we've even gone so far as to with this bslib project having support for Bootstrap 3 customization in the sense that if you have a legacy project that happens to break with the more modern version of Bootstrap, you can still say BS theme version 3 and then leverage like some of the custom theming features where you can specify like maybe a dark background color and a light foreground color and a different primary color and some custom fonts to get something that looks more like this and even use that with Bootstrap 3, not just a more modern version of Bootstrap. Oh, that's interesting because some people might be concerned that, oh, if I'm still on the older Bootstrap, do I get access to all these nice features that bslib is giving me? This sounds like, yes, that's not an issue in that sense.

Bootstrap versions and Bootswatch themes

Right. So, we fully like ship full bundles of the Bootstrap project for all of these major versions and all of them at least have, so you'll notice on the reference page here, your BS theme has an argument for specifying the version. You can also specify a boot swatch theme, which if you've ever used the Shiny themes package, then you're probably familiar with boot swatch themes and with more modern versions of Bootstrap, there's actually more boot swatch themes available. So, there's actually kind of a section on this, on this getting started article that kind of goes through the links to different boot swatch themes and kind of mentions like, hey, more modern versions of Bootstrap have more boot swatch themes available to them.

So, this is a new theme, boot swatch Minty, that has sort of like a nice primary green color and a pink secondary color and has both the usage for Shiny as well as R Markdown, but just to go back to the reference page for BS theme. So, yeah, sort of those two main controls is Bootstrap version and then boot swatch theme and then these named arguments are starting to get more into like the custom controls for custom fonts and custom colors. Sure. And these named arguments in BS theme are controls that we've basically said, we will support these controls across all Bootstrap versions. I see. So, you will be able to use these arguments with any version and be confident that it's going to work in a sensible way across those versions. But then the dot, dot, dot here is basically any Sass variable that Bootstrap has provided. So, this point here, customize like more specific variables for customizing more specific things, goes through this dot, dot, dot. So, in order to see like what theming options are available that are a little bit more specific, we have articles for both Bootstrap 5 now and Bootstrap 4 Sass variables where this is a big interactive table with hundreds of different settings for customizing everything from like the main border color setting to, you know, how much border radius there should be, what the background color should be on active components versus the foreground color.

So, you can get very, very detailed with how you approach your theming. And you're basically just providing a CSS unit or value to this BS theme function rather than having to go and like write custom CSS that target special classes that might change in the future. So, it's a much better structured way of, you know, doing custom styles compared to writing custom CSS rules. Yeah, I admit I am not a CSS expert. I just know enough to be dangerous, as I say. But this makes that transition to it a lot easier. And also, what you've done in bslib, you've also given a great interactive component for users to actually explore the impact of these parameters right away. And maybe we could dive into that a bit too. Right. So, yeah. So, I like how we're kind of going logically through the getting started article where it kind of goes through what's going on with bootstrap versions, as well as boot swatch themes. And then it gets into custom themes where we sort of talked about these main color and font settings.

Real-time theming widget

And then the next section here is what you're referring to, which is real-time theming, where here I have a little screen recording of this interactive theming widget that you can overlay over any Shiny application and choose, like, basic colors and accent colors and in real time see the new CSS styles on the page interactively. Sort of refreshing the CSS styles. And this even works with thematic in the sense that if you enable thematic, you could have static plots in your Shiny application that automatically sort of refresh themselves based on new styles in the Shiny app.

So, you can see a demo application, the same demo application with the screen recording that I have here by calling this BS theme preview. That will overlay this interactive widget with the demo application all in one shot. But you can also call this BS themer function inside of any Shiny application. And as long as you're using bootstrap core and higher, it will overlay this interactive theming widget that allows you to, in real time, theme your Shiny application. Very cool.

And as long as you're using bootstrap core and higher, it will overlay this interactive theming widget that allows you to, in real time, theme your Shiny application.

I actually put this to good use when I did a, I knew it wasn't going to win, of course, but I did a couple submissions to the Shiny contest this year. And I kind of knew in general what I was looking for, but then to be able to play with this interactively and select the colors that matched my aesthetics to the game I was kind of using as inspiration, this got me started so much faster than trying to build all that up from the ground up. So, I made great use of it and it paid off nicely. That's great to hear. Yeah, because I think not only will people, I think, find this useful for Shiny, but I think potentially the R Markdown community is going to find this really useful for, as you said, more quickly iterating through a custom theme for your documents.

So, you can do something similar where you'll have to call this BS themer function in an R Markdown document, but it has to be used with a Shiny runtime. That's kind of the one main catch here is you do need access to that Shiny runtime in order for the real-time theming widget, essentially, for it to be able to rerun SAS code and refresh CSS styles on the page. It needs to run through a Shiny runtime.

Under the hood: bslib and upstream Bootstrap

Excellent. Yeah, and there's a lot that's going on under the hood that you're not seeing as just a typical end user to make all this happen, because I know when we were preparing for this episode, you mentioned a lot of the work you've had to do with even just upstream bootstrap itself to accommodate certain things that was in the vision of what you and the Shiny team wanted to do with bslib and theming in general. Yeah, I mean, that gets into the territory of bslib being a fairly low-level tool, and we're just providing bootstrap to any project that wants to use bootstrap. So, this goes beyond just Shiny and R Markdown, but even just focusing on those two ecosystems. There's lots of projects like Flex Dashboard and flexdashboard, and, you know, other R Markdown packages that kind of extend the HTML document format in some sense that want to make use of Bootstrap. And what we want is for bslib is to do sort of all the hard work to allow these packages to keep on sort of using Bootstrap 3 features in a modern Bootstrap world. So we've had to provide some additional layers to kind of recognize Bootstrap 3 style HTML and CSS and make that still work in Bootstrap 4. So that has actually led to me actually making some contributions to Bootstrap itself and also, you know, kind of updating, you know, trying to cover all of our bases in terms of making sure that, like, all of the features that you're used to using in a Bootstrap 3 world will continue to work in modern Bootstrap world.

So, and, yeah, I guess that's maybe a nice time to also mention that the development version of flexdashboard has gained support for using bslib. Oh, that's exciting. Similar way to how I was just demonstrating its use with HTML documents, where you basically supply parameters to this theme parameter. And there's a whole article now on the new flexdashboard website that talks about theming flexdashboards. So this kind of is written

in such a way that it should be a gentle introduction to what Bootstrap is and what bslib is and how to use it with flexdashboard. So you do your typical thing of, say, output flexdashboard and then theme version 4 if you want to use Bootstrap 4. Or if you wanted to add on a boot swatch theme, say, Minty, we should actually update the screenshot here. This will actually look a little bit different now with the most recent version. But same sort of support for custom themes, where you, again, can supply custom colors to do, like, a dark mode look for your dashboard. And you can still make use of that sort of an intelligent ability to download and cache font files by using these different font settings in bslib and adding a special Google keyword and then just give it the font family that you want. And it will provide those font files for you.

I'm so amazed at how that works. That's just really cool. But that those YAML parameters, again, the kind of remind what you mentioned earlier, those are basically parameters that you would use in a call to the BS theme function, right?

Yeah, so this looks very, if you were to go back to the custom theme section here, you can kind of see these side by side.

Ah, yes. Okay, very good.

With the sort of normal programmatic Shiny usage, you use this font Google function, which also has some different arguments to it that are documented on this reference page. But if you wanted to compare these closely, you know, again, this is the programmatic use. And then we provided like a special YAML syntax for doing the same thing a little bit more in a YAML like way.

I see.

And yeah, so that YAML usage can be a hard thing to discover. So I suppose this is probably the best place to come and find it is this section with custom themes. Hopefully we'll make that a little bit more discoverable. I think, you know, one of the things that I've heard the IDE team and our markdown team wants a thing that they want to be working on is, you know, making YAML settings more discoverable in general for our markdown.

Yeah. So hopefully that's, you know, something that we can try to solve a little bit more generally in the future as well.

Yeah, it's amazing. I mean, it gets even just R itself, all the customization you can do. But honestly, yeah, the idea of being able to take advantage of this great work in Flex Dashboard is going to help a lot of people, especially people I work with who are now making, I'm even surprised by this sentence, but very sophisticated dashboards that almost, you know, if you look at it, you wouldn't even know that it technically was built with R markdown and not shining directly. But the fact that now they can change the appearance as kind of like that same experience. Yeah, it's going to make getting these kind of data driven outputs even easier to polish for those key customers that want a certain experience or certain things to be emphasized. So I'm I'm genuinely impressed by this for sure.

Well, thanks. Yeah, hopefully we're trying to make it as easy as possible, but it's, it's, you know, it's a tough world to be working in, in the sense of, you know, trying to make these kind of things easier where everyone has like a different set of needs.

That's true.

So I think what we've landed on should be generally useful for folks. But I think, you know, in some cases, you're still going to need to know how to like write custom CSS rules. And if you get really advanced with this stuff, you know, it's going to be worth actually learning about SAS itself. Cause it has some useful, you know, it's, it's more of a programming language than CSS, which allows you to do like more fancy, sophisticated things. So you'll probably still end up in a world where you need to add on additional custom rules to these projects.

And one thing that we've done recently with, you know, HTML document and flex dashboard, they've always had this CSS parameter where this has been kind of the typical way to you know, tack on custom rules to in our markdown document.

Sure.

And this parameter now has the ability to compile SAS files as well. So this dot S CSS extension is actually a SAS file that needs to run through a SAS compiler.

And now you can provide files like this, and you can do it in such a way that if you're using BS lib like this, say theme version four, then you can have SAS rules that reference those bootstrap SAS variables that we were talking about earlier. So, you know, you could have your custom CSS rule like BS for bright, actually reference these SAS variables that are provided by bootstrap and can be themed through this custom theme object. And then, you know, those rules will be defined for your usage inside of the document.

SASS, custom components, and real-time theming

Yeah, that's, it's funny, as we talk about SAS in general, if you just take away the last S in that abbreviation, that is a totally different language that my industry has to deal with. That's absolutely nothing to do with theming. I always got to chuckle out of that. But I am slowly starting to get the hang of this. I haven't, you know, I certainly haven't mastered it by any means. But it's certainly if you have to go into this level, and it's a much easier way to kind of have more cleaner syntax on these different rules, these different declaratives, than if you just had to account for all these different possibilities in one massive CSS file.

Right, right, right. Yeah. And you can almost view this as like, a nice way if you're working with like a front end developer, a front end developer who's more familiar with like working at the HTML, CSS and SAS level. One neat way to kind of think about the infrastructure that we've provided is you could, you know, have somebody define some sort of HTML component that you could then wrap in our code. So this is using HTML tools to like, create an HTML div with a class of person. Sure. And then there's a header and like two other divs for the title and company for this person. And then that, you know, front end developer could also write SAS rules that, you know, provide CSS styles for this person class. And these SAS rules could, you know, again, reference bootstrap SAS variables. So it's kind of binding to the semantics of bootstrap in the sense that it will inherit whatever the border styles are, or whatever kind of general setting there is for padding or border radiuses, and sort of define that general template that can then be customized through the BS theme object. Right. So it's a nice way to kind of decouple like the person defining the component from the person defining the custom theme.

So it's a nice way to kind of decouple like the person defining the component from the person defining the custom theme.

So like the HTML front end developer has provided the scaffolding to, you know, provide the HTML and general sort of generic CSS for these person cards. But then somebody else can come in and say, Oh, no, I actually want, you know, a slightly different background or foreground color, different primary color, and have that influence the end styles of a custom component.

Yeah, that's really slick. I hadn't even gone this far in exploring this. But if I ever get the good fortune of working with, you know, designers on some of the bigger projects I have with Shiny, this will be a great way to kind of bridge that gap and some of the things that they would want to contribute from their end and how I can kind of be able to take advantage of that. We're still in the BS lib kind of paradigm. So that's really slick. Right? Yeah. And we're hoping that, you know, kind of the Shiny widget developers of the world will eventually shift to this kind of a structure for providing CSS to their custom widgets, where we've provided a couple helper functions that allow your components to work with real time theming, in the sense of, you know, if you have your person components, you know, the typical way to approach this is to just provide like a pre compiled CSS file to define the styles for this component. But we provided this BS dependency defer, which kind of provides a recipe for going from a theme object to a CSS CSV file, which allows like the interactive themer to basically recompute the CSS files that define the rules for these components. So this is almost defining like a runtime dependency that can change after page load, where this BS dependency defer is taking in a function that it's a function of the theme that if it's a BS theme, it actually ends up compiling SAS code and generating a CSS dependency from that. And if you don't want to use this component with bootstrap, you can also just provide like a pre pre compiled CSS file. So just because you're writing a dynamically themable component that can derive its styles from bootstrap, you can still do this in such a way that you can write a component that works without bootstrap as well. So hopefully, this kind of provides a it's it's kind of a complex thing to kind of wrap your head around at first, but hopefully, it's digestible enough that we'll start to see more people sort of adopt this approach.

Because this is really the thing that makes if we go back to the real time theming screenshot here, like, this is what allows for instance, when I change the accent color here from pink to orange, if you watch the slider change to blue, and now orange, right? That's coming from a CSS file that is actually whenever we change that color in the real time themer is recompiling SAS code to provide a new CSS file. Yeah, so it's not like you had to in this app feed in like 100 different CSS files, each of a different color. This is all just taking advantage of SAS the dynamically inject that I kind of liken it to when I use the glue package to inject dynamic text in normal like long form text. I don't have to write all those text files myself and just let glue do it dynamically. This is a simplified example, but that's how I like in SAS to do it. It's just giving you that flexibility, having one rule for it instead of having the future, like try to think in your head how many different possibilities there are.

Yeah, yeah, that's, that's basically the main idea of it. It's a little bit more complex in the sense that we bring in the new CSS file and then disable and remove the old one, right after the new one has been brought in. But yeah, that's essentially the idea.

Yeah, I like I said, I'm hugely impressed with this. And it gives me cases where when I know I want to keep my applications dependencies somewhat streamlined, even from like the shiny architecting sense, that now I can treat the default shiny UI combined with bslib as a real alternative to creating a great UI UX experience in cases where I want to keep that dependency footprint smaller, and still get this great customization. And that can be that can be very helpful to certain situations. So again, I know it's early days for it. But it does sound like you've laid a lot of the foundation down for others in the shiny ecosystem to slowly adopt a similar mindset for some of the custom widgets or custom themes that they're producing in their packages. Yeah, yeah. So that's exactly it is. And where I view this as a contribution is not only like the cross, you know, bringing trying to bring the ecosystem to a modern bootstrap world, but also kind of laying that foundation to start thinking about custom theming a little bit differently, where we're really leveraging the runtime abilities of SAS and being able to compile these things at runtime, rather than, you know, at build time.

but also kind of laying that foundation to start thinking about custom theming a little bit differently, where we're really leveraging the runtime abilities of SAS and being able to compile these things at runtime, rather than, you know, at build time.

Absolutely. And we'll have again, we'll have links to all the documentation sites that you have here, I can tell there's been a lot of effort to make this so that we can follow along and go in that progression of starting simple and then going to different layers of customization. So I always think the best way to learn is by doing so I'm yeah, the example theming app right there is a great way to start and then you can drill down and get more fine tuned detail as your application needs. But it's exciting to be able to play with this. And like I said, with my contest submission, it didn't have to go very far for it, I just needed a custom font, and some custom colors, which again, were easily defined. But then I could use the default shiny UI to do it all. It was it was amazing. Right? Very cool.

It's great to hear. Yeah. And like I said, our RStudio global, this was all the talk in our breakout sessions for sure. But I certainly want our audience to know that this is definitely not been the only thing you've been focusing on. And I want to transition to something that actually has some history with folks like me and my particular industry as a way that the shiny team at RStudio's already been listening to feedback from certain users out there.

So we had a situation where for us, we're using shiny a lot more in life sciences. And some of the things that would come up time and time again, we're having the ability to have these production grade applications in the hands of stakeholders. But if they're being used for like key decisions being made on like, should we move forward with this treatment or things like that? We wanted a way to have in our apps a way to kind of capture what's being done in there. So that if we had to go back and say, okay, this decision a year ago, what did we do in the app to make that call? Instead of somebody like trying to manually record all the steps in a text file somewhere. A lot of us try like, very, how do you say like hocus pocus ways of pulling it off. And it was always kind of variable. But then about maybe a year or so after those initial conversations, you and Joe Chang had shared shinymeta, which recently just had a milestone release on CRAN. So for our audiences as familiar shinymeta, where you could give a little bit about that background and how that's been helping that vision of reproducibility from a shiny perspective.

shinymeta: reproducibility from Shiny

Yeah, so as you mentioned, shinymeta has just hit CRAN. So I should be updating this installation section of the website. So you now can grab it from CRAN.

And yeah, so shinymeta, you know, has started out, sort of the inspiration from it very much has some life science roots to it, where I kind of view it generally as this problem of, you know, you work on a team where maybe you as the shiny developer are getting some instruction from your project manager to say, you know, I want to add these specific features to the shiny application so that I or my other, you know, colleagues can, you know, explore what's in this shiny application, find the relevant bits of information from this model or from this data set. And then once we get to that point, we would love to have a reproducible R script to reproduce the outputs that the shiny app has generated.

Because, you know, shiny at least for a while has had the ability to bookmark state on a shiny application, which can kind of get you to the same, kind of attacks the same problem in the sense that at least you can return to a specific state that you deemed as worth saving in a shiny application. But that still sort of relies on the fact that there's going to be a hosted shiny application instance somewhere that's working. And in some cases, you know, actually implementing that bookmarking feature can also be a pretty complicated thing to do. I could fill an entire episode on the tricks I've done in our shiny bookmarking state implementation that would make probably Winston in your head spin a little bit. It's been a very important feature, nonetheless.

Sure. Yeah. And I don't want to market this project as like, you know, it's going to be easier than bookmarking in terms of providing a way for people to sort of save a certain state of your shiny application. Because actually adding shinymeta integration into a shiny app does involve kind of understanding this sort of custom way of meta programming, where you're supposed to use this dot operator, which conceptually, it's quite similar to bang bang from Arlang. Okay. But it's different. It's actually a little bit more complex in terms of understanding exactly what it's doing. So there's a whole article on this, on how to actually get your shiny app to generate code.

But the main idea is you basically any sort of reactive read, like an input dollar variable, or a read of a reactive expression, like this bear, any sort of reactive read like that, if you just wrap that read in this dot operator, that is kind of one of the two main pieces of adding shinymeta integration to your application. So that, you know, this example here is showing, you know, how I can choose different variables to choose a summary, which is shown in this render print output here. But then if you read an output, like this output dollar summary inside of an expand chain context that has been marked appropriately with these dot operators, then the shiny app will, like the result of this expand chain call will be the R code that shiny is essentially used to produce the output in the application.

the result of this expand chain call will be the R code that shiny is essentially used to produce the output in the application.

So that's a real minimal example of how, you know, you can go from targeting a specific output where you want to provide reproducible code to regenerate that output outside of the shiny application runtime and provide it to your end user in a way that they can just copy and paste that code into to a different R session.

And what do you what do you say to somebody that maybe has an existing application and they're intrigued by getting into shinymeta? What would you recommend they do to kind of ease that transition into maybe down the road, having an application that's fully compliant with this workflow?

Yeah, so it's going to be a daunting thing if you already have a big complex shiny application to undertake the project as though you want to export all the code in the shiny app, that's going to be a difficult undertaking if you kind of approach it that way. But if you break it down into smaller pieces and just say this is an important output that I want to provide reproducible code for, the way you could kind of approach it is, you know, maybe you identify this output summary as an important output and that's being created through this render print reactive context. The first thing to do would be to change this render print into a meta render which allows this render context to then have this ability to sort of spit out code at the end.

And then identify any reactive reads in that reactive expression. So the read of this bare reactive expression should then get the dot dot operator and then you just add that to all the reactive reads in this expression. And then you'll want to go back to all these reactive reads that you've marked with the dot dot operator and follow them back to the original creation. And in the case of reactive, we have a meta reactive variant which works just like reactive in the normal execution context but also gives it the ability to generate code when you call it inside of this expand chain context.

Interesting. So it's kind of like working backwards in the sense of like identifying an output and then identifying all the dependencies of the reactive expressions that got you to that point. And marking all the reactive reads with the dot dot operator and using like the meta variant of reactive or meta render. There's also meta observed if you're using observed statements. But I'm not going to lie. Like, you know, with a more like very complex application, you know, you might be better off just kind of starting over again in terms of, you know, writing the shinymeta aspect to it first and getting that working in a state, in a working state. And then, you know, just kind of copying over the rest of the logic to get the rest of the pieces working. Yeah. Might be a better way to kind of break down that problem.

Yeah. I see a lot of parallels, at least in my head, to like the idea of something that's new to Shiny modules. They may have heard me harp on it probably too many times to admit, but I've been beating that drum for a bit. And when you have an app that's been built, modules. They may have heard me harp on it probably too many times to admit, but I've been beating that drum for a bit. And when you have an app that's been built for maybe a long time, maybe for years, and you're able to kind of get away without them, but then you're at a point where you're tired of like reusing code everywhere and all that. I think there's similar principles where if you just think about converting the entire app to modules right away, that's not going to be smooth if you just think in that perspective. But if you take an important piece and just slowly go bit by bit, and then you can architect it in kind of a longer term play to take advantage of the module infrastructure, it sounds like this is kind of a similar thing where you just want to look at the most important outputs first that you want to reproduce, trace back to how they got there, and then kind of construct the shinymeta flow around that.

Yeah, and the adding modules to the equation is, of course, going to add another layer of complexity to the whole thing. So what I would recommend is when you're first starting to learn shinymeta, I would start with simple examples. I would start with this article, actually, where we put a lot of work into trying to come up with good examples that kind of succinctly get across the whole API of how shinymeta works and how those concepts kind of relate to metaprogramming concepts. So this actually isn't too long of a read. It's got quite a bit of code because it's a real example that kind of demonstrates all of the things that you might want to know about when it comes to shinymeta. There's even sort of a video with smooth transitions of changing this code from a real shiny app to a shinymeta app. Very nice.

So the idea is you identify domain logic, you capture the domain logic with the meta variants. There's even tools for ignoring code that you don't want to export to the user. You mark those reactive reads with the dot operator. There's even more sophisticated uses where you can evaluate code and then eventually call this expand chain to get the code behind. You can give it an output like this output dollar ply or reactive expressions as well if you just want to generate code behind a reactive expression. You can do that too. But I think this is a nice sort of succinct example that has a nice video demonstration on how to kind of modify your existing shiny app to a shinymeta app.

Yeah. So I'm going to be exploring more. But this is going to be my place to start because I am in that camp, so to speak, of a lot of large code bases behind my existing apps. But if I start fresh, then if I know it's going to be one of those things that's used in highly important decisions or, you know, informing important decisions that I can take that investment at that point and be set up for success with it. Right. Yeah. And on the actual website, I don't think we have any great examples of modules, but just want to make you and your viewers aware there is underneath the instance and examples folders

of the shinymeta source code. There's a modules example that demonstrates kind of like a basic modules example with shinymeta.

And essentially there's I mean, it does add complexity in the sense that with modules, you know, you have to think a little bit differently about how your reactive code is set up. Sure. But it's a pretty similar concept in the sense that you could start from like your render text that you're creating within a module and same sort of workflow in the sense that you put a meta render around it. You put the dot dot around reactive reads, and then you follow kind of the dependency of these things up the chain.

Yeah, it's definitely going to take some practice. But again, I said the same thing about modules, too. That took me quite a bit of practice to get the hang of. But we got examples to draw from at least. So we'll definitely have a link to shiny meta site and this specific example in the notes because I do remember perusing this once or twice in the early stages as I was thinking about that conversion, but I'll definitely be revisiting this.

Cool. Yeah. Yeah. And I just before we move on from shiny meta, I want to make sure we I also demonstrate that shiny meta also provides some tools for providing the reproducible code in a more convenient format than just like displaying the code in line in an application.

So here's a shiny app that allows me to like search for different R packages and get the number of downloads for those packages over time. So let me get that shiny. So I can see, you know, of course, shiny has a lot more downloads than Plotly. I could apply different transformations to this time series. So I'll do a weekly transformation. So there's less noise and the number of downloads.

And then you could provide a button like this download reports, which will actually populate an RMD file with the reproducible code to reproduce this interactive graph inside of an R Markdown document and also render the results. So here you can see it's taking some time to actually call like R Markdown render on the results and provide everything in a zip bundle that I can then open. And this will have both the RMD source file as well as the HTML results.

And sorry, my computer's freaking out a little bit. But that but that's huge, though, right, because then you can put this in the hands of other customers that may want to run these analyses later on, but keep that same exact interactions that were done at that earlier point. I could see this even being valuable for setting up kind of like a, you might say, a headless testing paradigm. Not that this replaces shiny test or anything, but it's a great way to kind of capture a use case and the steps you took to get there.

Right. Yeah. So this is one of the, when I'm given a talk on shinymeta, this is an example I like to demonstrate this idea of like exploration and then permanence. The exciting thing about this is, you know, you can have that permanence of saving the units of interest in this case, like Shiny and Plotly as two different R packages that I'm interested in. And also have a permanent record of the R code that was run to produce these results. And then apply that to like a dynamically updating data source that allows you to then use a service like provided by Connect, for instance, that allows you to like schedule this RMD file to re-render like once a week so that you have a constantly updating report of the packages that you're interested in.

The exciting thing about this is, you know, you can have that permanence of saving the units of interest in this case, like Shiny and Plotly as two different R packages that I'm interested in. And also have a permanent record of the R code that was run to produce these results.

So I do actually have on Connect this updating, but I've since forgotten the link of where that was. Oh, we can definitely put that in the show notes when we release this. Yeah. Yeah. Very cool.

Yeah, this this one, like I said, it was something that colleagues in my industry have been clamoring for and we've tried our own solutions for with varying degrees of success. But as we as we start to use Shiny in ways that maybe we're outside the box, so to speak, but give us that capability of understanding data, understanding insights and making actions based on those reviews. This does become quite important for traceability and reproducibility. So we're we're extremely excited to play with this a bit.

Advice on JavaScript and Shiny customization

And so I always like to conclude these with a kind of a more thought provoking question based on your experience and your journey with Shiny. I've always like we mentioned at the outset, your work with Plotly has been so impressive and not just what Plotly can do at a high level, but the learnings you've had to do with learning the HTML widget ecosystem and the tie ins with R and JavaScript. And viewers of my dev series may know I'm kind of on a journey, if you will, I'm climbing a little bit into getting more under the hood of customization of Shiny.

And one of the things I'm trying to master is kind of the bindings between R and JavaScript and passing things back and forth when maybe the widget I'm using isn't quite coming up with that same tie in that I need. And even recently on a live stream, I went into kind of the bowels of doing that. And it took me a while to grip my head around being able to set a custom input from JavaScript and be able to bring that back into Shiny. But eventually I figured it out and it was a nice win for me. But I know I have a long way to go. But how do you recommend others that are watching our episode today, starting that journey of kind of like slowly getting more customization into their applications via the power of JavaScript or other kind of advanced techniques from your side?

Yeah, so I think the most general piece of advice I can offer is to go to shiny.rstudio.com And I think we have at least a couple articles that should be, I think, under the like somewhere around Shiny extensions. These are some great articles that I wish had been around when I started learning these things where I was, you know, more in a space where there was maybe a few people even doing these things and I just had to follow very closely what they were doing in order to find out how to, you know, do something custom, like put in a custom D3 visualization inside of a Shiny app. You know, I was just basically forced to, you know, find examples that did what I wanted and read the source code and figure out how it works from there.

But I would say like hopefully a fair amount of your users can come to these sections and at least have a glance through them and see if some of the, you know, whether the article is loosely talking to the use case that they're looking for. So I would say like if you're looking to, you know, take some existing JavaScript library and use that JavaScript library inside of Shiny, a nice framework for doing that is this HTML widgets framework, which the nice thing about going the HTML widgets framework route is that it allows you to provide an R wrapper around a JavaScript library in such a way that you can use it inside of Shiny, you can use it inside of R Markdown, you can use it at the R console. So it makes it very versatile in that sense that, you know, this is what Plotly is built on, this is what Leaflet is built on, this is what hundreds of R packages that bind to JavaScript libraries build upon.

So if you think that this is going to be a longer term investment for you in terms of really investing in a JavaScript binding to some library, this would be a great direction to learn. But I think we have some other like more specific kind of use cases where if it's just like a one-off sort of thing that you want to do inside of a Shiny app, you should probably at least be aware of like kind of the main building blocks for sending data between a Shiny server and the client.

So I think this article will do a good job of that where I think it will speak to the thing that you discovered recently, which is set input value, which is a way of taking like a piece of data on the client and creating a Shiny input value using that piece of data. So that it's accessible in your reactive Shiny code.

And at that point, it's actually maybe worth discussing that I think especially if you are using a lot of Plotly and have interest in JavaScript, I did write this chapter in the book towards the end. Oh, actually, it's a whole part of several chapters. Well, even better. What I call event handling in JavaScript. I don't think it's super in-depth, but it's at least intended to be written at the typical R user that doesn't know much of anything about JavaScript or HTML, but at least has some motivation to maybe do something custom with a Plotly graph or, yeah, I mean, this is focused more on doing something custom with Plotly with JavaScript rather than through Shiny, but these techniques could be used inside of a Shiny app and some of these same techniques and ideas might be useful to you outside of Plotly as well, where I try to start out fairly simple with explaining how all these things are interconnected, like what JSON is, which is the sort of data structure that JavaScript works with, and talk about how you can use functions from HTML widgets like onRender to call a JavaScript function after the HTML widget has rendered.

So this is an example of just saying, I want to execute this JavaScript function to print to the console the DOM element that's holding this visualization. So this is a little video of me opening up the developer tools to then see in the console a log of this DOM element and showing techniques to bring that out as a global variable in this JavaScript runtime and looking at the pieces of data that are attached to this DOM element.

Nice. Boy, that's kind of why I like to do some of the things I'm doing with the streaming and hands-on development work. When you see this in action, it resonates a lot more than even just a wall of text saying like, open this, click this, this, this, this. Great job on making these. I've always liked that with the Plotly book, the great balance of the text and the hands-on demos. Yeah. Thanks. Yeah. Because I think it's really easy to forget. These are some pretty critical things to learn about if you're going down this world of front-end development, essentially. Right. Web page. Yeah.

Because that's one thing with Shiny, whether you realize it or not in the beginning, you are becoming a designer or an engineer of sorts with what you're giving to the users as a front-end. So having mastery or at least getting to a mastery of these kind of debugging paradigms and getting under the hood of what Shiny is doing, it helps you immensely with kind of understanding the big picture of what's happening here.

Right. And hopefully, like I was saying before, there should be some sort of shared or like some of the things that I'll go through here should be relevant if you're working with another JavaScript library where a lot of these sort of libraries will fire JavaScript events based on user interactions where this is using this on render function to then say, register on the Plotly hover event, this callback to print out data relevant to a hover event and then do the same thing for a click event and do the same thing for a brush event. So this kind of goes through me using the developer tools to see what data is emitted for each one of these user events.

Excellent. Yeah. And it goes without saying, we'll have a link to the Plotly book and the show notes. But again, I have both the printed copy and I have this in my bookmarks.

So whenever I do a Plotly plot, this is always open next to it. Yeah. And this is one advantage of using the free online version is, you know, I, I unfortunately in the printed form, I can't really get across these videos very well. So, yeah. If you, if you figure that out, you get a lot of money. I'll just say that. I want to figure it out with printed media, but that's very good to have both available to us.

Right. Yeah. And just one sort of last thing on this is, I just want to mention there's this whole section where it really kind of explores the possibilities of this general idea of attaching metadata to graphical markers in Plotly. So here's like creating a scatter plot where I'm attaching some metadata that keeps track of a Google search query to visit whenever I click on a point, and then use this on render method to say, when you render, register this click event to get the custom data, and then open a new tab to this URL. And then the video, you know, shows that in action.

Oh, that, that opens up a ton of possibilities. Yeah. I actually had a little fun with Plotly in one of my shiny contest apps where I had, it was like a horizontal kind of bar chart, but it ended up being kind of like a animated version of like race cars scrolling to like the number of points they have. But I was able to do it all in native shiny and hook into like the animation layer. But the book actually walked me through like 90% of the way there. And all I had to do was do a little more customization afterwards. And it was, it was a lot of fun to explore. And Plotly was the only way I can make that happen. I tried over packages. It didn't work. I was able to get the race car kind of little funny figures on it and have a little play button. And it was, it was great. So I love what's under the hood here. And the book covers immensely more details on all the different types of plots you can do.

Yeah. Well, yeah, yeah. I saw that example. It's, it's really fun. That's, that's really great to hear that, you know, somebody is finding some use out of this documentation. Yeah. Well, trust me, it is definitely more than just me, but I remember having that next to me and I was like, okay, I got to look at the animation chapter, look at the annotation chapter. And once I figured it out, once I had the data in the right format, smooth sailing from there. So I, yeah, I'm a big fan of it and I'll be using it, I'm sure in the foreseeable future of a lot of my apps going forward.

Wrapping up and how to reach Carson

So yeah, I could geek out a lot of the, a lot of this stuff all day with you, Carson. I want to make sure I respect your time, but if you, if our listeners or our viewers want to get ahold of you or maybe help with contributions to various things in the Shiny ecosystem, where would you like to point them to? Yeah. So you can just reach out to me on Twitter. I'm at, I'm on Twitter, I'm on GitHub at the username cpsevert. So that's a good place to reach me is to just tweet at me about stuff and I'm usually pretty responsive when people are tweeting at me. Otherwise, you know, you can always email me, you can find my email on my GitHub profile. I'm pretty sure it's up there. And you know, you can, my RStudio email is Carson at RStudio.com so you can reach out to me if you have any questions or want to chat about anything. I'd be happy to, you know, answer any questions you might have.

Excellent. Well, you are welcome back on the series anytime. I'm always learning a lot more whenever I talk to the brilliant Shiny team that you're a part of and yourself included. So I want to thank you again for taking the time to dive deep into some of the great work you've been doing. And yeah, I'll be watching the developments of bslib and everything very closely as I supercharge my apps in the future. So again, thank you so much for joining me, Carson. Yeah, my pleasure. It's been great. All right. Excellent. And we'll be back right after this.

Closing thoughts

All right, well, I certainly hope you were as impressed as I was with the immense work Carson has done with making bslib and even this early state so easy to get started with and so easy to take it to another level for your customization of your Shiny apps, visual theme and styling. It's just going to make that transition for those like me who are primarily statisticians who have learned this kind of software engineering mindset on the fly, especially with Shiny development. It makes it even easier for us to achieve the same effects that a professional designer might do if they had to spin up some custom HTML. So I'm really excited to play with that more.

It makes it even easier for us to achieve the same effects that a professional designer might do if they had to spin up some custom HTML.

And certainly, as I mentioned at the conclusion there, Plotly itself has been transformative to the fact that Carson's been behind each of these efforts and really getting into the nuts and bolts of how Shiny is conducting things on the back end. It's just it's just amazing to watch. So I really have fun talking with him. And as I said, he'll definitely be back on the Shiny dev series later on as we look at what's new coming up on the horizon.

So in the meantime, if you enjoyed watching this episode just as much as I did talking to Carson, you can check out the full show notes with the link below in this video and definitely check out our previous guests. We've had, of course, a mix of members of the Shiny team. We've had practitioners and consultants and authors of various packages in the Shiny ecosystem. Every time I bring these excellent guests on, I learn something new. And I hope you all have that same experience. So please, if you want to stay up to date with future episodes, you just hit that little subscribe button somewhere down there. And also keep an eye out on the Twitter feed. We have a dedicated Twitter account of at Shiny dev series. And I also cross post a lot of those things from my own Twitter account of at the artcast. And also, if you have suggestions, feel free to leave a comment in the video directly below. And you're also welcome to go to ShinyDevSeries.com and go to the contact page and fill out a little Shiny powered contact form and I'll get that feedback directly.

I'm actually going to be thinking about overhauling that site a little bit with some cool new R Markdown powered templates that I've been seeing a lot of great attention lately. So that might look a little different in the coming future. And if you like to see how I develop Shiny applications, kind of my process around that, I'm now trying to stream somewhat regularly on my Twitch channel. That's at twitch.tv slash rpodcast. And typically Wednesday nights, kind of later in the night, I work on Shiny projects and I'm trying to get one off the ground that's actually based on a calendar. And it's been fun to kind of explore new integrations. And best of all, those that show up on the chat have actually helped me debug a lot of interesting issues and give me good feedback. So if you're interested in watching that, yeah, you can see the link to all that in the show notes below.

So that'll wrap up episode 22 of the Shiny Developer Series. Again, I'm very thankful that you tuned into this. And if you like it, please tweet about it, share with your friends. We love engaging with the R and Shiny community and always welcome your feedback. All right, that will do it for me. So until next time, I will see you later. Bye for now.