Resources
Oct 31, 2024 |
57 min
|
5.3k views
|

Updates from Posit, with Hadley Wickham, Charlotte Wickham, George Stagg, and James Blair

6:45 Hadley introduces the conference.
10:37 Hadley on Posit, PBC. Who are we, and what we do.
20:41 Charlotte Wickham on Quarto, an open-source scientific and technical publishing system.
31:05 George Stagg on webR, R for WebAssembly. Execute R code in your web browser.
43:34 James Blair with the latest on Posit's partnerships with Databricks and Snowflake.

Please join us for our first Posit Conf 2024 keynote, where we’ll tell you about our mission, our products, and some of the exciting things we’ve been working on over the last year. Hadley Wickham, Chief Scientist, will talk briefly about Posit’s mission and products, before introducing the three speakers who will update you on some of the coolest projects we’ve worked on over the last year. James Blair, Senior Product Manager, will give you the latest on our partnerships with Databricks and Snowflake, and how we’re building seamless integrations that let you focus on data science instead of dealing with technical details. Charlotte Wickham, Developer Educator, will show you what’s new in Quarto, focusing on new ways to build beautiful PDFs with Typst. Finally, George Stagg, Senior Software Engineer, will tell you about the latest innovations in teaching using webR, a tool that lets you compile your R code into standalone HTML files.

Talk by Hadley Wickham, James Blair, Charlotte Wickham, and George Stagg

image: thumbnail.jpg

Transcript#

This transcript was generated automatically and may contain errors.

How do we keep food fresh? How can we reduce water usage in our daily lives? How do we make trash valuable? How do we innovate to make the world a better place? Is what we do.

So at Dow, we have this graph that measures our net carbon emissions of our entire product portfolio. We've made very ambitious targets related to our carbon neutrality, where by 2050 we want to reach complete carbon neutrality. And with this graph, we can see the various projects, investments, and opportunities we have to reduce our carbon footprint.

And what's also very special about that set of goals is that it's not purely aspirational. We know we have understood the technologies required. We have projects in place to account for pretty much 95% of that CO2 mitigation. That remaining 5%, it's going to require a lot of creativity and ingenuity from all different functions across Dow. It's going to be up to us to figure out how to get there.

I work at Dow because I love Dow's mission. We touch Dow products all the time. They're in food packaging, mattresses, car parts, adhesives. And I like to do science to make people's lives better. We're also very ambitious because we don't want to just deliver these solutions. We want to do it in a sustainable way. And so all of that involves great technical and technology gaps.

We have a very capable and innovative bunch of researchers that are able to get a lot done with just about any set of tooling that you throw out of them. However, the Posit tools, what they've enabled is an extremely low barrier to entry that's allowed people who have an idea to go test that idea in a matter of minutes and to be able to share the work product of exploring a concept, a Friday afternoon activity, and go and share that with some of their colleagues with relative ease.

So I think as scientists, we're very driven towards questioning. Questioning in the sense of making sure that we're doing the right thing or that we should be pursuing the thing that we want to pursue. It's our critical thinking superpower, and I think it's healthy. But once we started not only talking about the vision of what these tools could do for us, or started building prototypes, a lot of light bulbs started going off in people's head. Once you have those initial success stories, and then all of these exciting ideas were on the table, the conversation was not anymore, why should we do this? It's how do we do it better? How do we do it more intentionally?

But there is a moment when it all converges, and it clicks, and it starts flowing, and now you have this rhythm.

Since 2019, we've grown from this handful of researchers to over 300 people now have access to Posit Workbench internally, and even more are using apps that are deployed on Posit Connect. That growth has been in part through our investment in Posit Academy, where we're teaching people how to code using an example and a project that's tailored to the work that we expect that they'll use these tools for. But also, we've been very intentional about trying to grow this community. We also have a citizen data science program. We have more and more people asking us for these capabilities, asking us for these tools that they can go use to get more value out of their data, to improve their own productivity.

Seeing people who so often may hesitate to put on that, I'm a data scientist hat or label, but seeing what they're able to create gets me really excited. The goal is really to make sure that every person in the organization has some level of fluency, so they can understand and respect data generated the best possible way. When they are showing someone data, they will show with the highest standards of science and statistics, and that they document properly all the metadata around what they generated, so in the future, people can use them for advanced modeling the best possible way.

That connection between research and development, between science and doing good, this ability to use science to make the planet a bit better, and to leave it in better conditions for my kids is extremely important to me.

In a typical workday, it's easy to get lost in all of those meetings, in the busy calendars, or the kids at home. It's easy to get lost in that and forget about the impact that we're going to have. And it's a huge challenge that we have to face, because we're literally taking what previously we've looked at as trash, and now we want to stick it back into our products. Not only is it something we want to do, it's something that we are doing.

When I think about a career at Dow, I want to be able to look back and see the, maybe small, but real contribution I will have made to help us reach that. And I think we will.

Welcome to PositConf 2024

Good morning everyone. Welcome to PositConf 2024. I'm so excited to see so many of you all here in the room, and I know there are many more joining us online as well.

A few important pieces of information. The number one thing you need to know about is the app that has everything that's going on, regardless of whether it's our awesome speakers, or some of those fun social events we have planned for you. Also, two important links in the app. The first is in many of the sessions, you'll be able to ask questions of the speakers through Slido. And if you want to engage with others online, whether they're here in person or not, you can use our Discord server.

The first time this year, we're trying out something fun. We have a little bit of a conference competition going on. You can enter to be in to win prizes, like free tickets to next year's Conf, or sticker packs, or signed books. The competition is going to ask you some fun questions about you, and we're going to present those in a shiny Encordo app, of course. You'll be able to see those statistics live, updating in the lounge. If you haven't heard of the lounge, you've already walked through it on the way to the keynote. But this is the best place to sit down, grab a coffee, talk to our sponsors, and if you have any questions about Posit products, open source, or commercial, we'll have a bunch of folks there to answer any questions that you might have.

If you need to identify a Posit employee, the shirt color is a little subtler than it has been in the past, but we are wearing a bluish green, and you all are wearing a greenish green.

We really want everyone to feel safe and welcome at Posit Conf, so as part of your registration process, you've already agreed to our code of conduct. If you experience any problems, or see anyone else having problems, please let us know. You can approach any Posit employee, you can come to the registration desk, or you can email conf at posit.co.

We've also given you a bunch of information available on your lanyards and badges, so if you want to indicate whether you'd prefer hugs, or handshakes, or no contact, you can grab one of our contact pins. We ask you to respect folks' pronouns on their badges, and finally, if you see anyone wearing a red lanyard, that's an indication that I'd like to not appear in any photographs.

We also have a bunch of spaces reserved if you have kids with you, and they need somewhere to run around while you watch the talks on your computer. We have a family room on floor six. We also have a quiet zone there if you want to do some work or just chill out. On the fifth floor, we've got a meditation and prayer room, and a lactation room, and every room has gender-neutral bathrooms.

That brings me to the most important rule of PositConf, and that is the Pac-Man rule. The Pac-Man rule is if you're standing in a group, please make sure you leave Pac-Man's mouth open so that new people can join the group. We know this is a big conference, and it can be intimidating to meet people, so at lunch today and tomorrow, we have birds of a feather stations. This makes it easy to sit down and talk to someone because you've got a guaranteed shared common interest.

About Posit, PBC

Hopefully, you get the idea that community is important to us, and community is important to us not just at Conf, but to the company as a whole. I wanted to take a couple of minutes of your time while I'm up here in front of you all to talk a little bit about who we are, what we do, and then I'm going to introduce three of my colleagues who are going to tell you some of the particularly cool stuff we've been working on in the last few months.

So, to understand who we are, I think it's worth looking at our name. Now, this is not a particularly complicated name, but there are two parts, and I wanted to talk first about the second part, this PBC, or Public Benefit Corp, or B Corp. So, what is a PBC? Well, if you imagine all organizations falling on some sort of continuum, on one side, you've got corporations. In the US, at least, the sole goal of an LLC is to maximize shareholder revenue, shareholder value.

And this is not the worst thing, right? Corporations have been a tremendous economic engine for the world over the last hundred years, but at Posit, we don't just want to make money, we want to do something good in the world as well. And one way to do something good in the world is to be a charity. But being a charity is really hard, because you have to ask people for money, and they have to give you money out of the goodness of their hearts. And while there are many incredibly valuable charities, at Posit, we believe we can provide you some kind of like immediate benefits as well, as well as this longer-term mission to create really amazing open-source tools for data science.

We can give you some tools that actually pay off right away, and that you're hopefully willing to pay us for. And so that's why we're a PBC. A PBC kind of lies in the middle. It has responsibilities towards its shareholders, but also to its employees, to the environment, and to the community. And every PBC has to have some kind of fundamental mission as well. And so this is our mission, to create free and open-source software for data science, scientific research, and technical communication. This mission is something that's so important to us that is fundamentally encoded in our corporate DNA.

to create free and open-source software for data science, scientific research, and technical communication. This mission is something that's so important to us that is fundamentally encoded in our corporate DNA.

So that's the PBC side of the name. Well, what about the Posit side? Well, many of you know we used to be called RStudio, and that's because we mostly cared about R. In the very early days, certainly we were 100% R, but it hasn't been true for a very long time, because even over the last 10 years, we've started to produce tools for Python, and SQL, and Julia, and other languages.

And you may have heard JJ and Tarif talk about our goal of being a 100-year company. And when you think about that kind of time span, no matter how much you might love any programming language today, it seems pretty unlikely that it's going to be around in 100 years' time. So for Posit to exist for 100 years, we need to be willing to embrace many different languages over time. The other thing we need, if we want to be around for a long time, is money, and probably bags of it.

And that brings me to the kind of economic engine at the heart of Posit, which we call the virtuous cycle. So the virtuous cycle starts with us creating free and open-source software that anyone, anywhere in the world can use, regardless of means. And millions of people use those tools. And then some of those people go on to join organizations, larger organizations that have kind of special needs, and that leads them to our commercial products. And by buying our commercial products, you both give us the money to support further investment in free and open-source software, but you also give us data. Data about what are the challenges facing data sciences today that we can solve in both our open-source tools and our commercial tools.

So the way we kind of think a little,

kind of think about the split between the free and the paid is generally the tools we provide for doing data science are free. So that's tools like the Tidyverse and Tidymodels for doing data science in R or many of the hundreds of R packages that we maintain. It's also true for tools like Shiny, which allow you to create interactive apps for R and Python, or Quarto, which allow you to create data-driven documents and using R and Python and Julia and other languages.

When you want to bring those tools into your organization, that's where our commercial tools come into play. Because large organizations have special needs. They have needs for things like authentication and monitoring, security, for scaling and tuning and data governance.

And we also charge for products that live in the cloud, regardless of whether those are our own products, like Shiny Apps IO or Posit Cloud, or tools from our partners like Databricks and Snowflake, or for generalized compute services like AWS or Azure or Google's Compute Platform.

Posit's core beliefs

So at the heart of Posit, there are really three beliefs. First of all, we believe in open source, that the era of using proprietary software to do data science is over. That open source provides such a benefit, is such a great way of doing software development, because it brings together expertise from many, many people all around the world.

We also believe in code, and clearly not getting our fonts correct. We believe if you're doing data science every day, that the benefits of learning to code massively outweigh the costs of learning to code. Now, this is certainly not true for everyone, right? Not everyone uses data science every day, and there's a bunch of different user interfaces that might appeal to different types of people. But fundamentally, what we believe in and the audience that we want to serve are people who write code.

But fundamentally, what we believe in and the audience that we want to serve are people who write code.

And so what we want to do is take open source, code-first data science, and make it ready for the enterprise, so that you can take the same tools that you used for free when you're learning, and regardless of whether you join a 10-person little business or a 100,000-person massive organization, you can take those tools and continue to use them effectively.

Posit's commercial products

So what are those tools? Well, the first one, and to my mind at least, the most important, is Kinect, because it connects data scientists to decision makers. Kinect makes it easy to deploy whatever you create as a data scientist regardless of whether that's a Shiny app or a Flask API or a Quarto document or an R Markdown document, and get that out of your hands and into the hands of the people, the decision makers in your organization.

Posit Workbench is about helping data scientists, teams of data scientists, collaborate using shared compute and providing seamless authorization and authentication so you can just get the data you need without having to fuss around trying to find the right database driver or randomly Google and finding Stack Overflow answers from five years ago. And Workbench helps you do that regardless of whether you use RStudio or JupyterLab, Jupyter Notebooks, or VS Code.

Posit Package Manager is all about bringing open-source R packages and Python packages into the walled garden of your IT organization so that your IT folks don't need to worry about you installing random code from the internet and you can still use all of the tools that you're used to.

As James mentioned in the intro video, Posit Academy is all about creating new data scientists using a project-based apprenticeship program that brings together data scientist mentors from Posit with mentors inside your organization so that folks learn the way that you all do data science.

And then finally, we have Posit Cloud, which solves problems for organizations, often small and medium businesses and education, that don't want to run their own IT organization. It allows you to compute in the cloud and publish your results to share without having to have a big team of people in your organization running your own hardware.

So let's Posit PBC. We are a company that has a mission. We want to make free and open-source software to make data science, scientific communication, and technical publishing better. And we do that by selling products that hopefully also make your lives better inside companies as well. So this is a mission that really aligns with my personal mission and is one of the reasons that I joined our studio, I joined Posit, well, I joined our team, I joined Posit, well, I joined our studio, and I continue to work at Posit today.

Next up, I'd love to introduce three of my colleagues who are going to speak about some of the cool stuff that we've been doing as a company lately. In reverse order, we have Databricks and we have James Blair, who's gonna talk about Databricks and Snowflake. George Stack is gonna talk about WebR and Carlotte Wickham is gonna talk about Quarto. Over to you, Charlotte.

Charlotte Wickham on Quarto

It's a pleasure to be here. Coming to a conf keynote always takes me back to being a little kid on Christmas morning. I get so excited about waiting to see what presents are waiting for me under the tree. And back then, just like now, there's always someone else there trying to spoil it, my big brother Hadley.

So the present for me two and a half years ago was Quarto. But the real present has been seeing what you've built with it. And we've seen some absolutely beautiful websites like this one from Real World Data Science. And I've seen you exquisitely wrap up your reports with custom output like this one from Megan Hall and this one from the R for the rest of us team. You can learn more about both of those if you head to the pour some glitter on it session tomorrow.

I really love to see people use Quarto for documentation. It's a little bit more like getting a pair of socks for Christmas. It's like slightly underwhelming when you open them but every day you put them on, you get a little bit of joy. And the less time you spend worrying about how to build documentation, the more time you get to spend writing it and the more people you get using what you've built. And in this case, that's a tool for spatial equity and you can learn more about that from Gabriel Morrison in the data science case studies session.

I'm here today to talk about two things that the Quarto team has been working on in the last year. And the first comes from looking back, looking back to the R Markdown ecosystem and seeing what was missing in Quarto. And that was dashboards.

Quarto dashboards

So Quarto dashboards are an HTML format designed for the full page layout of plots and tables and values. And this is one I found out out in the wild, somebody already using it for the Shelby County Department of Housing. The fundamental unit in a Quarto dashboard is a card. And these cards get laid out in rows and columns.

What does it look like to build a Quarto dashboard? Take a look at this HTML document. It's got a table, it's got a plot, it's got a map. What does it take to turn that into a dashboard? Well, you simply swap out format HTML for format dashboard. All of the computational outputs are going to get put into cards and then those cards get laid out on the page.

By default, that happens in a single column. You can see they're all stacked on top of each other. What happens if you want a two-row layout? All you have to do is add two markdown headings. It's as simple as that.

You can put anything in a Quarto dashboard that you could normally put in a Quarto document. But there are some specific dashboard components. So, for example, there are these value boxes. These are ways to have a single value but in a visually attractive way. And there is this idea of input panels, which is a way to group together interactive inputs into sidebars or toolbars.

You can also add interactivity in a Quarto dashboard any way you can in a Quarto document, including native support for observable JS. You can use frameworks like Leaflet and Plotly and 3JS through HTML widgets in R and Jupyter widgets in Python. And, of course, you can use Shiny. You can add Shiny and Shiny for Python inputs and outputs directly in code cells.

You can learn more about dashboards by heading to quarto.org and checking out our guide. If you want to see a dashboard in action at Conf, head to the It's Not R or Python session. It's R and Python. You can hear Nick Crane and Alenka Frum talk about a dashboard they've built that's got both Python and R components.

Now, if you've been paying really close attention, you'll notice I've just recommended three sessions. They're all happening tomorrow at 1 p.m. And life is never easy. There's actually a fourth session also with a great Quarto talk. So you can also head along to level up your data science skills to hear Cynthia Huang talk about some interesting things she's been using Quarto for in terms of knowledge management.

And I don't know what I'm going to be doing at 1 p.m. tomorrow. Probably just sitting somewhere completely confused and disabled by too much choice.

Quarto and the future of PDF

The next thing I want to talk about is the future. The next thing I want to talk about comes from looking towards the future. In particular, the future of PDF.

Let me tell you about our PBC report. This is a report that Posit makes every two years that tracks our progress toward our mission. And it is primarily a PDF document. Here's the cover of that document. Inside, there's some pages that have this dark blue hex background. There's some pages that have a white background but with some full bleed photos. Text is laid out in two columns. Some pages have a banner at the top. There's plots in there.

This year, we made that report with Quarto. And if you've made PDFs with Quarto before, you know that they go through LaTeX on that journey. But we did not make this report by going through LaTeX. We used Typed. Typed is a new open source framework for producing PDFs. And it's already supported by Quarto. To use Typed, you simply swap out format PDF for format Typed.

We think Typed is the future of PDF because it's really easy to get started. Quarto bundles Typed, so there's no separate install to manage. You can do it today. Typed is really fast. So rendering a PDF via Typed is much faster than rendering a PDF via LaTeX. We think it's easier to learn. There's great documentation on the Typed site. And the syntax feels a lot more familiar to people that are coming from languages like R or Python or Julia. And it's very customizable. So our report being an example of that. It took a little work to get that customization, but you can produce really beautiful documents.

We think Typed is the future of PDF because it's really easy to get started.

So we think it's the future of PDF. It maybe isn't quite the present of PDF. Typed is still developing, and our support in Quarto is still developing too. But I do want to show you something really cool you can do in Typed right now.

So take a look at this table. This is an HTML table. It's built using the GreatTables package in Python. And an example from Grant Chalmers. It's on the GreatTables documentation site. And I think it's a beautiful table.

And GreatTables and GT and a lot of table packages really focus on absolutely beautiful HTML output. So what happens if you switch to Typed? Well, behind the scenes, Quarto takes all the CSS that makes that HTML table really beautiful, and it converts it to Typed properties. And what that means is when you get your PDF out, that table looks just as good as it did in HTML, but now it's in a PDF. And not only that, like Typed is sort of good enough to know that, hey, this table's pretty long. It's not going to fit on one page. I'll break it over two pages, and I'll repeat the headers and footers and make it all just work for you. It's really cool.

Community contributions to Quarto

The last thing I wanted to talk about is not what we've been working on, but what other people have been working on that enhances Quarto for everyone.

So when Quarto was built, it was great because it brought all these features of R marked down to Python users without requiring them to install R. But if you were a Julia user, the first thing you would see is, hey, you need to install Jupyter, and that's a Python thing. That is no longer true. There is a native Julia engine, so if you want to run Julia code without having to have an R install or a Python install, you can now do that. And that was an entirely external contribution from Julius Krumbegel.

The other contributions I want to talk about come in the form of extensions. And there are Quarto extensions to add QR codes to your slide, and there are Quarto extensions to add countdowns to your slide. There's already a Quarto extension that gives you Tufti-style output via types. And there are so many extensions

that you should check out this other contribution.

This is a Quarto extension listing built by Mikhail Kanwe. And it's a great place to go to see what's new in the extension world and what's popular in the extension world.

If you want to see some Quarto extensions here at Conf, you can see James Goldie talking about an extension he's built to bring the power of Svelte graphics to Quarto documents. And you can also hear Andrew Bray talk about an extension he's worked on with James to bring scroll retelling to Quarto documents.

If I haven't put enough on your schedule already, you can find the Quarto team in the lounge today at noon.

It's been an absolute pleasure telling you what we've been working on and showing what you've been working on. And I hope that together we can keep Quarto as the gift that keeps on giving.

Now you're going to hear from someone that's already given many gifts to the Quarto community. Please welcome George Stagg.

Introduction to WebR

Hello, my name is George Stagg and I'm the lead developer of the WebR system, a way to run R code directly inside your web browser.

So I've been working for Posit now for about two years and I'm still really excited to tell people that. Posit's mission of making great open source software really resonates with me. I've been working for Posit for about two years and I'm still really excited to tell people that. Posit's mission of making great open source software really resonates with me. I've been making open source software for over 10 years now. I started in research software developing for HPC

and over that time my approach to open source has always been no matter what you do, no matter what decisions I make, they must always be in the interest of the user.

And it was that philosophy that originally led me to create WebR. At the time I was supporting a group of students who were remote up and down the UK and we needed to evaluate R code in a consistent environment and capture those results. To me this felt like the best solution for the job.

If you've not seen WebR before, here's an example of a WebR application. It's a website you can visit and when you do an R console pops up and you can type code in and you can do visualizations and you can really experiment with R in a very easy way, very quickly.

This is really great for new users of R who don't have something like RStudio installed. And Hadley earlier talked about the idea of code first data science. What could be more code first than opening a web browser and typing in some R code?

How WebAssembly works

So how does this work? The main technology is called WebAssembly and what WebAssembly is, is a portable binary code format. So what does that mean?

Really this just means that if you take a piece of software and compile it, you get a big list of instructions to execute. The word portable here just means that those instructions, the binary is the same on every machine. And that's done by running that binary inside a web browser. It's really designed to run high performance applications in this way.

It does work with most modern web browsers on many devices. And one of the things it's really great at is security because it's really designed to run on the web. It runs your code in a containerized sandbox environment. In a web browser, you interact with WebAssembly through JavaScript and that has positives and negatives.

It means that WebR itself is a JavaScript application. So you need to write JavaScript to use it. And a positive of that is that it means it works great with other web technologies. A downside of that is that you probably don't love writing JavaScript code. At least not in the same way that the people in front of me love writing R code or they love writing Python code.

So for that reason, in addition to working on WebR, we produce tools built upon WebR to make your job easier.

ShinyLive and Quarto Live

Tools like ShinyLive. At PositConf last year, Joe Tang had a talk introducing ShinyLive for R built upon the WebR technology. And over the years since then, we've been continuing to update the system. There's been several releases. And you can use ShinyLive now in a web browser to interact with live Shiny examples. You can export existing apps using an R package.

Or you can use a Quarto extension to embed Shiny applications directly into your Quarto documents. Again, the website example is great. You can go on to a URL and type some Shiny code and it just appears. And I've actually seen this used at workshops to teach people how to use Shiny and to share examples.

And that takes me on to one of the demographics that have really taken ShinyLive and run with it. And that's the demographic of educational content educators. They've really embraced the idea of WebAssembly-powered educational content. And for that reason, we thought to ourselves, well, is there more we can do? Can we get more WebAssembly into Quarto documents?

And the answer to that is yes. And I've been working on a new extension for Quarto over the last six months.

And we're calling it Quarto Live. Quarto Live is a way to get interactive code examples in R in your Quarto documents. You can directly interact with that code and you can create code exercises so that if you're an educator, you can create very easy ways for students to interact with your code and get quality feedback.

Quarto Live is a way to get interactive code examples in R in your Quarto documents. You can directly interact with that code and you can create code exercises so that if you're an educator, you can create very easy ways for students to interact with your code and get quality feedback.

We've designed Quarto Live to be really, really easy to use both for authors and for the students and learners.

So what does it look like? So let's say you have a Quarto document and it looks a bit like this. You've got some R code and it's producing a visualization. Here we have some air quality data from 1970s New York and it's plotting some temperatures and some ozone levels.

Now, let's say you want to turn this into an interactive example. Much like in Charlotte's examples, you change the format into a live format and you change your R block into a web R block. And if you have the Quarto Live extension installed, that's it. You do nothing else. It just, it works.

You can see the output looks pretty much identical. That's by design. And the only real difference is that your source code above your output is now an editable code block that you can interact with and click the buttons to rerun your code.

Now, this is great for students because someone who's interacting with this content for the first time may be interested. They want to know more. They want to play with this example. Previously, they would have to copy that code and open our studio and switch to a different window. But now they can literally just change the code to plot wind instead of temperature, hit the button, and it's there in the same page inside their document.

By the way, one of the things I'm really excited about this is that those are no longer pictures that are distributed with your document. They're generated dynamically on the fly every time you open that document. That means they're kind of reproducible by default, right? Because the fact that you can see that image at all means the R code in your document has executed successfully and produced some output.

Creating exercises with Quarto Live

If you're an educator and you want to augment these code examples into an exercise, you give it an exercise label. At that point, you can then add things to make that exercise more and more interactive. Here's an example where we're asking a student to plot some data, right?

And what we're going to do is we're going to augment that with some setup code. So we create another WebR block. We tell it it's a setup block and this is the exercise that it's linked to. And what we've done is we've set up that data set for the students so they don't have to worry about it. They don't have to worry about loading that data. It's just there. They can just directly answer that question without having to worry about the extra parts. And that's because a setup block will run automatically every time when that student hits the run button.

So let's say the student tries to run their code. They see, okay, there's a blank. I need to answer this question. But they're struggling. They're new to this. They're not quite sure what's going on. Can we help?

Well, we could add a hint and we do that using a quarter block. Here's a quarter block with the class hint and we've, again, linked it to our exercise using that same shared exercise label. Inside there, you can write your hint and you'll notice it's not added to the document straight away. Instead, QuartoLive adds a button to the exercise saying hint and when the student clicks it, then they get the hint text containing your block. And if the student's still struggling, you can do the same thing with a solution block. You can write your Quarto code inside a solution block and when the student hits that button, they see that solution text.

What's really great about this is that this is just Quarto syntax. There's nothing new here. All of the tools that you know how to write in Quarto to make great content will work inside these blocks. Tools like call-out blocks, tools like collapsible panels, syntax highlighted code blocks, code annotations to really explain how a piece of code works. All of these things work inside a solution block and a hint block. So you can make all of these tools work with your exercises straight away.

Now that the student has seen the solution, they can then obviously continue to interact with the code exercise, paste in that code and run it and ah, I'd recognize that logo anyway.

A nice, well, maybe not so nice, an interesting thing about educational content is that lots of students now are not really interacting with this content using a laptop or a desktop computer with a big screen. They're interacting with this content on screens that look more like this.

And we've really designed WebR to work as best as it can on these constrained devices so that all those tools like visualizing and plotting and producing graphics, they work on something like a mobile phone or tablet, even if these devices themselves can't install something like R or RStudio. And much in the same way that Quarto itself resizes very nicely on these small screens, Quarto Live has again also been designed so that these exercises work well in these kind of environments. The idea here is that we want to be able to support education for any user, no matter what kind of device they're using.

Some more quick examples of the things you can do with Quarto Live. If you've ever used the LearnR or GradeThis system, you may be interested in writing algorithms to grade students' code. You can do that. Here's an example where we ask students to do some filtering and summarizing of some customer reviews of a cat bed.

And if the student makes a mistake, we can give some encouraging feedback to say, you know, try again, you're almost there. If the student gets the answer right, we can give some celebratory feedback to say, great job.

What's really great about this is that the R code that the student writes is really being evaluated. And your grading algorithms are also really being evaluated. And then these are all written in R code. So you can catch certain mistakes that you know students are going to make. Say, you know students are going to use a double ampersand here when they should use one. In addition to the error text from R, which may not be entirely clear to a new student what that means, you can add targeted feedback to say, oh, this is what you've done wrong and this is how you can fix it.

The tools that make this work are very dynamic and very general. They're not just designed to work with exercises like this. In a Quarto Live document, you can use observable JS inputs with WebAssembly blocks to create a dashboard-like experience deployed to some kind of static server, something like GitHub Pages, something like Quarto Pub. Again, these tools that authors already know how to use, they can deploy to these places without having to become an expert in DevOps, learn how to run a Shiny server and manage these Shiny applications. Instead, you can just fire and forget.

So, that's Quarto Live. It was released last week. It's available on GitHub. It's free and open source. It's a Quarto extension and I'm really super duper excited to see what you create with this kind of tool. Everything from educational content for colleges and universities to interactive demonstrations of R packages in documentation.

It also works with Python. It uses the PyDyed WebAssembly engine so you can also create exercises in Python too. And you can even share data between WebAssembly and PyDyed, between Web on PyDyed if that is something you're interested in. Okay, I'm going to head off now. Thank you very much for listening. I'm going to hand you over to our next speaker, James. Thank you.

James Blair introduces himself

Thank you. Yesterday, I noticed a particularly interesting sticker on Hadley's laptop. So, I asked him about it and he told me that it was for particularly strong contributors to the Tidyverse and then he didn't offer me one.

Which isn't really surprising and in fact kind of begs the question, what am I doing up here? And if you find out, please let me know.

I'd like to start today though by turning the clock back to an experience that we all likely had at some point early in our journey, which was something like this. For the very first time, we opened up an environment or a tool like RStudio or VS Code and we wrote some initial lines of R Python.

Now, if you were like me, some of those initial lines of code read in some sort of local data source and then allowed me to start exploring that data object using a new and exciting world of open source tools. This was my very first introduction to this data science life cycle. This idea of bringing data into an environment, use tools to explore, visualize, analyze, clean it up and then report out on the thing that I've learned.

This period of my life was a period of tremendous excitement. I still look back on this period of life and feel butterflies about how exciting it was to learn all that I was now capable of. And part of the reason that it was exciting at this point was because everything was very easily understood. The work that I was doing

was all happening in the scope of my own little laptop. The tools I was using and the data that I was accessing were all available right in front of me, easy to understand and think about.

However, over time, things have changed a little bit. The data moved to a database. I had to figure out how to access that. Then the tools I was using moved to some sort of cloud platform. Now the only gateway I had was no longer some sort of process running locally, but rather a web browser that I was using to access some service running somewhere else in the cloud.

Now, at the end of the day, I'm still doing this. I'm still accessing data. I'm still working with it, understanding it and then communicating some sort of result from that data out to somebody who's invested in that result.

But it can start to feel like to get there, I have to dig through an ever-growing pile of dirt. Now, we're fully capable people. We can dig in and get our hands dirty. But this is not the work that we want to be doing. Instead, we want to be working with data and we want to get there as quickly as possible.

Now, here at Posit, we've been hard at work doing the best job that we can to clear this out of the way so that you can get back to doing the work that you know and love.

One of the ways we've done that over the past year has been through our partnerships with leading cloud platforms like Snowflake and Databricks. We've looked at our community and the conversations that we have with each of you to determine ways that we can partner with these platforms to solve some of these unique challenges that exist, particularly for those of you that are working in large enterprise organizations.

We've identified kind of three key areas that we've been focused on. Infrastructure management, data access and control, and application deployment. I want to take a few minutes today to walk through some of the things that we've done over the past several months to address some of the challenges that we've observed in each of these areas.

Infrastructure management

Let's start with infrastructure management. Now, most of you out here today are end users of Posit products, whether that's purely open source users operating inside of the RStudio IDE in some environment or you're working as part of a large organization with access to our commercial tools.

However, some of you, either by choice or most of you likely by accident, have found yourself in an administrative role. This means that you're responsible for maintaining a set of infrastructure that other members of your team use to access Posit tools or other tools in the cloud.

As I speak with these individuals almost every week, I hear some version of this request. We want everything to just be managed and done for us.

Now, I can understand this request. As enterprises grow in their infrastructure complexity, the desire to simplify is increasing. And we see more and more organizations turning to cloud platforms to try to solve this exact problem. And it's easy to understand why.

If we take a look at the Posit Workbench documentation, you can see that it's rather complex in the variety of different ways it can be configured, set up and administered. Now, this is not complexity for complexity's sake. Rather, Workbench is designed to be highly flexible so that it can meet the needs of a variety of different organizations based on size, security requirements and other needs.

However, it can be difficult to manage and maintain, particularly for smaller teams. One of the ways that we've chosen to solve this is by partnering with organizations like Snowflake to provide Posit Workbench as an application that's available through the Snowflake Marketplace.

What this means is that existing Posit customers can go to the Snowflake Marketplace, subscribe to this application, input their existing Posit license key, update a few small configuration things and then provide users within their organization with access to a version of Posit Workbench that's all running directly inside of their Snowflake infrastructure.

This version of Posit Workbench will automatically be updated when new versions and new releases come out and it's a very low maintenance hands-off approach to delivering the end result that users need. And we're committed at Posit to continue to work with other cloud data platforms providing similar experience in other environments.

Data access and control

Now, once your infrastructure is set up and configured, the next thing you need is access to the data. As an R user accessing local data or as a Python user, accessing local data is typically pretty straightforward. Like the example we saw at the very beginning, there's a whole series of commands, tools and packages for reading data from a local data source, whether that's a CSV file or something else.

However, once the data moves location and becomes part of an infrastructure that's more governed and more secure than what you would typically find locally, the way that we connect starts to look like something like this. This is an example of some R code used to access Databricks to read some data.

There's a lot here. Now, it's not difficult to unpack. All of us are capable of writing code like this. But there's a lot of detail here that just provides opportunities for mistakes. I need to know what the ODBC driver is called. I need to understand what these different arguments are. And perhaps most critically, I need to make sure that I'm accessing this in a secure way that meets the security requirements of my organization, whether that's through a username and password or more often than not, some form of a security token that I don't have to remember, but I have to keep in a super secret location and update every 30 days.

That can become a challenge. So we've worked hard over the past several months to take this amount of work and turn it into something that looks more like this.

This approach dramatically simplifies our ability to connect to external data sources. Because again, until we have access to the data, the real work hasn't really started yet. We've accomplished this in a couple of different ways. One is through our contributions to the open source ODBC R package with functions like Databricks. And there's also a function called Snowflake that facilitate easy connections to these external resources. These functions are part of the open source package and are available to anyone.

We've also made some adjustments on the Posit Workbench side to make it much easier for individuals to gain access to these backend data resources.

Posit Workbench users today can now not only sign into Posit Workbench and start a new session and decide whether they want to use VS Code or RStudio or any number of other tools. They can also choose to sign in to their Databricks account or their Snowflake account.

Once they've gone through this sign-in process, which will just follow whatever the typical sign-in process is within your organization, Posit Workbench then will manage credentials for that data connection on your behalf. This means you no longer have to think about access tokens, passwords, how to manage those, how to pass those into the connection string. All that is taken care of for you and you can get in and immediately connect to these data resources.

Now, this doesn't just work for ODBC, although it works very well there. You'll also notice that it works for tools like SparklyR and interacting with Spark and Databricks. And it also works inside of VS Code for a number of Python frameworks and SDKs that have been provided by platforms like Snowflake and Databricks as well as their respective VS Code extensions.

Application deployment

Okay, so I've got access to the data. Now I'm able to do what I want to do. I explore the data. I understand what it's teaching me. I finally arrive at something that I want to share. So what do I share?

In many cases, we see people share emails, PowerPoint presentations, static plots, documents. But more and more frequently, we're observing that users are creating dynamic dashboards and interactive web applications that allow end users and business users to discover, explore, and arrive at their own conclusions.

PositConnect makes it really easy to share and deploy these kinds of applications across your organization. So I can build a dashboard that I share with the rest of my company.

However, as soon as I make this transition from saying I've built something to I want to share something, I once again have to think about authentication. And the problem has gotten bigger because now I'm not just thinking about my own authentication and how do I access the data. Now I'm thinking about how should everyone who visits this application access the data.

In a world where data security is of highest importance, the last thing I want to do is unintentionally publish an application that gives someone else unauthorized access to data they should not otherwise be able to see.

Now, this is not a new problem necessarily. In fact, there are well-established patterns for how to approach this. One of them is to use some sort of access token attached to a service account. So I create some sort of account inside of Databricks, for example, or Snowflake. I say here's the permissions for that account. I generate a token and then I input that into my Connect product, into my dashboard so that it will use that same credential set whenever somebody visits the application.

This pattern works very well, but it really has two distinct limitations. One is everybody visiting this dashboard will see the same data regardless of what their specific permission set is. They will see data governed by the permission set of the service account I've set up.

The other problem though is that if the number of applications I build grows, the amount of service accounts and access tokens I'm responsible for also grows. And now I'm spending far more time worrying about data access and security from my deployed applications than I am building and distributing and creating new applications.

We're excited to announce that starting last week, this is all gone. What we've now introduced in Posit Connect is native support for authentication into Databricks and Snowflake just like we have in Posit Workbench.

We're excited to announce that starting last week, this is all gone. What we've now introduced in Posit Connect is native support for authentication into Databricks and Snowflake just like we have in Posit Workbench.

What this means is now instead of configuring an access token, I can come into Posit Connect and specify that I want to make sure that whoever visits this application signs in to their Databricks or Snowflake account.

Then as a viewer, if I come to the application, I'll see something like this. This is my view. But if I send this to somebody else and say I want you to log into this application, they'll first be prompted to log into their Snowflake account. And once they've logged in their Snowflake account, they'll see the same application but with their own unique view powered by the permissions they've been assigned inside of Snowflake.

This means I don't have to worry about who has access to what data. Instead, I can build something and share it and then anyone who accesses it will have access to the data they should be able to see based on the permission that have been set within the database. This works today for Snowflake and Databricks and we're actually looking at other ways that we can continue to expand this functionality.

Now, at the end of the day, our mission at Posit, as Hadley stated earlier, is to create open source software for data science, scientific research, and technical communication. Now, one of our goals and my personal goal is to make sure that regardless of whether you're operating on the desktop or you're operating in the cloud or anywhere in between, you have the ability to do that in an environment that feels comfortable and familiar and is immediately and easily accessible.

If you'd like to learn more about some of the things that we've discussed this morning, our commercial products, our partnerships with organizations like Snowflake and Databricks or anything else, again, feel free to visit us in the lounge, which can be found right outside the doors in the back of this room. There'll be members from the Posit product teams there happy to answer questions and you can find me there as well. And with that, please join me in welcoming the one and only Hadley back on stage.

Positron and closing remarks

Thanks, James. So if you've been paying attention to kind of data science on the Internet or some form of social media, you might be wondering that there's something we didn't talk about today, and that's Positron.

So Positron is a new multilingual IDE that we've been working on at Posit. The goal is to take all of the things that you love about our studio and build them on top of a new modern platform, build on top of VS code. So as well as R and Python, we can continue to support data science in whatever languages come along in the future.

If you'd like to hear more about Positron, you're going to hear from the entire Positron development team in the next session after this break in Columbia A&B.

So with that, again, thank you for joining us all at PositConf in person and online. Please enjoy the rest of the conference. Thank you.