image: thumbnail.jpg
Live Q&A: August 30th at 11:30 ET | Pins Workflow Demo Q&A Session
The Q&A portion will begin at 11:30 ET in this YouTube Room and the demo *should* automatically bring you over here.
Thanks so much for joining us!
- Rachael
____________________________________________
Please join Ryan Johnson, Isabel Zimmerman, and Rachael Dempsey here for live Q&A following the end-to-end workflow demo on pins on August 30th.
If you end up here first, here's the link to the demo room: https://youtu.be/t8A-ysXinpE?feature=shared
Please use the YouTube Chat for Q&A or feel free to ask questions anonymously here: pos.it/demo-questions
If you'd like to add future end-to-end workflow demos to your calendar you can use this link: pos.it/team-demo
Follow-up links:
* Demo recording will be here: https://youtu.be/t8A-ysXinpE?feature=shared
* Posit Team: https://posit.co/products/enterprise/team/
* Talk to us directly: https://posit.co/schedule-a-call/?booking_calendar__c=RST_YT_Demo
* Posit Team demo resources: pos.it/demo-resources
Thanks for joining us!
Transcript#
This transcript was generated automatically and may contain errors.
Hey everybody, we're going to wait a minute here for everybody to come over from the demo. Thank you so much for joining us.
How's everyone's day going? Let's see if it pushed everybody over here. I just re-shared the Q&A link in the chat as well.
We'll give everybody, let's say two minutes here. Hey Sanjay, hey Sergei. Nice to see you all in the YouTube chat here.
Okay, I can see a good number of us are coming over here so I think we can get started. But hi everybody, so nice to see you here on YouTube. Hope everyone's having a great Wednesday. And thanks for joining us over here for the live Q&A. Thank you so much Ryan for the great Pins demo as well.
I guess we could all introduce ourselves here too. If I haven't met anybody yet, I'm Rachel Dempsey. I lead our customer marketing and pro community here at Posit. And Ryan and I are joined by a special guest here today. Isabel, if you want to introduce yourself.
Hi everyone, I am Isabel. I'm the maintainer of the Pins for Python package and also the setter for Python package. Which is a model ops package that lives very closely to Pins. So I've got a lot of love for this nifty little tool that I get to use all the time. It's quite useful for my everyday work. So thanks for having me.
That's awesome. Thank you again for joining us Isabel. Ryan, I know you introduced yourself before in the demo. But why don't you introduce yourself again here. Yeah, absolutely. So I'm Ryan. I'm a data science advisor here at Posit. And I'm the one whose voice you just heard for the last half hour. I really enjoy giving these demos. And I hope you all are finding them valuable. It's pretty hard to believe we've already been doing this for five months. But hopefully this structure works well for you.
I saw a lot of great feedback afterwards in the chat. And so we're certainly going to incorporate that feedback into future demos. I would love any additional feedback. And love all of your questions about Pins, workflows, Posit team, all that fun stuff.
Community check-in
Awesome. Well, I was talking to a few different community builders. And we were talking about fun questions to ask people in the community. And I would love to ask you all here, what's something that you're proud of this month? And maybe Ryan and Isabel, you could answer that for me too. Isabel, you want to go first?
I put everybody on the spot here. But I love hearing these answers. I think in a non-data science world, I'm so proud. I have a five-month-old puppy that is a little high energy. And she finally learned how to sit and down this month, which is very exciting. Maybe in a data science world, I have been a longtime contributor and user of Pins. But it was just within the last month that I was promoted to maintainer. So that is also something I'm very proud of and excited to continue working with.
Congratulations. That's awesome. Ryan, what about you?
I would say from like a non-data sciencey, non-Posit thing, my wife and I are expecting our first kid in December. And we procrastinated a lot in preparing for this kid. And I finally painted the baby room. And I am very proud of that. Congratulations.
Okay, I'll go. Non-data science, I've been taking learning guitar more seriously the past few months. And I've actually been practicing a lot more. Work-related, we're working on releasing a new Posit Connect page that I've worked on a lot, the first iteration of it. And hopefully that's going to be going live this week. So I'm really excited about that.
Okay. I don't know if anybody was writing in the chat what they're proud of, too. But we'd love to hear that. I really enjoyed it. It said, like, yay puppies, yay babies. Yeah.
Questions about pins
Awesome. Well, let's jump into some of the questions from earlier. As a reminder, you can ask questions anonymously, too. But you can also use the YouTube chat here. So on the screen it says the short link, POS.IT slash demo dash questions. That will bring you to Slido where you can ask anonymously. And we'll be hanging out here for 15, 20, however many minutes you all are asking questions.
But one of the questions from earlier was, are pins mainly for sharing data between users? Or can R code be shared or pinned as well?
Yeah, I can probably take that one. And, Isabel, you can add any additional comments. Pins are really great for sharing typically objects. So things like data sets, models, plots, objects in R and Python. In terms of code itself, we would certainly suggest some type of version control using Git, for example, publishing your source code to GitHub, which makes it super easy to share that source code with other users. So I wouldn't necessarily say pins is a great way for sharing code. But it is a fantastic way for sharing the objects that you're creating in R and Python.
Thank you. I would completely agree with that. If you're looking to use the best tool for the job, I would say version control for code. And then pins for a little slice of data or not necessarily large data sets. This is not a replacement for a database. But being able to share code quickly with others, probably not for pins.
So, Isabel, I know you helped me answer this question in Slido. But the question was, why do you paste the API key in the Jupyter notebook? Is that a security risk?
So this is a demo. This is not a security risk for us in this very moment. It is best practice to be using a package called .env in the Python world. In the R world, it will automatically load from your environment when you make the board connect. But in Python, if you use that .env package, you can use a .env file and it will load in all of your environment variables. Then you don't have anything in plain text. And you can share your screen with others and they won't see your API keys.
And .env is just like B-O-T-E-N-V? Yep. Kind of like the phonetic spelling of the period .env file. Okay. Awesome. Thank you.
Okay. Another question I'll copy over to show on the screen here. Is wondering about the speed with large data sets or objects and the relative merits of different object load methods. Local, directly from pins, loaded into memory from pin. Can either of you speak to that?
I can at least start in something Isabel already briefly mentioned. Pins is really great for sharing these like final products that you've built. Whether that be like a cleaned up data set or a polished model or just really pretty plot that you want to share with other people or other pieces of content, for example. It is not a substitute for storing terabytes of data, for example. And you should really consider using a database for stuff like that. So I always personally like, you know, if I have a raw data set, I clean it up. It's nice and pretty. And I want that data set to feed into a lot of different pieces of content. That's a perfect use case for pins, in my opinion. Isabel, anything else you want to add there?
No, I think that was great. There is a suggested limit on the pins documentation of 500 megabytes. That is what we have tested up to. And we know pins is super equipped to handle. So anything after that is a little bit at your own risk for the speed. And I guess the merit of loading from different places.
Thank you. I'm going to jump over to some of the YouTube questions here. And I see Rebecca. Hi, Rebecca. Asked, are there other features? So pins is newish. Are there other features in the pipeline? I see the delete old versions mentioned. Don't mean anything is lacking. Just curious.
So for between the two packages, there's a little bit of work right now of getting pins Python, which is even newer than pins are up to speed. Specifically looking at versions and non-version boards, looking at being able to de-parse boards. So move these into different files for like pipelining things with data burn specific. I don't know if there's really anything big on the horizon, but we're always looking for community input, especially when people are looking for new locations of boards. There are other locations beyond Posit Connect like AWS or Google Cloud. GitHub was one Kaggle was one as well as like local folders as well.
Just to follow up there. So is it 500 megabytes per pin object or 500 total in pin? Per pin object. Okay, cool. Thank you.
Let's see. Harlan, I see you had a question here. Can the data stay on the data server without having to be transferred to the pin server? This is just to keep data security model consistent without having to have two data servers.
Okay, so if I understand this question, there's another thing about two different servers here. Data server, which maybe has like a database on it. And then the pin server in this example would likely be Posit Connect. It's really going to be up to your use case. A lot of folks, you know, take their data from a database. They pull it in and clean it up and they can reload it to that same database. And that's totally fine. That might be a work well for your team. But if there's a, if you have Posit Connect and you have these polished data sets and you want to share that content with other pieces of content on the Connect server, you know, I think it makes a lot of sense to publish it to Posit Connect so that pin can be easily made available. But I don't think there's really any right answer to that specific question. It's kind of based on your team's need. Anything else you want to add, Isabel?
No, I think that was a good answer. And it might be helpful. I know I answered another question with this, but we're also happy to chat directly with you, like one-on-one, so we can talk about your specific use case. And I put a link in there just a few messages ago in the chat. So feel free to use that, even if you're just curious if other people across your company are using Connect as well.
I see, Joseph, you asked a question that says, is there any way to schedule a workflow to run based on an event? For example, every time a new data set is downloaded to a directory.
Yeah, so for the Connect server, at the moment, it's really based on timed scheduling. So you set a time and then you can set it to every day, every second. It could be as infrequent as every year, two years, if you had a workflow for that, which would be kind of odd. But how most folks kind of get around that is they can have it, you know, check for updates every minute. And if yes, kind of program these if statements. If yes, it can do something. If not, it cannot do something. But I currently, within Posit Connect, at least, there's no trigger to set off a workflow, other than just kind of a scheduled report.
Sending emails from Connect
Thank you. I see there's a few questions on sending emails. And so maybe we'll try and group some of these together. But one was, is there a way to send the email with the content in the body of the email rather than as an HTML attachment?
Yeah, I can speak to this one. So the answer is yes. So when we went through today's demo, I created a Quarto document and I placed that on Connect. I set that to run. And as you might've seen, when you send that email, it's typically going to be a direct link to that content on the Connect server. However, with R Markdown, there's another package called Blastula. And some of you may have heard of this package before, but what that actually allows you to do is to embed plots, tables, text directly in that email rather than a direct link. So the answer is yes, you can embed content from an R Markdown document using the Blastula package. And I'll actually, I'll find a link here because we have some documentation on you doing this. So, but yeah, the answer is yes. It sounds like that could be a fun end-to-end workflow example, like really customized emails.
Yeah, yeah, I think that would be really cool. Yeah, I'll say about that. I'll say we have some other fun stuff in the works, but I won't say anything else right now.
I see George asked a question over on Slido and said,
pins is awesome. If I used it, I would probably have a hundred plus pins. How would all of these be managed, searched, or categorized?
There's a few different ways you can do this, especially with using Connect. So Connect has some built-in organization tools, including the ability to add labels to various pieces of content, including pins. So for example, if you had maybe of those a hundred plus pins, maybe you had five or 10 of them associated with a specific project, you can easily give those a label and search for them easily within the Connect instance.
There's also other tools. Some of you may have heard of a tool called Connect Widgets, which allows you to take various pieces of content. It doesn't have to be a pin. It can be a pin, a Shiny app, a Quarto, or a Markdown, and essentially create a single document that organizes all that content. So you'd certainly have options for organizing your content on the Connect instance, and I would definitely suggest using the various labels and also Connect Widgets.
Awesome. I just shared the Connect Widgets link in the chat, too.
I know there's a great demo that our solutions engineers have put up collecting pins, so I might want to find this link for everyone because it is beautiful, and they have it for both R and Python. Is that with the bike share demo? It sure is.
Let me jump back over to YouTube. Oh, and Marcos asked, what was that email package called again? Blastula.
Rachel, I just sent you a link, if you don't mind just posting that on Slack. Oh, yep. We have so many ways of communicating with each other. Communicating with each other right now. I'm like, where am I?
Okay, another question over from YouTube. Can you use the usual pin read in a Shiny app, or does it have to be pin reactive read?
Do you want to take that, Isabel? I can also use that. I'm not a huge Shiny user, but I believe the way this works is pin read is if you want to read it once, and then pin reactive read is if you want it to update every single time some reactive component changes. Yeah, that's a perfect answer. That's exactly what I was going to say.
Arthur I see asked in YouTube, any plans to offer pins for other languages? Not currently, but if someone's interested in starting their own pins for another language, we would certainly love to support you in your journey.
I saw Libby mentioned in the chat, learning about Blastula from Matias in a data science hangout episode where he talked about integrating with Slack. And I will definitely share that one in the chat here as well. That was a great call out Libby.
Oh, well, here's the easy question for me. Is this Q&A recorded so we can re-listen to it? Yes, it is. So the recording will be made available immediately at the same link here.
Follow-up questions on emails and Blastula
Follow-up question on the emails. Could you just create the email on Workbench and have that run from Connect when the data is updated for the Connect pins?
Yeah, could you just create the email on Workbench and have that run from Connect? Yeah, so actually the link that Rachel posted a bit earlier, it's on our solutions.posit.co website, where we talk about leveraging Blastula and creating emails. The way that article actually works, it creates like a conditional email. So it only sends an email if, for example, the data is updated. And then it'll send an email. If it's not, it won't send an email. So I think you might be able to find some value checking out that article there. But you can certainly code up these solutions to meet your needs however you want.
Marco said, I need something like Blastula, but for Python. Isabelle, any ideas? That's a great comment. I will take that to heart. We're always looking for fun new packages that exist in R especially, because the Python world has so many great tools. But sometimes it's those little things that are so useful. But I think R has invested a lot of time in tiny little packages for everyday things. Thanks for the tip, Marco.
Drawbacks of using pins
Another question in Slido is, what are the drawbacks of using pins to store data? For example, would it be hard to find the docs if you have many?
I mean, pins is pretty awesome. I don't see a lot of drawbacks. Now, certainly you don't want to just take every single data set that your team has ever created and use and just pin it kind of all willy-nilly. There should be some kind of strategy for taking this finished object, whether it be data or a model, and serving as a pin for a specific purpose. I think that's kind of what you want to sit down with your team as you're considering these pins and what you should do with them. But not too many drawbacks that I can think of. It is your favorite package, I think, right? It's definitely up there. It's a top three for sure.
I think it could be hard to find certain pieces if you have one data set that you've pinned a thousand times. It would be difficult to parse through each of those individually. There is a little pin secret is you can add custom metadata. So maybe if you have a lot of data that you're scared you're not going to be able to find it again, you could add some custom metadata fields that you could search later. And that might help with the large iteration over a certain data set. And there are documentation on both of the websites on how to add custom metadata and read it back in really idiomatically.
There is a little pin secret is you can add custom metadata. So maybe if you have a lot of data that you're scared you're not going to be able to find it again, you could add some custom metadata fields that you could search later.
That's a really good point. Yeah, if you have those links, you can share with me. It's always fun to be able to share pin secrets.
Sharing boards and collaborating
I'm trying to remember if I had asked this question. So apologies if it's duplicate. How would one go about sharing a board in a project that has multiple collaborators that all need the same data? How does the project know about the board?
Sharing a board. So, okay. So is this multiple people working on a single project?
Yeah, well, the way I interpret it is if multiple people are working on maybe different types of content, like a shiny app that would need a pin or maybe different Python visualization or Quarto doc. Like how does that project know that this board exists with the pin?
So every time you're creating a new document or a new artifact, or what I think is what is being referred to as a project in this question, you're gonna have one line of code too if you wanna return it across multiple lines that says like board equals board connect or board equals board folder. And that is how that specific piece of code knows about the board. And you will run that every single time you create a new artifact. Let me know if that does not answer your question though and I'm happy to elaborate more.
Yeah, and I think like what Isabel mentioned, like we talked about pinning to Posit Connect. That is one type of board. There are plenty of other boards you can choose from based on how your team operates. So again, for these types of demos, we're really just trying to highlight the advantages of using Posit Connect as your board.
I was copying over into the chat what Isabel mentioned on custom metadata. We just shared the link there too.
Another question from YouTube was, so if you're on a team and you do the work in Posit Workbench, how do you pin the output for your team instead of it just being used by yourself?
Yeah, if you work in Workbench, how do you pin the output for your team instead of yourself? Yeah, so that's actually kind of like a perfect use case for Posit Connect. So Connect, you know, the take home message is that it takes anything you publish to it, a pin, a shiny app, quarter document, anything, and makes it easily, but also kind of, you can totally customize who has access to it and who you can share it with. So we talked about, I'm not sure if we covered a lot of detail in today's demo, but once it's hosted on Connect, there are sharing settings. And so if you want to take that pinned output and you want to share it with your teammate, other people that have access to Posit Connect, you can be very explicit and kind of call them out individually. You can open it up to everyone who has access to Connect Server, or you can open it up to the world as well. You have a lot of options for how you can share this pinned object with your colleagues.
Pin versioning and archival
Oh, what is the, I think we touched upon this just a little bit, but what is the pin archival process?
Yeah, and I guess maybe I, just to make sure I totally understand the question, I think it's referring to probably more like the pin versioning. And so as you take a pin and that data will essentially change, most data does, you can pin it to the same location and it doesn't overwrite the original data. It just creates another version on top of it. And so you can essentially have multiple archive versions of the same pin in one location rather than having like 20,000 data sets.
And I know as we've been teasing this demo on like LinkedIn and stuff, we talked about how do you take a data set or anything and make it like the final version? Like, what do you call it? Is it data final, data final one, data final two, data final, I'm totally serious dot CSV. This, the versioning allows you to just have it in one location and the most recent version will always be up to date, but you can always go back to previous versions as well. And you can kind of set thresholds like how many you want to save. And Isabel, I'm not sure if there's like, there are ways to trim certain versions as well. Is that correct? That is correct. You can use something called pin prune, I believe is the function that you would be looking for to remove pins at a certain interval.
Yeah. I loved seeing everybody's responses when I posted about that, like data final, data final, final. Someone said their favorite one was presentation V7 final, don't use this one.
Size limits and feedback
Rebecca, I think I maybe missed this question earlier, but you said, I see the 500 megabyte in the documentation. Is there a way, or maybe it exists to throw a warning if you're trying to pin something larger than that?
I don't know off the top of my head if it does throw a warning or not. Connect will fail if you pin something larger or like so large that it can't be handled. I don't know if we would ever have like a hard error just because that is a little bit of a fuzzy, like if you pin something that's 501 megabytes, I don't want your entire pin to fail, but this is great feedback that maybe it should be either something subtle or maybe a few more times in documentation or somewhere because yes, it is hard to remember all those nitty gritty pieces as you're trying to just complete the end to end workflow. Yeah, that's good feedback. Thank you.
Contact and follow-up
I see someone had asked in Slido, could you provide an email address for follow-up questions? So I don't think I'll regret sharing this here, but you can email me directly. So it's just rachel at posit.co and I'm gonna put that in the Slido as well. And I'm happy to connect you with the right person on the team or help make sure we answer your questions too. I guess I could copy that over here into YouTube as well.
And if you have specific questions on the open source side, feel free to open an issue. It doesn't have to be like a bug or a feature request. It can also be like, hey, I've been thinking about pins in this way. Does this use case make sense or something like that? We love questions like that. And we'd be happy to get back to you.
That's great. Thank you as well. Another question is about, and we didn't mention conference yet, but the Posit Conference is coming up in just a few weeks here and would love to see many of you there. But one of the questions was, will some of this be included in the Posit Conference in Chicago? If so, which sessions?
The only thing I can say, we do have a kind of like a professional tool lounge where we can just hang out. You don't have to be a customer of Posit. You can also just, if you're a, you know, you love Posit and you love data science, a lot of Posit employees are just hanging out and we love talking about this stuff.
But I actually will be there. And I think at least twice we will be hosting some demos as well, some live demos. And Rachel and I were actually just talking about this yesterday. And we think like incorporating pins into one of these demos, since obviously as you all can see, there's a lot of interest in this package and these various workflows that it enables.
So that's the only thing I can speak of. I'm sure there's sessions and talks that are going to talk about pins. I don't know if Isabel, if you know any off the top of your head.
I do not. I know there's a few great workshops that include it.
I'm sure it's in there somewhere. I just put the link to the talks and workshops so you can search there. And I think you should just be able to search on pins as well and see which ones.
But I will second what Brian said. I'm really excited for the lounge. That will be where I'm hanging out at the Posit conference too.
I guess I'll say it here in case anybody's going to be there. It's going to be right next to the main keynote room, so you won't miss us there. But there'll be a demo screen. So I had just slacked Ryan earlier and said, hey, can we do one of these pins demos there in the lounge too?
Let me just say, if there's a question you've asked in YouTube and I've missed it from earlier, feel free to just copy and paste it in again or bring it to the top here. So I'm sorry if I've missed any.
And I'm going to go jump over to Slido real quick and see. Oh, this was one I missed.
Downloading pins and sharing boards
Can somebody download a pin data set or plot directly from the board? They said, I can see this being useful for a not so technical supervisor.
With Posit Connect, you absolutely can. There's, when you click on the pinned object, there will be a link to directly download the pinned object. I'm not sure for other boards, but I know you can at least do that for Connect.
I'm also not sure for other boards, but Posit Connect plays so nicely with pins. It's beautiful.
I see another question from Christopher, which is, if I create a board locally with the intention to share with other users, does the board need to be created in a shared drive or can other users connect to a board in my file system?
My understanding is that if you do intend to share that with others, it would have to be on a shared drive. But you can use things like Dropbox and S3 Buckets for more collaborative boards, including Posit Connect, or you can share to a local file, either on your personal computer or if that file is on a shared network, you can share it that way.
Wrapping up
I just realized we were over the top of the hour and didn't even notice how quickly that went by. But I think we have gotten to most of the questions. But again, if there's anything that we missed, feel free to reach out to me directly here. We can also continue using the Slido link, so if you wanted to just keep asking questions here.
But for some people who had one-off questions where you wanted to talk to us specifically about your use case or your team's environment, I did make this little short link where you could schedule a call with us too and chat one-on-one.
Let me put this in the chat here.
But thank you, Ryan and Isabel, here for answering all these questions and joining us today. Thank you, Ryan, for the great demo as well. And thank you all for joining. Hopefully, I'll get to see some of you in person at the Posit Conference too, or even virtually if the conference is available virtually as well.
Thanks, everybody. Thanks. Bye, everybody. Have a great rest of the day.
