image: thumbnail.jpg
Data Science Hangout | Mythili Krishnaraj, AXA XL | Platform Governance With a Shared Vision
Mythili provided a really helpful view into the IT/technology perspective and shared the steps to platform governance at AXA XL.
✨ At (24:14) Mythili shares some guidance on how their team manages 200 users:
With onboarding of users, they are put into different AD Groups. Some users might only use Workbench and are not publishing content to Connect so they are put in different AD Groups.
Something that was initially lacking for us is that we didn’t know who was successfully onboarding or when someone left and we needed to revoke access to reallocate the license.
The governance was thought about as what can we do in terms of each steps?
First, with the license:
1️⃣ How do you get them onboard?
2️⃣ How do you keep an audit trail of what they are doing?
3️⃣ How can we tell when they are leaving?
One other consideration was - what if someone is not able to use the license for 5-6 months? We started putting a timeline around it. If they don’t use the platform for 6 months, we get in touch with them and find out whether they need the license. Otherwise, we take it back.
The team has also started their own R training to support new users. We have helped them with a sort of Wiki to put together all of our R resources so that they know who to go to for a request, how to connect to a data source, etc. That sort of knowledge sharing was a requirement.
In terms of governance, we have also segregated the environments now: we have separate development and production. In development, we also have staging and non-prod.
When they are ready to publish a model, they come through a ticket. We go through it to see what they are deploying, which are the data sources, have they used any hard coded passwords, etc.
There is also a validation framework which we are using. Are they using licensed libraries? Are they using proper packages?
✨ A few tips also shared at (17:06) for IT and data science teams to successfully work together:
Help the IT team understand the value of what you’re doing. Maybe your eventual output takes a week, a month, or even six months. It’s very important for them to be able to understand that value of what you’re going to bring in.
Explain to IT the flexibility that you need. Analytics and data science is all about the speed that comes with any platform.
It’s always a two-way street. Let them know that you’re here to support the governance, security and stability as well. Say, “I’m going to be really supportive of your guardrails but at the same time I’m expecting XYZ from your team.”
Collaboration is required. Each team knows how we work now and there are sometimes tradeoffs for both sides, but we are able to deal with that because we want to do what’s best for the business. This objective is the same both for technologists and the data science world.
Where the conflict comes in is when there is no shared vision. To get that shared vision, both teams need to talk.
🎙This conversation also reminded me of the meetup with Gordon Shotwell at Socure on Creating Secure Systems for Growth: https://www.youtube.com/watch?v=UnLpB4IDpZU
🎙Follow-up blog post on that meetup above as well, “How Data Scientists and Security Teams Can Effectively Work Together” https://www.rstudio.com/blog/how-data-scientists-and-security-teams-can-work-together/
Where to find more?
► Subscribe to Our Channel Here: https://bit.ly/2TzgcOu
► Data Science Hangout site: rstudio.com/data-science-hangout
► Add the weekly Data Science Hangout to your calendar: rstd.io/datasciencehangout
Follow Us Here:
Website: https://www.rstudio.com
LinkedIn:https://www.linkedin.com/company/rstudio
Twitter: https://twitter.com/rstudio
Transcript#
This transcript was generated automatically and may contain errors.
Welcome, everybody, to the Data Science Hangout. If you're joining for the first time, it is so nice to meet you. I'm Rachel. The Data Science Hangout is an open space for the whole data science community to connect and chat about data science leadership, questions you're facing and what's going on in the world of data science. So these sessions are recorded and shared to the RStudio YouTube, as well as the RStudio Data Science Hangout site.
Try to get them up at least a week after each session. So you can always go back and rewatch or find helpful resources. We also have a LinkedIn group for the Hangout, too. So if you ever want to continue a certain discussion, or if anybody wants to talk to each other in there, other than me just talking in there, please feel free to join that group. And Tyler or Hannah will share it in a second in the chat. Together, we're all dedicated to creating a welcoming environment for everybody here. So we love when everyone can participate and we can hear from everybody. So there's three ways that you can ask questions today. You can jump in by raising your hand on Zoom. You can put questions in the Zoom chat. Feel free to put a little star next to it if you want me to read it, or I can call on you to introduce yourselves and add some context, too. Lastly, we also have a Slido link where you can ask questions anonymously. And Hannah will share that in the chat as well. I just want to reiterate before we get started that we love to hear from everybody, no matter your level of experience or area of work, too.
And today, I am so excited to be joined by my co-host for the day, Mythili Krishnaraj, Global Delivery Lead, Pricing and Analytics Platform at AXA XL. And Mythili, I would love to have you just start by introducing yourself and telling us a little bit about your role, the company, and maybe also something you like to do in your free time outside of work.
Sure, sure. Thank you, Rachel. That's quite a warm welcome. And it's nice to be here because being, you know, a customer on the other side of RStudio, and I keep telling this to everyone, that it's become a product which I started loving so much. So I'm glad to be in the Data Science Hangout today. And I'm Mythili Krishnaraj. I'm part of Global Delivery Lead for AXA XL for Pricing and Analytics Platform. I've been in the industry for more than 20 years in terms of insurance and technology, have played different roles in terms of data and technology. Today, I'm here to share some of the experience with RStudio we had over the two years in terms of the maturity of the platform within the organization.
And free time, yes, I am a busy mom with two kids, but my older one is 16 and the younger one is nine now. So I'm getting a lot more time, I can say. I started my Executive MBA in March this year. So pretty busy with all that. But I love reading. And I love also like some artwork like glass art or crotchet or sewing. So that's what about me and I'm so, so happy to be here today. Thank you.
Looking ahead: cloud migration and team work
Thank you. So while we wait for questions to come in from everybody, I'm curious, what's something that you're really excited about thinking about the data science team in the next year ahead?
Okay, so it's a very exciting year for us. Because when I was talking about RStudio, it's been on on premise servers for almost from 2018. I joined the team two years back, taking the management of completely the platform. But it's been on on prem and we are in the process of migrating everything to cloud. We started the work this year and this is going to be one of our strategic platform for the pricing and analytics world. So I wanted to see the product getting evolved. And again, that's going to be a completely different experience in terms of again, bringing the maturity. And also, you know, always the operational cost comes for discussion, whenever we do such migrations and getting the platform there. So I'm really looking forward for that.
That's great. So to be able to put us all in kind of your team's mindset, would you be able to give us an example of some of the work that your team's doing?
Sure. So when I said I'm in delivery, global delivery, lead pricing analytics means like platform means a lot for different organizations, let me put it in, you know, in a easier box. So the platform lead comes with responsibilities in terms of security, because AXA XL being, you know, one of the global players, and also insurance and commercial space. It's very much data privacy, GDPR, and all the other security regulatory is quite hard for us. We have to be very, very careful with the data we handle, and also with all the integrations of different systems. So my role is to ensure that I'm aligning to all the security policies that is put forward by information security, as well as it also comes with operational work, like all the data analytics work and all the data science work, pricing work, because we use RStudio also for pricing.
Because in insurance, if you see, we have pricing as one of the important functionality of underwriting. So we use it for that as well. So we have to ensure that the platform is stable. And there is one property model we have for pricing, which I can say that 24 by seven, it should be up running. So that is, again, it comes under my responsibility. And apart from that, you know, definitely the budgets, right? So anything you run in terms of platform, it's not that you have huge investment to do, but you have to make sure that you have the right platform to have huge investment to do. So with the budgets, like given budgets, and also considering what is coming in terms of expansion from different users, I have to ensure that everything goes by line. So that's quite, you know, in a summary. And along with that, it comes all the other responsibilities that's tied to it.
Thank you. That's really helpful. I see Alice just started off with some questions in the chat. And Alice mentioned she can't unmute right now. So I'm going to read the question for her. But it's, hi, I'm curious, what were the key factors that led you to want to migrate to the cloud? And how many data folks do you have in the organization?
Okay, so the main reason why we are moving to cloud is not only for this particular platform. It's the whole decision of the corporate taken by AXA and then AXA XL. By 2023, we wanted all the data centers to be gone. And we went with private cloud. And now it's more of hybrid, choosing private and public. It's the decision of moving to cloud is again, you know, different reasons. One is the operational cost, definitely. Because if you see, we had, so when I say AXA XL, we are part of XL group and XL group acquired Catlin. Then we were again, you know, acquired by AXA. So we have like, say, three integrations gone into this whole of this acquisition. And because of that, the technology stack is huge. And we are trying to streamline everything. And in terms of the cost also, we wanted to say that, okay, move it to cloud. And cloud is also, you know, way of looking into saying by demand, you put on the infrastructure rather than just giving the maximum. So that is the thought process behind. But I will also, yes, agree to a point that cloud could be pricey. It could come up with its own, you know, different way of configuring it. And but since we know how much we have spent all these years in terms of on-prem, we have something like on comparative basis. And we'll try to be within that.
But I can't promise saying that definitely it's going to be cheaper than the on-prem.
Platform governance and maturity journey
Thank you. I had a question as well. I know we get to talk about this a little bit. And we talked a few weeks ago about it, but I know you did a lot around the governance and security of the platform. And I would love to just hear what your experience has been like going through that process, because I know there's a lot of people who might be listening in who are just starting off with that in their own organizations.
Sure. So, you know, it's quite, you know, I'm going to be talking about this, I think, throughout my career, because this is something which I have really enjoyed doing it. So the platform, when I started managing it two years back, it was, you know, all managed by the business users. So there was a bit of, you know, hidden detail, I can say, like a black box. We don't know what was the models that were sitting there. Also, we don't know who were the users, because it was being expanded to different departments within the organization. It started like a POC. And I think it started expanding to different business units, because they saw the benefits that was coming out of it. And then it was also picked up by one of the biggest property model pricing. So it had its own, you know, visibility in terms of, oh, we can do some in this platform, all these things. And from always for the users who have been on Excel sheet for some time, you're giving a platform which comes with its own benefits, they will love to use it. So that's how it started onboarding.
There was very, very thin line of support already there. When I say support, that was in terms of making sure the service doesn't go down, and also onboarding the new users. So that was the minimal thing I saw when I entered this unit. And then I have to take the maturity as my priority, because as I said earlier, the operational cost is something, you know, as a delivery leader, I have to make sure that we are within that, and also security, because tech debt and security became priority after AXA acquisition for us. So with all this as my main objectives, I said without even knowing what is there inside the black box, it will be very difficult. So we started with the journey of the maturity of what I was talking about. It was due diligence that first I wanted to do in terms of understanding what has gone in the system, and also how to bring the stabilization of the platform, because we also had complaints that it was mainly for the property pricing, the platform was coming down, and they were not knowing the reason what was happening.
Is it with the service? Is it with the network? Is it with the application? So that's where I think RStudio team also really helped us, and Kevin Hayden is here. So he was so helpful. He got the engineers, and we did a complete architecture review. So due diligence was going on one side, the inventory. I will also tell how I did the inventory, because that might be useful for many of them here. So it's a huge organization, different departments. I don't know anyone in person as well. So what we did was, and they're all spread across the global region. So what we said was, okay, let's start doing a questionnaire. So it was a simple questionnaire prepared with, you know, a lot more questions for us to first understand who are you, what are you doing here, and what is your pipeline, and how many, you know, models you have, what is your data sources, and that was the type of the questions. And we sent it out. We collected all the information, which was very, very helpful for us to say that, okay, this is what the platform is getting used for. And then one by one, we need to come out with the plan of, we call that as application currency, because, you know, we also quantify that whatever changes you bring into the platform. So we started with those projects to get to the maturity level. And there were quite a lot that has been done over the two years, and still I'm saying that we are still in progress. I won't say it's closed, because, you know, for a platform like this, where we have a number of users, we can talk about IDE users, we have around 200 to 250 users. And in terms of Connect viewers, we have around 1,250. So for such a platform, you know, it never goes. There is still work, we are, it's going.
Did you learn a lot more about the team through that survey? I think that's a great idea of surveying the users to see how they're actually using it. Yes, definitely, because I want to hear both, you know, the positive and the negative side, like what were they getting from technology, because I belong to technology. So I want to understand. One thing I'm very much, you know, I can say that that's something I keep it as, you know, one of my main objectives. Also, business users are doing something in a platform means that they are already, you know, creating some business value. It's all coming back. So I'm keen to help the business community. But at the same time, being from technology, I need to ensure, you know, the technology guidelines are also followed. So I think that particular survey helped us to think what are the challenges they are seeing from technology, and also to understand, you know, what they want us to do going forward in terms of their pipeline. And also certain things like how are they connecting to data sources, and you know, so it was both. It's a very, very insightful, I think, answers which we got.
Relationship between data scientists and IT
I'm so excited to have you here today to be able to provide that like IT and technology background and your perspective, because I know a lot of times there's questions on here about working with IT for the first time. And as a data scientist, how do you like build relationships with IT? And I see somebody asked anonymously, what's the relationship between data scientists and IT or platform team like at your company?
I will say very, very collaborative. See, one thing I believe in is technology cannot achieve what they want to achieve without the business. And it's the same for business users
also, the data scientists, they can't do what they want without the technology, because some of these requests which is coming from them. Simple example, I'll give you in terms of the packages they need, for example, right? Sometimes when it comes to us, we need to understand the importance of it and quickly turn it around. And at the same time, their deployment. So now recently, we implemented proper deployment release management for all our models. So nothing goes to production without going through the deployment, which we call it as release management.
And there, you know, I can understand being 20 years in technology. For other applications, it's okay, we get that particular build after, say, three days or four days, or even a month planning can be done. But for analytics, they want something to the market very quickly. They wanted to give the scoring models to underwriters. They wanted to give the pricing models very quickly in terms of, you know, change in inflation and things like that. So what we did was I have to come up with a different plan of release management, saying that, okay, let's put, you know, let's customize it. We won't treat every model as an application and go through the entire change management. We will go through a very, very thin line, only the minimal, right? The basics, what we need to do in terms of the guidelines. So we have to customize everything for our data scientists.
And I think they saw that when first, when I talked about bringing in guardrails and, you know, all that, they were thinking, oh, this is going to be complicating what we do now. We are not going to have the flexibility and we are not going to have that efficiency, which we had earlier. But now they are seeing why did I do it and how come it has streamlined their process. And now it's, you know, it's become like more easy for me to go and tell them this is what I'm planning to do for the next phase. So I took time, but I think now it's been two years and without their help, I will not be able to do anything.
Tips for working with the IT team
It's a really, really helpful perspective to hear. Do you have any, any tips for, for everybody listening in if they're just getting started working with the IT team?
Yes. So, you know, the tips is really for the IT team to understand the value of what you're doing, because everything you do, like data scientists, there is something going to be an output. It could take one week, it could take a month, or even it could take six months, which you are doing an R&D. But for them to understand the value we're going to bring in is going to be very, very important. And also explaining to them the flexibility you need, because analytics and, you know, the data science is all about the speed that comes with any platform. It could be any platform, right? So that is something which you need to explain to them. I'm fine. I'm here to support in terms of the governance or security or stability, what you want to bring in, even the operational cost. But at the same time, can I get this? So it's always, you know, two ways.
So if you say, I'm going to be really supportive of your guardrails, but at the same time, I'm expecting this from your team, I think it's going to be more good. And definitely I feel that that collaboration is required. Like I said, otherwise, it's very, very difficult to make this platform mature. And one thing what I'm seeing now is after this journey, two years journey, the platform has matured. And now we both know how we work. I mean, each team knows how we work. And there are sometimes trade-offs, both sides, but we are able to take up that. The reason being, finally, we wanted to give the best to the output for our business and for the business to earn the profits what they want. So finally, the objective is same, both for technology and the data scientist world. But I think where that sort of conflict comes in is, you know, when objectives, there is no shared vision. So to get that shared vision, I think both teams need to talk. That's very important.
there is no shared vision. So to get that shared vision, I think both teams need to talk. That's very important.
Why RStudio was chosen over other platforms
Absolutely. I see there's an anonymous question that came in, and I think it'd be helpful to clarify for people here. They said, how did the decision of using RStudio as the cloud service provider come to fruition? How did you overcome the question of why not AWS or Azure or Google kind of questions? So could you clarify a little bit about the cloud?
Yeah, definitely. So I think it's, I mean, I won't take the entire credit. Like I said, it's the users, it's the business users. I think I mentioned this to Kevin sometime back. They saw the flexibility that comes with RStudio. When I say flexibility, it's not like just, you know, without any guardrails going and using it. They know that entire governance is there, but what they saw it is like the IDE. Now, after the workbench, we have the launcher, so which they felt that it's one of the good functionality that came out because, you know, you have Python, Jupyter Notebook, anything they can launch and also package management. And the other thing is the, you know, the ease, easy of like day going and developing something and then publishing it to Shiny. So the property model, which I'm talking about, is entirely built on Shiny. And like past two years, you know, we have not had any major failures in terms of the platform, even though the users are expanding and, you know, we have it on prem with servers, still the business is able to do what they wanted to do, the data scientists. So that's one of the reasons, you know, their inputs convinced even our architecture to think that we cannot lose RStudio.
And another advantage is the RStudio completely, you know, the way it can be installed, like how we are doing now. An IDE now, if we want to copy the image and set it up, we also have very good scenarios. We have got the complete IDE set up within two, three days. So that is something which is going to be helpful even when we go in cloud. It's going to be more placed on containers, but if you wanted to do something very quickly, it can be launched. So that is something, you know, it was useful for us to convince. Yes, there was a debate about, you know, even using other competitive platform and also we had Azure and all that, but finally RStudio won the case.
And I know this is an anonymous question, so I'm just guessing as the follow-up, but just to clarify, so you took the RStudio team environment and installed it in the cloud provider of your choice though? Yes, that's right. It's going to be on containers and we have still not decided the entire infrastructure architecture. It's ongoing now, but we have decided that RStudio is going to be one of our strategic solutions.
IT team structure
Awesome. I see Robert just asked a question. He said he's eating lunch, so I will ask, what is the structure of your IT team? If you could share, who specifically are you collaborating with on projects?
So, technology team is quite, IT team is quite huge. Now, I am part of DIA with data and analytics department, but I used to work for global technology till March this year. Since we are almost, you know, all the time we are involved with pricing and analytics team, I am a part of data science. Our department has been moved, but IT with analytics team, I am a part of data science. I was working in global technology and even now, which we are integrating, you know, all our applications and also like security, governance, everything, still I have to be, you know, liaising with the technology team. So, in technology team, since I said it's a huge organization, we have infrastructure support separately. Under infrastructure, we have Unix, rental team separate, and we have network team separate, storage team separate, and then we have DBA team separate. So, these are all the technology teams who I have to deal with. And then my team comes with tech lead, application manager, and the support team. So, that's how the, it is structured. So, when I say technology, it is starting from procurement where we deal with, you know, getting the licenses. It's the entire team that I'm talking about when I say technology.
Managing governance for 200 users
Great. Thank you. And then just a reminder to everyone, if you joined a little bit later, you can also ask questions anonymously through Slido. And Hannah will share that again in the chat right now too. I realize for Zoom, the chat doesn't show up that happened previously when you jump on. But another question was, it sounds like you have a lot of users and with 200 users, that's what you said, how did you manage the governance of that?
Okay. So, when we did the due diligence also, you know, there were already some good things happening. Like the onboarding of users was not just the license was given. They were put in different AD groups. So, that helped us. So, there will be some users who just wanted to use only IDE and not publish any content to the connect. Like say they don't build anything in Shiny. So, they were like put into different AD groups. So, that was there. But one thing what it was lacking is we didn't know who were the users onboarding. And even if there is a lever, we didn't know that person has left and we need to revoke their access completely so that we could take the license back. So, the governance was thought about, okay, what can we do in terms of each step? So, first is the license. How do you get them like onboard? And then how do you keep an audit trail of when are they leaving? What are they doing? And say there will be some users, they get the license, but unfortunately, they are not able to use the application for say five months, six months. So, we started putting a timeline around it. If they don't use the platform for six months, we wanted to get in touch with them and find out whether they need the license. Otherwise, we take the license out of them.
So, everything has been put in terms of onboarding. And in terms of governance, like I said, we have segregated the environments now. So, we have the development and the production separate. So, in development also, we have staging and non-prod. So, when they are developing the models, when they are going to publish it, they need to come through the, again, a ticket, which comes to us, to my team. And then we go through it. What are they deploying? And what is the model? And which is the data sources? And have they used any sort of hard code passwords? Because those are all the things we have filtered now. And there is also a validation framework, which we are using in terms of validating, okay, what's happening? Are they using licensed libraries? Are they using the proper packages? Packages is one good thing, because we have, again, it only can go through my team. So, any packages that is requested, my team will be installing them. And now, if we need to, say, download it from some website, we need to whitelist it. That is a separate team. We need to send the request for whitelisting, and I need to approve it. Otherwise, all of our packages come from M-Brand, the Microsoft brand. And that's how we are managing the governance for new users.
That's really interesting that you can look to see, oh, you haven't used this in six months. What is the problem here? How can we help also? So, I'm curious, is there like a program in place for people in that position, like to then help them learn or understand why they're not getting value? Yes. So, there is. Even the business departments have started doing it, mainly the data scientists and the pricing model actuaries. Actuaries are the ones who are really using this platform, apart from the data scientists, the machine learning engineers. So, what they have is they have started their own R training, because when they get in, they are doing it. And we have helped them with some sort of wiki, like putting all of our resources together, like even because they need to know when I'm on board, whom should I go for request and what happens after that. And if I want to connect to a data source, how will I do it? Because that also, my team has to help them. So, we have put everything together and we have given it for the business users as well. And also, we have kept it in our platform so that they can see that.
So, that sort of knowledge sharing was required. So, there is quite enough support and I think there is a lot of investment that is being done for actuaries, because that's why you see 250 users now. They all are, I mean, they know R, they are experts now. So, we are helping wherever we have to. I see, Frank, you had a follow-up question. Do you want to jump in?
Package approval and governance
I would love to. You mentioned that you approve the packages, right? Packages that get used. The thought that popped into my mind, and perhaps I'm the only one that does this, but if I'm trying to solve a problem, I will Google, I will find a potential solution, install the package, try it. Ah, it didn't work. Find something else, try another package, right? Find the one that works for your problem in the moment, might not be the most efficient. So, what is that like? Do you approve everything in CRAN and then stuff that's like randomly on GitHub that might not be in CRAN, you have to approve?
See, there is no hard rule that they have to use only the packages from MRAN. If they choose from some other website, like say when you Google and you find it, it comes for whitelisting. And this was implemented recently. So, we need to send out the request to make it whitelisting so that we can use that link to download the package. But one step, what is in terms of governance is it goes through my approval. So, I need to ask several questions. One thing I have to make sure is there is no vulnerabilities that we are exposing, right? So, if that is the case, and mostly I will say that we have not come across any time that the users have requested something and we are not able to get that for them. Any website it could be, we see there is, you know, it is a proper website and it has its, you know, we know that it's not going to cause us any problems, then we are happy to go ahead with that. Do you have an SLA perhaps? No, I mean, to be honest, it's a very small team
because like I said, always operational cost is also something we wanted to keep it within. So, it's a very small team I hold. But we go by, you know, the business priority. If they say something, no, it's very, very critical because the next week I need to release this for, you know, for the scoring or, you know, for underwriters, then we try to give that as a priority. So, that's how we keep and now, I mean, for past year, we are using Kanban board where we keep all our tasks, everything, including installation of packages and it could be anything related to governance and security. We are following agile model there and we pull out the tasks based on the priorities. And we know who's working on that. If it doesn't come back, why it's not coming back. So, I think in terms of project management also, we have picked up this agile way of doing it. So, it's very quick.
Very sensible. Thank you. No problems. Thanks. It seems like a lot of people really like that question, Frank. So, it might be helpful to dive a little bit deeper into that one too.
Was there a time where the data scientists had to explain like the importance of these packages and what these packages really mean to the IT team who maybe didn't yet know what they were or the value of them? Yes, sometimes they need to because, you know, they could be asking it from some other websites outside. So, sometimes, yes, they need to explain why. It could be very, very useful for them in terms of some functions. And like I said, this is one of the main platforms for our actuaries as well as the data scientists. So, we can't say, you know, just bluntly no to it. So, we also put in time to see is there anything that we could do about it. Is there an alternative? If it is not there, then I'm happy to, you know, approve the whitelisting. And also, there is something else we have discussed also. In case we wanted that website to be used to install a package but we are thinking that we are not very sure whether that has any sort of risks that it is posing, then what we do is we do the installation of the package. Then again, we remove it from whitelisting so that no one is using it further. So, that's how we are trying to do. But like I said, many times, I think I can say 99% it has gone through without any issues.
Open source, security, and governance
Thank you. I see, Melissa, you had a question in the Zoom chat. Do you want to jump in?
Yeah, sure. I'm wondering how you've handled the conversation with IT if you have about open source technology and the risk that that might pose to company security. Have you ever run into that? And really, you or anyone on the call, how did you approach that?
Actually, I mean, definitely, if anyone wants to jump in, I will also love to hear. For AXA, Excel open source was allowed earlier but then it started becoming very strict that we are not supposed to be downloading anything. That's why about all this whitelisting, we can't use it without making it approved by all the info security group as well. So, after my approval, it goes through info security. They have to approve, then only it goes through. So, that was put in. But earlier, they had to go to open source and they can download it. But now, it's very, very strict. But one thing I wanted to tell us, I'm not very familiar with all the packages, but I can tell you that there is around 1,000 packages that is getting used currently now. So, I'm thinking that most of those packages are available in Microsoft brand. So, I think most of the functionality which our users are using, I think it's coming from validated package list. So, that way, we didn't end up going through it. And there has been times also R was downloaded from open source. So, that was also happening. Python also was similar. But now, for past year and a half, there is no way they can do that. It has to be going through a proper governance and then only they can download it. It's not only my team restricting it, it's also the infrastructure team restricting it.
Thank you. I see a few others. Can I jump in, Rachel? Yeah, sure, Manju.
Thanks. So, I'm really surprised that scientists get guardrailed in that sense, because you do you do need like a free mind to be able to bring about solutions. And the interesting thing is basically open source is being used in other places where like with the JavaScript and like the front end, where there's a lot of vulnerability. And that is the actual space which needs to be guardrailed. And yeah, and basically, I think that that's been like the roadblock for me as a data scientist in every corporation, you know, trying to bring solutions to the forefront, yet I feel inhibited and I feel I am, there's roadblocks there.
Yeah. So, Manju, the whole thing, right? See, it's again, like I said in the start, it's looking at both ways, right? So, I agree to you definitely, because you lose that sort of, you know, the flexibility to get what you want to do as a data scientist. But the whole thing is, it's not like I said, a very hard rule that nothing should be got from open source or nothing can be installed. We also have a process of, you know, finding alternatives. Other ways, there is also a process called going through, you know, complete security process, the toll gate, and then, you know, explaining why do we need this. And then, you know, we have something called as a RAM process. And if that is being set, a year is set on it, like say expiry date is set on it, every year it will be reviewed. And if we have already got an alternative, they'll close that. Otherwise, they will continue using it. So, because not only our platform, I manage another application, which is for risk maps. And there we had Python, which we have to go into the next version. But their utilities are still having in the old version of Python. So, we have got some, you know, buy-in there from security. So, it's always, you know, the business explaining to us what is the need for it. And then, we becoming, you know, like almost intermediate in this process. And we going and putting that forward for security and getting the approval.
So, there are, you know, we have like channels, which we can use in terms of when such situation happens. And the other thing what I wanted to say is, see, the risks organizations are having is huge. Coming from insurance organization, if something goes wrong, even one data gets leaked, the amount of penalty that the organization has to pay is very huge, which, you know, we can't even quantify how much is that. So, that's the risk we hold. And the data privacy is like really the topmost priority for us. And we have to ensure that for different applications. So, this is one of the applications, but there is quite a lot. And the governance is required. Otherwise, it will be too hard after a few years. Because we won't even know from where the packages have been downloaded and what each package is bringing the vulnerability. And it becomes very, very difficult to manage. It's not so much about the packages. I mean, I, I, it's open source is a mindset. It's, it's a mindset more so than like a process, right? And when we are trying to do the best for the organization, it has that flexibility and that alignment needs to happen. So, that we can bring about the perfect solutions for the organization. And it's the trust in people that ultimately matters. And I have seen a lot of data has got breached because of like whatever reasons outsourcing or whatever. And yeah, so, but the reality is it's an, it's a mindset that needs to be nurtured within corporations.
the reality is it's an, it's a mindset that needs to be nurtured within corporations.
No, definitely. I mean, I, I understand where you come from. Yes, it, it has to be nurtured. But I think it's a balance. And if we can achieve that balance, it's good for both sides, right? See something like you said, if everyone is being bit conscious, what they are exactly doing, am I every step I do, if I'm being conscious that no, I'm not going to pose a risk, then it's fine. It's absolutely fine. But when it comes to the reality and everyone doing their jobs, sometimes it's because of the time pressure or the timelines pressure also, there are chances that they might forget to check something. And, you know, even though I'm working in technology all these years, I am still not an expert of security. So, that's why we sometimes go to our security advisors to find out whether is this open source okay and whether can we allow it. And that's why they keep one more approval after me that the infosecurity has to approve it. So, I think it's again, you know, like I said, it's a balance. There should be, if you really need something, there should not be a complete blunt no. There should be channels to help you. So, if organizations could do that, I think it's really nurturing both mindsets, right? One side, we making sure, you know, we are aligning to all of our security. At the same time, we are also giving the flexibility for the data scientists.
Absolutely. I love that. So, what is the, like, what's the channels that you recommend for teams to, like, get support or what companies need to create for teams to get support when they're kind of in this roadblock space? Definitely the waiver, right? So, when we say something is very, very important for data scientists and they can't, they don't get it, say, from the reputed or known sites, they wanted to get something, I will say, please listen to the data scientist and there should be a group who's sitting and reviewing that, right? So, they can put a waiver there. It can be approved by their own management, say, from who's requesting it, the data science managers or their leadership can look into it and say, yes, it is needed because this person is going to create this business value and we can't just say no to it. So, let's put this form and fill out the form, submit it, let the security review it or technology review it and they can give you a waiver and you can put a time limit on that. If you don't want to give it, say, for five years, that's fine. Every year you can review it and you can see if there is no vulnerabilities posed, then leave it to go. So, I think we always, I think about this, right? When a person is requesting for something, finally, it is a business value he or she is bringing to the company. So, they need to listen and they have to give them what is the other alternative, right? Okay, if we can't download from this or we can download, but can you please fill out this form and can you please get approval from your own management? So, something like that, it is there for every organization. I think it definitely will achieve a balance.
Mithali, I have a statement. Yeah. This is my observation. So, it is not so much about the packages. Yeah, it could be anything. Yeah. No, within corporations, it is not so much about the packages. It is more so about getting access to the sources and it is more so about the quality and that's where the roadblocks are currently rather than the packages themselves. You see that, yeah, and where the problems lie within organizations. So, when you say access, Manju, are you talking about data sources? Yes, the data sources, like how they are being aligned and where they are within, like which stuff and things like that. That's the actual, those are the actual issues which need to be addressed.
Again, I think it's the maturity of the technology, right? Because, say, okay, I can give you an example, right? Say a data source has been accessed all the years using, say, a SQL server ID which you got it created, okay? You're accessing a SQL server and they have given you an ID for your name, okay? You have been using all that for many years and you have been sharing your ID and password even with other of your peers. That is, it poses a risk because the reason is the data that might be sitting in that data source could be confidential or it could be even, you know, very much it shouldn't be shared. It should be secured, right? But if it is given to you and your name is registered, it's fine. Only Manju is accessing it. But say you are sharing that ID and password with someone else, right? It is a risk. So, what happens in those scenarios is the organization comes out with their policy, okay? So, you wanted to access the data, that's fine, but we'll create a service account and that password will not be given to you. It will be in CyberArk or something where the rotation of passwords happen and every 60 days it gets renewed. So, that's what I'm saying for every, instead of just telling to you, no, you can't access that, they have to come out with an idea of how can it be accessed with governance. So, that's where, you know, the governance plays a bigger role in entire, you know, technology world.
Dev, staging, and production environments
I love this discussion. I want to make sure we get to a lot of the other questions that are here too. No, but one quick follow-up question from that was do you have separate dev and prod environments? Yes, we do. So, we have dev, prod, and also we have staging environments because some of the things that our team, like, wanted to test, say, for example, we were, like, moving to Workbench last year. So, we wanted to install everything. We wanted to try out, we use one environment where we don't have data scientists at all. So, we have such an environment and also we have, like, for the users to try out anything in terms of their trial, like, the lower environment. Once everything is ready, that's where they submit the ticket. Okay, everything is done and I have my data sources and I have my algorithm ready and now I need to deploy this or publish this content, then it goes through the
quickest way, right? That's what I explained. We have customized it because, say, there is a dashboard which someone has built it for within two days and it has to be immediately going to production and we see it's a very simple dashboard. Not many external data sources are there and there is no sort of any security that we need to validate. Then what we do is we give an SLA of even one to three hours. It will be published in the production, but say if it is a model which they also have built it for, say, six months, three months or nine months and it has got quite a lot of data sources and also there is some hard-coded passwords sitting in the code, then that goes through its, you know, little bit of validation. Again, I'm not talking about it will take a week. Again, there the SLA is two to three days.
Thank you. That's very helpful. An anonymous question from earlier was, how far out into the future do technology teams at your org have to think about when it comes to adopting new technology? We have to think about a little bit. I mean, I can say, yes, we all have to think about it. It's not only my organization, but at the same time, we also have legacy quite a lot. The reason being even new technologies come in, it's not easy for us to shift because of the expansion that happened over the days. So it's been, you know, not that easy for us to think about a new technology and immediately adopt that. And again, now the procurement process again, we need to go through. We can conduct POCs, we can conduct MVP, and if it is all fine, then we can submit the proposal. But again, there we are talking about budget. So we need a business sponsor who's supporting it. And what is the reason that we are shifting to a new technology? There should be a proper business case with both tangible and intangible benefits, then only we can move.
Deployment process and CI/CD discussion
Thank you. I just realized on that DevTestProd environment question, I see, Javier, you had a related question. Do you want to jump in?
Hey, thanks for having me. I've loved this conversation. It's been awesome. So and some of some of my, like, some of the question that I asked, I think was already discussed and answered. But let's say I'm a data scientist at your company, and I have pretty much built an entire, you know, ETL pipeline, and like a shiny app, like, I've built everything. But your team owns the deployment of this tool to production. Yes, that's right. Let's say I have to make an update, which could take me 30 seconds to do. So I, you know, submit the commit, or, you know, it's on GitHub or GitLab. And I reach out to you. Do the data scientists at your company have the ability to push that change to prod themselves? Or do they still need to file a JIRA ticket and, you know, wait up to two weeks for this?
Okay, so they have the ability to... Yeah, so definitely no waiting time of two weeks. And there is also, you know, we wanted to give them access from, like, automated checkout from GitHub and going into production. That is also possible. And since we are moving into cloud, the DevOps is going to get involved. And it's going to be all automated, the CI CD pipeline. So then it becomes very quick. But now on-prem also, that's what I said, the maximum we are taking is only two to three days. Otherwise, it's like, say, two hours or three hours, even one hour. Say there is a change, like you explained to us that it has to be done. Maybe instead of 30 seconds, it's going to take an hour. That's all. So it's not a major difference. But we also have a checkout process from GitHub, which is going to be very automated.
Do you... Has this been defined in your internal, like, SOC2 controls or SOCs controls? Do you have, you know, this mechanism defined in your, you know, internal documentation?
So for us, because like I said, since I am owning pricing and analytics platform, I have customized this release process a little bit different. What our information security calls out is you definitely need to have non-production and production. You can't do your development in production. Please do it and then go through release management. We have a change request that we need to submit. But what I have done is since it is data scientists and pricing actuaries, they can't wait for every change, the change request to be raised and going through the entire process. If I have an auditing mechanism that I can document it, then I have, you know, the flexibility to customize it. That's why I have done it that way. So if it is a small change, no implications, and it can go within one hour, we send it within one hour. They need not go through the entire change request process that we do for all the other applications.
Okay. I may reach out to you on LinkedIn, just to continue this conversation there, if you don't mind. No, definitely. I'm happy to help. I told this to Rachel. I mean, anything that could be a takeaway from this meeting, please feel free to reach out to me. Whatever information I can share with you guys, definitely I will love to do it.
Thank you so much. No problem.
And I just put Maithili's LinkedIn in there because it looks like the wrong LinkedIn got linked on the data science hangout page. That push should be going up in a few minutes, but just in case there's a delay there, we'll put it into the chat. Sure, sure.
Open-source package validation and governance maturity
Trevor, I see you had a question you just put into the chat, I think was a follow-up on the open-source evaluation process. Do you want to jump in?
Yeah. Hi, everyone. Hi. Oh, this is really interesting. Very, very nice conversation we're having. I'm just, I don't know if I could work in your environment. Yeah, you are welcome, Trevor.
Because data science is a creative process sometimes. And you know, you're trying to find a package that meets your needs at the moment. And if each time I want to try a new package, it may take me two to three days. That's creating a bottleneck that is really preventing us from being efficient. So it's more a comment than a question. But the real question is around, so given that open-source packages can be unique and many variations, how do you go about guaranteeing two to three days? Because a package may be simple with one function, or you could have a package with several functions. What are you looking for when you're looking at the packages? And how do you guarantee, given your small team, as you describe, the ability to meet your SLA of really two to three days?
See, it's a very good question. It is not easy. Like I said, it's a small team. But you know, one thing what we have done is there is a validation framework we have done, like a validation framework, where we have done a validation framework. And like scripting on the server side, which will go through very quickly in terms of your validation of your libraries and packages. And then we have like also there is a scanning team, which does the scanning for hard-coded passwords or anything. So it is a, I will say everything is not manual. There is like, we have to come up, see, that is the maturity I'm talking about when I started the discussions. Everything like governance, security, everything has to be done manual. It's going to take its own time. It's like the testing times, right? When you automate your testing, it's very quick. So similar things for governance also. If you can make anything automated in terms of validating them, then any team can turn around, say deployment to production within a few days. But you have to come out with those ideas. So that's why I keep saying this is like a journey. We have taken up the journey like last year, but still we are working on it. And you know, the next big thing for coming for us is the NFS storage cleanup. There is also cleanup going on parallelly as we do all these things. But I always motivate the team to come up with some sort of automation in terms of the scripting so that we can make this validation very quickly.
If you can make anything automated in terms of validating them, then any team can turn around, say deployment to production within a few days.
Yeah. So, so, so, so follow up question. So your data science teams, our team, our teams that you work with, what is your job, job satisfaction? Like, I mean, what are Actually, that's a good question. You know, I can't just claim here that, oh, they are all very happy that I'm doing this sort of, you know, work. But you know, one thing, like I said, right, they are collaborating a lot compared to, you know, when I started, when I approached them and I talked about governance. So I feel that now they are able to understand it's both ways because tomorrow when we are moving into cloud, all the work that I have been doing for this past two years is going to be very helpful because we did an inventory of completely all the applications and also the data sources, what they are connected to. And if tomorrow, if we go to them and we could have asked all this detail, maybe there is, you know, a chance that they won't even know. And they will be saying they are going to take few weeks to find it out. So there is both ways, right? I have made their life bit easier with all this inventory and the governance, which I put forward. And now since the deployment is done through release management, we also having a tracker for that. So we know the models when it is developed, when it is changed, which dates it's been changed, and it's going through the ticket to get it to production. So all this data that is getting gathered with the governance and security is going to help them.
So I think they should be happy. That is my assumption, but yeah, maybe I have to take a 360 feedback and find out. I think it's also really helpful for everyone to understand that this came as a request from the business to be able to use RStudio and by you going through that process and putting the governance around it, it allowed them to be able to use the tool that they wanted to, instead of being forced into something else too. Yes, that's right.
They have been very collaborative. Otherwise, I could have not done this. Nothing I could have done because when we sent out the inventory the first time, they came up with their responses very quickly, what exactly they are looking forward for the technology to do for them, and also giving the information what they had. So I think it has been quite helpful from their side.
Digital transformation and future technology adoption
Absolutely. I see there was a slide of a question I missed a little bit earlier that was, is digital transformation still a word that's used as an org-wide initiative? If not, what are the new org-wide initiatives to future-proof your business?
I mean, digital transformation is quite, it's becoming a buzzword across the industry. So not everything we do is called digital transformation. Sometimes it is done, but without even naming it as digital transformation. But there are also teams where we have digital transformation teams on the data side. So there are teams by that name, but I won't call it as every initiative we pick up, it's called a digital transformation. There is so much that's going on without it getting tagged under digital.
Great. Thank you. I did just have a question about your environment, if I get to ask one with a few minutes left. So I know you said you have both R and Python users now. And so are both sets of users in the same environment now, or do you also manage different tools for each?
No, they are all in the same environment. And even before the Workbench, there was some very little usage of Python. That time they had a server where they were using it. But now after this, we don't have separate environments. We have R IDE as one of the main thing because Workbench, we purchased it last year. So after that, I think it's only our platform that's getting used now. But there are also, in other departments, there are other competitor platforms, but we have to see when we move to cloud, what is getting segregated. R is going to be there, definitely RStudio is going to be there. But along with that, what other platforms are going to be also given to the community, we are not very sure of.
And is there, so I know you did the survey to figure out how people are using certain tools, but if there's other future languages or things that people want to use, how do they start to introduce that?
It has to go through the POC process. And earlier on, I mean, I'm talking about like, because now it's my 13th year in the organization, but before that, we used to have like anything you wanted to try, you can try and then go and put that as a business case and get it. But now we need to go through the procurement just to make sure even the POC departments, I mean, they know that they have to go through procurement, they have to reach out to the vendor, get the product and try it. And if we like it, then go with it. So there, if they wanted new languages or anything, or they want new platform, it has to go through the procurement process. And like I said, it also has to be justified. Why are we moving into new technology and the benefits of it?
Awesome. I don't see any other questions, but I just want to say thank you so much, Manjuli, for joining us. It was so helpful to hear your perspective and experience here. No, thank you very much. Thank you very much for the RStudio team as well, because I think throughout this journey I'm talking about, I don't know how many times I've asked my team to reach out to RStudio and especially Kevin and say that, oh, this is very much urgent for us. Can you please help us? So I should really thank even RStudio team for that. And yeah, I look forward to working long time in this. Let's see how that goes.
That's awesome. You just reminded me, I wanted to say this as well. If anybody is ever curious if your team is using RStudio's products or if you want to be connected with your RStudio account manager, I can't believe how many times that there will be teams who reach out,
who just don't know that their colleagues are using our tools yet. Feel free to reach out to me in the least salesy way possible. I can just let you know. I'll put my LinkedIn on here too, if you ever just want to ask and are curious. But thank you all so much for joining today and for all the great questions too. We will see you back hopefully next week. We also have an awesome meetup next Tuesday at 12 Eastern time with Julia Silvey and Isabel Zimmerman on using MLOps with Vetiver in R and Python. So I am also just going to put the short link real quick to that meetup if you want to just add it to your calendar too.
Awesome. Well, thank you, everybody. Have a great rest of the day. Bye.