View Full Transcript
Episode Transcript
[00:00:05] Speaker A: All right, well, welcome to this episode and season three of our show, true Data ops. I'm your host, Kent Graziano, the Data Warrior. Each episode, we try to bring you a podcast discussing the world of data ops with the people that are making data ops what it is today. So be sure to look up and subscribe to the DataOps Live YouTube channel where you can find all the recordings from our past episodes. If you missed any of our prior episodes, now is a good time to catch up as we start season three. Better yet, you can go to truedataops.org and subscribe to the podcast, and then you'll get proactive notifications.
My guest today is a dataops practitioner, snowflake, data superhero, and a senior data architect at National Grid, David Garrison. Welcome to the show, David.
[00:00:53] Speaker B: Yeah, thanks for having me.
[00:00:56] Speaker A: So for those that don't know you, could you give us a little bit about your background in data architecture and data ops and you know, what you've been up to?
[00:01:04] Speaker B: Yeah, sure.
Well, I've been in data pretty much since I got out of college, so 20, 1212 years, I've had most roles that have the word data in the title. Data analyst, bi data engineer, data architect, Snowflake, data superhero.
So one thing that I love about data is that it's such a transferable skill for different industries.
So it's, you know, you can hop from mobile games to real estate.
So ultimately, a lot of my experience in dataops data architecture comes from making a lot of mistakes, seeing other people make mistakes. Try not to make the same mistake twice if it can be helped.
[00:01:58] Speaker A: Yeah, that's one of those repeatable things you don't want to repeat.
[00:02:01] Speaker B: Right.
[00:02:04] Speaker A: Well, for this season, we want to take a step back and we're thinking about how the world of true data ops has evolved over the last couple of years and what we've learned. So you've been a practitioner in the field and worked on data ops on several different organizations and industries. So I'm really looking forward to kind of hearing your thoughts on all this for our listeners. If you've not looked at the seven pillars of true dataops recently, you can find [email protected].
seven pillars. So that's a good reference for you.
Now. It's been like four years since we first published the Truedata ops.org website, the philosophy and the seven pillars, also the dummies guide to data ops. So in that time, do you think that the seven pillars of true data ops still resonate with people?
[00:03:02] Speaker B: I certainly think so, I mean, I've seen various different iterations of these types of pillars, seven pillars, six pillars, five foundations.
So I think there's certainly a lot of value there in kind of defining these are the fundamental aspects.
Yeah, I think they hold true.
[00:03:29] Speaker A: So we've got this AI and Genai stuff going on all the time. So how do you think that plays with the concepts of data ops, true data ops, and what we need to do to make things work and really be effective with all of this new technology and kind of the speed at which things are going today?
[00:03:53] Speaker B: Sure.
It's an interesting question.
My initial gut reaction is that like, it hasn't really changed things fundamentally.
I suppose if anything, AI makes it a little bit more important because if you're not setting up a test environment and managing source control, then you're not doing data science.
But I think if anything, it's been undervalued in, I don't know, decades past. So I don't know if it's gotten more important, but maybe there's more talk about it, more awareness, more conversations about it.
[00:04:35] Speaker A: Yeah, I think that part of it, like you said, you mentioned fundamentals, and we get talking about AI and Genai and it seems like things that were fundamental principles of data governance, data quality, data modeling, all of that that has been not out there for literally decades, have kind of become more important because things are moving so fast.
I mean, it's one of those good news, bad news things, right? Is things are moving so fast that if you do something fundamentally wrong, you probably find out pretty quick. But at the same time, if you're generating things using Genai and you don't understand the fundamentals, do you even know that you actually built something that's not quite right or isn't going to get you the right result if you don't have some sort of controls in place?
[00:05:31] Speaker B: Yeah, well, Genii is something that kind of inherently, just because of the scale of it, requires scaling up. And if you're at that point of scaling up, you need to be following a lot of these, you know, principles.
I think that kind of goes for any company that's scaling up, even if you're not using AI. It's just that most companies that are getting that big at that scale are getting into data science a lot more. So.
[00:05:58] Speaker A: But even if, like you said, even if they're not scaling up to that point, if they're just, you know, building their, their data analytics platform on Snowflake, what have you seen in that area? It's like you know, how, how important has it been for them to follow some of these data ops principles?
[00:06:18] Speaker B: Yeah, well, I can't say that I've worked at a company that has gotten all seven of these principles right.
There's always something lacking. I haven't worked at your big Amazon's, Google's, Microsoft's, but.
[00:06:34] Speaker A: Yeah.
[00:06:37] Speaker B: But I mean, if a company did have all of those pillars right, then I might not be inclined to leave. So maybe that's why they're hiring us.
It's tough to find.
[00:06:54] Speaker A: So what kind of things have you seen organizations struggle with?
[00:06:59] Speaker B: The biggest ones that I see struggling with are really getting into CI CD for data. Like they'll have CI CD for web applications and whatnot. But data seems to be a hard challenge there.
Really getting into the automation of testing, you'll see good testing practices, but really getting to that step of having it be automatic and having it be test driven can be a challenge.
Some of them, I'm just coming from the world of Snowflake. The thing that I see the most often is actually issues with grant management governance.
One of my biggest frustrations with Snowflake and I've told them this and I should maybe tell them this more often, but they have excellent tools for handling very fine grain grant management, RBAC controls, security management, but they don't provide built in tools for the structure of following best practices. So being able to set up a read only role for a database requires managing those fine grained controls.
So having that structure around creating business roles in Snowflake in particular is something that I see frequently. So part of that just comes from the fact that I've been working with Snowflake so much in the last six, seven years.
[00:08:30] Speaker A: Yeah, so, yeah, so some of that's not, like you said, it's not built into Snowflake. The capability is there, but you got to do a lot of building yourself in order to make it work. So have you found any reasonable ways of managing that?
[00:08:49] Speaker B: I have.
I mean, certainly there are a few tools that help provide that structure, provide the semantic layers and templates.
Dataops live is one of them.
[00:09:04] Speaker A: It's okay, you're allowed to say that on this show.
[00:09:06] Speaker B: Yeah, yeah.
Big fan of dataops live for that purpose. I don't get to use it at every job I work at, even now at National Grid, part of the company uses dataops live, but that's a big part of the company doesn't and that's the part that I usually end up working with.
So, yeah, and, you know, there's an.
[00:09:29] Speaker A: Easier way, but you're not going there.
[00:09:32] Speaker B: Yeah, that can be frustrating for sure, but I've certainly seen homegrown solutions work as well. Even just building out templates and having good documentation can be the difference maker, but sometimes that gets in the way as much as not.
[00:09:52] Speaker A: Yeah, yeah. I mean, what you're talking about, like being able to have a template for people who I'll say, don't fundamentally understand the underpinnings and the fine grained level, but know that, okay, we need read only access to the analytics for the finance team. How do we set that up? Right, because we don't want the finance people necessarily going in and changing the data in Snowflake, but they certainly need to be able to access it, whether it's via tableau or power Bi or even in some of the built in tools inside of Snowflake.
I can see that that'd be a fairly big challenge, especially coming from a business perspective rather than a technical perspective.
[00:10:38] Speaker B: Right. Yeah. And you can certainly set up scripts that, you know, say, I want a read only role, create that role, this is what it's called, and then run through all the objects and make, you know, make it appropriate for an analyst, but that requires a decent understanding of the business and any database standards that that company is using. The environment, you know, it tends to be a little bit specific to the company unless you're using something a little more standard industry best practice.
[00:11:12] Speaker A: Yeah, and I guess that's really the challenge there, because even if you've got some scripts that say you wrote a bunch of scripts and you hand them off to the data analyst or the data architect or the data engineer and one of these other teams, how do they know they ran it right? How they know that it's being kept up, you know, the governance portion of it. So, yeah, it's one of, one of the seven pillars there. I think you mentioned that before. It's like. Yeah, that the governance, the security and then change control on just that sort of thing. Right. It's like, forget about the change control on the schemas, right. You got, you need change control on the schemas and the table definitions and the load processes and the pipelines and all that. But you got to have the change control even on the governance aspect.
[00:12:02] Speaker B: Pretty much every company I've worked at has had at least one admin who just goes and runs transcripts.
[00:12:08] Speaker A: Wow.
[00:12:09] Speaker B: And doesn't get them into source control. Doesn't, I mean, they'll usually have a ticket or they'll log it but there's always a little bit of disconnect between what the grants are in the system and what's been documented and what's re runnable, repeatable.
[00:12:25] Speaker A: Yeah, yeah, yeah. So I guess that's, that's another piece. When you start looking at like, the pillar for collaboration and self service is if you don't have a good framework around that, then it's a lot harder to do the collaboration. I mean, people may think they're doing self service, it's like, yes, it's working fine for them, right. But it's very, very centered on that particular individual and what they know needs to be run and all that. But then there's not that cross team collaboration, let alone the governance like approval process. Is there an approval process or somebody just puts a Jira ticket in and says, hey, we need analyst number five over here in finance needs access, please go grant them access.
Okay. And they execute that and say, okay, they close the Jira ticket. But did anybody do any due diligence? What's really the role of analyst five? And should they have access to all of the finance data or should they only have access to part of the finance data? And that level of granularity is missing in a lot of the homegrown stuff.
[00:13:41] Speaker B: Yeah, we certainly have our, at national grid, we have plenty of layers of approvals.
If anything, it sometimes gets in the way. But translating those approvals into documented scripts and governed scripts, source control, the challenge.
[00:14:02] Speaker A: Yeah, I think like you said, in a lot of organizations, these kinds of controls and things are in place for application development, but have never really been in place on the data side.
Maybe they used an ELt tool that had version control built into it, but there wasn't any governance. They just went in and the developer went in and drew a couple of different lines and hit save and hit deploy.
Okay. It's been versioned in the repository, but nobody actually looked at it before it got deployed.
Right.
[00:14:44] Speaker B: Sure.
That layer of approvals and code reviews mixed in is certainly important and often lacking.
[00:14:53] Speaker A: Yeah.
[00:14:53] Speaker B: And I mean, something.
[00:15:12] Speaker A: Uh oh, looks like. Oh, there you are. You're back.
I'm not hearing you. So we lost, lost your audio.
Sorry about that, folks. Every once in a while, that's the downside of live.
Okay, I hear you.
[00:15:30] Speaker B: Okay, there we go.
[00:15:31] Speaker A: Okay, we're back. Okay, you were in the middle of saying something, then you went gone.
[00:15:35] Speaker B: Yep.
Let me fix my video settings. Okay. No, that one made me go away.
[00:15:42] Speaker A: That made you go away. So your video was fine.
[00:15:45] Speaker B: Okay.
[00:15:46] Speaker A: And your audio?
[00:15:54] Speaker B: Yeah, it's gonna. There we go.
[00:15:56] Speaker A: There we go. Now I got you back.
[00:15:58] Speaker B: So sorry, I bumped a cord.
[00:16:00] Speaker A: Don't touch anything.
[00:16:01] Speaker B: Not gonna touch that again.
[00:16:04] Speaker A: All right, I want to do a quick walkthrough on our seven pillars. You've got ELT and the spirit of ELT, agility and CI CD component design and maintainability, environment management, governance, security change control, which we've been talking a lot about, automated testing and monitoring that you mentioned, and collaboration and self service. So I guess my question is.
[00:16:35] Speaker B: Do.
[00:16:35] Speaker A: You think that covers it? If everybody ideally, were to do all of these things, does that really cover what's going to make us successful in building out our data states?
[00:16:49] Speaker B: I think it's a good list.
I would probably separate a couple of them. Like, I think governance and security are kind of separate enough, at least. But again, I work a lot on the governance side on my day to day, so I see that as kind of more separate from security.
[00:17:15] Speaker A: Yeah, yeah, yeah.
[00:17:17] Speaker B: And I also kind of think of source control as being very tied to CI CD. So I don't know, if I were laying it out, I might do it a little bit differently. But I think you've got all the right points here and it's really half a dozen, one, six, and the other.
[00:17:35] Speaker A: Yeah, it's kind of what needs to be covered effectively. Right.
So is there anything that you think is missing? Do you think every. Do we really have it all covered, like you said, other than maybe you could split governance and security into separate things?
[00:17:57] Speaker B: I don't know of any major pillar that I would say is missing these days. I often see these types of pillars include something specific for AIh. And I think there's some merit to that because machine learning, engineering, the process of training and retraining models, is itself kind of its own level of infrastructure and operations.
So I think it's valid to have that.
[00:18:26] Speaker A: Mlops, right?
[00:18:28] Speaker B: Yeah. So I think that's a valid thing to have, but it doesn't apply to everyone in the same way that most of these do pretty much apply to most companies and most data projects.
[00:18:42] Speaker A: So you've been a consultant a fair amount in your career as well as, and you currently work at National Grid, but how do you approach the buy versus build conversation with them? Because I know obviously you're familiar with dataops Live, which is obviously a buy scenario.
[00:19:04] Speaker B: Yeah, I do certainly see buy as being kind of too quickly dismissed by a lot of teams companies. I'm sure many are familiar with the manager who will balk at $10,000 for a tool to solve a problem, but will happily throw 10,000 person hours at that same problem just because it comes from a different budget.
I've certainly seen that where buying the tool comes out of my team's budget, but paying for an employee comes out of some other visibility of numbers can certainly come into play there, and that's frustrating to see.
But of course, buying a tool isn't exactly a 0% person hours proposition either. There's certainly valid concerns of getting locked into a tool, buying something that you don't actually need.
Sometimes you just have a marketing guy trying to sell a tool who's just over the top, and it's hard to identify if what they're talking about is just marketing a good sales pitch or if it's actually the tool that you need.
So I certainly don't like to see buying a tool as the default option either.
In my personal experience, the biggest headaches that I've had have come from trying to split the difference, which is to say paying or buying a team of consultants to build the tool.
And I'm sure there's occasions where that can make sense, but I've yet to personally see one.
[00:20:48] Speaker A: Okay, good. Yeah, because I haven't. Because I know that I've seen that approach as well, where they specifically rejected buying a tool that did at least 50% of what we needed.
[00:21:04] Speaker B: Right.
[00:21:04] Speaker A: It might have covered, you know, half of the, half of the seven pillars, we'll say, in this case. And then they spend even more money. They have consultants that are actually doing the work and writing scripts and all of that. But then when the budget runs out for them and they leave, you've got all this stuff that nobody knows how to use.
It's not well documented.
You might have hired this little side, hired a tableau expert to build all this stuff out, but then when the tableau expert left, nobody knew how to modify the reports, nobody knew how to modify the displays.
Imagine doing like manual, which you've experienced, the CI CD, was it Jenkins and cucumber, and all of these other vegetable names, right.
[00:21:57] Speaker B: Or worst case that I've seen is like a data vault implementation, that then you lose your data vault expertise.
And so you now have an entire modeling system that nobody at the company knows how to maintain or make changes to.
[00:22:15] Speaker A: And they probably rejected buying one of the data wild automation tools like data vault or Datavault builder.
[00:22:21] Speaker B: Of course, that would come from the tool budget. We don't have money for that.
[00:22:25] Speaker A: Yeah, it's funny, all these years that I've gone through doing this sort of thing, and I've certainly experienced it where we built spreadsheets to even show, here's the cost of the people and how many hours we think it's going to take. Here's the cost of the tool, and it's clear Roi. Okay, yes. There's ramp up time with the tool. Yes. But there's development time with building it yourself that takes even longer, no matter what the developer says, as you know. Right. Developers. Oh, yeah. I can build that tool in two months. Okay. That's two months to initially write the code, and then it's nine months of iterations and testing, and it doesn't quite do what we want. And. And then that guy leaves. Right, right. He's like, okay, now what do we do? But I had never, I guess it had never come through as clear as what you said is, like, it's a different budget.
[00:23:17] Speaker B: Yeah, I see it all the time. I've seen it.
[00:23:20] Speaker A: So much sense now.
[00:23:22] Speaker B: Like, yeah, the. The guy who's looking at that spreadsheet says this money for the tool comes straight out of my department's budget, and so I can't afford it. And then they'll go and look at the spreadsheet of, you know, the person hours and where that those salaries come from. Like, oh, that's fine. I'll pay as much as you want on that side of the things.
[00:23:46] Speaker A: Yeah, yeah, that's. And so the buy versus building is. Yeah, it's not a straightforward technical conversation, like you say, in some cases, totally.
It's budgetary, but it may even be political at that point because who owns the budget for that? And. Yeah, that's a different challenge. Right. It's like, so it's gonna have a very compelling case for buying at national.
[00:24:16] Speaker B: Grid, too, because we deal with utilities and government regulations. We also have a security component to it as well, getting sign off to even hand over metadata, to even open up channels that could be used for sharing, even if they're not used for sharing. Just opening up those channels requires a ton of security review, and that takes time, too. So for us, sometimes the default decision is, well, let's just build the thing because then we don't have to even get into the politics, which is kind of ironic in a way. It ends up taking just as long or longer.
But it's interesting how, like, that default option changes because of non monetary factors.
[00:25:09] Speaker A: Yeah, thought about that. But, you know, a lot of times we talk about people, processes and things, thing and technology. And the technology in many cases is the simplest of them. It's the people and processes that get in the way of the progress make it harder to do some of these things.
[00:25:31] Speaker B: Somebody doing construction in the room?
[00:25:33] Speaker A: Yes, unfortunately, yeah. I asked them not to work on the thing right behind me. They don't do anything there for the next 30 minutes, so, yeah, unfortunately, you know, workers got to do what they got to do. Yeah.
It's bound to happen again. That's what happens with these live things.
Anyway. So what do you got coming up next? Are you going to be going to any events you're going to or organizing?
[00:26:07] Speaker B: I'm running a few events, so I run the Boston Snowflake user group, and we're doing stuff monthly. We've got a few things coming up. We're doing a snowflake build event workshop.
We do socials every month, so that's certainly taking up a fair bit of my get out and do technical things.
[00:26:31] Speaker A: So you do little social meetups as well as the technical meetups?
[00:26:36] Speaker B: Yeah, typically every other month is a social. We'll go to a pub or we'll go to a brewery or something and just meet up, get a chance for people to talk.
Snowflake provides a little bit of budget for food and drinks, and then on the alternate months, we will do workshops, presentations, lightning talks, various structured meetup events. The next one coming up in November is a workshop that me and the other organizer will be actually running, but we bring in speakers for all sorts of topics, all snowflake related, and then we occasionally also.
[00:27:34] Speaker A: Snowflake user group.
[00:27:36] Speaker B: Yeah, yeah. Then we occasionally also work with some of the other user groups. We've done a collab with the remote data vault user group.
We're looking at doing one with the women in Snowflake user group as well.
[00:27:51] Speaker A: Oh, great. Good. What's the best way for people to get a hold of you?
[00:27:59] Speaker B: I'm most responsive on LinkedIn. That's the place where I will actually see your message if you want to get ahold of me. I'm also at the Boston Snowflake user group.
We have a slack channel for that as well. That requires getting an invite sent out, so that's not a great place to start, but LinkedIn, great.
[00:28:22] Speaker A: And yeah, the QR code is right there on the screen for everybody if they want to track you down.
All right, well, given I got this note, noisy construction going on outside, I think we're going to call this one done here. Thank you so much for being on, David, and providing your insights.
It's interesting. Like I said, I have a new perspective now on the whole buy versus build thing and the different budgets that I hadn't even thought of before, so that's a good one.
I want to thank everyone else for who's online joining those of you watching replay later, just a reminder to join me again in two weeks. And my guest is going to be thought leader and industry analyst Wayne Ekerson, who's also one of the co authors of the data ops for Dummies book and the philosophy and seven pillars of true data ops. So, as always, be sure to like the replays from today's show once they're up on the YouTube channel. Tell your friends about the True Data Ops podcast, and don't forget to go to truedataops.org and subscribe to the podcast so you don't miss any episodes. Until next time, this is Kent Graciano, the Data warrior, signing off. For now.
