0:00[music]

0:09[music] I recently worked for like a gigantic company. Um, they make a lot of shoes.

0:16They have a swoosh and that's what they are. Um, we'll get into that in just a sec. So, but before we get into like where I've been or anything else like that, we'll talk more about me. Um, a lot of times when people come up on stage, they like to talk about themselves. So, I'm going to do that,

0:29too. Um, so who am I? I'm an author. I'm a mentor. I teach. I'm an engineer. I like to cook because engineering is difficult and like cutting onions at the end of a day feels really good. Um, I also do hot sauce. Um, does anybody in here like hot sauce?

0:42>> Yes. I was in New Orleans last week and a lot of people like hot sauce there, too. So, it always feels like a nice thing to be able to, you know, it's something we all have in common. Um, I also like adventure sports. I say that because I don't like kind of traditional sports. I like snowboarding. I like

0:57watching people like throw themselves downhill on the Red Bull Channel like on a mountain bike. It's like what are you doing? That's just scary stuff. Um but it's super fun to watch them not, you know, die. Um so aside from that, um I've been writing for a long time and it's like another thing that was like sort of kind of like a stressfree

1:15situation. It's like, you know, the end of a hard day, what do you want to do?

1:18You want to go write be on another deadline. Um so I've written a bunch of stuff with like O'Reilly. Um I've done stuff with AP press as well. um uh was a data bricks beacon back in 2020 which is like a nice like kind of boost for my morale during like the pandemic and then I've been a data bricks MVP like I guess

1:35now for the past like five years. Um so now we start to talk about the swoosh stories. Um so this whole entire talk um I like the company that I used to work at. I just want to preface everything that I say with that first. Um so we'll

1:52figure stuff out from there. But so once upon a time, right, we're talking about stories and today is a day about stories. So once upon a time, I used to work for one of the most iconic brands in the world and we had problems. And if you think about problems, like who here works in an enterprise?

2:08Okay, good. So they know, you know what problems we could have? Uh, who else here like works with companies that sell products to enterprises?

2:17Awesome. So, more people. So, like there's a lot of problems that enterprises have because for the most part those companies have been around for a long time. Uh, when I worked at the swoosh company that I can just say because it's Nike, like when I worked at Nike, they were around for 50 years.

2:30They're still around right now. And so, in 50 years, right, like you think about the amount of stuff that's changed, it's everything. Um, and so when you're, you know, essentially sending mail back and forth across the world, um, to get things accomplished, it's really hard to kind of change those practices to then be like, oh, like, well, now we're going

2:47to automate that because people are afraid of change and change is scary. And so we think about data problems. We had problems at Nike. We also had data problems at Nike because you have, you know, essentially like accumulated so much, you know, baggage, technical debt, different types of like, you know, vendor solutions that come in and stay

3:05forever. Like if you're an enterprise, you're going to buy a solution and you're going to let it run forever because eventually everybody on that team leaves and then there's no one there to run it. So no one knows how it's being used. So it just will, you know, just you keep paying the check year after year and that's okay. But we

3:19also had people problems. And so a lot of times like people problem and data problems come together in the same sort of way because you know the same problems you have with data is usually about the trust of the data. And the same problem in a large company is usually with you know do you trust or not trust the people that you're working

3:34with like trust is you know one of the most important things and it's really hard to build. And so if I talk about lessons, which I've already started talking about, but just to kind of rephrase, this is essentially me trying to talk about four lessons when there's many more than four lessons, but these are just bite-siz stories and you know,

3:51jokes cuz data fun. Okay, so data decisions. Uh, who here uses data to make any decisions? Awesome. Probably everybody. That's good. Late in the day, we're making it through. Uh, after me, there's only the panel and then there's a reception. So we you know we'll figure that that out too. So if I want to make data good, I need to build trust, right? And

4:16if I have like decision indecision, which is a hard thing to say because I also have dyslexia, so I'm reading this all backwards. I'm like, ah, that's weird, right? But like most of the people who work with data on like a daily basis, you want it to work. You want it to be trustworthy. You want it to, you know, be consistent, you know,

4:32through releases, things like that. Um, but if you've been, you know, essentially harmed by, you know, the sins of the past or like you have like accumulated scar tissue of like things that keep on breaking, um, then there's a ton of problems that you just really can't get past. And so, like a lot of like the reason why I was, you know,

4:47anyways, like I'll say the name again, the reason why I joined Nike in 2022 is essentially to help like change the engineering culture. So, I used to come like I used to work at companies that, you know, were not retail. I worked at Twilio, did a lot of stuff with like, you know, Telephony and like global, you

5:02know, eyes and ears across like our entire stack. I helped build the voice insights team when I was over there. And we did a lot of fun stuff with like Spark and we had just mountains and mountains and [ __ ] tons of data. Um, so those problems are not necessarily the same thing that you have like when you're working with, you know,

5:17e-commerce. Like I have, you know, a finite number of users. I've got a finite number, you know, of SKUs or products that I'm going to keep in a database. And I've got a finite number of places where I can actually send anything. And so those problems aren't as bad as like, you know, having like, you know, 50 billion to a couple

5:33trillion, you know, individual unique messages a day like when I was at Twilio. Those types of things continue to roll over and you have to solve different types of problems. So the bigger problems we had at Nike was essentially just on communication and people. And so right now to kind of go into you know I don't know maybe just me

5:49just talking but so you know if you're thinking about like you know coming into a new company a lot of times people give you time to figure out what the company is like. Um I had 90 days when I first joined and I joined as a distinguished software engineer. It's like figure out what is all screwed up. Figure out the

6:05thing that is the easiest to fix. uh we used to call it ground fruit because it's like instead of like you know low hanging fruit it's something you're stepping on because it's very easy to step or trip over that those are simple problems and so it's like don't solve the hard problems first solve the simplest problems and then build trust

6:21right so usually like the lies we tell ourselves are more like this guy so it's like you know bigger is better right enterprises it's like let's just keep scaling indefinitely so uh who here is like manager level director level senior director

6:37VP, senior VP, right? It keeps on going up. It ladders up. But the big problem is that like there's, you know, at a certain point there's not room for more people. So then you subdivide. And so in a lot of companies like at Nike, you end up getting like 25 different organizations within like just global tech itself.

6:54So here company that

6:59people are head nodding like [ __ ] this [ __ ] right? But um so a matrix is essentially like a really hard way of getting any like decisions made, right?

7:07It's not not one person, it's many people. And so it's like you're not selling to one person, you're selling to an entire community. So like every idea you have, you're essentially selling it like any startup is trying to sell new software, but it's like, well, we just need this thing to work. Can't we just do that? And it's like, well, give us

7:22nine months to figure this out, right? It's going to go through debate. It's sort of like this inquisition. Um, so bigger is better is like that kind of idea where it's like the more we grow, the harder things should be. It's sort of kind of like a question, right? And so, you know, we're here at small data,

7:37so we think about data, right? We've got less data. Less data is kind of puzzling because people are like, but I, you know, what about big data, right? And so it's like bigger data is kind of like that dream of this dude that Gemini created that you think is super cute.

7:48Um, but he's like dreaming of this like world where it's like I could, you know, what if I had more, you know, what if we scaled? What if we made things more complicated? Uh what if that complexity grew to the point where it became like this like monstrosity and what if that monstrosity just kept on growing? Um

8:01does anybody watch South Park? Um there's the Trapper Keeper episode where it's like things just keep growing and it's like you're accumulating all this debt but then you're just taking over everything else as well. And that's sort of kind of like what happens if you have a ton of data, you know, warehouses that just kind of grow over time and you have

8:16a ton of teams that don't talk to each other. Uh eventually all you're doing is copying data from every point to new points and that's just all you keep on doing that forever. Um, and then you have like, you know, terabytes, then you have pabytes, then you get zetabytes and everything else like that. And then things fall apart and you delete it all

8:30and you try again. Um, so questions, right? It's like, well, that's weird. Um, what if, you know, what if we could do something different, right? It's like, what if we could do a lot with a little? Um, this is actually something that we talked about a lot like when I was at Nike. Um, it's like, you know,

8:45Gasp, we have made things so challenging that we can't get a single thing done. It's going to be at least nine months.

8:51Everyone always said, "Well, not 9 months, it's 18 months." Didn't matter what it was, it'd always be an 18-month deal. So then everyone's like, "Screw it. We're going to stay and we're going to do exactly what we are going to do and nothing's going to ever change." Um, but it's like, well, what if we changed the status quo? And like a lot of

9:05problems that companies have as like they grow over time is this, you know, this little kind of insidious problem of like, you know, h [sighs] what I'd have to do something. And it's like, well, yeah, you're getting paid to do stuff, right? It's like that's like what we're everyone here is getting paid for something or wants to get paid for

9:22something, right? And so, you know, you usually want to do something that's gonna actually be like exciting. Like my analogy that I usually kind of talk about is like um you know, your job is like a mattress, right? This is kind of weird, but anyways, like you spend most of your life in bed because we sleep, you know, 8 hours a night unless you're

9:37on the pager. Um then you don't sleep, but when you do sleep, it feels really good. But a team should also be something that you're comfortable being around, you know, for the other side of your day as well because essentially that's your family, right? So it's like great, that's awesome, that's nice. But going back to like the actual main

9:53storyline, it's like, you know, when I first joined Nike, there was a lot of work. There was a lot of stuff. There was things going on. Everybody had things to do. It was non-stop, but we're going nowhere, right? It's sort of like um does everybody I'm old, so anyways.

10:07Uh back in the day, there used to be a little Garmin before people had iPhones, and if the map didn't exist, it would spin forever, and it'd be kind of fun to watch, but then you you'd still be lost.

10:16And so, we were kind of charting a course to nowhere. And so, charting a course to nowhere, you feel good. Things are happening, you're getting paid, work is being done, but you're not really changing anything. And so, with that said, and I'm going to totally run out of time, so we can skip through some of this stuff. Um, I wrote like a two

10:32series blog post on like a lot of like the opportunities and like the whole entire, you know, event of my three and a half years at Nike. Um, but one of the things that like I found is like if you want to really make change happen a lot of times instead of going for ground fruit, it's like go for something that

10:47would actually like change everything in the company itself. Um, I've been doing streaming for like over a decade now.

10:54Um, and one of the hardest problems that like we had at Nike was essentially like consumer behavior and like our click stream. And so click streams are difficult because you know you've got tons and tons and tons of different types of events to encapsulate everything. Like I'm standing that's great. Now I've moved. Well, I need I need a new event for like essentially

11:10everything that I'm doing. And it becomes very complicated to really kind of distill that information down into something that you can really do anything with. Um so uh who here does anybody know about like e-commerce in general? Like has anyone bought something on like Amazon? Great. Lots of people. Um, so there's something called core buy and the core buy is essentially

11:30like that funnel that people take where they're like, "Oh, that's neat. I will pay money and that's it." Right? Core by flow. It's pretty simple. You know that somebody has to do something to put something into a cart to be able to take that something from a cart and then turn it into a physical good or a digital

11:45good because we sold NFTs but we won't talk about that. Um, and so aside from the NFTs, which were I apologize for that on behalf of the company, but um, aside from that, um, if you have something that shows up at your house in a box from Nike, sometimes it shows up on time, sometimes it doesn't, but we

12:02got a little bit better at that. Um, but all of that, like all that information had to kind of come together. And the big problem was that none of it did um, until I showed up. Not that I helped fix everything because I was in a leadership position. And a leadership you you're like, "This is a great idea." And people

12:17are like, I don't care, but you try your best. Um, I was also like an IC and like um we're having like conversations before like back like backstage and it's like as an IC depending on certain companies like technology kind of wins and like engineering wins. In other companies, management's like well that's nice, that's cute, we're going to keep

12:34on doing this other thing. And so anyways, I'll probably kind of skip skip over this part because of trauma, but anyways, like as like as an engineer, um it was very hard to get things accomplished. And so we had to chart a course and we had to build a culture. Uh because we're kind of, you know, getting

12:51right out of like the, you know, getting right out of the pandemic. And getting out of the pandemic meant that people were finally physically showing up in person again. It had to be nice. Like in Zoom, it's very easy to be mean because you can turn your camera off and you be like, "I hate you." It's like great. Um

13:03that's cool. But in person, people are usually nicer. And also like in person, like I'm not saying people should go back to the office. People should sometimes meet up in real life like we're doing right now because it's nice to be able to understand like social interactions and like how to like, you know, kind of behave as a team, have

13:18ideas together, things like that. Um, I always found that like a team that like eats together usually kind of stays together and you have like a nice little family. Um, so going on to teams, right?

13:26What about the right team and like what if they cared? Uh, who here has worked for any company where you feel like you had a team where you're like, "That was the goddamn best team I've ever worked with." Awesome. This is like beautiful because I've seen so many hands raised.

13:39Like finding like those types of teams is like uh usually it's like a once or twice in a career type thing, but usually you're not just handed that team. Like usually like you have a smaller team and then you're kind of handpicking people that are filling very specific roles to make that team a good team. Um, and they usually have to care.

13:56So like earlier today I think Sam had put some stuff up on the board where he was talking about like curiosity right like very good teams of very curious people who want to solve really difficult problems and so if you can solve really difficult problems as a team usually you're not doing that in isolation either you're doing that kind

14:11of as like a collaborative effort and so like before we had LLM to do that we had people to do that and then you know now we have LLMs and people so make your decisions um but good data right because I've got five minutes left we can skip through some stuff um who here has ever dealt with bad data everybody right of

14:28course um data itself is difficult like it's you know it has to represent something that is tangible even though it's you know essentially you know some sort of you know just it's just bytes um but if you think about being able to kind of like encapsulate any sort of story like the right data helps you kind of patch those things back together

14:46again and so if you think about a clickstream or anything that's like a time series analysis like you know there's a start there's a finish things occur in between and it kind of tells tells a full kind of you know a cohesive story which is always good but the data is the thing that is actually the one

15:00telling that story and so it doesn't have to be difficult uh we use protobuff I work at a company called buff going to say that real quick I can get you know get away from talking about the company that I work at but I like protobuff it's really great um and I at least had to just plug that real quick so good data

15:16at Nike we use protobuff um protobuff is like very binary compact it's very good for streaming um there's a lot of cool tools around that I've got four minutes left. So if anybody wants to ask me a question about part later we can do that. So good data helps make things work right and then so if you think

15:33about you know having your data work right there's another conversation today um about you know testing your data pipelines like you would software um software releases either go out and cause problems or they don't go out and you don't cause any problems. The same thing with, you know, a pipeline, right? Like I want things to work between releases. Therefore, I

15:54should care about the things that I am building. Um, and so if you work on a team where people don't care, you have a lot more pager problems. And like Holden had like the pager up before. I hated being on a pager. So I had one night where I got a thousand pages because everything kept on falling over. And at

16:08a certain point, you're like, "Fuck this. I'm going to, you know, I don't even want to be part of this company anymore." It's like, what am I doing?

16:13Right? This is what am I doing with my life? Everyone else is having fun watching Netflix and I'm sitting here trying to get Elastic Search to get back up and running. Um, okay. So, 3 minutes.

16:23We're going to keep going as fast as possible. So, uh, simple data architectures. A lot of times people will show like these complex, you know, graphs of all these wonderful things they're proud of. Um, and that's wonderful. And then we talk about like duct DB and things where it's like, but what if it was just this? And so like

16:39the work that I worked on for the past like 15 years now has been essentially like how do you take something that's complicated and make it really really simple and then how do you do that like at scale over and over and over again which is just like software only with like only with pip uh pipelines they

16:52can't talk. Um, and so like if you think about how do you like unwind 50 years of like problems at a company that you know sells shoes, a lot of times you simplify like kind of the you know collaborative chaos of what was actually being built and so simple data architectures are a great way of doing that. I do have an

17:10architectural diagram at some point. Um, hold on one sec. So speaking of architectural diagrams, everyone can be excited. Yay. Fruits of our labor. So I've kind of very quickly gone through like you know things happened it's in the blog post we fixed some things we had some cool ideas and this was essentially like at the end of the day

17:28like ingestion for about 10,000 different pipelines all following the same sort of thing. So you have clients, a client sends data of some sort that's analytical. Um that's doing being done with Protobuff and gRPC. You've got an intake API that says this is good or this is bad. Simple. That's awesome. Uh there's also something called proto validate that's open source that makes

17:49it even easier. You're essentially compiling your rules. You don't have to write validation. And then things go into CFKA and then life gets easy. Um, and you know, put any technology you want in after things are coming off of CFKA and this will usually work across the board for everything. Um, you don't even need CFKA depending on like the

18:05size of the data as well. You can just go from gateway API service directly to database as well and just, you know, get rid of most of the stuff in like that foundational trust by design box wherever wherever you know whatever screen you're looking at. Um, and then things just kind of work. And so like there's this notion right in like

18:20software of like I want to create a library. It's generic. It can now be anything you want. You can do the same thing with a pipeline and people have done that and there's probably people in this room that are selling it as a service and it really works. Um, which is nice and you know then you don't have

18:33to worry about debugging everything differently because it's all the same thing. Um, you know, just a different pager, different, you know, different schedule. Um, so quick leadership lessons because we've got a minute left.

18:43So take risks, break rules, erode the status quo. Um, I like taking risks, you know, I'm into snowboarding. I take risks every time I throw myself down the mountain. Um, but also like if you're, you know, working at a company and you want things to change and you're hoping that somebody else will do that, you're the one who can do that. And so

18:59sometimes it's worth taking a risk and betting your job. So like if you think about things that are important to you, they're probably important to other people and a lot of times, you know, there's a hill that you're willing to die on or there's like, you know, an argument that you're willing to get fired over. Um, I've done that twice

19:14when I was at Nike and it worked out. Um, Papa bearing it. Um, when I used to work at Twilio, there's a guy Joshua Bartlett. Um, and hopefully I just said his last name correctly. Um, he always talked about Papa Barrying it. So, it's like if you at a certain point have risen up the career, you know, your

19:30career ladder wherever, you know, wherever you're working, you have clout in a company and you have the ability to stand up for people who can't. So, do it because, you know, pay it forward. um you know turn those little sad bear cubs into somebody who is happy but you know protect and insulate that little family and we don't really have to worry about

19:48the results amount of time. I'll just really quickly go through it. So, we used to have 52 uh 52,000 uh clickstream events. We turned into 133 because of core buy flow. That saved us a lot of time. Uh used to be $4,000 a day went down to $16 a day to run our pipelines.

20:03Uh just for literally that was just for click stream. Doesn't nothing else on top of that. Um our costs were very significant. But anyways, uh lead time for change 90 days to one week. Great. Um on time call per engineer per week who was actually on call. 70% of the time that people were on call before they were triaging stuff

20:21and pulling their hair out. Like I don't have a ton of hair so I'm wearing a hat um from being on call. Um and so if I was the 1% I'd have hair and I'd be much happier. Um so final lesson um no one to leave right at a certain point it's okay to like walk away from a company. You've

20:36done well. Usually everybody who's like left a company knows when that time is. You feel it. It feels right. So, sort of like a wolf kind of going off to die in the woods, but not in the same way. Um, just no one to leave. And so, with that, that's it. So, thank you.

20:57>> [music]