Transcript of the App Performance Café episode with Mark Finkle, from Tumblr, originally published on July 9, 2020. More details in the podcast episode page.
Rui Costa:
Hi, welcome to the App Performance Cafe, a podcast on mobile app performance. My name is Rui and our guest for today is Mark Finkle from Tumblr. We had such an interesting conversation around the key technique of real user monitoring or RUM, which is fundamental to understand how your users are experiencing your app in the real world. And why is this so relevant to define proper performance budgets. Then Mark will share with us his experience with progressive image formats and the impact it has on performance. As well as why you should look at the 99th percentile latency-wise because that's probably driving a user experience, even if you don't think it is. And finally, Mark would share with us, which for me kind of struck my mind, how can we use a lithe version of the app to actually the original one performance-wise. Hope you enjoy the conversation and don't forget to follow us on the usual podcast platforms, as well as visit performancecafe.codavel.com
Episode Start
Hi, everyone. Welcome to the App Performance Cafe, a podcast on mobile app performance. I'm Rui and I'm very honored to have you with me, Mark Finkle from Tumblr to tell us about his view on mobile app performance. Thank you so much, Mark, for taking the invitation. Can you start by telling us a little bit about yourself and what you do at Tumblr?
Mark Finkle:
Sure. At Tumblr, I'm the Director of the Platform Engineering Group. It's a lot of the infrastructure side; including infrastructure for mobile apps, as well as some of the data infrastructure and other things like that. Before Tumblr, I led the Firefox mobile group at Mozilla. And before that I've been writing software for a little while.
Rui Costa:
Yeah. So, I guess that one of the things that made me send you an invite was precisely that you had that experience coming from a browser, Firefox, and then merging into a mobile application - which I guess it's not the different - but for sure there will be significant differences in how you look at performance and what you can do about performance in those two use cases.
Mark, so I always start with this introductory question about why, in your perspective, why should we care about mobile app performance?
Mark Finkle:
I think one way to look at it is it is a big part of your user experience. If you care about how users perceive and use your application then performance and quality should go hand in hand, and should play as big of a role as features as well. I think it also says a little bit about the engineering rigor that a team actually spends on the code and the effort that they put into what they're building, if they're also focusing on the performance aspects of things as well. If you're not focusing on performance and quality, then what kind of job are you actually doing with the features and other kinds of work that you are putting into the application? So I think it kind of gives you an insight into the engineering practices and the way the product is put together by looking at the fit and finish and things like performance.
Rui Costa:
Wow. Yeah, I have the same feeling. In our case we do see a lot of correlation between performance and precisely like the quality and the effort that you put in building up strong, robust, software in general. And in the case of a mobile app, I guess it's even more or challenging.
So I guess that the question is. How can we measure that? So from my perspective, you have the old fashioned way to do it - you give your app to a set of users, you do this focus group, you're watching your users using the app - from that you get some feedback on how the app is behaving and how you can optimize the app. But how do you do that in scale? So how do you see, what are your users experiencing with the Tumblr app, let's say? In the real world - so different users, different locations, different goals when using the application, different devices and different metrics. So how do you see all of that in the real world?
Mark Finkle:
We end up using a lot of, this is probably standard practice for any application of scale, is to actually measure all of this: all of the events, all of the performance metrics, even user flows for some sample of your user base. Doesn't need to be the entire user base, but sometimes it is depending on how much data you want to ingest. But actually collecting this data and I think it's called real user monitoring, RUM for short, gives you insight into what the user experience is like for different sets of users in the real world. Like I said, you're probably going to be sampling data in some cases, because there's a lot of data coming in, especially if you're collecting event flows, as well as some of the performance aspects of your application. So you do need to worry about storage and ingestion and things like that. Also you're probably storing the raw events, cause you want to always be able to slice and dice the data, looking across different dimensions or facets of the data. You never really want to take all of those data and just aggregate it or roll it up into some numbers and then not be able to dig into it to find specific sets of users or experiences that you want to actually focus on.
So, like we said, you're going to be collecting this data. It's gonna have metadata associated with it, such as some form of location, maybe the network speed, the type of device, the amount of memory in the device - all of these different aspects that you can then try to subset and figure out what is the experience of the startup time or the way images load for different sets of users. That kind of plays into some of the topics that you've had on before on performance budgets, it's pretty easy to talk to your product people or even leadership of the company and say, "How long do we expect the application to take to start up?", or "How long do we expect this screen to not display an image while it's loading?" And your CEO or CTO or your product people are going to come back and give you some kind of probably realistic number of what their own expectations are.
And real user monitoring allows you to take those performance budgets and see how well we do in the real world? Because it's pretty interesting. A lot of people end up running their application locally on wifi at the office on a nice phone. And when they're testing things, the app starts up really quick - you know, maybe two seconds. Or as you're scrolling through the feed, images are appearing. Like they, maybe that you don't even see them appear. They're just always there. It's like, how did this happen? This is amazing. And then you take a look at some of your user data and you see that - oh my gosh, 25% of the image requests take longer than 10 seconds. That doesn't match up with what I see, but that is literally what your users are seeing. Or the application might take 14 seconds to start up in some cases, how did this happen? Because this doesn't match up with what I see in the office. So, getting that real user data does give you a lot of insight into how those performance budgets are actually being met by the application with your real users.
Rui Costa:
Taking a step back. So, how do you define what events to track? So you're not tracking everything otherwise it's unbearable from the point of view of storage on the device and actually sending the data to the cloud. How do you pick what to measure? Is it just a single event? Do you also measure like a succession of different events within the app? How do you do that? How do you make those decisions?
Mark Finkle:
I think the one place to start is thinking about what your product does. What is the purpose of the product in some ways. For Tumblr or other types of social apps, it's about getting a feed and probably a lot of images. So you want that to be crisp and responsive. You want scrolling to happen in a smooth way. So from a product standpoint, you can actually think about the focus areas, and then you can put more measurement in some areas than others. The startup time is another one. People end up caring a lot about the way that you - especially the way that new applications work, modern applications in a way. It's not really modern, I mean, this has been around for awhile - multi-threaded is a real issue on devices, especially native. You're creating a web app, JavaScript is a single threaded thing, but there are still other issues going on with network requests and whatnot.
But in a native mobile app you're probably using threads, background processes in some ways. And all of those things actually work against you whenever you're trying to figure out what the performance of startup actually is. So breaking up startup time into some constituent components or pieces is usually a good way to look at things, so you're not looking at a number. Perhaps start up time for you is when the application starts and the screen is visible and the user can interact with it. Okay, so you have an endpoint. But if for some reason it takes 10 seconds to get to that point and now you want to start debugging it. You really want to be able to send a little bit more information along. You don't want to have to put new data points into the application and then ship it in order to be able to do this. If you know, startup time is your focus area, go ahead and instrument it a little bit more broadly.
So you can say I'm going to put a duration from when the application code starts, by the operating system, to when my main function, or whatever that ends up being, starts getting hit. So now I can actually measure potentially how long it takes for the OS to actually jumpstart my app. I'm going to start another duration: from when my main function gets called to when the application is initialized. I do a lot of STK initializations,I might reconfiguration from disc or memory or wherever, or even network - for crying out loud - and I want to know what that period duration takes to look like. Okay, now I'm doing my network, my main network call, or I'm reading from cache to load the feed of whatever I'm displaying on the main page. So I want to figure out what that duration is, and then there's the rendering, right? I'm rendering the posts or whatever the images and when that's finished, hopefully we're at time to interaction and we're done. So in that case, you've got startup time, but you've got it broken up into a couple of constituent pieces. And the odd thing here is those are probably running in parallel in some way, shape or form. So they're not sequential, and having a map of that is helpful for you to be able to start understanding exactly how the application works in the field. How early or how late is the feed being read from the network. What other competing network calls are happening at the same time? As you're trying to read the feed? Are you blocking on the configuration being loaded from this, or if you need to download a new one from network? Those are the kinds of insights that you can get by looking at some of this data when you've broken it up into different traces or durations.
Rui Costa:
Yeah, in this multi-threading world it's always very, very challenging to actually understand what in fact is what, right? Everything's happening at the same time and breaking it down, as you said, if we have everything properly instrumented and measured, we may be able to replicate what happening in our own office and then learn from that. And that's, I believe, one of the most challenging components when working on optimizing performance within an app.
You mentioned time to interactivity. So in an application like Tumblr, what are your key goals with respect to interactivity? So when do you say that your application is interactive?
Mark Finkle:
Right now we look at when the first initial pagination of the main feed is rendered. By that time it's also interactive, it's probably technically interactive even before then just because the text being rendered in the posts is a different pipeline and the image is being rendered in the post. And honestly one of the Holy Grails that we really want to figure out is instead of just rendering, we want to understand everything that is on the screen being rendered at the same time and when that's finished. Because if you really think about it from a user experience standpoint, if I'm opening up an application and it's displaying a post that has four images in it, and those are animated GIFs and is my expectation that the app is ready to go when the text of the post is visible to me and I can actually start scrolling, or whenever all of the images are rendered and animating as well.
So that can be a tricky part to try to encapsulate all of those different threads and pipelines right now. There are times whenever you have to say this is good enough, it's good enough for an answer, right? And maybe we don't have the Holy Grail of startup time,with all the components meshing together. But it's good enough for us to figure out what relative performance looks like, and if it gets better or worse over time, we at least know where we're working against ourselves in that case.
Rui Costa:
An application like Tumblr, I would say that - at least this is my perspective - you are heavy in terms of media, like images, at least I'd say. So I'm very positive that you have big, big challenges when delivering a good experience with such a heavy media, a savvy media application, like Tumblr. How do you tackle that? So, yesterday we were talking about this post with 12 GIFs and everyone expects that to happen. Everything simultaneously, everything loads immediately - that's obviously not achievable always at least. So how do you tackle that? So how do you guys optimize the experience in this such a savvy application.
Mark Finkle:
Working with the user data, again, helps us figure out our performance budgets. And even though if we have a post that has 10 GIFs all animating and it looks great, whenever we're running in the office, we can tell from the data that this doesn't always happen. That in some cases it can take 10 to 15 seconds for some images, just single images to load and in some places, in some situations. So we know that the user experience for everyone is not the same. And we do use some of the data that we collect, this RUM data that we collect, to help change and mold some of the user experience.
In the case of images we've definitely had to add some changes and a lot of work by a lot of teams. One of the teams is in charge of content delivery, worked on just even the GIF format itself, not the smallest format - it's an old format. It's very versatile, but it doesn't end up with the smallest file formats that you're going to try to send down over a network connection. So, looking for alternate formats, a lot of other applications will use MP4 instead of GIF because of the size reduction, basically plays it as a video. In our case, we felt that the user experience there might not be what we want it because playing 10 or even 5 videos at the same time creates a bit of a challenge. In fact, some hardware we were running on wouldn't even do it, it would actually crash if we tried to run more than two or three videos at a time. So we had to look for alternate ways to handle that experience, and that team settled on a format called webp. Which is still an animated or static format, but uses some of the same techniques that you'd see in MP4 to reduce the size significantly of a GIF. we want to download the applications faster - let's make the image smaller. That will take less time. And in fact it does it, it's quite a significant change to go from GIF to webp. But it still didn't solve all the problems.
We had to look at other ways to be able to do things as well. And in some situations we actually decided that in really low bandwidth situations, cause we're monitoring bandwidth - not just the type of network, you know; wifi, 4G whatever, but the actual bandwidth as well - cause it could be hotel wifi, which is notoriously, not that great. We decided that in low bandwidth situations let's change the experience entirely. Let's gracefully degrade so that we would not animate to begin with. We would show static placeholders and if the person wanted to see these images animating, they could just tap on the images and it would do the full download and go ahead and start animating the images.
This helps us in a lot of ways, especially for scrolling. It's in those situations where if you want to scroll through a feed and see a bunch of gray boxes - great, right? That probably not a user experience that we want. So instead, in low bandwidth situations, just static images, instead of the animated images, it creates a better experience as you're scrolling through a feed and, "Oh, here's a post that has something I want to see." I can tap on it and it starts animating. The end goal there is as good as performance or as good of experience as we can give with the current situation, which isn't ideal. And those kinds of situations crop up because we collect data and we understand what users are actually experiencing.
Rui Costa:
Yeah. So, thank you for sharing that. When I look at Tumblr, I always think, how can you make such an interactive and such a good experience, which such a heavy and challenging - for me as a network engineer's point of view - like so many network calls with so many images, animated images: it's quite impressive to see the outcome working pretty well.
So then the question I have is - so we're measuring all of this. We are monitoring, or instrumenting key pieces of the user experience. But how do you look at the data? So for example, how do we handle outliers, what are the roles of looking at percentiles and the distribution of what you're measuring?
Mark Finkle:
Well, whenever you're collecting real data from users, you're going to get weird data as well. Sometimes the code is not instrumented exactly the way you think it's going to work out.
Rui Costa:
Yeah - been there, done that.
Mark Finkle:
And I get some very strange looking numbers. So I think one thing that we have learned, and some of the folks on our teams have really done a good job at is trying to vet the data. As you're starting to look at this data coming in, ask yourself, does this even seem realistic? Is there something, is there a problem here? Do we want to include all of this data or is some of it an outlier? Like, does it really take two minutes for the application to start up? Or is there some kind of weird thing happening with background processes and timing and threads and the code just didn't do what we thought it was going to do.
So at some point, you're probably going to put some kind of limits on the data that you're even going to analyze to begin with. Anything that falls out of those bounds based on common sense, and hopefully some group discussion, is ignored and you just focus on, hopefully it's still a sizable chunk of what looks like normal data within the normal realm of possibility. And once you're doing that, you really don't want to be dealing with averages as much. Some of the data teams we have at both Tumblr and Mozilla have always looked at percentiles as even a first cut. And there are ways that you can even go deeper than percentiles and look at heat maps, and all kinds of other different ways to look at latency.
Percentiles is a very easy way to start and you can think about things in terms of what percent of my network requests are greater than a certain latency? Or a duration or whatever you're actually measuring. And it's kind of interesting when you start thinking about that, because most people don't go immediately to 100 percentile or the 99th percentile or the 95th percentile, because that seems crazy. You know, why would you look at such high percentile numbers? But as you start digging into this a little more and you think about it, and you see a lot of what other people have written about this. If you consider Tumblr and you think about what is the 90th percentile latency for an image download and what if it's six seconds and you think, "Well, I never see that", right? Well, if you're scrolling through a Tumblr dashboard, you probably end up seeing a hundred images at some point depending on how far you scroll. So technically you probably are in that bucket, you do experience that. And these are also things that we really need to keep in mind when we're thinking about this data. And we think abou how often do I actually experienced some of these things. That's why thinking about things like percentiles and the 99th or 90th even percentile is a pretty useful model to go after. You know, we look at even the 75th percentile for a lot of our typical alerting numbers. And we will alert on some of this stuff as well. We have both an alpha and a beta cycle, which is also sending data as well as the production cycle. We will take a look in the production systems and if startup time, for example, seems to regress by more than 5%, you know, we alert on it and we'll start digging into what happened here and what can we do to fix it?
Rui Costa:
Yeah. So before going into that, because I'm really curious about CI and the relationship between CI and real user monitoring, but you're talking about the 99% percentile. I remember a few years ago, maybe I was listening to a talk by Jill Tan and actually wrote a post on that. So, and it's so simple - the example with the google.com page, it's such a simple page. It involves 31 ATTP or - by the time I wrote it involved 31 different HTTP requests - so this actually means that when you open google.com, the probability that you will experience something like the 99 percentile is actually above 26%. So one out of four people will experience 99% percentile experience in google.com: such as simple, simplistic page. So it's quite striking when you look at the data from this perspective. And that's why you should definitely look at details - so look at the higher percentiles.
Going back to - how do you relate this and use this information and in the development environment. So in your CI cycles, how do you do that? If you do it at all?
Mark Finkle:
We had done that at various times. It can be as tricky to keep up and running, as a RUM system is also tricky to keep up and running. There are third parties that make this a lot easier as well. In both cases, both in RUM data collection, as well as running performance tests in CI. One of the first things with CI, continuous integration, and performance testing that you'll probably come across is don't expect it to mirror what you see in the field. That's not its purpose. The purpose of running performance tests in CI is to give you a signal of when something goes wrong.
While RUM data has immense amounts of variation in it for different types of devices, networks, you know even feeds, right? The feed that you get on your dashboard is different than the one I get. So the content itself is causing variation in CI. You want to minimize all of that as much as possible. And you want to run on the - at least they consistent type of device or device class. You want to make sure that you're giving it a consistent content that you're measuring if you're going to load. Either, hopefully some kind of network proxy or a mockup, so that you're not actually calling out over the network in order to do this because that also adds variation. We want to try to minimize other applications that are running on this device or delete them entirely if you can, just to keep that aspect of noise down. And then what you're left with is being able to look at some of those components that we talked about - maybe with startup time, like how long does it take for the OS to even start my app? How long does it take for me to initialize my app? How long does it take for me to JSON the code, the content that I'm loading? - all of these things. You now have a much cleaner, hopefully a less noisy signal. And then you can use the that clean signal to catch any kind of variations that happen whenever someone checks in a piece of code that suddenly changes one of those metrics by whatever percent you want to alert on. And you can use that then to control code before it shifts. It's not always the way that you're going to catch some of these things.
In our own experience, the running in CI is really great for dealing with your own initialization. "Oh, we added a new SDK. It blocks on something on the main thread. Holy crap, ow we've got a one second bump in application init time". But it's not as easy to look for issues where the content has suddenly changed in some ways. Maybe you have a problem where you've actually introduced a regression in the way you lay out bullet items in a text post, right? And that wasn't something that it was in your CI content, although if you try to carry a lot of variations, hopefully, maybe you did put that in there. But if you didn't, and one of these posts became a viral post and it's being loaded by a lot of people, suddenly you're gonna end up seeing something weird happening in your real user data. So CI I will find a certain class of regressions, but in other cases you really need to also look for just the fragmentation that happens to real users. And we do get a lot of benefit out of our alpha and beta cycles because we have a sizable enough audience using those two products, which basically last a week each before we ended up going to production. So that if we're paying attention and we have alerting on this stuff, there are times when we can catch some of this before it actually hits production as well.
Rui Costa:
We do something very similar like this often, but stages with real usage is fundamental because as you said, something very small like bullet points can indeed create a massive problem. And it's very tricky to find that in the replicable system in your own office. So you do have to take it out and test it out - absolutely.
So I guess that now the question is, so you're measuring all of this. You're monitoring user behavior and experience in this way, but then how can you use this and leverage this information to actually make changes to the actual mobile app?
Mark Finkle:
Well, it kind of goes back to the performance budget idea again, I think. I liked that tool as a way to start conversations with leadership and product, so that you can frame things in a way that they'll find some importance in it as well, you know? And it kind of goes back to that thing we were talking about where you're looking at how can you gracefully degrade some of the characteristics and behavior of the application, because what really matters is the user experience. And you need to embrace the fact that there are actually multiple types of user experiences.
Let me change that and say there are multiple types of good user experiences. Obviously there's plenty of bad user experiences as well, but there isn't just the one ideal user experience - we should be thinking about that. That's the Holy Grail of what we're working on, in terms of the application's polish and the way it runs and what the user can see. But if a set of situations happen that we can't get that user experience, that ideal user experience we need to still be thinking about, well what's a good secondary experience, or even what's the one what's the user experience of a lower than secondary even.
So the example of let's not animate images in some situations, let's just download the static placeholders, which are significantly smaller and that allows the scroll experience to be really good. You know, still hitting our 60 frames a second - we hope. But it does mean that the experience for the user now is that if they want to see the animation, they need to tap on the set of images to get them to kickstart. There are other ways that you can look at things as well. Once you go past that brute force optimization, like we talked about with - Oh, let's go from GIF to webp, then you need to start making the experience trade,right? It's it's, uh,
Rui Costa:
Brute force …
Mark Finkle:
I do think of it as brute force because sometimes it also feels like I'm trying to wring blood from a stone, right. What can I get? Can I get this any faster? But at some point you're going to hit the wall and you need to start dealing with user experience and perception changes instead. And there are different optimizations you can start thinking about locally. You're probably looking at, in our case in many other cases where images are a big deal in an application, you're probably using some sort of image framework to handle all of the different formats and the networking, and caching, right? These frameworks have in-memory caches and local disk caches as well. How can you leverage those to be a more performance application as well? Because the fastest network connection or network download is one that doesn't need to happen.
So if you are displaying a lot of avatars, for example, these are relatively small images, they might cache well, and if you end up with a UI that shows maybe two or three different and avatar sizes. Then you can quickly fill up your local cache with a small set of real avatars just at various sizes. So maybe one of the tricks you do is really going to download one size of avatar and then we're just going to resize it to fit the right screen UI. That means you probably end up with making more space in the local cache, which means you can load more avatar images, which means less network requests.
Another even - it starts the borderline, it does get a little bit borderline with some of the user experience trade offs you need to do, but we've even played around with ourselves of - in really bad network situations where if you want to display like a post image, not an avatar, but an actual image or media that's in the app - instead of asking for the 500 pixel image that you ideally would want to display in that section, maybe you ask for a 250. And you upscale - it's not as great of quality, but if this is a really bad 3G connection that maybe the image would just never load in terms of what the user would experience, because it just feels like it would never load. You know how when you're trying to watch that image load, it seems to take forever.
But making some of these kinds of changes are ways that you can potentially still make a user experience that's better than garbage. And another way that we like ... we did end up changing and doing a couple things that I think a lot of other different types of applications do when it comes to, especially when it comes to images. As I mentioned before, instead of always sending down GIFs, we will look at using webp but also MP4 if necessary. So in that we use this extension called GIFV, which you see a lot of other applications doing. We'll also do something similar on the static image side, where you end up seeing a lot of PNG or JPEG type images. JPEG is interesting in that it's progressive - it has a progressive format. So it's in a lot of cases, it's all about giving the user some kind of feedback that a thing is happening and a progressive image load does that. It allows you even over a crappy network connection, to be able to see that first frame - even though it's really blurry - and then it'll start coming in. That's better than waiting for the entire image to display and you're looking at a gray box or something. So in many cases, we'll convert PNGs to JPEG and progressive JPEG in fact, and then use a PNJ extension. So again, you need to look at the content type whenever you're downloading these things, to figure out what actual image type it really is. But this is just another way of - in this case it is the brute force attempt because progressive JPEGs tend to be smaller anyway - but also the perception trick as well, because you're giving them that first image frame as quickly as possible over potentially poor network connections. What else? I think those are some of the things you can do. I know you and I talked also about the phenomenon of light applications.
Rui Costa:
Yes.
Mark Finkle:
Once you get past some of the tricks and perception tools, you'll probably start talking about - maybe we should make a light app instead, we should just start over again and make a light app. And I used to try to block this conversation whenever it would come up because light applications use will probably require a lot of engineering resources. Look at companies that create light apps, right? They've got large engineering teams. Engineers to spare. You probably don't.
So I would typically try to push this conversation away, but nowadays I actually embrace it because I think that if someone starts talking about how we should make a light app these days, I'll say, "Okay, what's going to be in that light app?" What's different? How are we going to do things different than we do in our main application, this heavyweight beast, right? And there'll usually be a few items, bullet points that will come up - hey, we should try this and try that and try that - and I'm nodding along and saying, "You should file tickets on that and figure out how we can actually do that in the heavy beast". Because these are all great ideas. And if we could do it in a light app, in a brand new app with zero people using it and need to build it up in terms of a user base, why don't we actually try some of those tricks in the current application which already has a user base which would love to have a better user experience in some of these cases. And we do end up getting some good ideas that we try to actually prioritize and get into the application.
Rui Costa:
I believe that's a very, very clever way to put the engineering team thinking in this challenge in a challenging way. In the sense that this is not something that I'm doing just for the purpose of picking some items from the pipeline - and that's it. This is actually a challenging problem. From the motivation standpoint on when to think about performance and optimizing your app, that's probably one of the most clever ideas I've ever heard about, which is: think about a lighter version of the app and now out of that --- of ideas, pick some that you can actually use and the in-deep app - in the original application. In the end do you feel that the outcome was positive? So do you feel that the engineering team gets more motivated when they have this approach?
Mark Finkle:
Sometimes. I think it's a mixed bag, because when you look at the motivation of an engineer, like we're engineers, right? So we ended up sharing the same motivation, a light application represents greenfield development, right? So not only is it, how can we create a better user experience? It's also, how can we get rid of our legacy code? So light applications are also a means to get rid of legacy code. And if that's the case though, trying to prioritize tech dev or legacy code is maybe something that still should be done. It's just not usually called out in one of the bullet items. It's sort of the, the hidden thing that - Oh, if we do a light app, obviously we won't have any of our legacy code, too.
The other thing is the thought of a light application, a new greenfield, it creates an emotional response - like a euphoria to think about it. Trying to channel that into - okay, you've got these six great ideas for how we can, you know - tricks or changes that we can make to the light app. Let's do these into the beast, the legacy code - that the euphoria drains away a bit, right? Because, ugh, you mean, I need to work in the legacy code and this isn't as fun as thinking about doing something brand new. And I totally get that as well.
You know, that probably the last thing is prioritization, right? If you create six bullet items of good ideas, some of those are gonna have maybe a better return on investment than others in terms of, when we're talking about building a light application, it's kind of just talking and not actually doing, but whenever we're actually thinking about: okay, let's take these six items and turn them into tickets and get it prioritized with product to see what we're going to do. Now, it becomes reality. And it has to go through the same gauntlet of- well, we've got these three other things we want to do too. So, how are these six cool things that deal with light applications? How are those match up? And maybe you only get one or two things actually in the implementation cycle. So, that's sort of the way the reality of engineering goes, though.
Rui Costa:
It's absolutely challenging. I do relate to that a lot.
Mark, thank you so much. It was very, very enlightening conversation. I always like to end the episode with a teasing question for the guests. Imagine you're walking down the street, or let's say at Starbucks, and you meet someone starting a career in performance engineering, and you have one minute to tell them about the key takeaways from your experience around this topic - what would that be?
Mark Finkle:
I think framing discussions in terms of user experience is the key, because the pure technical aspect of making performance important often gets overshadowed with other work that is also important. And if you can frame your performance work in terms of user experience, then it gives you a better voice to deal with product and other parts of the organization that also have competing priorities as well. Performance will always be seen as something that maybe we'll take care of, if you don't create a good framework for being able to talk about it and its impact on end users.
Rui Costa:
Yeah, absolutely. That's why I love the concept of performance budgets, for example. That's absolutely the case. Mark, thank you so much and thank you all for listening. See you next week!
Hope you have enjoyed the conversation. It was such an enlightening view for Mark on the performance and real user monitoring. I'll leave Mark's LinkedIn address in the description of this episode. Don't forget to follow us on the usual podcast platforms like Spotify or Apple podcasts and visit our brand new performancecafe.codavel.com.
See you next week!