Continuous monitoring and mushy requirements lead to performance doom. Firm up your requirements, and pass/fail them automatically.
|
|
When it comes to advising teams and organizations on how to approach web performance and scaling issues as a group, I frequently see two problems:
- There are no defined requirements. The app should be "fast" and "scalable", but no one agrees on what those words mean.
- Performance is continuously monitored, usually by the one person on the team who can understand the dashboards involved. No one else ever looks at the dashboards.
This situation sets a team up for a web perf failure. In this situation, there is really no way for performance work to ever get into the team's development pipeline. Since there are no standards for failure, any perceived performance change can always be punted into the future. In addition, there's probably no understanding of whether or not the app is slow at all, as that knowledge is locked away in the head of the one team member who made and monitors the performance dashboards.
A continuous monitoring approach without any defined thresholds, agreed upon by the entire organization, means that the frog is boiled until customers are complaining and the application is barely usable.
Instead, I recommend a different approach that I'm calling "red/green" performance. In a red/green setup, the team agrees on performance metrics for the entire application - usually just a handful of 4 or 5 will do, with special cases for the 10% of the application that needs them.
Once you have hard, metric-driven requirements, you can turn those into "alerts" or "red/green" or "pass/fail" notifications or monitors, which can be watched by a computer, rather than a human being. When these metrics are red or failing, the team treats this as a bug, and then follows the normal "bug" workflow (opening a JIRA ticket, GitHub issue, or whatever).
Where most teams fail is step 1 (coming up with requirements) rather than step 2 (monitoring those requirements and setting up alerts on failure).
In general, most websites should have the following performance requirements:
1. Largest Contentful Paint on "cold" load (no cache, first visit).
2. Navigation time (page to page or tab to tab) on "warm" load (post-first-page).
3. Cost of deployment per request (e.g. "we will spend $1,000 per month per 1,000 requests per minute we receive on average")
Using these numbers, we can monitor these metrics either synthetically (using a tool like webpagetest.org) or using real user monitoring (using a tool like New Relic Browser or Datadog RUM).
With high level numbers like these, we can "back in" to lower-level metrics as well, such as server response times.
For example, say you have a requirement that page navigation (hot cache, moving from one screen to another) should take 1 second or less, measured as the time between user input and a completed render/paint. As part of that interaction, you have to go to the server, get some HTML, and then render/paint the page. If the parsing, layout and painting of the page takes 500 milliseconds, and your average customer has about a 100 millisecond round-trip time between themselves and your servers, that leaves you 400 milliseconds of server response time.
To help the team agree on thresholds for performance metrics, you can even set up a synthetic test scenario using "sleep" calls on the backend or some creative Javascript to make pages load more slowly. What's the difference between a 2 second and a 3 second page load? Test it! Which one feels better? Do you even notice the difference?
Development teams are good at dealing with pass/fail scenarios. We do it every day when evaluating the correctness of our programs via automated tests. Moving performance into a similar workflow requires extra work up front to define requirements, but provides smooth sailing afterward.
Until next week,
-Nate
|
|
|
|