Computer-centric metrics are easy, human-centric metrics are hard
|
|
I've been thinking a lot about human-centric performance metrics lately.
Almost all our traditional performance metrics center the technical implementation.
Rails developers, in particular, track server response time. It's the one performance metric that all my clients know before our engagement starts.
It's easy to track. It's also inadequate.
It isn't even a good proxy for the "time to first byte" or "server response time" as perceived by a client. Here are some components of time to first byte which aren't included in server response time:
1. Time to set up a SSL connection
2. Time to resolve DNS
3. Physical distance (round-trip time on fiber) from the nearest CDN point of presence, or if you don't use a CDN, your server.
4. Client connection latency (i.e. higher on cellular)
5. Backbone routing (CDNs usually do this faster than ISPs, see Cloudflare Argo)
Even then, time to first byte is still a computer-centric metric. When your customer's receives the first byte of the HTML document, is the page ready to go at the point? Is the user ready to read and interact with your site? Of course not! It's not even ready by the time the last byte gets there!
Google is now proposing a soft navigation standard. This would broaden Core Web Vitals to finally cover Single Page Applications. It will also lead to a common set of tooling for SPA performance. Companies like Datadog or Sentry will ship standard instrumentation for SPAs, regardless of the framework used. This is good news in some respects. It will make instrumenting SPAs easier than it is today.
Yet, it is yet another set of computer-centric metrics.
Navigations are not the only important user latency experience on the web. They were in 1993, when JavaScript didn't exist (those halcyon days), and the only thing you could click on was a blue anchor tag.
But, in 2023, SPAs can and do involve themselves in every single user input. Those user inputs often do not result in the URL changing. The soft navigation standard would not track these interactions.
Clicking anything and waiting on any change is an important performance experience. Why does the soft navigation standard need the URL to change, when nowadays browsers hide the URL 80% of the time?
It is easy to be computer-centric. Computers are deterministic and perform according to specification. It is thus quite simple to write a standard for "When The Page Is Loaded" based on these behaviors. Humans, of course, are squishy. You ask 100 people when a page is loaded, you'll get 100 different answers.
My favorite work in the human centric performance field is this research from Wikipedia. It used a survey to determine whether pages loaded quickly - and the data was quite illuminating.
Look for opportunities to introduce more of the human side in your performance work. We make web-pages for humans, unfortunately, and not HTML parsers.
Why the SQL Shotgun Works
I received an email from reader (and prolific /r/ruby commenter) Jonathan Rochkind. He was responding to two sentences in my previous newsletter:
> is that it is very rare for a Rails application to spend more than 50% of it's time waiting on I/O
That [...] matches my experience. But… how does that go with your earlier comment:
> Database access and querying is roughly 80% of the time of every controller action,
Were you leaving, say, view rendering out of “controller action time”? Or do you mean something different by “DB access” than I assume, does not somehow fit into “waiting on I/O”?
I did mean something different by "database access". Let's restate the observation and proceed:
80% of the wall time of the average Rails web application response is preparing data.
I tried to be a bit more careful with my wording this time around ;)
We spend the majority of our time in a Rails response dealing with data. Dealing with data does not mean waiting on I/O 100% of the time. Waiting on I/O is not the only task relating to data that a Rails application must complete.
Here are some of the CPU-bound tasks related to "preparing data" that I can think of:
- Turning ActiveRecord relations into SQL query strings
- ActiveRecord QueryCache access
- Filtering, sorting, ordering in Ruby
- Turning database query results into rich Ruby objects
- Calculating pagination - page numbers, number of pages, etc.
- Preloading - that is, turning 2 separate database queries into 1 eagerly-loaded Relation.
These are all the traditional responsibilties of a "controller" in an MVC architecture. Using these words in a Rails context is confusing. The words "model", "view" and "controller" in Rails are quite different. The code that takes data from one or many sources, sorts it, filters it, and otherwise turns it into one big Hash or Array of "data": that is the controller.
The view, in the Platonic ideal of MVC, takes this Big Ball of Data and turns it into HTML or JSON. In Rails, our "views" do some of this "controller-like" work, such as filtering, ordering. Our lazy ActiveRecord relations run queries when the view code is running, rather than eagerly in the controller.
The SQL shotgun method presumes that at the root of all these "preparing data" tasks, there is a SQL query. And, if you can get rid of that SQL query, you also get rid of all the upstream tasks that are CPU bound. You don't need to spin CPU cycles generating a SQL query string if you don't need to run a SQL query.
Until next week,
Nate
|
|
|
|