The latest Speedometer, a benchmark measuring web responsiveness, is out in version 3.0. However, some browser makers have questioned how helpful such benchmarking is in the era of shared rendering engines.

Speedometer is a tool to help browser makers spot regressions and improve performance. Its intention is to simulate what users actually do on websites, including editing rich text, reading a news site, working with a to-do list, etc.

While the benchmark's makers noted that all the various components and processes used to make up the web can't be simulated, a 2023 snapshot of the HTTP Archive helped to inform discussion regarding what JavaScript UI frameworks are most common in complex web apps. Other workloads have been added to get as close as possible to what users do with their browsers.

Speedometer 1.0 turned up in 2014, released by the WebKit team. 2.0, the result of a collaboration between Apple and the Chrome team, arrived in 2018. Speedometer 3.0 includes added contributions from Microsoft and Mozilla and, according to the Speedometer team, is "a collaborative effort between the three major browser engines: Blink, Gecko, and WebKit."

Things have changed in the decade since the initial release of the Speedometer benchmark. While Chrome was still topping the charts back then, according to StatCounter, it could not be called dominant, and made up just 38.5 percent of the market in June 2014. Internet Explorer came second at 15.4 percent, with Firefox and Safari fighting it out at 13.3 percent and 13.8 percent respectively.

Today, the browser landscape is quite different: Chrome is far ahead of the competition with a 65 percent share and Safari in second place at 18.7 percent. With so many browsers now using the Chromium engine for rendering, performance testing of the browser itself is not the measure it used to be.

Tarquin Wilton-Jones, a security expert at browser-maker Vivaldi, noted that with a few notable exceptions, a majority of browsers use the same core engine. Compiler options might cause minor differences, "but in practical terms, the differences are negligible when browsers all share the same core."

"For an engine maker, these [benchmarks] are likely to be useful to detect performance regressions, and that's a good thing," he added.

However, "Using a different computer with a different amount of memory, a different processor or a different operating system makes the biggest difference of all, and these tests do not cover any of that.

"More importantly, they do not test the browser itself; the parts that really make a difference to the user experience. All the little bits of UI that are completely different. All the little shortcuts, and additional functionality, any feature that helps the user be more productive with their time. These are what make the biggest difference for a user, not whether canvas takes 0.0001 seconds longer to calculate 100 Bézier curves." ®

https://www.theregister.com//2024/03/12/speedometer_30_adds_new_workloads/