The four pillars of game performance quality
This is the second of our two Performance IQ bulletins dedicated to the essentials of GPM. PIQ 21.05 provided a quick reference glossary of all the fundamental concepts that every gaming professional needs to understand.
Here we go into greater depth, unpacking and explaining the key GameBench benchmarks that define and differentiate the “Great” (enthusiast), “Good” (standard), “Basic” (casual), and “Poor” (out of scope) levels of actual gamer experience. These are best understood using the four pillars of experience quality: Fluidity, Responsiveness, Image Consistency, and Duration.
Where did these recommended benchmarks come from? They’re derived from eight years of dedicated GameBench performance analysis of experience quality, based on real gameplay – over 70,000 hours of testing, across over 20,000 game sessions, with over 7,000 device models tested.
When the FPS dips below the developer’s target or becomes unstable, users immediately begin to notice stuttering images and lag in the game. The three key performance metrics here are Median FPS, Minimum FPS, and FPS Variability.
The middle-most frame rate during gameplay, representing the typical visual fluidity that the gamer experiences. (Higher is better.)
The worst frame rate experienced during gameplay, typically happens during a moment of heightened gamer activity or a computational bottleneck. (Higher is better.)
The average jump between consecutive frame rate readings taken each second, reflecting the amount of variation in visual fluidity that the player experienced, (Lower is better.)
When a game is described as “laggy” or “slow”, players’ actions are not immediately acted upon, and the immersive experience is naturally compromised. This is a responsiveness issue, and the primary performance metric here is User Perceived Latency.
User Perceived Latency
The measured time for a response to a user input, rated according to fixed benchmarks set by research. Example: time between a player clicking the mouse button, and seeing the muzzle flash of an in-game gun.
Disruptions in consistency of image presentation (for example, macroblocking, a common distortion that appears as large pixel blocks) directly undermine the enjoyment and quality of gameplay.
Figure 1: Here we see results of around 0.33-0.39 for both the Note 10 and iPhone 11 Pro Max, reflecting artefacts typical of upscaling from a 720p native image to a 1080p display image without effective anti-aliasing.
Our Jaggy Index
Algorithms are used to monitor key metrics of image consistency across a gameplay session. Two complexity indices, fractal and spatial, help to spot differences like texture detail and lighting effects in a scene.
Our own GameBench Jaggy Index (see Figure 1) measures the degree to which low-res gameplay creates aliasing artefacts in geometrical elements containing straight lines. (Lower is better.)
This is the obvious cornerstone of mobile gamer experience, and while battery life continues to improve year-on-year, the need for carefully managed trade-offs between power and performance remains. Here the key performance metrics are Available Gameplay Hours and Loading Time.
Available Gameplay Hours
The average mA consumption of the whole system, measured during gameplay and combined with battery capacity (mAh) to rate the expected number of gameplay hours on a charge.
The time taken to launch an app after it has been removed from memory, but is not freshly installed (i.e., a “cold launch” but not a “first launch”). Measured from tapping the game icon on the home screen to seeing the first interactive screen.
These two latest Performance IQ bulletins naturally go together, with a third to follow that combines both our GPM glossary and these key benchmarks. Keep an eye on your inbox!
Performance IQ by GameBench is a hassle-free, one-minute read that keeps you abreast of precisely what you need to know. Every two weeks, right to your inbox. Here’s a quick and easy sign up for your personal invite to the sharp end of great gamer experience.
The intelligence behind outstanding performance