All The Metrics That Matter: Part One | Glossary of Key GPM Terms

Essential terminology

Game Performance Management is not just a science … It’s also something of an art. Pioneering GPM since 2013, GameBench has identified, defined and captured all the metrics that matter to ensure the best possible user experience. These are the fundamental concepts – along with some helpful extra links, and a summary of the best GPM practices – that every gaming professional needs to understand.

Battery drain
User-perceived battery power as measured by the device OS. Corroborated with milliamp readings (where available).

Battery ratings
The average mA consumption of the whole system, measured during gameplay and combined with battery capacity (mAh) to rate the expected number of gameplay hours on a charge.


CPU usage
The amount of work the CPU is doing as a percentage, normalised against the available CPU cores and their given frequencies.

Frame rate (FPS)
The number of frames shown in a given second.

Frame times
Tied to frame rate, the number of milliseconds to draw a single given frame. For example, a frame every 16.6667ms reflects 60 frames per second.

Frame rate variability
The average jump between consecutive frame rate readings taken each second, reflecting the amount of variation in visual fluidity that a gamer experienced (Lower is better).

Frame rendering
What the user sees while playing a game. Measured using Median FPS, FPS Stability, and Variability Index.

Gameplay
The core, interactive component of a game, excluding loading screens, menu screens, and advertisements.

GPU latency
The demand that a game’s software makes on the graphical processor.

Image consistency ratings
Algorithms are used to monitor key metrics of image consistency across a gameplay session. Differences between tests – e.g., in average edge or color complexity – are highlighted.

Input latency
The measured time for a response to a user input, rated according to fixed benchmarks set by research. Example: Time between a player clicking the mouse button, and seeing the muzzle flash of an in-game gun.


Jank
In GPM, an isolated, long pause between two frames, usually caused by dropped frames. (“Janky” is also used, less specifically, by gamers to denote poor game quality.)

Jitter
Negative impacts on video and audio quality, caused by delays in data packet arrival, due commonly to network congestion and/or route changes. (A key network metric, see also packet loss and latency.)

Launch Time
The time taken to launch an app after it has been removed from memory, but is not freshly installed (i.e., a “cold launch” but not a “first launch”). Measured from tapping the game icon on the home screen to seeing the first interactive screen.

Median frame rate
The middle-most frame rate during gameplay, representing the typical visual fluidity that the gamer experiences (Higher is better).

Minimum frame rate
The worst frame rate experienced during gameplay, typically happens during a moment of heightened gamer activity or a computational bottleneck (Higher is better).

Network latency
Also known as lag, the time for a packet of data to be captured, transmitted and processed, then received and decoded. (See also packet loss and jitter.)

Packet loss
A key network metric (see also jitter and latency) when packets of data fail to reach their destination. Measured as a percentage of packets lost against number of packets sent.

Performance rating
Graphical performance rated by measuring multiple frame rate metrics during gameplay (median frame rate, minimum variability) and compared to established gaming benchmarks and user data analysis.

Pixel shader load
The operating load for running pixel (fragment) shaders, which update colours and textures on scene geometry.

Vertex shader load
The operating load for running vertex shaders which shape scene geometry.

Best practices

Experience analytics
Data should reflect the experience of real gamers who are playing to win, i.e. no bots, scripts or device farms.

Natural gameplay
Each gamer should follow a strict methodology that does not interfere with natural gameplay, but does ensure that certain parameters are matched across tests (e.g., game and device configuration, game scenarios covered). Minimum sessions of 15 minutes are essential.

Testing methods
The testing elements of GPM should be accomplished using objective tools, with no reliance on traditional testers or subjective opinions.

Game-device pair
Every gaming experience relies on multiple hardware and software working in harmony, so valid testing is always done in game-device pairs.

Validated metrics
Metrics should be validated using multiple independent methods. However GPM relies to some extent on the accuracy of underlying metrics produced by e.g., Windows, Android or iOS operating systems, or cloud gaming platform telemetry.

Real-world devices
Real-world, unrooted and non-jailbroken devices should be used, reflecting actual gamer experience as closely as possible.

This is the first of three Performance IQ bulletins dedicated to the essentials of GPM. PIQ 21.06 will unpack and explain the key benchmarks that define and differentiate the “Great” (enthusiast), “Good” (standard), “Basic” (casual) and “Poor” (out of scope) levels of actual gamer experience. What’s more, we’ll be bringing you both drops combined into a single, simple pdf for reference.