Performance IQ Logo

Network and Game Performance – Time to Close the Gap

GameBench Staff

PIQ 21.09

Network and Game PerformanceA new playing field

If 2021 has been the year of anything in gaming, the shift of focus to networks is surely a big headline. Quite suddenly, across the CSP (communication service provider) space, ISPs and mobile operators are playing catch up, or planning new offerings that serve the notoriously demanding needs of the gaming audience. Or, in many cases, both.
Cloud Gaming, of course, has rightly captured a lot of our attention. But its undoubtedly immense promise still lies some way in the future, and through cold commercial eyes, it would be easy to make the case for “watch, wait and see”. After all, current estimates for cloud gaming’s growth, even with a CAGR of close to 50% from 2021, suggest a market value of little over $5bn by 2025. Certainly a lot of money. But not to every player in the ecosystem.

Clearly, there’s more going on here, and we don’t have to look far for much larger and immediate trends. It’s well-known, for example, that mobile games already represent 50% of global gaming revenues, and there are currently some 1 billion online gamers worldwide discover more now about this directly at the website. And, startlingly, in 2021 over 50% of gamers report playing an MMO weekly. There are also some authentic and best online casinos on the website, click to play Casino Slots Forum now! Learn about the biggest historical slot wins.

Network ExpectationsUpping our game

So game performance over networks has landed, and landed hard. Naturally, GameBench is supporting many of the leading ISPs and mobile operators, as they get fully to grips with gaming – no longer as just a proxy for overall network performance, but as a critical market sector in its own right, and a driver of critical new offerings, revenue and future growth.
In a previous Performance IQ bulletin, we asked “What Will it Take to be the #1 Mobile Network?”. The overall conclusions – derived directly from our own extensive international work this year testing major networks against real game play – were stark and sobering. 5G, at least for now, is far from the magic bullet that has commonly been assumed. And while, as we would have anticipated, ISPs tend to perform well overall, most, if not all, mobile networks have a way to go yet.

Performance measurement and network operatorsAnother league

Performance measurement is nothing remotely new, of course, for any network operator. But, as our own clients have rapidly come to see, the traditional metrics and methodologies that serve more than adequately for communications, and even the less-demanding forms of streaming media, fail to measure what matters, uniquely and passionately, to the gamer.

top game complaints
Figure 1: The video and network performance dynamics that link the variables in the engineering layer with real- world gamer experience.

This is due, not merely to these ever-increasing expectations, but to the complexity of the overall gaming ecosystem, which ranges across developers and publishers, through platforms and networks, to device manufacturers and silicon vendors. Each one of these verticals contains its own particular performance variables. Furthermore, each one of these variables interacts with all the others, to create a perfect storm of performance – and therefore, experience – variability.
We begin to understand the nature and scale of the gaming challenge for networks, as they engage with the requirements of this hugely lucrative, profoundly demanding environment.

Getting fit for purpose

Traditional network testing methods measure only the performance of the network itself, excluding other critical performance drivers such as individual games and devices and their interplay. They also calculate the game performance of the network using proxies that are more than adequate for traditional telecoms functions – voice, text and some streamed media – but unfit for gaming’s far more demanding requirements.
GameBench captures and computes user experience metrics using video capture software on the device during actual gameplay. As illustrated in Figure 1 above, these metrics accurately reflect the true user experience, unlike the data extracted using a traditional average or aggregated engineering approach. In Figure 2 below, we lay out the differences between network performance and gaming performance metrics in full.

MetricsNetwork Methodologies
Median Download SpeedPredict game user experience using indirect metrics –  Median FTP downloads, UDP packet loss and jitter measured using a reference server (not the actual cloud gaming platform server). Measure RTT network latency during web and app testing to reference server.
Packet Loss
Jitter
Latency
Mean opinion score (MOS) modelPredicted game experience MOS score using crowdsourced FTP download and ping data collected.
MetricsGameBench Methodology
Animation FluidityCapture and store actual gameplay frame rate – calculate variability and sustained fps drops to grade fluidity experience.
Application LatencyMeasure actual application gameplay latency (ie: Time between OS processing input and reaction rendered on screen).
Image Quality & ConsistencyCalculate Fractal and Spatial complexity from captured gameplay images – calculate actual % degradation in image quality compared to a reference.
Launch timesCalculate actual time game took to reach gameplay directories.
Streaming Data RateMeasure the actual network usage by the local client application to/from the cloud gaming platform server.
Figure 2. The stark contrast between the demands of traditional network performance metrics, and those required for confident game performance management.

As we can clearly now see, not only are performance proxies an entirely inadequate approach for networks to measure and benchmark their current and future gaming experience, but the inherent complexity of the entire ecosystem, driven by the interplay between both new and emerging variables, demands an unprecedented degree of rigour.

Performance IQ by GameBench is a hassle-free, one-minute read that keeps you abreast of precisely what you need to know. Every two weeks, right to your inbox. Here’s a quick and easy sign up for your personal invite to the sharp end of great gamer experience.



And of course, get in touch anytime for an informal chat about your performance needs.

 

The Intelligence Behind Oustanding Gaming, Streaming and Network PerformanceThe intelligence behind outstanding performance