Although we did not collect the test data and so cannot be sure that all necessary variables were controlled across tests, we did use our standard methodology for analysis within the GameBench dashboard.
This involves, for example, using the Chart Selection feature to highlight gameplay sections only, ignoring menus and load screens. It also means we only recorded minimum frame rates that occured while gameplay animation was onscreen (including animation that took place behind an in-game prompt or pop-up).
GameBench Labs is the service side of GameBench Ltd, offering:
- Data collection, using crowd data from users of the free version of GameBench, plus hired testers in a range of markets who have been trained to use GameBench tools in combination with a strict game-testing methodology;
- Product ratings, based on GameBench’s industry-recognised system for objectively grading gaming experiences, useful for a top-level summary of results, competitor analysis, internal KPI-setting and external marketing;
- Performance analysis, including competitor analysis and highlighting likely bottlenecks and optimisation issues;
GameBench Labs analysed an unreleased 60fps build of Fortnite on the Samsung Galaxy Note9, showing that this frame rate option was not yet optimised or market-ready for that device. Due to excessive variability and severe frame rate drops, the 60fps build was no smoother than the 30fps build, and some gamers might actually perceive it to be slightly less smooth.
GameBench Labs’ report focused on the problem high-altitude views within the game (e.g. aerial views while flying) causing significant frame rate drops, likely to due to a draw distance and number of draw calls that were inappropriate for the Note9. An example is shown in the GameBench interface screenshot below, showing a severe FPS drop correlated with a flying “umbrella” section of gameplay: