Address frame policy benchmark flakes (#155130)

Recently the microbenchmarks were flakey, but from an older bug. Turns out, `LiveTestWidgetsFlutterBindingFramePolicy` is defaulted to `fadePointers` with this fun note:

> This can result in additional frames being pumped beyond those that
the test itself requests, which can cause differences in behavior

Both `text_intrinsic_bench` and `build_bench` use a similar pattern:
* Load stocks app
* Open the menu
* Switch to `benchmark` frame policy

What happens, rarely, is that
`LiveTestWidgetsFlutterBinding.pumpBenchmark()` will call (async) `handleBeginFrame` and `handleDrawFrame`. `handleDrawFrame` juggles a tri-state boolean (null, false, true). This boolean is only reset to `null` when handleDrawFrame is called back to back, say, from an extra frame that was scheduled.

1. Switch tri-state boolean to an enum, its easier to read
2. remove asserts that compile away in benchmarks (`--profile`)
3. use `Error.throwWithStackTrace` to keep stack traces.

I've been running this test on device lab hardware for hundreds of runs and have not hit a failure yet.

Fixes #150542
Fixes #150543 - throw stack!
This commit is contained in:
John McDole
2024-09-12 16:19:15 -07:00
committed by GitHub
parent 74caead4cd
commit b755641559
4 changed files with 25 additions and 11 deletions

View File

@@ -33,7 +33,7 @@ Future<void> execute() async {
await benchmarkWidgets((WidgetTester tester) async {
runApp(intrinsicTextHeight);
// Wait for the UI to stabilize.
await tester.pump(const Duration(seconds: 1));
await tester.pumpAndSettle(const Duration(seconds: 1));
final TestViewConfiguration big = TestViewConfiguration.fromView(
size: const Size(360.0, 640.0),

View File

@@ -29,7 +29,7 @@ Future<List<double>> runBuildBenchmark() async {
await tester.pump(const Duration(seconds: 1)); // Complete startup animation
await tester.tapAt(const Offset(20.0, 40.0)); // Open drawer
await tester.pump(); // Start drawer animation
await tester.pump(const Duration(seconds: 1)); // Complete drawer animation
await tester.pumpAndSettle(const Duration(seconds: 1)); // Complete drawer animation
final Element appState = tester.element(find.byType(stocks.StocksApp));
binding.framePolicy = LiveTestWidgetsFlutterBindingFramePolicy.benchmark;