Commit Graph

2022 Commits

Author SHA1 Message Date
gaaclarke
947367aaa2 Added standard deviation to rasterizer results. (#131781)
This calculation is important if you want to calculate the probability that one thing is better than another.
2023-08-02 16:40:07 +00:00
Polina Cherkasova
35213ceabd Upgrade Flutter libraries. (#131700) 2023-08-01 12:59:47 -07:00
Jackson Gardner
b928b3c1c3 Reland --omit-type-checks for benchmarks. (#131493)
Because the cost of type checks dominate our dart2wasm benchmarks, we've
decided to pass `--omit-type-checks` for now.

This was previously reverted because the skwasm benchmarks were broken
in general for a separate reason, and my getting rid of `bringup: true`
broke the tree. I ended up fixing the benchmarks and getting rid of
`bringup: true` in a separate commit, so this just adds the flag only.
2023-07-31 13:17:34 -07:00
Daco Harkes
fceaa005cd [doc] Fix module_test_ios comments (#131470) 2023-07-31 11:05:11 +00:00
Jason Simmons
8a84437989 Manual roll to engine commit 9b14c382 using Dart SDK version 3.2.x (#131371)
Dart SDK 3.2.x requires a new version of the dart_internal package.
2023-07-27 17:33:07 +00:00
keyonghan
cfafb9105f Add DeviceLab build+test separation doc (#131365)
This PR add instructions to separate DeviceLab tasks to build and test model.
2023-07-27 15:16:09 +00:00
Loïc Sharma
81b0e9f111 Revert "Run benchmarks with --omit-type-checks" (#131353)
Reverts flutter/flutter#131102 as it closed the tree due to `Linux web_benchmarks_skwasm` failures. The test seems to be timing out without completing.

Example failure: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_benchmarks_skwasm/1/overview
2023-07-26 19:23:37 +00:00
Jackson Gardner
8046e13373 Run benchmarks with --omit-type-checks (#131102)
We've decided to use the `--omit-type-checks` flag for our dart2wasm benchmarks. Right now, many of the benchmark results are dominated by type checks and most of what we are actually trying to measure get drowned out in the noise.
2023-07-26 17:10:07 +00:00
Polina Cherkasova
30c005f7f4 Upgrade to newer leak_tracker. (#131085) 2023-07-21 14:26:42 -07:00
flutter-pub-roller-bot
edcaa335d1 Roll pub packages (#130821)
This PR was generated by `flutter update-packages --force-upgrade`.
2023-07-18 21:55:17 +00:00
Victoria Ashworth
a903f1defb Add verbose engine logs for iOS start up tests (#130511)
Added to be able to view logs from https://github.com/flutter/engine/pull/43616.

Added to debug https://github.com/flutter/flutter/issues/129836
2023-07-14 16:51:50 +00:00
Polina Cherkasova
5870159ed7 Upgrade leak_tracker. (#130507) 2023-07-13 18:40:50 +00:00
Alexander Aprelev
3b8f6c4020 Upgrade framework pub dependencies, roll engine with rolled dart sdk (#130163)
Manual roll is needed because incoming dart sdk requires updated version
vm_snapshot_analysis (>=0.7.4).


5ae09b8b4f...7c83ea3e85

```
7c83ea3e85 Reland "Manual roll Dart SDK from 2d98d9e27dae to 0b07debd5862 (21 revisions) (#43457)" (#43472)
9ef3e8d533 Roll Skia from 5eba922297bb to 93c92f97f5ab (2 revisions) (#43471)
```

Remove implementation of SuitePlatform from the test as well. Remove use
of fake cwd from SuitePlatform as it can't be properly faked.
2023-07-07 13:55:35 -07:00
Victoria Ashworth
6683468f0b Add debugging for iOS startup test flakes (#130099)
Adding debugging for https://github.com/flutter/flutter/issues/129836.

Takes a screenshot when startup test takes too long (10 minutes).

Also, removes some old debugging and add new debugging message.
2023-07-07 16:49:00 +00:00
Jackson Gardner
1b887c72b5 Skwasm benchmarks. (#129681)
This enables benchmarks for the Skwasm renderer, compiled with
dart2wasm.

Platform views aren't supported in Skwasm yet, so we are skipping those
benchmarks for now.
2023-06-28 17:16:19 -07:00
Flutter GitHub Bot
ba636881eb Roll pub packages (#129380)
This PR was generated by `flutter update-packages --force-upgrade`.
2023-06-28 14:07:50 -07:00
Brandon DeRosier
b898d1d270 [devicelab] Add platform view scroll benchmarks for Impeller on Android (#129455)
For tracking perf improvements with changes like
https://github.com/flutter/engine/pull/43152.
2023-06-23 18:40:17 -07:00
Mouad Debbar
a162d7546a flutter update-packages --force-upgrade (#128908)
- Bumps `vm_service` from `11.6.0` to `11.7.1`
- Bumps `web` from `0.1.3-beta` to `0.1.4-beta` and adds it everywhere.
- Moves `js` from `dependencies` to `dev_dependencies`
2023-06-15 18:17:09 +00:00
Polina Cherkasova
05d1cde0dd Address leak tracker breaking changes. (#128623) 2023-06-12 11:11:35 -04:00
Jackson Gardner
7c15a26eab Reland "Migrate benchmarks to package:web" (#128266)
This attempts to reland https://github.com/flutter/flutter/pull/126848

This was reverted because it made some unexpected changes to our perf measurements. After landing https://github.com/flutter/flutter/pull/127900, we have much less noise in our benchmarks, so I'd like to reland this and see if there is still a significant measurement difference.
2023-06-08 22:25:40 +00:00
Victoria Ashworth
cd18c8c02f Workaround for Dart VM timeout (#127875)
Workaround solution for: https://github.com/flutter/flutter/issues/121231
See https://github.com/flutter/flutter/issues/120808#issuecomment-1551826299 Error Case 2 for more information.

Sometimes the `ios-deploy` process does not return the logs from the application. We've been unable to figure out why. This is a solution to workaround that by using `idevicesyslog` alongside `ios-deploy` as a backup in getting the log for the Dart VM url. As explained in https://github.com/flutter/flutter/issues/120808#issuecomment-1551826299, when error case 2 happens, the `idevicesyslog` does successfully find the Dart VM.

Also, in the comments of the code it mentions `syslog` is not written on iOS 13+, this was added in response to this issue: https://github.com/flutter/flutter/issues/41133.

However, `idevicesyslog` does in fact work (at least for iOS 16), we use it to collect device logs for our CI tests already: 1dc26f80f0/dev/devicelab/lib/framework/devices.dart (L998-L1006)
2023-06-02 17:17:57 +00:00
Camille Simon
6268c04bd7 [Android] Modify Linux_android flutter_engine_group_performance to uninstall app before attempting to install (#127429)
Changes `Linux_android flutter_engine_group_performance` to uninstall the app that it uses for testing before attempting to install it again to ensure proper cleanup.

Attempt at fixing https://github.com/flutter/flutter/issues/127011.
2023-05-31 16:40:03 +00:00
Jackson Gardner
e8f4d803ba Improve web benchmarks measurements (#127900)
By default, the browser fuzzes the timer APIs such that they have a granularity of approximately 100 microseconds (this is due to Spectre mitigation techniques). However, many of the thing we are trying to measure actually have a much finer granularity than 100 microseconds. As a result, many of our benchmarks are extremely noisy and don't provide accurate data.

By serving the initial script files with the `Cross-Origin-Opener-Policy: same-origin` and `Cross-Origin-Embedder-Policy: require-corp` HTTP headers, the browser runs the benchmarks in a `crossOriginIsolated` context, which restores the fine granularity of APIs such as `performance.now()` to microsecond precision.

Also, we were considering anything an outlier that was more than one standard deviation away from the mean. In a normal distribution, that means we are only capturing 68% of the data and the rest are considered outliers. This is not ideal. Doing two standard deviations away captures 95% of the data, and the outliers are in the remaining 5%, which seems much more reasonable.
2023-05-31 02:29:02 +00:00
Christopher Fujino
0763d61f56 [flutter_tools] manually roll pub deps (#127447)
Fixes https://github.com/flutter/flutter/issues/127226
2023-05-30 23:34:52 +00:00
Phil Quitslund
5bf6318688 Update collection-fors to prefer final (as per updated prefer_final_in_for_each) (#127511)
The newly updated lint will soon flag for-each in collections.

See discussion: https://github.com/dart-lang/linter/pull/4383

/cc @goderbauer
2023-05-26 23:34:36 +00:00
chunhtai
c687dcd56f Migrates android semanitcs integration to integration test (#127128)
I think the flake is due to setclipboard or semantics update race condition. I migrated the test to use integration test package which relies less on timing

fixes https://github.com/flutter/flutter/issues/124636
2023-05-23 22:33:19 +00:00
joshualitt
3001d6ab17 Revert "Migrate benchmarks to package:web" (#127207)
Reverts flutter/flutter#126848

Triggered some kind of measuring discrepancy / performance regression.
2023-05-19 10:20:44 -07:00
joshualitt
4fea3ef588 Migrate benchmarks to package:web (#126848) 2023-05-18 14:02:12 -07:00
Michael Goderbauer
5e1ba701ed enable no_literal_bool_comparisons lint (#126647) 2023-05-16 16:14:23 +00:00
Polina Cherkasova
ea5eddb5a9 Upgrade leak_tracker to 5.0.0 (#126367)
Fixes https://github.com/flutter/flutter/issues/126259

Updated pubspec:
345f0bffbf/packages/flutter/pubspec.yaml
2023-05-16 04:39:26 +00:00
Jonah Williams
41abe998ee [devicelab] add drawPoints benchmark (#126728)
Add a benchmark that measures the improvements from https://github.com/flutter/engine/pull/41803
2023-05-15 19:14:58 +00:00
Zachary Anderson
674b1ad0ae Remove obsolete iOS benchmark (#126862)
Fixes https://github.com/flutter/flutter/issues/126861
2023-05-15 19:12:56 +00:00
stuartmorgan
d3c3dde1b6 Remove duplicate plugin tests (#126582)
Removes two plugin tests that aren't actually doing anything useful.

When they were added they were testing v2-embedding plugins in a v1-embedding app, but support for creating v1-embedding apps--and the environment variable that these tests were using to, in theory, trigger them--was removed several years ago in #61203. That means that these tests are in practice exact duplicates of the copies just above without the ennvironment variables.

Fixes https://github.com/flutter/flutter/issues/122200
2023-05-15 17:55:55 +00:00
Tomasz Gucio
99c7e9f088 Add spaces after flow control statements (#126320) 2023-05-15 11:07:30 +02:00
keyonghan
8ab782d08a Migrate gallery ios tests to build+test (#111164)
Part of https://github.com/flutter/flutter/issues/103542
2023-05-10 00:50:11 +00:00
Pierre-Louis
65dfb555c0 Update packages (#126140)
In particular, update pin for `material_color_utilities` to `0.5.0`.
2023-05-08 09:51:28 +02:00
Jenn Magder
4f3c989bd3 Remove complex_layout_*__compile benchmarks (#126023)
Fixes https://github.com/flutter/flutter/issues/126020.
2023-05-03 23:49:01 +00:00
Zachary Anderson
aef7929f3d Report total_ui_gc_time to SkiaPerf (#125919)
Not sure why this wasn't already being reported.
2023-05-02 22:09:33 -07:00
Chris Yang
9fcb42d0dd Update xcode to 14e222b (#125716)
Framework part of https://github.com/flutter/flutter/issues/125814
2023-05-02 19:56:17 +00:00
Flutter GitHub Bot
73a343f191 Roll pub packages (#125698)
This PR was generated by `flutter update-packages --force-upgrade`.
2023-04-28 19:30:06 +00:00
Zachary Anderson
50f83fc277 Add back one Skia test on iOS (#125663)
Fixes https://github.com/flutter/flutter/issues/125648
2023-04-28 10:41:33 -07:00
Flutter GitHub Bot
888b208d62 Roll pub packages (#125447)
This PR was generated by `flutter update-packages --force-upgrade`.
2023-04-28 17:31:45 +00:00
Taha Tesser
457e98a1e7 Add Sliders to macrobenchmarks (#125296)
This adds a performance test for `Slider` and `RangeSlider` to run on Android.
This is to measure upcoming changes to these widgets and compare their performance. 

https://user-images.githubusercontent.com/48603081/233633349-0bcad3c3-04a9-42dd-acd0-46b76ce51178.mp4
2023-04-24 23:25:28 +00:00
Jenn Magder
baafa1991d Combine flutter_tool_startup devicelab tests (#125226)
`flutter_tool_startup__windows/linux/macos` were identical. Use the same task for all three.
2023-04-21 22:33:07 +00:00
Flutter GitHub Bot
d85e2fb810 Roll pub packages (#125225)
This PR was generated by `flutter update-packages --force-upgrade`.
2023-04-20 18:28:10 +00:00
Flutter GitHub Bot
d300b86e80 Roll pub packages (#124709)
Roll pub packages
2023-04-13 20:54:52 +00:00
Jenn Magder
c60f202308 Force plugin transitive dependencies to an ARC compatible version (#124349)
Force plugin transitive dependencies to an ARC compatible version
2023-04-07 18:07:23 +00:00
Flutter GitHub Bot
cc4b455521 Roll pub packages (#124364)
Roll pub packages
2023-04-07 17:19:24 +00:00
Victoria Ashworth
c52042fbfc Make logs verbose for gallery tests (#124271)
Make logs verbose for gallery tests
2023-04-07 15:47:20 +00:00
Flutter GitHub Bot
0046a25e39 Roll pub packages (#123899)
Roll pub packages
2023-04-03 16:56:56 +00:00