Commit Graph

148 Commits

Author SHA1 Message Date
Michael Goderbauer
5491c8c146 Auto-format Framework (#160545)
This auto-formats all *.dart files in the repository outside of the
`engine` subdirectory and enforces that these files stay formatted with
a presubmit check.

**Reviewers:** Please carefully review all the commits except for the
one titled "formatted". The "formatted" commit was auto-generated by
running `dev/tools/format.sh -a -f`. The other commits were hand-crafted
to prepare the repo for the formatting change. I recommend reviewing the
commits one-by-one via the "Commits" tab and avoiding Github's "Files
changed" tab as it will likely slow down your browser because of the
size of this PR.

---------

Co-authored-by: Kate Lovett <katelovett@google.com>
Co-authored-by: LongCatIsLooong <31859944+LongCatIsLooong@users.noreply.github.com>
2024-12-19 20:06:21 +00:00
Kate Lovett
7b0b03bc0d Adjust timeout for devicelab tests (#156553)
Fixes https://github.com/flutter/flutter/issues/156456

There is a known long running test that has been bumping up against the default timeout. This adjusts the perTestTimeout for the devicelab tests to add 15 seconds to stop this from flaking.

The test is question: 82ebb74c64/dev/devicelab/test/run_test.dart (L76)

`test/run_test.dart: run.dart script prints a message after a few seconds when failing to connect (this test takes >10s)`
2024-10-15 20:47:03 +00:00
Ben Konyi
2c84e63ba7 [ Cocoon ] Wait for task results to be received by the task runner before shutting down the task process (#156002)
Prior to this fix, `_TaskRunner.run` would immediately cleanup the
keep-alive port once the task completed, which would result in the
isolate shutting down as soon as the task result was returned from
`ext.cocoonRunTask` callback in the form of a
`ServiceExtensionResponse`. Since the service extension response is
processed by the service isolate, it was possible for the VM to start
shutting down before the service isolate could send the task result data
back to the task runner.

This change introduces a new service extension,
`ext.cocoonTaskResultReceived`, that the task runner invokes after it
receives the task result from `ext.cocoonRunTask`, notifying the task
process that it can close the keep-alive port and shutdown.

Fixes https://github.com/flutter/flutter/issues/155475
2024-10-02 17:01:24 -04:00
Matan Lurey
f070ffc53b Get reasonable output when a stream test fails. (#154377)
Debugging https://github.com/flutter/flutter/issues/154268 was hard; when I removed `--disable-dart-dev` I would get the following error from this test:

<img width="627" alt="Screenshot 2024-08-29 at 4 25 43 PM" src="https://github.com/user-attachments/assets/2cc3266e-cc6e-425f-b909-12d7556ff110">

After my change, this is what the error looks like:

<img width="1055" alt="Screenshot 2024-08-29 at 4 26 50 PM" src="https://github.com/user-attachments/assets/8b0ecccc-a7c9-4da7-bf22-15cef24c4cc7">

In general we should embrace more of the test package and matchers when appropriate.
2024-09-03 21:36:24 +00:00
Jason Simmons
203a19e82f Ensure that the output path used by perf_tests_test matches the path used by perf_tests if the FLUTTER_TEST_OUTPUTS_DIR variable is set (#153888)
A recent change to the flutter_drone recipe is setting FLUTTER_TEST_OUTPUTS_DIR in the environment (see https://flutter.googlesource.com/recipes/+/b111cd3ef2297e66905cc48c2cfedce0cf1ba46f)

If FLUTTER_TEST_OUTPUTS_DIR is set, then perf_tests will use an output path based on that variable.  perf_tests_test needs to use the same logic when looking for the expected output file.
2024-08-22 03:30:24 +00:00
Reid Baker
1eb7cd2c73 allow adb to set canfail then use canFail=true for clearing logs (#150517)
Fixes https://github.com/flutter/flutter/issues/150093

New tests added to cover that we at least pass the arguments we expect to adb. 

The test for #150093  is not ideal in that it does not verify the behavior of a failed process but instead ensures we set the parameter that contains the behavior we want. 

devicelab code and tests are not setup to enable fake process or fake output from stdin/stderr and hang if adb or no hardware are present.
2024-06-24 19:13:24 +00:00
Ian Hickson
5e216d44ca Simplify devicelab logic and fix tests (#139122)
- fix https://github.com/flutter/flutter/issues/53707 by having the test not expect a timeout but instead actually look for the retry message
- simplify the `--task` option to only accept task names rather than also accepting paths
- remove some obsolete options that referred to the manifest which no longer seems to exist
2023-11-29 08:41:27 +00:00
Callum Moffat
2925525e35 Upload frame_request_pending_latency [attempt #2] (#135890)
These values are generated since https://github.com/flutter/flutter/pull/135279, but I didn't know to add the new keys to this list to get them to upload. 

Failed to do so in #135645, I believe the mistake there was putting them in `_kCommonScoreKeys`, which is also used in "E2E" tests, that don't get full trace data, only high level `FrameTiming` packets.

Part of https://github.com/flutter/flutter/issues/129150
2023-10-05 21:54:27 +00:00
Zachary Anderson
63b98d55b0 Revert "Upload generated frame-request-pending stats" (#135672)
Reverts flutter/flutter#135645
2023-09-28 07:23:25 -07:00
Callum Moffat
dd341d4295 Upload generated frame-request-pending stats (#135645)
These values are generated since https://github.com/flutter/flutter/pull/135279, but I didn't know to add the new keys to this list to get them to upload.

Part of https://github.com/flutter/flutter/issues/129150
2023-09-28 13:05:10 +00:00
Matan Lurey
a7997f606e Update dev/devicelab/** to provide --local-engine-host. (#132342)
Partial work towards https://github.com/flutter/flutter/issues/132245.

I have to admit I don't totally understand what I've updated, or whether
there are more integration points needed.
2023-08-10 17:25:30 -07:00
Zachary Anderson
aef7929f3d Report total_ui_gc_time to SkiaPerf (#125919)
Not sure why this wasn't already being reported.
2023-05-02 22:09:33 -07:00
Ben Konyi
ecd7518df5 Reland "Remove references to Observatory (#118577)" (#121606)
This reverts commit 275ab9c69b.
2023-02-28 11:57:04 -05:00
Michael Goderbauer
275ab9c69b Revert "Reland "Remove references to Observatory (#118577)" (#121215)" (#121555)
Revert "Reland "Remove references to Observatory (#118577)""
2023-02-27 23:46:53 +00:00
Ben Konyi
fbae472fc3 Reland "Remove references to Observatory (#118577)" (#121215)
This reverts commit 298d8c76ba.
2023-02-27 09:26:43 -05:00
Casey Hillers
298d8c76ba Revert "Remove references to Observatory (#118577)" (#120929)
This reverts commit 2df140f40d.
2023-02-16 21:28:30 -08:00
Ben Konyi
2df140f40d Remove references to Observatory (#118577)
Observatory is being deprecated for Dart 3.0 so it should no longer be
referenced in tooling messaging / flags.

See https://github.com/dart-lang/sdk/issues/50233
2023-02-13 14:29:30 -05:00
Michael Thomsen
253e3c0856 Migrate device lab runner to nulls safety (#110582) 2022-08-30 19:46:05 +02:00
keyonghan
8027842393 Copy artifacts to applicationBinaryPath when specified for build+test separation (#109879) 2022-08-23 16:32:39 +00:00
Michael Goderbauer
10a7c9ba22 Unify analysis options (#108462) 2022-07-28 09:07:49 -07:00
Alexandre Ardhuin
93a74ca9f2 remove unused members in executable libs (#107464) 2022-07-12 21:41:08 +00:00
Alexandre Ardhuin
07f1c20474 add missing trailing commas in list/set/map literals (#102585) 2022-04-27 09:15:35 +02:00
keyonghan
647044c969 Use separate metric file names for same tests running on different platforms (#101317)
* use separate file names for same test

* use string buffer
2022-04-05 09:16:04 -07:00
Michael Goderbauer
195a1cc413 Prepare dev (minus flutter_gallery) for use_super_parameters (#100516) 2022-03-29 12:52:42 -07:00
Chris Yang
e6bd208196 Reland "Add the refresh rate fields to perf_test #99710" (#99854) 2022-03-21 15:55:13 -07:00
David Iglesias
93c0c043cf [devicelab] Web benchmarks now run on Chromium 89+ (#98629) 2022-02-16 19:04:14 -08:00
keyonghan
e8bc5c5ace Support Samsung phone when getting device wakefulness status (#98146) 2022-02-11 12:05:18 -08:00
Michael Goderbauer
5684b9f69e Enable no_leading_underscores_for_local_identifiers (#96422) 2022-01-21 14:43:59 -08:00
Ian Hickson
57f097b532 Refactor devicelab bin/run.dart and other cleanup (#96320) 2022-01-10 13:40:17 -08:00
keyonghan
33be68d9bf Stop sending metrics to builder name (#92989) 2021-11-05 16:28:02 -07:00
keyonghan
30d3866ab9 Add extra benchmark metrics to test name in addition to builder name (#92530) 2021-10-27 15:33:06 -07:00
keyonghan
7de92e2653 Add devicelab benchmark tags support (#92141) 2021-10-20 15:03:06 -07:00
Ian Hickson
61a0add286 Enable avoid_redundant_argument_values lint (#91409) (#91462) 2021-10-08 09:25:14 -07:00
Zachary Anderson
b9d2177da0 Revert "Enable avoid_redundant_argument_values lint (#91409)" (#91461)
This reverts commit 5fd259be24.
2021-10-07 21:11:07 -07:00
Ian Hickson
5fd259be24 Enable avoid_redundant_argument_values lint (#91409) 2021-10-07 20:13:02 -07:00
Anis Alibegić
a753d09cc9 Fixed several typos (#89485) 2021-09-07 14:56:04 -07:00
keyonghan
e311e3b3ef skip staging update (#89137) 2021-08-30 09:26:05 -07:00
keyonghan
db4c104c8e Use task name when uploading metrics to skia perf (#89004) 2021-08-26 22:31:02 -07:00
keyonghan
b67e264203 Revert "Skip staging test update to cocoon in test runner (#88835)" (#88971)
This reverts commit 884dfc260d.
2021-08-26 09:41:22 -07:00
Casey Hillers
b0950fd567 Revert "Use bucket to check staging test instead of builder name (#88908)" (#88969)
This reverts commit d36f246dcc.
2021-08-26 09:19:50 -07:00
keyonghan
d36f246dcc Use bucket to check staging test instead of builder name (#88908) 2021-08-25 20:36:02 -07:00
keyonghan
884dfc260d Skip staging test update to cocoon in test runner (#88835) 2021-08-25 11:36:03 -07:00
keyonghan
b468be3030 Use default value for ResultData when uploading metrics from test runner (#88749) 2021-08-24 12:04:03 -07:00
keyonghan
208d312ff0 Upload devicelab test metrics from test runner (#88447) 2021-08-20 12:57:03 -07:00
yusufm
fe52e56614 Adding a timeout and retry to upload results step. (#87306)
* Adding a timeout and retry to upload results step.

* Adding tests for retries and timeout.

* Adding async to call

* Fixing sleep to trigger the timeout, and adding addition timeout test.

* Fixing minor typo.

* Fixing some minor flutter style issues.

Co-authored-by: yusufm <mohsinally@google.com>
2021-08-03 13:40:28 -07:00
Casey Hillers
79f377dabe [devicelab] Only upload results on master (#87125) 2021-07-27 12:49:05 -07:00
Darren Austin
8fcace1d6c Migrate devicelab tests and test runners to null safety. (#85999)
* Migrate devicelab tests and test runners to null safety.
2021-07-23 13:37:57 -07:00
keyonghan
3aeb794298 Extend test runner command to update test flaky status (#86513) 2021-07-19 09:46:03 -07:00
keyonghan
4d96a3fd50 Rerun devicelab task from test runner (#86394) 2021-07-14 15:23:10 -07:00
Dan Field
e36e62e9a6 Revert "Revert "Audit hashCode overrides outside of packages/flutter (#85370)" (#85451)" (#85567)
This reverts commit 2be0d57fa2.
2021-06-30 09:46:54 -07:00