Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ui: new benchmark start screen #826

Merged
merged 9 commits into from
Dec 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/result-spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ This file explains the full format for results generated by this app.
This format is used when saving results locally on device, when uploading them to the online database,
and when fetching them from the online database.

Here is an example result file: [extended_result_unittest.json](../flutter_common/test/data/extended_result_unittest.json)
Here is an example result file: [extended_result_unittest.json](../flutter/unit_test/data/extended_result_unittest.json)

## General structure

Expand Down
7 changes: 0 additions & 7 deletions flutter/android/app/src/main/AndroidManifest.xml
Original file line number Diff line number Diff line change
Expand Up @@ -54,13 +54,6 @@
to determine the Window background behind the Flutter UI. -->
<meta-data android:name="io.flutter.embedding.android.NormalTheme"
android:resource="@style/NormalTheme"/>
<!-- Displays an Android View that continues showing the launch screen
Drawable until Flutter paints its first frame, then this splash
screen fades out. A splash screen is useful to avoid any visual
gap between the end of Android's launch screen and the painting of
Flutter's first frame. -->
<meta-data android:name="io.flutter.embedding.android.SplashScreenDrawable"
android:resource="@drawable/launch_background"/>
<intent-filter>
<action android:name="android.intent.action.MAIN"/>
<category android:name="android.intent.category.LAUNCHER"/>
Expand Down
17 changes: 8 additions & 9 deletions flutter/flutter.mk
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ flutter/backend-list:
RESULT_JSON_SAMPLE_PATH?=output/extended_result_example.json
.PHONY: flutter/result/gen-extended-sample
flutter/result/gen-extended-sample:
cd flutter_common && \
cd flutter && \
${_start_args} dart run \
--define=jsonFileName=../${RESULT_JSON_SAMPLE_PATH} \
lib/data/generation_helpers/write_json_sample.main.dart
Expand All @@ -105,14 +105,17 @@ flutter/result/gen-schema: flutter/result/gen-extended-sample
quicktype ${RESULT_JSON_SAMPLE_PATH} \
--lang schema \
--out ${RESULT_JSON_SCHEMA_PATH}
cd flutter_common && \
cd flutter && \
${_start_args} dart run \
--define=schemaPath=../${RESULT_JSON_SCHEMA_PATH} \
lib/data/generation_helpers/edit_json_schema.main.dart

.PHONY: flutter/result/json
flutter/result/json:
cd flutter_common && ${_start_args} flutter --no-version-check pub run \
@echo "Generate .g.dart files for the @JsonSerializable annotation"
@# https://github.com/dart-lang/build/issues/2835#issuecomment-1047849076
cd flutter && ${_start_args} flutter packages pub get
cd flutter && ${_start_args} flutter --no-version-check pub run \
build_runner build --delete-conflicting-outputs

.PHONY: flutter/build-info
Expand Down Expand Up @@ -157,9 +160,7 @@ flutter/set-windows-build-number:
.PHONY: flutter/pub
flutter/pub:
[ -z "${FLUTTER_FORCE_PUB_GET}" ] || rm -rf output/flutter/pub
make \
output/flutter/pub/flutter.stamp \
output/flutter/pub/flutter_common.stamp
make output/flutter/pub/flutter.stamp
[ -z "${FLUTTER_FORCE_PUB_GET}" ] || rm -rf output/flutter/pub
output/flutter/pub/%.stamp: %/pubspec.yaml
cd $(shell basename $@ .stamp) && ${_start_args} flutter --no-version-check pub get
Expand All @@ -168,8 +169,7 @@ output/flutter/pub/%.stamp: %/pubspec.yaml

.PHONY: flutter/test/unit
flutter/test/unit:
cd flutter && ${_start_args} flutter --no-version-check test --no-pub test -r expanded
cd flutter_common && ${_start_args} flutter --no-version-check test --no-pub test -r expanded
cd flutter && ${_start_args} flutter --no-version-check test --no-pub unit_test/* -r expanded

ifneq (${FLUTTER_TEST_DEVICE},)
flutter_test_device_arg=--device-id "${FLUTTER_TEST_DEVICE}"
Expand Down Expand Up @@ -201,6 +201,5 @@ flutter/run:
.PHONY: flutter/clean
flutter/clean:
cd flutter && ${_start_args} flutter --no-version-check clean
cd flutter_common && ${_start_args} flutter --no-version-check clean
rm -rf output/flutter/pub

4 changes: 2 additions & 2 deletions flutter/integration_test/expected_throughput.dart
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ const Map<String, Map<String, Interval>> _objectDetection = {
_kIphoneOnMacbookM1: Interval(min: 9, max: 16),
},
_kCoreMLBackend: {
_kIphoneOnGitHubAction: Interval(min: 1.5, max: 4),
_kIphoneOnGitHubAction: Interval(min: 1.0, max: 4),
},
_kPixelBackend: {
_kPixel6: Interval(min: 300, max: 490),
Expand Down Expand Up @@ -104,7 +104,7 @@ const Map<String, Map<String, Interval>> _imageClassificationOffline = {
_kIphoneOnMacbookM1: Interval(min: 30, max: 45),
},
_kCoreMLBackend: {
_kIphoneOnGitHubAction: Interval(min: 3, max: 15),
_kIphoneOnGitHubAction: Interval(min: 3, max: 20),
},
_kPixelBackend: {
_kPixel6: Interval(min: 1000, max: 1700),
Expand Down
6 changes: 3 additions & 3 deletions flutter/integration_test/first_test.dart
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ import 'package:integration_test/integration_test.dart';
import 'package:mlperfbench/benchmark/run_mode.dart';
import 'package:mlperfbench/firebase/firebase_manager.dart';
import 'package:mlperfbench/store.dart';
import 'package:mlperfbench_common/data/environment/environment_info.dart';
import 'package:mlperfbench_common/data/extended_result.dart';
import 'package:mlperfbench_common/data/results/benchmark_result.dart';
import 'package:mlperfbench/data/environment/environment_info.dart';
import 'package:mlperfbench/data/extended_result.dart';
import 'package:mlperfbench/data/results/benchmark_result.dart';
import 'package:shared_preferences/shared_preferences.dart';

import 'expected_accuracy.dart';
Expand Down
4 changes: 2 additions & 2 deletions flutter/integration_test/utils.dart
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ import 'package:mlperfbench/ui/home/benchmark_start_screen.dart';
import 'package:provider/provider.dart';

import 'package:mlperfbench/benchmark/state.dart';
import 'package:mlperfbench/ui/home/result_screen.dart';
import 'package:mlperfbench_common/data/extended_result.dart';
import 'package:mlperfbench/ui/home/benchmark_result_screen.dart';
import 'package:mlperfbench/data/extended_result.dart';
import 'package:mlperfbench/resources/result_manager.dart' as result_manager;
import 'package:mlperfbench/resources/resource_manager.dart'
as resource_manager;
Expand Down
2 changes: 1 addition & 1 deletion flutter/ios/Podfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,6 @@ SPEC CHECKSUMS:
url_launcher_ios: 08a3dfac5fb39e8759aeb0abbd5d9480f30fc8b4
wakelock: d0fc7c864128eac40eba1617cb5264d9c940b46f

PODFILE CHECKSUM: 00cc1108992a37101ac41f0db0bdc6d05d92ccab
PODFILE CHECKSUM: 7efefa9b0888bed7351e3ede8a076c7bd0e778fa

COCOAPODS: 1.12.1
1 change: 1 addition & 0 deletions flutter/lib/app_constants.dart
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ class AppColors {
static const lightRedText = Color.fromARGB(255, 255, 120, 100);
static const darkText = Colors.black;
static const darkRedText = Colors.red;
static const darBlue = Color(0xFF0B3A61);

static const dialogBackground = Colors.white;
static const snackBarBackground = Color(0xFFEDEDED);
Expand Down
2 changes: 1 addition & 1 deletion flutter/lib/backend/bridge/ffi_run.dart
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@
import 'dart:ffi';

import 'package:ffi/ffi.dart';
import 'package:mlperfbench_common/data/results/benchmark_result.dart';

import 'package:mlperfbench/backend/bridge/handle.dart';
import 'package:mlperfbench/backend/bridge/run_result.dart';
import 'package:mlperfbench/backend/bridge/run_settings.dart';
import 'package:mlperfbench/data/results/benchmark_result.dart';

class _RunIn extends Struct {
external Pointer<Utf8> backend_model_path;
Expand Down
2 changes: 1 addition & 1 deletion flutter/lib/backend/bridge/run_result.dart
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import 'package:mlperfbench_common/data/results/benchmark_result.dart';
import 'package:mlperfbench/data/results/benchmark_result.dart';

class NativeRunResult {
final Accuracy? accuracy1;
Expand Down
3 changes: 1 addition & 2 deletions flutter/lib/backend/list.dart
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
import 'dart:io' show Platform;

import 'package:mlperfbench_common/data/environment/environment_info.dart';

import 'package:mlperfbench/backend/bridge/ffi_match.dart';
import 'package:mlperfbench/data/environment/environment_info.dart';
import 'package:mlperfbench/device_info.dart';
import 'package:mlperfbench/protos/backend_setting.pb.dart' as pb;

Expand Down
2 changes: 1 addition & 1 deletion flutter/lib/benchmark/benchmark.dart
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
import 'package:collection/collection.dart';
import 'package:mlperfbench_common/data/results/benchmark_result.dart';

import 'package:mlperfbench/app_constants.dart';
import 'package:mlperfbench/backend/bridge/run_settings.dart';
import 'package:mlperfbench/benchmark/info.dart';
import 'package:mlperfbench/benchmark/run_mode.dart';
import 'package:mlperfbench/data/results/benchmark_result.dart';
import 'package:mlperfbench/device_info.dart';
import 'package:mlperfbench/protos/backend_setting.pb.dart' as pb;
import 'package:mlperfbench/protos/mlperf_task.pb.dart' as pb;
Expand Down
3 changes: 1 addition & 2 deletions flutter/lib/benchmark/run_info.dart
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
import 'package:mlperfbench_common/data/results/benchmark_result.dart';

import 'package:mlperfbench/backend/bridge/run_result.dart';
import 'package:mlperfbench/backend/bridge/run_settings.dart';
import 'package:mlperfbench/backend/loadgen_info.dart';
import 'package:mlperfbench/data/results/benchmark_result.dart';

class RunInfo {
final RunSettings settings;
Expand Down
8 changes: 7 additions & 1 deletion flutter/lib/benchmark/state.dart
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@ import 'package:flutter/cupertino.dart';
import 'package:flutter/foundation.dart' show ChangeNotifier;
import 'package:flutter/material.dart';

import 'package:mlperfbench_common/data/extended_result.dart';
import 'package:wakelock/wakelock.dart';

import 'package:mlperfbench/backend/bridge/isolate.dart';
import 'package:mlperfbench/backend/list.dart';
import 'package:mlperfbench/benchmark/benchmark.dart';
import 'package:mlperfbench/board_decoder.dart';
import 'package:mlperfbench/build_info.dart';
import 'package:mlperfbench/data/extended_result.dart';
import 'package:mlperfbench/resources/config_manager.dart';
import 'package:mlperfbench/resources/resource_manager.dart';
import 'package:mlperfbench/resources/validation_helper.dart';
Expand Down Expand Up @@ -229,6 +229,12 @@ class BenchmarkState extends ChangeNotifier {
throw StateError('unreachable');
}

Future<void> resetBenchmarkState() async {
_doneRunning = null;
resetCurrentResults();
notifyListeners();
}

Future<void> runBenchmarks() async {
assert(resourceManager.done, 'Resource manager is not done.');
assert(_doneRunning != false, '_doneRunning is false');
Expand Down
2 changes: 1 addition & 1 deletion flutter/lib/build_info.dart
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import 'package:mlperfbench_common/data/build_info/build_info.dart';
import 'package:package_info_plus/package_info_plus.dart';

import 'package:mlperfbench/app_constants.dart';
import 'package:mlperfbench/backend/list.dart';
import 'package:mlperfbench/data/build_info/build_info.dart';

part 'build_info.gen.dart';

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import 'package:mlperfbench_common/data/extended_result.dart';
import 'package:mlperfbench_common/data/result_filter.dart';
import 'package:mlperfbench_common/data/result_sort.dart';
import 'package:mlperfbench_common/data/results/benchmark_result.dart';
import 'package:mlperfbench/data/extended_result.dart';
import 'package:mlperfbench/data/result_filter.dart';
import 'package:mlperfbench/data/result_sort.dart';
import 'package:mlperfbench/data/results/benchmark_result.dart';

class BenchmarksDataProvider {
List<ExtendedResult> results;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import 'package:json_annotation/json_annotation.dart';

import 'package:mlperfbench_common/data/environment/env_android.dart';
import 'package:mlperfbench_common/data/environment/env_ios.dart';
import 'package:mlperfbench_common/data/environment/env_windows.dart';
import 'package:mlperfbench/data/environment/env_android.dart';
import 'package:mlperfbench/data/environment/env_ios.dart';
import 'package:mlperfbench/data/environment/env_windows.dart';

part 'environment_info.g.dart';

Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import 'package:json_annotation/json_annotation.dart';

import 'package:mlperfbench_common/data/build_info/build_info.dart';
import 'package:mlperfbench_common/data/environment/environment_info.dart';
import 'package:mlperfbench_common/data/meta_info.dart';
import 'package:mlperfbench_common/data/results/benchmark_result.dart';
import 'package:mlperfbench/data/build_info/build_info.dart';
import 'package:mlperfbench/data/environment/environment_info.dart';
import 'package:mlperfbench/data/meta_info.dart';
import 'package:mlperfbench/data/results/benchmark_result.dart';

part 'extended_result.g.dart';

Expand Down
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
import 'package:uuid/uuid.dart';

import 'package:mlperfbench_common/data/build_info/build_info.dart';
import 'package:mlperfbench_common/data/environment/env_android.dart';
import 'package:mlperfbench_common/data/environment/env_ios.dart';
import 'package:mlperfbench_common/data/environment/env_windows.dart';
import 'package:mlperfbench_common/data/environment/environment_info.dart';
import 'package:mlperfbench_common/data/extended_result.dart';
import 'package:mlperfbench_common/data/meta_info.dart';
import 'package:mlperfbench_common/data/results/backend_info.dart';
import 'package:mlperfbench_common/data/results/backend_settings.dart';
import 'package:mlperfbench_common/data/results/backend_settings_extra.dart';
import 'package:mlperfbench_common/data/results/benchmark_result.dart';
import 'package:mlperfbench_common/data/results/dataset_info.dart';
import 'package:mlperfbench/data/build_info/build_info.dart';
import 'package:mlperfbench/data/environment/env_android.dart';
import 'package:mlperfbench/data/environment/env_ios.dart';
import 'package:mlperfbench/data/environment/env_windows.dart';
import 'package:mlperfbench/data/environment/environment_info.dart';
import 'package:mlperfbench/data/extended_result.dart';
import 'package:mlperfbench/data/meta_info.dart';
import 'package:mlperfbench/data/results/backend_info.dart';
import 'package:mlperfbench/data/results/backend_settings.dart';
import 'package:mlperfbench/data/results/backend_settings_extra.dart';
import 'package:mlperfbench/data/results/benchmark_result.dart';
import 'package:mlperfbench/data/results/dataset_info.dart';

/// Generates sample data.
///
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import 'dart:convert';
import 'dart:io';

import 'package:mlperfbench_common/data/extended_result.dart';
import 'package:mlperfbench_common/data/generation_helpers/sample_generator.dart';
import 'package:mlperfbench/data/extended_result.dart';
import 'package:mlperfbench/data/generation_helpers/sample_generator.dart';

//
// This file generates a sample of result data and writes it into specified place.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import 'package:json_annotation/json_annotation.dart';

import 'package:mlperfbench_common/constants.dart';
import 'package:mlperfbench_common/data/environment/environment_info.dart';
import 'package:mlperfbench_common/data/extended_result.dart';
import 'package:mlperfbench_common/data/results/benchmark_result.dart';
import 'package:mlperfbench/data/environment/environment_info.dart';
import 'package:mlperfbench/data/extended_result.dart';
import 'package:mlperfbench/data/results/benchmark_result.dart';
import 'package:mlperfbench/data/string_constants.dart';

part 'result_filter.g.dart';

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import 'package:mlperfbench_common/data/results/benchmark_result.dart';
import 'package:mlperfbench/data/results/benchmark_result.dart';

enum SortByEnum { dateAsc, dateDesc, taskThroughputAsc, taskThroughputDesc }

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import 'package:json_annotation/json_annotation.dart';

import 'package:mlperfbench_common/constants.dart';
import 'package:mlperfbench_common/data/results/backend_settings_extra.dart';
import 'package:mlperfbench/data/results/backend_settings_extra.dart';
import 'package:mlperfbench/data/string_constants.dart';

part 'backend_settings.g.dart';

Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import 'package:json_annotation/json_annotation.dart';

import 'package:mlperfbench_common/data/results/backend_info.dart';
import 'package:mlperfbench_common/data/results/backend_settings.dart';
import 'package:mlperfbench_common/data/results/dataset_info.dart';
import 'package:mlperfbench/data/results/backend_info.dart';
import 'package:mlperfbench/data/results/backend_settings.dart';
import 'package:mlperfbench/data/results/dataset_info.dart';

part 'benchmark_result.g.dart';

Expand Down
8 changes: 4 additions & 4 deletions flutter/lib/device_info.dart
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@ import 'package:flutter/services.dart';

import 'package:device_info_plus/device_info_plus.dart';
import 'package:device_marketing_names/device_marketing_names.dart';
import 'package:mlperfbench_common/data/environment/env_android.dart';
import 'package:mlperfbench_common/data/environment/env_ios.dart';
import 'package:mlperfbench_common/data/environment/env_windows.dart';
import 'package:mlperfbench_common/data/environment/environment_info.dart';
import 'package:process_run/shell.dart';

import 'package:mlperfbench/backend/bridge/ffi_cpuinfo.dart';
import 'package:mlperfbench/data/environment/env_android.dart';
import 'package:mlperfbench/data/environment/env_ios.dart';
import 'package:mlperfbench/data/environment/env_windows.dart';
import 'package:mlperfbench/data/environment/environment_info.dart';

class DeviceInfo {
final EnvironmentInfo envInfo;
Expand Down
2 changes: 1 addition & 1 deletion flutter/lib/firebase/firebase_manager.dart
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ import 'package:firebase_core/firebase_core.dart';
import 'package:firebase_storage/firebase_storage.dart';
import 'package:firebase_ui_auth/firebase_ui_auth.dart';
import 'package:intl/intl.dart';
import 'package:mlperfbench_common/data/extended_result.dart';

import 'package:mlperfbench/data/extended_result.dart';
import 'package:mlperfbench/firebase/firebase_auth_service.dart';
import 'package:mlperfbench/firebase/firebase_crashlytics_service.dart';
import 'package:mlperfbench/firebase/firebase_options.gen.dart';
Expand Down
2 changes: 1 addition & 1 deletion flutter/lib/l10n/app_en.arb
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@

"menuHome": "MLPerf Mobile",
"menuHistory": "History",
"menuBenchmarkConfiguration": "Benchmark Configuration",
"menuSettings": "Settings",
"menuAbout": "About",
"menuProfile": "Profile",
Expand All @@ -25,6 +24,7 @@
"mainScreenGo": "GO",
"mainScreenMeasureTitle": "Measure your device performance for:",
"mainScreenWaitFinish": "Wait for benchmark to finish",
"mainScreenBenchmarkSelected": "Benchmarks (<selected>/<total> selected)",

"progressMeasuring": "Measuring...",
"progressDontClose": "Don't close the app!",
Expand Down
Loading