Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 15 additions & 2 deletions tools/ensemble_test_runner/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,13 @@ initialState:
name:
first: John
last: Doe
# Ensemble encrypted public storage. Keys are logical keys; values are
# encrypted at bootstrap and persisted internally as enc_<key>.
secureStorage:
onboardingComplete: true
# Platform FlutterSecureStorage, used by keychain/readSecurely actions.
keychain:
authToken: test-token
steps:
- expectVisible:
id: greeting_text
Expand Down Expand Up @@ -123,7 +130,9 @@ logStorage:

`mocks` and `initialState` in `config.yaml` apply to every test. Test-file
`mocks` / `initialState` values override suite values for the same API name or
storage/keychain/env key.
storage/secureStorage/keychain/env key. `storage`, `secureStorage`, and
`keychain` are separate backends: `secureStorage` is Ensemble's encrypted
public-storage namespace, while `keychain` is platform secure storage.

When `devices` is set, each test expands to one run per device (ids look like
`home[android_nl]` when there is more than one device). Device `locale` sets
Expand Down Expand Up @@ -337,6 +346,10 @@ override suite keys:
initialState:
storage:
apiUrl: http://ensemble.test/ws/NeMo/Intf/lan:getMIBs
secureStorage:
onboardingComplete: true
keychain:
authToken: test-token
env:
APP_LOCALE: nl

Expand All @@ -355,7 +368,7 @@ steps:
### Reusable authenticated session

The session producer runs once. After it passes, the runner captures public
storage, keychain values, and locale in memory. Each consumer restores that
storage, encrypted secure-storage values, keychain values, and locale in memory. Each consumer restores that
snapshot, runs its `setup`, and mounts a fresh requested screen.

```yaml
Expand Down
2 changes: 1 addition & 1 deletion tools/ensemble_test_runner/STEP_VOCABULARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ with `--report=json` or `--report-file=build/ensemble_test_results.json`.
Each `*.test.yaml` file is a single test case and must provide:

- `startScreen` — cold-starts the app on the given screen and runs steps
- `session` — optional; runs the referenced test once, restores its captured storage/keychain/locale for this test, runs `setup`, and mounts the requested `startScreen`
- `session` — optional; runs the referenced test once, restores its captured public storage/secure storage/keychain/locale for this test, runs `setup`, and mounts the requested `startScreen`
- `retry` — number of additional attempts after a failed run, e.g. `retry: 3`

Root-level `setup` supports `httpRequest`, `group`, and `optional`. It runs
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@
},
"initialState": {
"$ref": "#/$defs/initialState",
"description": "Suite-wide storage/keychain/env applied before each test initialState. Test values override suite values per key."
"description": "Suite-wide storage/secureStorage/keychain/env applied before each test initialState. Test values override suite values per key."
},
"profiles": {
"type": "object",
Expand Down Expand Up @@ -213,6 +213,18 @@
}
]
},
"secureStorage": {
"oneOf": [
{
"type": "object",
"additionalProperties": true
},
{
"type": "string",
"pattern": "^\\$\\{(inputs|scenario)\\.[A-Za-z0-9_.-]+\\}$"
}
]
},
"keychain": {
"oneOf": [
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,18 @@
}
]
},
"secureStorage": {
"oneOf": [
{
"type": "object",
"additionalProperties": true
},
{
"type": "string",
"pattern": "^\\$\\{(inputs|scenario)\\.[A-Za-z0-9_.-]+\\}$"
}
]
},
"keychain": {
"oneOf": [
{
Expand Down
12 changes: 10 additions & 2 deletions tools/ensemble_test_runner/lib/assertions/assertion_engine.dart
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,11 @@ class AssertionEngine {
);
}
final isEnabled = _readSemantics(
() => tester.getSemantics(finder).hasFlag(SemanticsFlag.isEnabled),
() => tester
.getSemantics(finder)
.getSemanticsData()
.flagsCollection
.isEnabled,
);
if (isEnabled != enabled) {
throw EnsembleTestFailure(
Expand Down Expand Up @@ -358,7 +362,11 @@ class AssertionEngine {
throw EnsembleTestFailure('expectChecked: widget "$id" not found.');
}
final isChecked = _readSemantics(
() => tester.getSemantics(finder).hasFlag(SemanticsFlag.isChecked),
() => tester
.getSemantics(finder)
.getSemanticsData()
.flagsCollection
.isChecked,
);
if (isChecked != expected) {
throw EnsembleTestFailure(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -635,7 +635,7 @@ class EnsembleTestExecutionPlanner {
}

/// Suite [initialState] is the base; test values override per key within
/// `storage`, `keychain`, and `env`.
/// `storage`, `secureStorage`, `keychain`, and `env`.
@visibleForTesting
static Map<String, dynamic> mergedInitialState(
Map<String, dynamic> suite,
Expand All @@ -654,7 +654,7 @@ class EnsembleTestExecutionPlanner {
}

final merged = <String, dynamic>{};
for (final key in const ['storage', 'keychain', 'env']) {
for (final key in const ['storage', 'secureStorage', 'keychain', 'env']) {
final value = section(key);
if (value.isNotEmpty) {
merged[key] = value;
Expand Down
Loading
Loading