diff --git a/src/docs/Coding-Standards/PowerShell/Testing.md b/src/docs/Coding-Standards/PowerShell/Testing.md new file mode 100644 index 0000000..3cb776a --- /dev/null +++ b/src/docs/Coding-Standards/PowerShell/Testing.md @@ -0,0 +1,58 @@ +--- +title: PowerShell Testing +description: Pester test naming and the Simple, Standard, and Advanced profiles for PowerShell modules. +--- + +# PowerShell Testing + +PowerShell tests build on the [testing baseline](../Testing.md): they are test-first, locally runnable, deterministic, isolated, and automated in CI. Use [Pester](https://pester.dev/) and name ordinary test files `*.Tests.ps1`. + +## Module test profiles + +Simple, Standard, and Advanced are documentation profiles: conventions for arranging module-local tests, not selectable Process-PSModule modes. All three feed the same filesystem discovery engine; there is no layout setting in `.github/PSModule.yml`. + +Choose the smallest profile that keeps the suite easy to navigate. + +### Simple + +Use one root-level test file named after the module: + +```text +tests/ +└── .Tests.ps1 +``` + +The Simple layout fits a module whose tests remain readable as one suite. + +### Standard + +Use one root-level test file per public function group: + +```text +tests/ +├── .Tests.ps1 +├── .Tests.ps1 +└── .Tests.ps1 (optional) +``` + +`` names the public function group the file covers. When public functions are organized under `src/functions/public//`, use the same group name for the test file. + +Ungrouped public functions and cross-cutting module behavior may use separate root-level `*.Tests.ps1` suites where appropriate. Name each suite after the functions or behavior it covers. + +### Advanced + +The Advanced profile is recommended when parts of the suite need independent Pester configurations, containers, or ordinary test files. Organize those parts in subdirectories: + +```text +tests/ +├── Unit/ # No configuration or containers: all test files +│ ├── GroupOne.Tests.ps1 +│ └── GroupTwo.Tests.ps1 +├── Integration/ +│ └── Integration.Configuration.ps1 +└── Compatibility/ + ├── Linux.Container.ps1 + └── Windows.Container.ps1 +``` + +Different directories may use different forms. Process-PSModule recursively applies its [per-directory discovery precedence](../../Frameworks/Process-PSModule/pipeline-stages.md#module-local-test-discovery), including sibling suppression, unique test-name requirements, and root-only workflow phases. diff --git a/src/docs/Coding-Standards/PowerShell/index.md b/src/docs/Coding-Standards/PowerShell/index.md index 24cd48a..26c7771 100644 --- a/src/docs/Coding-Standards/PowerShell/index.md +++ b/src/docs/Coding-Standards/PowerShell/index.md @@ -16,6 +16,7 @@ This standard builds on the [language-agnostic baseline](../index.md); where the | [Functions](Functions.md) | Advanced functions — CmdletBinding, typed and validated parameters, pipeline blocks, ShouldProcess, and required comment-based help. | | [Classes](Classes.md) | When to reach for a PowerShell class, and how to structure its members, constructors, and documentation. | | [Scripts](Scripts.md) | Structure for standalone .ps1 scripts — requirements, parameters, help, and keeping the script thin. | +| [PowerShell Testing](Testing.md) | Pester test naming and the Simple, Standard, and Advanced profiles for PowerShell modules. | | [Messaging](Messaging.md) | Write-Verbose for user-facing operational progress and normal troubleshooting; Write-Debug for developer-focused internals and deep diagnostics. | | [Version Constraints](Version-Constraints.md) | Express module and package version constraints as NuGet version ranges — the canonical notation across PSResourceGet, .NET package references, and (mapped) #Requires and module manifests. | | [Module Requirements](Requires-Modules.md) | Valid `#Requires -Modules` version specifications — minimum, major-lock (with the `N.*` wildcard), exact, and GUID identity pinning — with an executable proof. | @@ -80,4 +81,4 @@ Beyond the basics, these language-specific habits keep PowerShell correct and fa The toolchain enforces this standard in CI — it does not define it. The rules above are the source of truth; each tool's configuration is derived from them: - **[PSScriptAnalyzer](https://github.com/PowerShell/PSScriptAnalyzer)** is the linter and formatter; its settings are derived from this standard, so passing it cleanly means matching the standard. Let it format — do not hand-format. -- **[Pester](https://pester.dev/)** is the test framework; test files are named `*.Tests.ps1`. See the [Testing baseline](../Testing.md). +- **[Pester](https://pester.dev/)** is the test framework; test files are named `*.Tests.ps1`. See [PowerShell Testing](Testing.md) for module test layouts. diff --git a/src/docs/Frameworks/Process-PSModule/configuration.md b/src/docs/Frameworks/Process-PSModule/configuration.md index 71af957..86e1b9e 100644 --- a/src/docs/Frameworks/Process-PSModule/configuration.md +++ b/src/docs/Frameworks/Process-PSModule/configuration.md @@ -13,6 +13,15 @@ Plan phase, Process-PSModule enriches this input into an internal runtime `Setti consume. Internal runtime paths in workflow docs (for example, `Settings.Publish.Module.Resolution.*`) describe that enriched inter-workflow contract, not a different authoring format for repository settings files. +Simple, Standard, and Advanced test profiles are repository conventions, not settings. `.github/PSModule.yml` has no +layout or suite-matrix selector; Process-PSModule [discovers the files under `tests/` recursively](pipeline-stages.md#module-local-test-discovery) +and computes its internal `Settings.Test.Module.Suites` matrix from them. + +Test discovery and change triggering are separate. The default `ImportantFilePatterns` match only `^src/` and +`^README\.md$`, so a test-only change does not enter the important-change build, test, and publish path. Repositories +that need test or automation changes to exercise that path must [add `^tests/` and any relevant settings or workflow +paths](usage.md#customizing-important-file-patterns) while retaining every default they still need. + The following settings are available in the settings file: | Name | Type | Description | Default | diff --git a/src/docs/Frameworks/Process-PSModule/pipeline-stages.md b/src/docs/Frameworks/Process-PSModule/pipeline-stages.md index 349f01e..2dce5a9 100644 --- a/src/docs/Frameworks/Process-PSModule/pipeline-stages.md +++ b/src/docs/Frameworks/Process-PSModule/pipeline-stages.md @@ -19,8 +19,8 @@ situational awareness, it calculates the next module version. The user-facing settings file stays in `.github/PSModule.yml`. The workflow enriches that input into an internal runtime `Settings` object passed between jobs. In this runtime contract, execution decisions are phase-owned (`*.Enabled`), test -suite matrices are defined under each owning test phase, and resolved version metadata is stored under -`Settings.Publish.Module.Resolution`. +suite matrices are computed under each owning test phase, and resolved version metadata is stored under +`Settings.Publish.Module.Resolution`. The `*.Suites` values are workflow outputs, not authorable layout settings. ### Internal runtime settings contract @@ -37,9 +37,9 @@ suite matrices are defined under each owning test phase, and resolved version me | `Settings.Test.CodeCoverage.Enabled` | Whether code coverage aggregation/enforcement runs. | | `Settings.Publish.Module.Enabled` | Whether module publication/release runs. | | `Settings.Publish.Site.Enabled` | Whether documentation publication runs. | -| `Settings.Test.SourceCode.Suites` | Source-code test suite matrix. | -| `Settings.Test.PSModule.Suites` | Framework test suite matrix. | -| `Settings.Test.Module.Suites` | Module-local test suite matrix. | +| `Settings.Test.SourceCode.Suites` | Computed source-code test suite matrix. | +| `Settings.Test.PSModule.Suites` | Computed framework test suite matrix. | +| `Settings.Test.Module.Suites` | Computed module-local test suite matrix. | | `Settings.Publish.Module.Resolution.Version` | Resolved semantic version used for build and publish. | | `Settings.Publish.Module.Resolution.Prerelease` | Whether the resolved version is prerelease. | | `Settings.Publish.Module.Resolution.FullVersion` | Resolved full version string. | @@ -109,24 +109,40 @@ The [PSModule - SourceCode tests](https://github.com/PSModule/Process-PSModule/b [workflow](https://github.com/PSModule/Process-PSModule/blob/main/.github/workflows/Test-ModuleLocal.yml) -- Imports and tests the module in parallel (matrix) using Pester tests from the module repository. +- Imports and tests the module in parallel (matrix) using module-local Pester tests. +- Discovers module-local tests recursively under `tests/`, applying the [per-directory precedence](#module-local-test-discovery) independently at every level. - Module test files declare a Pester **6.x** requirement via `#Requires -Modules @{ ModuleName = 'Pester'; ModuleVersion = '6.0.0'; MaximumVersion = '6.*' }` — a convention module authors add to each `*.Tests.ps1`, not something this pipeline injects. The [Invoke-Pester](https://github.com/PSModule/Invoke-Pester) action installs a matching `6.x`, so minor and patch updates flow in automatically while a new major stays a deliberate, reviewed change. - Supports setup and teardown scripts executed via separate dedicated jobs: - - `BeforeAll`: Runs once before all test matrix jobs to set up the test environment (e.g., deploy infrastructure, download test data). - - `AfterAll`: Runs once after all test matrix jobs complete to clean up the test environment (e.g., remove test resources, clean up databases). -- Setup/teardown scripts are automatically detected in test directories and executed with the same environment variables as the tests. -- This produces a JSON-based report that is used by [Get-PesterTestResults](#get-test-results) evaluate the results of the tests. + - `BeforeAll`: Runs root `tests/BeforeAll.ps1` once before all test matrix jobs to set up the test environment (e.g., deploy infrastructure, download test data). + - `AfterAll`: Runs root `tests/AfterAll.ps1` once after all test matrix jobs complete to clean up the test environment (e.g., remove test resources, clean up databases). +- Setup and teardown detection is not recursive; nested files with those names are not workflow phases. +- This produces a JSON-based report that is used by [Get-PesterTestResults](#get-test-results) to evaluate the results of the tests. + +### Module-local test discovery + +Simple, Standard, and Advanced are [documentation profiles](../../Coding-Standards/PowerShell/Testing.md#module-test-profiles), not selectable workflow modes. The same discovery engine handles every profile. `.github/PSModule.yml` has no test-layout or suite-matrix setting; `Settings.Test.Module.Suites` is computed internally from the repository files. + +Process-PSModule inspects `tests/` recursively. Within each directory it uses the first matching form: + +1. Exactly one `*.Configuration.ps1`. Discovery fails when a directory contains more than one. When selected, sibling `*.Container.ps1` and `*.Tests.ps1` files are not independently selected. +2. Otherwise, one or more `*.Container.ps1`. When selected, sibling `*.Tests.ps1` files are not independently selected. +3. Otherwise, all `*.Tests.ps1`. + +The selected form takes precedence only in that directory. Child directories are still inspected independently, so a repository may mix configurations, containers, and ordinary test files across different directories. + +Every discovered artifact needs a unique prefix before its first dot because that prefix becomes `TestName`. For example, `Users.Unit.Tests.ps1` and `Users.Integration.Tests.ps1` both become `Users`; use distinct prefixes such as `UsersUnit` and `UsersIntegration`. ### Setup and Teardown Scripts The workflow supports automatic execution of setup and teardown scripts for module tests: -- Scripts are automatically detected and executed if present. +- The exact root paths `tests/BeforeAll.ps1` and `tests/AfterAll.ps1` are detected and executed if present. - If no scripts are found, the workflow continues normally. +- Detection is not recursive. #### Setup - `BeforeAll.ps1` -- Place in your test directories (`tests/BeforeAll.ps1`). +- Place at the root test path `tests/BeforeAll.ps1`. - Runs once before all test matrix jobs to prepare the test environment. - Deploy test infrastructure, download test data, initialize databases, or configure services. - Has access to the same environment variables as your tests (secrets, GitHub token, etc.). @@ -143,7 +159,7 @@ Write-Host "Test environment ready!" #### Teardown - `AfterAll.ps1` -- Place in your test directories (`tests/AfterAll.ps1`). +- Place at the root test path `tests/AfterAll.ps1`. - Runs once after all test matrix jobs complete to clean up the test environment. - Remove test resources, clean up databases, stop services, or upload artifacts. - Has access to the same environment variables as your tests. diff --git a/src/docs/Frameworks/Process-PSModule/repository-structure.md b/src/docs/Frameworks/Process-PSModule/repository-structure.md index 8db4700..2428d93 100644 --- a/src/docs/Frameworks/Process-PSModule/repository-structure.md +++ b/src/docs/Frameworks/Process-PSModule/repository-structure.md @@ -26,19 +26,23 @@ Process-PSModule expects repositories to follow the staged layout produced by Te ├── icon/ # Icon assets linked from manifest and documentation │ └── icon.png # Default module icon (PNG format) ├── src/ # Module source, see "Module source code structure" below -├── tests/ # Pester suites executed during validation +├── tests/ # Pester suites; the Simple layout is shown │ ├── AfterAll.ps1 (optional) # Cleanup script for ModuleLocal runs │ ├── BeforeAll.ps1 (optional) # Setup script for ModuleLocal runs -│ └── .Tests.ps1 # Primary test entry point +│ └── .Tests.ps1 # Simple: one root-level module suite ├── .gitattributes # Normalizes line endings across platforms ├── .gitignore # Excludes build artifacts from source control ├── LICENSE # License text surfaced in manifest metadata └── README.md # Repository overview rendered on GitHub and docs landing ``` +The tree shows the [Simple PowerShell test profile](../../Coding-Standards/PowerShell/Testing.md#simple), not an exclusive test-file shape. Standard keeps one root-level `tests/.Tests.ps1` file per public function group. Advanced uses recursively discovered subdirectories, and layouts may mix across directories. Process-PSModule defines the exact [per-directory precedence and sibling suppression](pipeline-stages.md#module-local-test-discovery). + +These names describe repository conventions, not settings. `.github/PSModule.yml` does not select a test profile. The optional `tests/BeforeAll.ps1` and `tests/AfterAll.ps1` files are root-only workflow phases and are not discovered recursively. + Key expectations: -- Keep at least one exported function under `src/functions/public/` and corresponding tests in `tests/`. +- Keep at least one exported function under `src/functions/public/` and corresponding tests in `tests/` using a [documented test profile](../../Coding-Standards/PowerShell/Testing.md#module-test-profiles). - Keep documentation site configuration in `.github/zensical.toml`. - Optional folders (`assemblies`, `formats`, `types`, `variables`, and others) are processed automatically when present. - Markdown files in `src/functions/public` subfolders become documentation pages alongside generated help. diff --git a/src/docs/Frameworks/Process-PSModule/usage.md b/src/docs/Frameworks/Process-PSModule/usage.md index 5b74fef..d0ae5f3 100644 --- a/src/docs/Frameworks/Process-PSModule/usage.md +++ b/src/docs/Frameworks/Process-PSModule/usage.md @@ -137,11 +137,11 @@ $env:CONFLUENCE_SITE # from the "variables" map (not masked) The same `TestData` keys are exported before every module-local phase runs: -- `BeforeAll-ModuleLocal` runs `tests/BeforeAll.ps1` before the module-local test matrix. -- `Test-ModuleLocal` runs the module's Pester tests. -- `AfterAll-ModuleLocal` runs `tests/AfterAll.ps1` after the module-local test matrix, including cleanup paths. +- `BeforeAll-ModuleLocal` runs root `tests/BeforeAll.ps1` before the module-local test matrix. +- `Test-ModuleLocal` discovers and runs module-local Pester tests recursively. +- `AfterAll-ModuleLocal` runs root `tests/AfterAll.ps1` after the module-local test matrix, including cleanup paths. -Setup scripts, tests, and teardown scripts should therefore use the same environment variable names. +Setup and teardown detection is not recursive. These root scripts and the discovered tests should use the same environment variable names. If `$env:` is available in one phase but missing in another, treat that as a Process-PSModule propagation bug rather than a caller contract difference. @@ -223,11 +223,11 @@ This table shows when each job runs based on the trigger scenario: ## Important file change detection -The workflow automatically detects whether a pull request contains changes to "important" files that warrant a new -release. This prevents unnecessary releases when only non-functional files (such as workflow configurations, linter -settings, or test files) are modified. +The workflow automatically detects whether a pull request contains changes to "important" files that should enter the +build, test, and publish path. This prevents unnecessary work and releases when only files outside the configured +patterns are modified. -### Files that trigger releases +### Files that trigger the important-change path By default, the following regular expression patterns identify important files: @@ -244,11 +244,18 @@ To override the default patterns, set `ImportantFilePatterns` in your settings f ImportantFilePatterns: - '^src/' - '^README\.md$' - - '^examples/' + - '^tests/' + - '^\.github/PSModule\.yml$' + - '^\.github/workflows/' ``` When configured, the provided list fully replaces the defaults. Include the default patterns in your list if you still -want them to trigger releases. +want them to trigger the build, test, and publish path. + +Recursive [module-local test discovery](pipeline-stages.md#module-local-test-discovery) does not change this trigger. +With the defaults, a test-only change does not run the important-change build, test, and publish stages because +`^tests/` is not matched. Add `^tests/` when those changes must exercise the path, plus each settings, workflow, or +other automation path whose changes need the same validation. Include only paths that should trigger all three stages. To disable file-change triggering entirely (so that no file changes ever trigger a release), set an empty list in the settings file: diff --git a/src/zensical.toml b/src/zensical.toml index 667f6f0..fdc6ac3 100644 --- a/src/zensical.toml +++ b/src/zensical.toml @@ -104,6 +104,7 @@ nav = [ {"Functions" = "Coding-Standards/PowerShell/Functions.md"}, {"Classes" = "Coding-Standards/PowerShell/Classes.md"}, {"Scripts" = "Coding-Standards/PowerShell/Scripts.md"}, + {"Testing" = "Coding-Standards/PowerShell/Testing.md"}, {"Messaging" = "Coding-Standards/PowerShell/Messaging.md"}, {"Version Constraints" = "Coding-Standards/PowerShell/Version-Constraints.md"}, ]},