Sync with Microsoft ONNX Runtime - 18082026 - #1262
Merged
Merged
Conversation
This pull request strengthens validation for padding values in pooling operations to ensure they are non-negative, preventing invalid configurations and potential runtime errors. It also adds unit tests to verify that negative padding values are correctly rejected, both at the ONNX operator level and in the underlying MLAS pooling implementation. Validation improvements: * Added explicit checks in `pool_attributes.h` to enforce that all padding values (`pads`) are non-negative when `auto_pad` is not set, raising an error if this condition is violated. * Updated `pooling.cpp` to throw a `std::invalid_argument` exception if any padding value is negative, providing an additional safeguard at the MLAS pooling layer. Testing enhancements: * Added new unit tests in `pool_op_test.cc` to verify that negative padding values cause the expected errors, both for the ONNX MaxPool operator and for the MLAS pooling backend.
This pull request introduces a new session option to control the replay of saved runtime optimizations from ORT format models, enhancing security and configurability. By default, replaying saved runtime optimizations is now disabled and must be explicitly enabled for trusted models. The changes also update tests and documentation to reflect this new behavior. **Session Option for Saved Runtime Optimizations** * Added a new session option `session.enable_saved_runtime_optimizations` (constant `kOrtSessionOptionsConfigEnableSavedRuntimeOptimizations`) to control whether saved runtime optimizations are replayed. The default is "0" (disabled), and it must be set to "1" to enable replay for trusted models. * Updated the `OrtFormatLoadOptions` struct so that saved runtime optimizations are ignored by default unless explicitly enabled. * Modified the model loading logic in `InferenceSession::LoadOrtModelWithLoader` to check the new session option and set `ignore_saved_runtime_optimizations` accordingly. [[1]](diffhunk://#diff-3e2227e1225091e8b74c02688e23b21630d1393dd395e15966558901538dd2c7R1989-R1993) [[2]](diffhunk://#diff-3e2227e1225091e8b74c02688e23b21630d1393dd395e15966558901538dd2c7L2037) **Test and Documentation Updates** * Updated unit tests to cover scenarios with saved runtime optimizations both enabled and disabled, including adding explicit tests for the default (disabled) behavior and updating test helpers to accept the new option. [[1]](diffhunk://#diff-091729c3958c3945dfe649e7a1c3313b941a3da9a27662f047cc35c948196e49L538-R539) [[2]](diffhunk://#diff-091729c3958c3945dfe649e7a1c3313b941a3da9a27662f047cc35c948196e49R563-R565) [[3]](diffhunk://#diff-091729c3958c3945dfe649e7a1c3313b941a3da9a27662f047cc35c948196e49R618-R633) [[4]](diffhunk://#diff-091729c3958c3945dfe649e7a1c3313b941a3da9a27662f047cc35c948196e49L628-R644) [[5]](diffhunk://#diff-911fb882df494c58a77b6107296e96e0b8117b93e5c3223ccab942cd8d9ac271L233-R241) [[6]](diffhunk://#diff-911fb882df494c58a77b6107296e96e0b8117b93e5c3223ccab942cd8d9ac271L284-R288) [[7]](diffhunk://#diff-f8ff296cfa4648f2de15ec12ea7224d956e3e275740149b113a6a88281f814cfR237-R238) [[8]](diffhunk://#diff-f8ff296cfa4648f2de15ec12ea7224d956e3e275740149b113a6a88281f814cfR261-R262) [[9]](diffhunk://#diff-f8ff296cfa4648f2de15ec12ea7224d956e3e275740149b113a6a88281f814cfR333-R345) * Updated documentation and warning messages in the Python model conversion tool and model readme files to clarify that replay must be enabled explicitly and only for trusted models. [[1]](diffhunk://#diff-dca4da8cb704da861d8ac89bfaab276d8a34635d8b5af899ddeb47f39ffadee6R4-R6) [[2]](diffhunk://#diff-10b36566cefcd506f2d737f967628e96f32259e24449bcca56ff1de48d78065dL216-R217) [[3]](diffhunk://#diff-10b36566cefcd506f2d737f967628e96f32259e24449bcca56ff1de48d78065dR311-R316)
Add support for the WebGPU subgroup-size-control extension, which allows compute pipelines to request a fixed subgroup size via WGSL @subgroup_size(N) attribute. As an example, the MatMulNBits kernel has been enforced to use a subgroup size of 32 on Intel.
Avoid expanding the PagedAttention block table when its block size already matches XQA’s 128-token page size. For 128-token blocks, the existing block table is passed directly to XQA. Larger block sizes continue using the existing expanded page-table path. This removes an unnecessary temporary allocation and conversion kernel from each eligible XQA invocation.
microsoft#31660) …titioning The NCHWc convolution algorithms (pointwise and direct) partition work uniformly by item index over FilterSetCount x OutputHeight items, but the cost of an item is proportional to its FilterSet's FilterCount: full sets process FilterSetSize (4) NCHWc blocks while a ragged last set (when Cout/16 % 4 != 0) processes as few as 1. Threads landing on full sets do up to 4x the FLOPs of threads on the tail set, capping scheduling efficiency at 62-75% for common shapes (e.g. Cout=96: sets of 4+2 blocks -> 0.75 efficiency at 4 threads). Replace the uniform index split with a split proportional to FLOP cost, measured in block-rows (one NCHWc output block x one output row): MlasPartitionWork distributes cost intervals, and CostToWorkIndex maps cost boundaries back to work-item boundaries with exact coverage and no overlap. Applied to both MLAS_NCHWC_CONV_POINTWISE_ALGORITHM and MLAS_NCHWC_CONV_NCHWC_ALGORITHM; the NCHW first-layer, depthwise, and pooling algorithms keep the uniform split (no imbalance there). Outputs are bitwise identical: repartitioning moves whole output-row items between threads without changing any element's accumulation order. Also: - Add session config "mlas.nchwc_conv_max_input_channel_batch" to override the pointwise algorithm's input-channel batch (default 128, rounded up to a BlockSize multiple; 0/unset keeps the default) via MLAS_BACKEND_KERNEL_SELECTOR_CONFIG, for perf experimentation. - Fix copy-paste guards in the SconvKernelAvx512F.S ReLU post-process (FilterCount 2 -> 3/4 for the zmm14/18/22 and zmm15/19/23 rows); behavior was accidentally correct but the FilterCount=2/OutputCount=6 variant emitted 6 vmaxps on dead registers. Measured on AMD Ryzen AI MAX+ 395 (Zen 5, AVX-512), batch=1 fp32, intra_op=4, 512-iteration medians vs unmodified main: yolox_tiny 8.009 -> 7.149 ms (-10.7%) 3x3-conv heavy, 86% of its 3x3 time in ragged shapes mv3_large 0.962 -> 0.922 ms (-4.2%) mv3_small 0.458 -> 0.455 ms (-1% noise) mobileclip unchanged (+/-1% noise; its conv shapes are 64-aligned) Hottest yolox shape (96->96 3x3 @52x52): 939 -> 1167 GFLOP/s (91% of machine peak). All 29,910 onnxruntime_mlas_test cases pass. ### Description <!-- Describe your changes. --> ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
ai-fw-intg
requested review from
Jaswanth51,
ankitm3k,
jatinwadhwa921 and
vthaniel
August 17, 2026 20:36
hdharpure9922
self-requested a review
August 18, 2026 04:49
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated daily backmerge from ORT main to ovep-develop. No conflicts detected. Do NOT squash or rebase - use merge commit only.