ONVIF Agentic Service - #809
Conversation
| <mediaobject> | ||
| <imageobject> | ||
| <?oxy_comment_start author="Ottavio Campana" timestamp="20260729T095751+0200" comment="Image to be redone from scratch"?> | ||
| <imagedata fileref="media/AI/stack_overview.svg" contentwidth="140mm" |
|
PR specifies a persistent HTTP GET /mcp carrying newline-delimited notifications. MCP 2026-07-28 explicitly removed the GET stream. Requests use POST; long-lived notifications use the SSE response to subscriptions/listen. Independent notifications/message events cannot simply be placed on a permanent GET stream. Official MCP Streamable HTTP specification. |
|
server/discover is incorrectly mandatory as the first call. A server must support it, but a modern client may call any operation directly and handle UnsupportedProtocolVersionError. The PR currently says the client shall call it first. |
@willysagefalk Do you have a reference for that? |
|
@ocampana-videotec Yes. The normative reference is the MCP 2026-07-28 specification, Discovery — When to Call: https://modelcontextprotocol.io/specification/2026-07-28/server/discover#when-to-call It states that calling The same section only recommends calling it first for a dual-era client over stdio, as a backward-compatibility probe. Since the ONVIF proposal uses HTTP POST, that stdio-specific recommendation does not apply. I therefore suggest replacing:
with something along these lines:
|
Corrected. |
|
@willysagefalk , following our conversation, I just added chapter 9 about metadata. As you can see it defines abstract concepts and for Observations and Provenance it proposes two specialization. SOSA/SSN was included to accommodate @DammnAI 's feedback in https://github.com/onvif/wg_AI/pull/9 , while the JLDS-based provenance is my proposal based on an already existing specification. If any member wants to model an Observation, they can simply propose a similar derivation. |
|
Thanks for adding the metadata chapter. I suggest that we now use this PR as the working integration point and complete the model here. I reviewed head The following changes are needed before the draft can become an implementable specification. Blocking implementation issues
Information-model changes requiring WG agreement
One editorial cleanup: the PR description should describe the minimum information model in vendor-neutral ONVIF terms. The abstract-class and specialization direction is useful. The main work now is to make the boundaries explicit and ensure the transport, JSON-LD, schema, and signature mechanisms can be implemented and tested consistently. |
|
Sorry @willysagefalk but we need to find a more efficient way to tackle the document, we cannot just copy&paste tons of lines from an LLM. For the future, we need IMHO to be shorter a more focused. Point 1: I will study it, it looks correct. Point 2: I have no idea right now. It's generated by the linkml tools... Point 3: I intentionally got rid of ecdsa-rdfc-2019 in order to point our specs to the ONVIF Security Baseline. Point 4: The yaml file is broken, I know it very weak, there is even a note in the AI.xml that that part will have to be removed and be put in the ai.yaml. We are simply too far now to allocate time to it. Point 5: what is the problem here? beyond a style issue, I really see no problem for the observation to be to top level container. Point 6: how is it defined in SOSA? We do not want to diverge. If this is not good enough, then we have a reason to ditch SOSA Point 7: this looks like point 5. What is the use case we are trying to solve? Point 8: I agree and in fact we are starting to work on Annex A. The problem is that before creating an example we must clarify how it will work. Therefore, the immediately actionable items are: 1, 3 and 6. |
|
I suggest extending the model with a neutral Assertion type alongside Observation, together with an ObjectObservation specialization: InformationEntity ObjectObservation represents an observation concerning one persistent PhysicalObject. It groups the Claims produced within that observation context and may include observed positions or other object-related information. Assertion is not an Observation. It expresses a proposition, relationship or conclusion and can reference Observations as supporting evidence. Statement and Claim are concrete specializations of Assertion. For example, in RE-ID: ObjectObservation A ObjectObservation B Re-identification Claim SOSAObservation remains available for SOSA interoperability, but it does not need to be the foundation of the ONVIF information model. ObjectObservation and SOSAObservation describe different aspects, so a sensor-based ObjectObservation may also be mapped to SOSA. |
|
@willysagefalk let's start clarifying the definition of the abstract classes involved in metadata streaming. Once we clarify them, we can specialize stuff like ObjectObservation, PhysicaObject and so on. My definitions:
I do have a few ideas I am working for for statement, but I do not have a clear understanding of the difference between Claim and Assertion. My idea is that, if we are talking about abstract classes, assertions, claims and statements are the same thing. If that's the case we could have To my eyes, InformationEntity is an empy container, it's just a waste of bandwidth. We do not need to wrap a rich container (Observation) with an empty conatiner (InformationEntitiy) From assertion we could derive a statement, and for the statement use the idea of the predicate we were discussing at the Budapest meeting. |
|
InformationEntity is not intended as a serialized wrapper around an Observation, so it adds no bandwidth. It is only the neutral abstract superclass that allows both Observations and Assertions to be top-level information types. An Observation represents an observation/perception context and may contain Assertions produced during that observation. An Assertion represents a proposition or conclusion and may reference one or more Observations as evidence. For example, a RE-ID Assertion that two observations concern the same physical object cannot naturally belong to only one of those observations. I agree that we should first settle these two abstract definitions. We can then clarify the precise distinction between Statement and Claim and add specializations such as ObjectObservation. |
|
I am still having difficulties to understand clearly the difference between an Assertion and an Observation I am good with "An Observation represents an observation/perception context and may contain Assertions produced during that observation". But then I see a logical problem in "An Assertion represents a proposition or conclusion and Why may? Observations may exist without Assertions, but in my understanding Assertions cannot exist outside of an Observation. It is a must, not a may in my opinion. Assertions without observation are, de facto, Observations. Do you agree on this? |
| configure and control the intelligent functions as defined in this specification. The device | ||
| provides AI configuration through the AI service. | ||
| <?oxy_custom_start type="oxy_content_highlight" color="255,255,0"?>WSDL for this service is | ||
| specified in <link xmlns:xlink="http://www.w3.org/1999/xlink" |
There was a problem hiding this comment.
We dont have wsdl for the AI service. Pls remove media.wsdl.
There was a problem hiding this comment.
This part is still untouched because we will have a yaml file. Once we agree on the content and we will have a file, I'll patch it.
| LinkML YAML source. This compiled JSON Schema is used in two places:</para> | ||
| <itemizedlist> | ||
| <listitem> | ||
| <para>First, it is embedded in the <code>inputSchema</code> field of each MCP Tool |
There was a problem hiding this comment.
If json schema is used finally, do we really need intermediate LinkML ?
There was a problem hiding this comment.
it's the same think with XSD. you need it to generate XML
| </listitem> | ||
| <listitem> | ||
| <para>Second, a copy of the compiled schema is stored in the ONVIF Client Schema | ||
| Registry after schema discovery. The registry uses the compiled schema to validate |
There was a problem hiding this comment.
Does it mean that client has to keep different schema versions for each of the camera it talks to ?
| <code>notifications/message</code> frame shall carry exactly one metadata event. Batch | ||
| delivery of multiple events in a single frame is not permitted.</para> | ||
| <para>When the Streamable HTTP channel is closed, the ONVIF Device shall retain pending | ||
| metadata events in a FIFO buffer until the buffer capacity is exhausted. When the buffer |
There was a problem hiding this comment.
Do we have to maintain this event session in the device, why cant we have something like onvif event service, where when a client reconnects it gets only the current state of property event. This will reduce the burden on device to maintain session.
|
After yet another round of discussion with @willysagefalk , we decided to:
The only critical point is statements about statements. My current proposal is that an Therefore, the class structure becomes classDiagram
class InformationItem {
<<Abstract>>
+float confidence
+Timestamp productionTime
+Provenance provenance
+Extensible extensible
}
class Timestamp {
<<Abstract>>
+Extensible extensible
}
class ExactTimestamp {
+DateTime absoluteTime
+Extensible extensible
}
class RelativeTimestamp {
+DateTime baseTime
+Integer offsetMinutes
+Extensible extensible
}
class Provenance {
<<Abstract>>
+Extensible extensible
}
class JDLSProvenance {
+Extensible extensible
}
class Location {
<<Abstract>>
+Extensible extensible
}
class GPSLocation {
+float latitude
+float longitude
+float height
+Extensible extensible
}
class Observation {
<<Abstract>>
+Location observedSpace
}
class PeopleDetectionObservation {
+string input
+Procedure procedure
+string[] featureVector
+Extensible extensible
}
class Assertion {
<<Abstract>>
+InformationItem[] supportedBy
}
class ReidentificationAssertion {
+Procedure procedure
+Extensible extensible
}
class Procedure {
+string procedureModel
+string procedureVersion
+Extensible extensible
}
class Extensible {
<<Abstract>>
+VendorExtension[] vendorExtensions
}
class VendorExtension {
+string vendor_namespace
+string name
+Any value
}
%% Core Associations
InformationItem "1" --> "1" Timestamp
InformationItem "1" --> "1" Provenance
Observation "1" --> "1" Location
Assertion "1" --> "1..*" InformationItem
PeopleDetectionObservation "1" --> "1" Procedure
ReidentificationAssertion "1" --> "1" Procedure
%% Extension Associations
Extensible "1" --> "0..*" VendorExtension : vendorExtensions
InformationItem "1" --> "0..1" Extensible
Timestamp "1" --> "0..1" Extensible
Provenance "1" --> "0..1" Extensible
Location "1" --> "0..1" Extensible
Procedure "1" --> "0..1" Extensible
%% Inheritance
InformationItem <|-- Observation
InformationItem <|-- Assertion
Timestamp <|-- ExactTimestamp
Timestamp <|-- RelativeTimestamp
Provenance <|-- JDLSProvenance
Location <|-- GPSLocation
Observation <|-- PeopleDetectionObservation
Assertion <|-- ReidentificationAssertion
%% Styles: Orange Group
style InformationItem fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Timestamp fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Provenance fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Location fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Observation fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Assertion fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Extensible fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style VendorExtension fill:#fff3e0,stroke:#f57c00,stroke-width:2px
%% Styles: Blue Group
style ExactTimestamp fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style RelativeTimestamp fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style JDLSProvenance fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style GPSLocation fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style PeopleDetectionObservation fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style ReidentificationAssertion fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style Procedure fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
As previously, orange boxes to be discussed here, blue boxes to be discussed in WG AI Let's try to see it from a JSON-LD perspective. It all starts from an graph TD
O[O<br>timestamp_o]
The we have an graph TD
A["A<br>timestamp_a"] --> O["O<br>timestamp_o"]
The sanity constraint is Then we have a statement on a statement graph TD
A1["A1<br>timestamp_a1"] --> A2["A2<br>timestamp_a2"]
A2["A2<br>timestamp_a2"] --> O["O<br>timestamp_o"]
This holds until Finally, we can have ideally more complex structure, they will always be a direct loop-less graph under the causality constraint: flowchart LR
%% Asse del tempo a SINISTRA (linea continua senza nodi intermedi)
subgraph Axis [" "]
direction BT
T_start[" "] -->|Time| T_end[" "]
end
%% Grafico principale (dall'alto verso il basso)
subgraph Graph [" "]
direction TD
A1["A1<br>timestamp_a1"] --> A2["A2<br>timestamp_a2"]
A1 --> A3["A3<br>timestamp_a3"]
A1 --> O2["O2<br>timestamp_o2"]
A2 --> O1["O1<br>timestamp_o1"]
A2 --> O3["O3<br>timestamp_o3"]
A3 --> O3["O3<br>timestamp_o3"]
end
%% Posiziona l'asse a SINISTRA del grafico
Axis ~~~ Graph
%% Stili per nascondere i nodi vuoti e i riquadri
style Axis fill:none,stroke:none
style Graph fill:none,stroke:none
style T_start fill:none,stroke:none
style T_end fill:none,stroke:none
Therefore the parser should parse the JSON as long as:
|
|
PeopleDetectionObservation is the wrong abstraction here because it classifies an observation by the analytic that produced it. “People detection” is a procedure/capability; it is not the thing being observed. ObjectObservation: evidence concerning one observed object. classDiagram
direction TB
class Observation {
<<Abstract>>
+URI id
+TemporalEntity observedTime
+URI observedSpace
+URI producedBy
+ObservationResult result
}
class ObjectObservation {
+ObjectSnapshot observedObject
}
class SpaceObservation {
+SpaceObservationResult result
}
class ObservationProcess {
+URI id
+URI procedure
+URI[] inputs
+URI producer
+DateTime executionTime
}
class Procedure {
<<Abstract>>
+string model
+string version
+Map parameters
}
class PeopleDetectionProcedure {
+string[] detectableObjectTypes
}
class ObjectSnapshot {
+URI objectId
+ObjectType objectType
+Geometry geometry
+float confidence
+string[] featureVector
}
class SpaceObservationResult {
+integer objectCount
+float occupancy
}
class Assertion {
<<Abstract>>
+URI[] supportedBy
+float confidence
+URI producedBy
}
class ReidentificationAssertion {
+URI objectA
+URI objectB
}
Observation <|-- ObjectObservation
Observation <|-- SpaceObservation
Assertion <|-- ReidentificationAssertion
Procedure <|-- PeopleDetectionProcedure
Observation --> ObservationProcess : producedBy
ObservationProcess --> Procedure : executed
ReidentificationAssertion --> ObservationProcess : producedBy
Assertion --> Observation : supportedBy
|
|
The InformationEnvelope is a bounded container for one or more observations or assertions. Its manifest records provenance and cryptographic hashes for each information item, while the signature authenticates the manifest and thereby protects the integrity and origin of all enclosed items. This is an example mermaid of how it could look like. But would like to have Björn and Nils to work on that part classDiagram
direction TB
class InformationEnvelope {
+URI id
+string formatVersion
+InformationItem content
}
class Manifest {
+URI id
+DateTime creationTime
+URI claimGenerator
}
class ManifestEntry {
+URI itemId
+string mediaType
+string hashAlgorithm
+string hashValue
}
class Signature {
+string algorithm
+string keyIdentifier
+string signatureValue
+string certificateChain
+DateTime signingTime
}
class Provenance {
+URI producer
+URI process
+URI procedure
+URI model
+string modelVersion
+URI[] inputs
}
class InformationItem {
<<Abstract>>
+URI id
+DateTime productionTime
}
class Observation {
<<Abstract>>
+TemporalEntity observedTime
+URI observedSpace
+URI producedBy
}
class Assertion {
<<Abstract>>
+float confidence
+URI[] supportedBy
+URI producedBy
}
InformationEnvelope "1" *-- "1" Manifest : manifest
InformationEnvelope "1" *-- "1..*" InformationItem : items
Manifest "1" *-- "1..*" ManifestEntry : describes
Manifest "1" *-- "1" Provenance : provenance
Manifest "1" *-- "1" Signature : claim signature
ManifestEntry --> InformationItem : binds by hash
Signature --> Manifest : authenticates
InformationItem <|-- Observation
InformationItem <|-- Assertion
|
flowchart BT
O1["ObjectObservation O1"]
O2["ObjectObservation O2"]
O3["SpaceObservation O3"]
A2["Assertion A2"]
A3["Assertion A3"]
A1["Assertion A1"]
A2 -->|supportedBy| O1
A2 -->|supportedBy| O3
A3 -->|supportedBy| O3
A1 -->|derivedFrom| A2
A1 -->|derivedFrom| A3
A1 -->|supportedBy| O2
Graph validation checks |
|
An observation should reference an ObservedSpace, not directly use Location or GPSLocation. Location provides the semantic site context—such as building, floor, room or entrance—while ObservedSpace describes the actual observable area and its geometric extent. Relative timestamps should be added only if a common temporal reference model is defined. classDiagram
direction TB
class InformationItem {
<<Abstract>>
+URI id
+DateTime productionTime
}
class Observation {
<<Abstract>>
}
class ObjectObservation {
+URI observedObject
}
class SpaceObservation {
}
class ValidTime {
<<Abstract>>
}
class Instant {
+DateTime time
}
class Interval {
+DateTime startTime
+DateTime endTime
}
class Location {
+URI id
+string name
+string locationType
}
class ObservedSpace {
+URI id
+string name
+boolean mobile
+boolean derived
}
class SpatialRegion {
<<Abstract>>
+URI id
}
class CoordinateFrame {
+URI id
+string coordinateSystem
+string orientationFrame
+string originDescription
+string unit
}
InformationItem <|-- Observation
Observation <|-- ObjectObservation
Observation <|-- SpaceObservation
Observation "1" --> "1" ValidTime : observedTime
Observation "1" --> "1" ObservedSpace : observedSpace
ValidTime <|-- Instant
ValidTime <|-- Interval
Location "0..*" --> "0..1" Location : containedIn
ObservedSpace "0..*" --> "0..1" Location : locatedIn
ObservedSpace "1" --> "0..1" SpatialRegion : extent
SpatialRegion "1" --> "1" CoordinateFrame : expressedIn
|
| <section xml:id="sec_service_discovery"> | ||
| <title>Service Discovery</title> | ||
| <para>An ONVIF Device that supports the AI Service shall advertise the MCP service endpoint | ||
| URL through the ONVIF Core service discovery mechanism. In the ONVIF Core |
There was a problem hiding this comment.
I suggest enhancing WS-Discovery by including an additional endpoint for mcp, for example:
wsd:XAddrshttp://192.168.1.100:80/mcp</wsd:XAddrs>
This way, we can avoid making a GetServices call just to determine whether MCP is supported.
There was a problem hiding this comment.
Avoiding an additional GetServices request would be useful, but adding the MCP URL as another untyped XAddr would be ambiguous. XAddrs identifies transport addresses for the discovered target service, so the client could not reliably distinguish the Device Service endpoint from the MCP endpoint or depend on their ordering.
If we want this optimization, I think ONVIF should define an explicit WS-Discovery Type, Scope, or extension that advertises AI/MCP support and maps it to the MCP endpoint. Otherwise, GetServices should remain the authoritative and interoperable mechanism for obtaining the typed AI Service endpoint.
The example should also use https://, since the MCP endpoint is protected by TLS.
There was a problem hiding this comment.
Yes @willysagefalk, I agree with your observation. It would be better if we define a specific type, scope, or distinct tag for the MCP service rather than relying on XAddrs. That way, clients could reliably distinguish the MCP endpoint.
|
@willysagefalk we need to start splitting between orange e blue parts. To get to a draft of the service, we must define the orange ones and then the rest can be modelled in WG AI. So, please, let's focus on that and the fact that they have the right extension points so that WG AI can do its work. Then, if we can have al leat a blue item for every orange class it is much better, but not necessary About your points:
|
|
Thanks, Ottavio. I agree that we should now focus on the orange foundational classes and ensure that they provide the necessary extension points for WG AI. The blue classes should primarily be examples used to validate that the foundation works.
WG AI can later specialize this into
With this split, I think the orange foundation should cover:
|
|
Reorganized architecture after today's discussion with @willysagefalk classDiagram
class InformationItem {
<<Abstract>>
+float confidence
+Timestamp productionTime
+Provenance provenance
+Signature signature
+Extensible extensible
}
class InformationEnvelope {
+InformationItem[] items
+Signature signature
}
class Signature {
<<Abstract>>
+Extensible extensible
}
class JsonLDSignature
class Timestamp {
<<Abstract>>
+Extensible extensible
}
class ExactTimestamp {
+DateTime absoluteTime
+Extensible extensible
}
class RelativeTimestamp {
+DateTime baseTime
+Integer offsetMinutes
+Extensible extensible
}
class Provenance {
<<Abstract>>
+Extensible extensible
}
class Location {
<<Abstract>>
+Extensible extensible
}
class GPSLocation {
+float latitude
+float longitude
+float height
+Extensible extensible
}
class Observation {
<<Abstract>>
+Location observedSpace
}
class PeopleDetectionObservation {
+string input
+Procedure procedure
+string[] featureVector
+Extensible extensible
}
class Assertion {
<<Abstract>>
+InformationItem[] supportedBy
}
class ReidentificationAssertion {
+Procedure procedure
+Extensible extensible
}
class Procedure {
+string procedureModel
+string procedureVersion
+Extensible extensible
}
class Extensible {
<<Abstract>>
+VendorExtension[] vendorExtensions
}
class VendorExtension {
+string vendor_namespace
+string name
+Any value
}
%% Core Associations
InformationEnvelope "1" --> "1..*" InformationItem
InformationEnvelope "1" --> "0..1" Signature
InformationItem "1" --> "1" Timestamp
InformationItem "1" --> "1" Provenance
InformationItem "1" --> "0..1" Signature
Observation "1" --> "1" Location
Assertion "1" --> "1..*" InformationItem
PeopleDetectionObservation "1" --> "1" Procedure
ReidentificationAssertion "1" --> "1" Procedure
%% Extension Associations
Extensible "1" --> "0..*" VendorExtension
InformationItem "1" --> "0..1" Extensible
Timestamp "1" --> "0..1" Extensible
Provenance "1" --> "0..1" Extensible
Location "1" --> "0..1" Extensible
Procedure "1" --> "0..1" Extensible
Signature "1" --> "0..1" Extensible
%% Inheritance
InformationItem <|-- Observation
InformationItem <|-- Assertion
Timestamp <|-- ExactTimestamp
Timestamp <|-- RelativeTimestamp
Signature <|-- JsonLDSignature
Location <|-- GPSLocation
Observation <|-- PeopleDetectionObservation
Assertion <|-- ReidentificationAssertion
%% Styles: Orange Group
style InformationItem fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style InformationEnvelope fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Signature fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style JsonLDSignature fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Timestamp fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Provenance fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Location fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Observation fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Assertion fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Extensible fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style VendorExtension fill:#fff3e0,stroke:#f57c00,stroke-width:2px
%% Styles: Blue Group
style ExactTimestamp fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style RelativeTimestamp fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style GPSLocation fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style PeopleDetectionObservation fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style ReidentificationAssertion fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style Procedure fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
Items to be put in the AI Service to nurture the discussion classDiagram
class InformationItem {
<<Abstract>>
+float confidence
+Timestamp productionTime
+Provenance provenance
+Signature signature
+Extensible extensible
}
class InformationEnvelope {
+InformationItem[] items
+Signature signature
}
class Signature {
<<Abstract>>
+Extensible extensible
}
class JsonLDSignature
class Timestamp {
<<Abstract>>
+Extensible extensible
}
class Provenance {
<<Abstract>>
+Extensible extensible
}
class Location {
<<Abstract>>
+Extensible extensible
}
class Observation {
<<Abstract>>
+Location observedSpace
}
class Assertion {
<<Abstract>>
+InformationItem[] supportedBy
}
class Extensible {
<<Abstract>>
+VendorExtension[] vendorExtensions
}
class VendorExtension {
+string vendor_namespace
+string name
+Any value
}
%% Core Associations
InformationEnvelope "1" --> "1..*" InformationItem
InformationEnvelope "1" --> "0..1" Signature
InformationItem "1" --> "1" Timestamp
InformationItem "1" --> "1" Provenance
InformationItem "1" --> "0..1" Signature
Observation "1" --> "1" Location
Assertion "1" --> "1..*" InformationItem
%% Extension Associations
Extensible "1" --> "0..*" VendorExtension
InformationItem "1" --> "0..1" Extensible
Timestamp "1" --> "0..1" Extensible
Provenance "1" --> "0..1" Extensible
Location "1" --> "0..1" Extensible
Signature "1" --> "0..1" Extensible
%% Inheritance
InformationItem <|-- Observation
InformationItem <|-- Assertion
Signature <|-- JsonLDSignature
%% Styles: Orange Group
style InformationItem fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style InformationEnvelope fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Signature fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style JsonLDSignature fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Timestamp fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Provenance fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Location fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Observation fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Assertion fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Extensible fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style VendorExtension fill:#fff3e0,stroke:#f57c00,stroke-width:2px
|
|
Reorganized my UML also. We agreed on the core part of Ottavios post. classDiagram
class InformationEnvelope {
+EnvelopeId id
+Timestamp productionTime
+Provenance packagingProvenance
+InformationItem[] items
+Manifest manifest
+Signature envelopeSignature
+Extensible extensible
}
class Manifest {
+ItemDescriptor[] entries
+string canonicalizationMethod
+Extensible extensible
}
class ItemDescriptor {
+InformationItemRef item
+string itemType
+HashValue contentHash
}
class InformationItem {
<<Abstract>>
+ItemId id
+Timestamp productionTime
+Provenance provenance
+Signature itemSignature
+Extensible extensible
}
class InformationItemRef {
+ItemId id
+URI envelopeLocator
}
class Signature {
<<Abstract>>
+string signatureScheme
+VerificationMethodRef verificationMethod
+IRI targetId
+HashValue signedHash
+bytes signatureValue
+Extensible extensible
}
class Timestamp {
<<Abstract>>
+Extensible extensible
}
class Provenance {
<<Abstract>>
+ProducerRef producer
+Procedure procedure
+InformationItemRef[] inputs
+Extensible extensible
}
class Procedure {
<<Abstract>>
+IRI id
+string version
+Extensible extensible
}
class Location {
<<Abstract>>
+IRI id
+CoordinateFrameRef coordinateFrame
+Extensible extensible
}
class ObservationResult {
<<Abstract>>
+Extensible extensible
}
class Observation {
<<Abstract>>
+Timestamp observedTime
+Location observedSpace
+ObservationResult result
}
class Statement {
<<Abstract>>
+Extensible extensible
}
class Assertion {
<<Abstract>>
+float confidence
+Statement statement
+InformationItemRef[] basedOn
}
class Extensible {
+Extension[] extensions
}
class Extension {
+IRI type
+URI schema
+JsonValue value
}
class DataIntegrityProof {
+string cryptosuite
+string proofPurpose
+DateTime created
+string proofValue
}
class ExactTimestamp {
+DateTime absoluteTime
}
class RelativeTimestamp {
+ClockRef clock
+Integer offsetTicks
+Integer ticksPerSecond
}
class GPSLocation {
+float latitude
+float longitude
+float height
}
class ObservedSpaceLocation {
+ObservedSpaceRef observedSpace
}
class AIModelProcedure {
+ModelRef model
+string modelVersion
+Parameter[] parameters
}
class ObjectObservation {
+ObjectObservationResult result
}
class ObjectObservationResult {
+PhysicalObjectRef object
+Classification classification
+Geometry geometry
+float confidence
+Uncertainty uncertainty
+FeatureVectorRef featureVector
}
class SpaceObservation {
+SpaceObservationResult result
}
class SpaceObservationResult {
+IRI observedProperty
+JsonValue value
+float confidence
+Uncertainty uncertainty
}
class IdentityStatement {
+EntityRef subjectA
+IRI relationship
+EntityRef subjectB
}
class ReidentificationAssertion {
+IdentityStatement statement
+PhysicalObjectRef resolvedIdentity
}
%% Envelope containment and authenticity
InformationEnvelope "1" *-- "1..*" InformationItem : contains
InformationEnvelope "1" *-- "1" Manifest : describes
Manifest "1" *-- "1..*" ItemDescriptor : entries
ItemDescriptor "1" --> "1" InformationItemRef : identifies
InformationEnvelope "1" --> "1" Provenance : packaging provenance
InformationEnvelope "1" --> "0..1" Signature : envelope signature
InformationItem "1" --> "1" Provenance : item provenance
InformationItem "1" --> "0..1" Signature : item signature
%% Provenance and graph references
Provenance "1" --> "0..1" Procedure : used procedure
Provenance "1" --> "0..*" InformationItemRef : input references
Assertion "1" --> "1..*" InformationItemRef : based on
%% Observation and assertion structure
Observation "1" *-- "1" Timestamp : observed time
Observation "1" --> "1" Location : observed space
Observation "1" *-- "1" ObservationResult : result
Assertion "1" *-- "1" Statement : proposition
%% Extension mechanism
Extensible "1" *-- "0..*" Extension : extensions
%% Abstract inheritance
InformationItem <|-- Observation
InformationItem <|-- Assertion
Signature <|-- DataIntegrityProof
Timestamp <|-- ExactTimestamp
Timestamp <|-- RelativeTimestamp
Location <|-- GPSLocation
Location <|-- ObservedSpaceLocation
Procedure <|-- AIModelProcedure
ObservationResult <|-- ObjectObservationResult
ObservationResult <|-- SpaceObservationResult
Observation <|-- ObjectObservation
Observation <|-- SpaceObservation
Statement <|-- IdentityStatement
Assertion <|-- ReidentificationAssertion
%% Concrete specialization constraints
ObjectObservation "1" --> "1" ObjectObservationResult : result type
SpaceObservation "1" --> "1" SpaceObservationResult : result type
ReidentificationAssertion "1" --> "1" IdentityStatement : statement type
%% Orange foundational classes
style InformationEnvelope fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Manifest fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style ItemDescriptor fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style InformationItem fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style InformationItemRef fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Signature fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Timestamp fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Provenance fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Procedure fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Location fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style ObservationResult fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Observation fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Statement fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Assertion fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Extensible fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Extension fill:#fff3e0,stroke:#f57c00,stroke-width:2px
%% Blue concrete examples
style DataIntegrityProof fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style ExactTimestamp fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style RelativeTimestamp fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style GPSLocation fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style ObservedSpaceLocation fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style AIModelProcedure fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style ObjectObservation fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style ObjectObservationResult fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style SpaceObservation fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style SpaceObservationResult fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style IdentityStatement fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style ReidentificationAssertion fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
|
|
Quick question @willysagefalk , why did you add orange boxes? For me an orange box is a box to be defined in this PR and/or effort, while a blue box is to be defined by WG AI. Therefore:
Should me blue in my opinion. Remember the goal: we want to have a draft that we can share in Prague, without depriving WG AI of its working topics. |
|
Corrected blue/orange classDiagram
class InformationEnvelope {
+EnvelopeId id
+Timestamp productionTime
+Provenance packagingProvenance
+InformationItem[] items
+Manifest manifest
+Signature envelopeSignature
+Extensible extensible
}
class Manifest {
+ItemDescriptor[] entries
+string canonicalizationMethod
+Extensible extensible
}
class ItemDescriptor {
+InformationItemRef item
+string itemType
+HashValue contentHash
}
class InformationItem {
<<Abstract>>
+ItemId id
+Timestamp productionTime
+Provenance provenance
+Signature itemSignature
+Extensible extensible
}
class InformationItemRef {
+ItemId id
+URI envelopeLocator
}
class Signature {
<<Abstract>>
+string signatureScheme
+VerificationMethodRef verificationMethod
+IRI targetId
+HashValue signedHash
+bytes signatureValue
+Extensible extensible
}
class Timestamp {
<<Abstract>>
+Extensible extensible
}
class Provenance {
<<Abstract>>
+ProducerRef producer
+Procedure procedure
+InformationItemRef[] inputs
+Extensible extensible
}
class Procedure {
<<Abstract>>
+IRI id
+string version
+Extensible extensible
}
class Location {
<<Abstract>>
+IRI id
+CoordinateFrameRef coordinateFrame
+Extensible extensible
}
class ObservationResult {
<<Abstract>>
+Extensible extensible
}
class Observation {
<<Abstract>>
+Timestamp observedTime
+Location observedSpace
+ObservationResult result
}
class Statement {
<<Abstract>>
+Extensible extensible
}
class Assertion {
<<Abstract>>
+float confidence
+Statement statement
+InformationItemRef[] basedOn
}
class Extensible {
+Extension[] extensions
}
class Extension {
+IRI type
+URI schema
+JsonValue value
}
class DataIntegrityProof {
+string cryptosuite
+string proofPurpose
+DateTime created
+string proofValue
}
class ExactTimestamp {
+DateTime absoluteTime
}
class RelativeTimestamp {
+ClockRef clock
+Integer offsetTicks
+Integer ticksPerSecond
}
class GPSLocation {
+float latitude
+float longitude
+float height
}
class ObservedSpaceLocation {
+ObservedSpaceRef observedSpace
}
class AIModelProcedure {
+ModelRef model
+string modelVersion
+Parameter[] parameters
}
class ObjectObservation {
+ObjectObservationResult result
}
class ObjectObservationResult {
+PhysicalObjectRef object
+Classification classification
+Geometry geometry
+float confidence
+Uncertainty uncertainty
+FeatureVectorRef featureVector
}
class SpaceObservation {
+SpaceObservationResult result
}
class SpaceObservationResult {
+IRI observedProperty
+JsonValue value
+float confidence
+Uncertainty uncertainty
}
class IdentityStatement {
+EntityRef subjectA
+IRI relationship
+EntityRef subjectB
}
class ReidentificationAssertion {
+IdentityStatement statement
+PhysicalObjectRef resolvedIdentity
}
%% Envelope containment and authenticity
InformationEnvelope "1" *-- "1..*" InformationItem : contains
InformationEnvelope "1" *-- "1" Manifest : describes
Manifest "1" *-- "1..*" ItemDescriptor : entries
ItemDescriptor "1" --> "1" InformationItemRef : identifies
InformationEnvelope "1" --> "1" Provenance : packaging provenance
InformationEnvelope "1" --> "0..1" Signature : envelope signature
InformationItem "1" --> "1" Provenance : item provenance
InformationItem "1" --> "0..1" Signature : item signature
%% Provenance and graph references
Provenance "1" --> "0..1" Procedure : used procedure
Provenance "1" --> "0..*" InformationItemRef : input references
Assertion "1" --> "1..*" InformationItemRef : based on
%% Observation and assertion structure
Observation "1" *-- "1" Timestamp : observed time
Observation "1" --> "1" Location : observed space
Observation "1" *-- "1" ObservationResult : result
Assertion "1" *-- "1" Statement : proposition
%% Extension mechanism
Extensible "1" *-- "0..*" Extension : extensions
%% Abstract inheritance
InformationItem <|-- Observation
InformationItem <|-- Assertion
Signature <|-- DataIntegrityProof
Timestamp <|-- ExactTimestamp
Timestamp <|-- RelativeTimestamp
Location <|-- GPSLocation
Location <|-- ObservedSpaceLocation
Procedure <|-- AIModelProcedure
ObservationResult <|-- ObjectObservationResult
ObservationResult <|-- SpaceObservationResult
Observation <|-- ObjectObservation
Observation <|-- SpaceObservation
Statement <|-- IdentityStatement
Assertion <|-- ReidentificationAssertion
%% Concrete specialization constraints
ObjectObservation "1" --> "1" ObjectObservationResult : result type
SpaceObservation "1" --> "1" SpaceObservationResult : result type
ReidentificationAssertion "1" --> "1" IdentityStatement : statement type
%% Orange: foundation proposed for the Prague draft
style InformationEnvelope fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style InformationItem fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Signature fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Timestamp fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Provenance fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Procedure fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Location fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Observation fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Assertion fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Extensible fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style Extension fill:#fff3e0,stroke:#f57c00,stroke-width:2px
%% Blue: definitions and specializations for continued WG AI work
style Manifest fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style ItemDescriptor fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style InformationItemRef fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style Statement fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style ObservationResult fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style DataIntegrityProof fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style ExactTimestamp fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style RelativeTimestamp fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style GPSLocation fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style ObservedSpaceLocation fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style AIModelProcedure fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style ObjectObservation fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style ObjectObservationResult fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style SpaceObservation fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style SpaceObservationResult fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style IdentityStatement fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
style ReidentificationAssertion fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
|
|
So @willysagefalk after you last change I think we have one last pending task before we focus on the document here and how it integrates with the rest of the ONVIF specs: how do we hand the work on the blue boxes over to WG AI with order? I expect the disucssion to be big, I think we should parition the blu boxes in multiple issues in the repo of WG AI. Do you have a proposal for splitting the discussion? PS: in your schema I think there is a weakness In your definition of So, first of all you have a conflict between the base class and the derived classes. Second, if you keep My proposal is to remove the generic |
|
Yes. I suggest one tracking issue preserving the agreed orange architecture, with four blue work items: 1. Observation and Assertion information modelDefine Observation and Assertion together because their semantics and graph relationships must be consistent. Scope includes:
2. Time, Space and TopologyDefine the shared temporal and spatial model used by Observations and Assertions. Scope includes:
3. InformationEnvelope, Authenticity and ProvenanceDefine how individual InformationItems and collections of items are bound, authenticated and traced to their origin. Scope includes:
4. SOSA/SSN InteroperabilityDefine SOSA as an optional interoperability mapping or specialization. Scope includes:
Suggested order
Each issue should identify its expected LinkML output, dependencies and at least one end-to-end example. The tracking issue can then show how the four work packages assemble into the complete WG AI information model. |
|
The foundation should define only AuthenticityProof. The WG shall evaluate at least an embedded W3C Data Integrity proof and a manifest-based proof. Data Integrity is suitable for independently protected JSON-LD items; a manifest-based profile is likely better for batched envelopes, extraction, provenance chains and multiple producers. |
I disagree on this, I see no value in having two mechanisms of the same thing, Verifiable Credential Data Integrity already exists and does not need to be invented from scratch. The manifest approach is chatter, not exsisting and most important, it is not clear what problem solves that Verifiable Credential Data Integrity does not. Anyway, as discussed in today's telco, I will open a work item in video enhancement. WG AI does not have the compenecy in cryptography that we need to discuss this |
4 participants
Initial draft pull request to gather feedback on the AI specs being written right now.
Current status
AI.xmlis been writtenSecurity.xmlTentative scheduling
Topics to be included
Work items generated during this discussion
I expect this PR to be used as a shared notepad about the specs begin developed. Once ready, this will be closed a new a formal PR will be created.