feat: Configurable OCSF schema version for SIEM compatibility (v1.1/v1.3)

Problem

OpenShell hardcodes OCSF_VERSION = "1.7.0" in crates/openshell-ocsf/src/lib.rs. The two largest OCSF consumers in the market expect earlier schema versions:

AWS Security Lake: requires OCSF v1.1.0
Splunk OCSF-CIM Add-On (splunkbase.splunk.com/app/6943): targets v1.1.0-v1.3.0
CrowdStrike Falcon Next-Gen SIEM: ingests via Security Lake (v1.1.0)

OCSF v1.7.0 is not backward-compatible with v1.1.0 (field renames, new required fields, structural changes). Enterprise customers with existing OCSF-based security automation cannot use OpenShell events without building a custom transformation layer.

What works vs. what doesn't

In a PoC integrating OpenShell OCSF events with Splunk:

What worked: Posting v1.7.0 events as raw JSON to Splunk HEC (sourcetype=ocsf) and building custom SPL dashboards that query field names directly (class_uid, sandbox_id, dst_endpoint.hostname, etc.). Splunk HEC accepts any JSON regardless of schema version.
What didn't work: The Splunk OCSF-CIM Add-On could not map v1.7.0 events to CIM data models. The Add-On expects v1.1.0-v1.3.0 field names and structure. This means customers with existing OCSF-CIM dashboards, correlation rules, or automated workflows built on the Add-On cannot use OpenShell events without either (a) custom dashboards that bypass CIM, or (b) an external transformation layer.

The distinction matters because "events appear in Splunk" is not the same as "events integrate with the customer's existing OCSF automation." The raw JSON approach works for a PoC but breaks in production environments where security teams have invested in CIM-based workflows.

Proposed Solution

Add a gateway-level configuration option to control the OCSF schema version used for event emission:

[openshell.gateway]
ocsf_schema_version = "1.1"  # or "1.3", "1.7" (default)

When set to an older version:

Downgrade field names to match the target schema (e.g., renamed fields between versions)
Omit fields that don't exist in the target version
Set metadata.version to the configured version

The default should remain the latest version (currently 1.7.0).

Alternatives Considered

External transformation layer (Cribl, custom Lambda): Works but adds operational complexity and cost for every deployment. The transformation should happen at the source.
Wait for SIEMs to catch up: Splunk and AWS move slowly on OCSF version adoption. v1.7.0 support may be months or years away.
Custom dashboards over raw JSON: Works for PoCs but doesn't integrate with existing CIM-based automation, correlation rules, or compliance workflows.

Impact

Enterprise deployments evaluating OpenShell for production need SIEM integration. The top 3 OCSF consumers all require v1.1.0. This blocks adoption unless operators build custom dashboards (losing CIM integration) or deploy external transformation pipelines.

Is there an opportunity to engage with AWS and Splunk here? What are they going to do when AI related events need to be supported? Having to support sufficiently older schemas feels like a step backwards.

@johntmyers

@johntmyers We could, but that would usually take longer, and we know customers don't upgrade as fast even if support for newer schemas is introduced, so they will want something that works today. We will look more into versioning

feat(ocsf): emit AI inference events via ai_operation profile on Http…

Scoping: Field Compatibility Matrix

Did a field-by-field diff across v1.1.0, v1.3.0, and v1.8.0 (our current schema). Here's what a downgrade filter actually needs to do for the fields OpenShell emits today.

Fields we emit vs. schema availability:

Field	v1.1.0	v1.3.0	v1.8.0	Action for v1.1/v1.3
`class_uid`, `class_name`, `category_uid`, `category_name`	yes	yes	yes	passthrough
`activity_id`, `activity_name`, `type_uid`, `type_name`	yes	yes	yes	passthrough
`time`, `severity_id`, `severity`	yes	yes	yes	passthrough
`status_id`, `status`, `message`, `status_detail`	yes	yes	yes	passthrough
`metadata` (version, product, profiles)	yes	yes	yes	rewrite `metadata.version` to target version
`device`	on class	on class	profile-gated	passthrough (exists in all versions, just moved to profile in v1.7+)
`container`	no	no	profile-gated	strip (not in v1.1 or v1.3 schema)
`ai_model`	no	no	profile-gated (v1.8+)	strip
`unmapped`	yes	yes	yes	passthrough
`actor`, `dst_endpoint`, `src_endpoint`	yes	yes	yes	passthrough
`http_request`, `http_response`	yes	yes	yes	passthrough
`firewall_rule`	yes	yes	yes (profile)	passthrough
`proxy_endpoint`	yes	yes	yes (profile)	passthrough
`observation_point_id`	no	no	v1.8+	strip
`ai_operation` in profiles list	no	no	v1.8+	strip from profiles array

Summary: the downgrade is small. For v1.1/v1.3 output, the filter needs to:

Strip ai_model, container, observation_point_id fields
Remove ai_operation and container from metadata.profiles
Set metadata.version to the target version

No field renames needed. The core event structure (class UIDs, activity IDs, HTTP fields, network fields) is identical across all three versions. The differences are all profile-gated additions.

Implementation approach: A fn downgrade_event(json: &mut serde_json::Value, target: &str) in the JSONL layer that runs after serialization and before writing. ~50 lines of code. Gateway config ocsf_schema_version flows to the sandbox via the settings bundle (same path as ocsf_json_enabled).

Reference: PR #2664 moved AI inference events from a custom class (7001) to the official ai_operation profile on HttpActivity (4002), and bumped the vendored schema to v1.8.0. The downgrade filter from this issue would strip the profile fields when targeting older versions, leaving a clean HttpActivity event that v1.1 SIEMs can consume.

@zanetworker

Nice analysis @zanetworker. Would 1.8 be the new default vs 1.7? Overall no issues with the downgrade approach.

@johntmyers

@johntmyers yes, the default would be 1.8 then with configurability for older release via gateway config for customers whose SIEMs need older schemas

feat(ocsf): emit AI inference events via ai_operation profile on ApiA…

feat(ocsf): configurable schema version for SIEM backward compatibility

@zanetworker

I had my agent read this issue and reword it based on an updated template for issues I'm working on; @zanetworker can you review and see if this captures the issue appropriately?

User Story

As a security or platform operator integrating OpenShell events with an OCSF-based SIEM, I want to select an OCSF schema version supported by my security platform so that OpenShell events work with my existing security operations workflows.

Problem Statement

OpenShell emits OCSF events using only its latest supported schema version. Operators cannot select an earlier schema version required by integrations such as AWS Security Lake or the Splunk OCSF-CIM Add-On. Although these platforms may accept the events as raw JSON, their schema-aware mappings cannot process them correctly.

Impact / Why This Matters

Without configurable schema output, security teams must deploy and maintain an external transformation pipeline or build custom dashboards that bypass their existing OCSF and CIM integrations. These workarounds add operational cost, duplicate established field mappings, and prevent OpenShell events from participating in existing correlation rules, compliance reporting, alerts, and automated response workflows.

Waiting for every downstream platform and customer deployment to adopt the latest OCSF schema does not address current production requirements. For organizations that standardize on an older supported schema, this incompatibility can block production adoption of OpenShell.

Proposed Design

Add a gateway configuration option that selects the OCSF schema version used for structured event output:

  [openshell.gateway]
  ocsf_schema_version = "1.1"

OpenShell should support the schema versions needed by major downstream integrations, initially "1.1", "1.3", and the latest schema supported by OpenShell. If the setting is omitted, OpenShell should continue using its latest supported schema.

OpenShell should reject unsupported values with an actionable configuration error that identifies the invalid value and lists the supported versions. Selecting an older version must produce events that validate against that schema, rather than only changing the version reported in event metadata.

Acceptance Criteria

After configuring the gateway:

All OpenShell OCSF events produced by sandboxes managed by that gateway conform to the selected schema.
Event metadata reports the selected schema version.
Information unavailable in the selected schema is omitted while supported core event information remains available.

feat(ocsf): emit AI inference events via ai_operation profile on ApiA…

@krishicks

@krishicks looks good! Would you like me to change the description of my issue accoringly?

feat(ocsf): emit AI inference events via ai_operation profile on ApiA…

feat(ocsf): configurable schema version for SIEM backward compatibility

feat(ocsf): configurable schema version for SIEM backward compatibili…

Navigation Menu

Repository navigation

feat: Configurable OCSF schema version for SIEM compatibility (v1.1/v1.3) #2662

Problem

What works vs. what doesn't

Proposed Solution

Alternatives Considered

Impact

Scoping: Field Compatibility Matrix

User Story

Problem Statement

Impact / Why This Matters

Proposed Design

Acceptance Criteria

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

feat: Configurable OCSF schema version for SIEM compatibility (v1.1/v1.3) #2662

Description

Problem

What works vs. what doesn't

Proposed Solution

Alternatives Considered

Impact

Activity

johntmyers commented on Aug 9, 2026

zanetworker commented on Aug 10, 2026

zanetworker commented on Aug 10, 2026

Scoping: Field Compatibility Matrix

johntmyers commented on Aug 11, 2026

zanetworker commented on Aug 11, 2026

krishicks commented on Aug 13, 2026

User Story

Problem Statement

Impact / Why This Matters

Proposed Design

Acceptance Criteria

zanetworker commented on Aug 15, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions