Skip to content

feat(provider-tck): inject the harness and backend control through our own ObjectFactory - #1926

Merged
aepfli merged 6 commits into
chore/tck-seamsfrom
chore/tck-object-factory
Oct 17, 2026
Merged

aepfli merged 6 commits into
chore/tck-seamsfrom
chore/tck-object-factory

Conversation

@aepfli

@aepfli aepfli commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

Draft, stacked on #1925 — the shape works, but the trade is a judgement call worth arguing before it merges.

Six commits, all aimed at the public API surface. A step class asks for what it needs:

public MySteps(TckState state, ProviderTckHarness harness, BackendControl backend) { … }

1. Our own ObjectFactory

PicoFactory offers no way to register an instance, and its isInstantiable refuses anything abstract, so an interface-typed parameter can never be satisfied. It is final, and delegating to it does not help either, since nothing can be added to its container. ObjectFactory is @API(STABLE); we register through META-INF/services, and the suite already selected the factory explicitly.

Scoping. Container per scenario, as the SPI specifies — TckState has no reset method, so the fresh container is the only thing keeping scenario N's thrown and events out of N+1. It holds four step instances and that state, no I/O. The harness, the backend control and the Compose stack are suite-scoped and registered by reference; stop() disposes the container and never touches them. That is why two flagd suites share one JVM with a Compose stack each.

What it costs. Three stock behaviours had to be reproduced, each found by a failing test rather than by reading the source:

  • addClass registers constructor dependency types recursively — TckState is a parameter, not glue (126 UnsatisfiableDependencies).
  • A run may have no suite lifecycle at all — ExtensionPointTest selects only the extension glue.
  • Cucumber's component rule: public, concrete, not an inner class.

All three failed loudly, which is the good case; the risk worth weighing is the quiet variant, a dependency resolving to the wrong instance. Separately, an adopter who sets cucumber.object-factory themselves silently loses the injection.

It does not buy package-private step classes: MethodScanner applies its own public check during discovery, before the factory is consulted. GlueVisibilityTest from #1925 stays necessary.

2. The gate folds into Capability

CapabilityGate is gone; its four rules are Capability.requireDeclared(uri, tags, declared) — one public entry point, beside requireDeclarable(). The tag-only overload is package-private, as only tests in this package use it.

The reason to fold rather than move it to internal.steps: the rules read fromTag(), reserved(), tag(), inexpressible() and inexpressibleBecause(). A gate outside this package needs all five public — five leaked accessors instead of one entry point. (Relocating the two test classes would be mechanical; that is not what decides it.)

ProviderTck.gateScenario() goes with it, since ProviderSteps now receives the harness.

3. The lifecycle moved instead of going public

ProviderTck is three public constants and a private constructor — 192 lines to 48. startIfNeeded(), stop(), isRunning(), harness(), backendControl() and harness discovery are a package-private TckLifecycle in internal.steps, the package both callers already live in: ProviderSteps' @BeforeAll/@AfterAll, and the object factory, which moved there for that reason and stays public only because ServiceLoader requires it. No new public members.

Not driven from the ObjectFactory, though that looks like its natural home: start()/stop() are per scenario, so it could own construction but never suite teardown, and it cannot tell "suite not started yet" from "no suite wanted" — a launcher selecting only the extension glue would start Docker and then fail discovery. Hence the isRunning() guard rather than an eager lookup.

4. Capability narrowed to what adopters call

tag(), reserved(), inexpressible() and fromTag() are package-private; their only callers are KnownDeviation and the tests, both in this package. Across the four adoptions, the one method anything outside calls is declarableExcept().

Eight public members down to four: declarable() and declarableExcept() for harnesses, and requireDeclarable()/requireDeclared(), public only because internal.steps is a different package.

Free only pre-release — tck is not on Maven Central yet. Promoting a member later is source-compatible; demoting is not. This commit has to be reverted if the gate moves out of Capability.

5. A test was holding harnessClassOf open

TckSuiteListener.harnessClassOf() was public for one line in ExtensionPointTest, which borrowed it to find the suite node in a discovered plan. The listener calls it only from its own two hooks, so it is private now and the test names TckSuiteFixture itself.

Moving the test would have cost more than it saved -- it also reads three package-private root-package constants, so relocating it to internal trades one public method for three public constants. Subclassing could not have helped either: a static cannot be widened by inheritance.

Sweeping every public member in main for test-only callers turned up five candidates; four were false positives, since composeFile, declarableExcept, tracked and untracked are called by the adoptions.

6. ControlApi becomes BackendType

The old name described the thing by the API that drives it. What it actually says is whether the run drove a real backend over the control API or a provider with no backend inside this process -- a backend type. So the enum is BackendType and controlApi() is backendType(), and it stays public, because BackendControl can be implemented elsewhere and such an implementation has to declare which kind it is.

The enum being closed is what makes that safe: an implementor picks one of two values rather than inventing a third no report can validate. A public closed enum is therefore the whole answer -- there is no gap left for a sealed type to close, which matters because this module targets release 11.

It is not redundant with InProcessControl, which means in-process and supplies the provider. ControllableBackendControl in this module is in-process without being one, so deriving the value from the runtime type would report "http" for an in-process run -- a silently invalid conformance claim. wireValue() stays public with it, as the @JsonValue hook the report serialises through.

The report's JSON key is unaffected, coming from the field name on ConformanceReport.Backend. feat/provider-tck-report carries its own copy of the enum and a call site in TckRuntime, and picks the rename up there. Two javadoc comments also still sent adopters to the package-private HttpBackendControl; they point at BackendControl#overHttp now.

What is public, after all six

Read off javap over the compiled classes, not off the source. Outside the glue the module exposes thirteen types, all in dev.openfeature.contrib.tools.tck:

role types
extend one of these ProviderTckTest, ContainerizedProviderTckTest (or implement ProviderTckHarness)
declare with these Capability (4 members), KnownDeviation
the backend seam BackendControl, InProcessControl, BackendType, BackendEndpoint
reachable from step classes TckState, FlagUnderTest, ProviderEventRecord
glue composition ProviderTck -- three constants

Two more are public only structurally, both under internal: TckSuiteListener, whose type ServiceLoader requires and whose currentSuite() is read from internal.steps, and TckValues.convert(), crossing the same boundary.

Package-private, so invisible to an adopter: TckLifecycle, HttpBackendControl, InProcessBackendControl, CanonicalFlags, ReportNames.

Verification

  • mvn -pl tools/tck test javadoc:javadoc — 250 tests, 0 failures on each commit, including ExtensionPointTest's no-suite launch.
  • flagd, testbed v3.10.3, with tools/tck built in the reactor: both suites 65 scenarios, 59 passed, 2 skipped, 4 failed — the four this adoption already documents, the numeric-coercion gap and the two flags the testbed does not serve, identical across both resolvers.

An earlier run of mine used -pl providers/flagd, which resolves tools/tck from the local repository instead of the reactor: it exercised a stale jar and proved nothing. The numbers above are therefore a first measurement, not a confirmation, and there is no base-branch number taken the same way yet. That is the last thing I want before this leaves draft.

Still open

internal.steps is what forces requireDeclarable() and requireDeclared() public, and the same boundary forced five members public in #1923. A module-info.java, or collapsing the package into the one it serves, is the real fix — worth deciding before this merges rather than after.

extensions/README.md documented ProviderTck.harness() and backendControl(), which commit 1 makes package-private; it now documents the constructor injection that replaced them.

@coderabbitai

coderabbitai Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2450b577-a924-493b-8694-4887862d7de8

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

…r own ObjectFactory

A step class now asks for what it needs:

    public MySteps(TckState state, ProviderTckHarness harness, BackendControl backend)

Cucumber's stock PicoFactory cannot do this. It registers glue classes and their
constructor dependencies, but offers no way to register an instance, and its
isInstantiable refuses anything abstract -- so an interface-typed parameter can
never be satisfied and a step class had to reach the harness through a static
accessor. It is also final, so there is nothing to subclass, and wrapping it by
delegation does not help: nothing can be added to its container.

ObjectFactory is @API(STABLE) and its javadoc says the factory "facilitates the
creation of both the glue classes and dependencies", so this is the extension
point used as documented rather than a component replaced on a hunch. Registered
via META-INF/services and selected by ProviderTck.OBJECT_FACTORY, which the
suite already set explicitly.

ProviderTck.harness() and backendControl() are package-private as a result: the
factory uses them, the steps receive them. Declared capabilities need no
registration of their own, being capabilities() on the injected harness.

Three behaviours of the stock factory had to be reproduced, each found by a
failing test rather than by reading it:

  * addClass registers constructor dependency types recursively. TckState is not
    glue, it is a parameter of every step class, so without this every one failed
    with UnsatisfiableDependencies -- 126 errors.
  * a run can have no suite lifecycle at all. ExtensionPointTest launches
    Cucumber with only the extension glue and never reaches @BeforeAll, so
    resolving the harness unconditionally in start() killed every scenario in it.
    Registration is now conditional on a suite running.
  * Cucumber's own rule for a component -- public, concrete, not an inner class
    -- which is mirrored rather than reinvented.

What this does NOT buy is package-private step classes. MethodScanner applies its
own public check during discovery, before the factory is consulted, so owning
instantiation does not get us discovery. GlueVisibilityTest stays.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
@aepfli
aepfli force-pushed the chore/tck-object-factory branch 2 times, most recently from 5602a8d to 3e3f88b Compare October 16, 2026 22:57
CapabilityGate is gone. Its four rules are Capability.requireDeclared(), beside
requireDeclarable(), which was already there and does the same kind of job.

Every rule the gate applies is about this vocabulary: a tag unknown to the enum,
a tag marked reserved, a capability the SDK cannot express, a capability the
provider did not declare. It read inexpressibleBecause() to say why a skip is the
SDK's limit rather than the provider declining, and that accessor is one only
Capability should expose -- so a gate living anywhere else has to be handed it
across a package boundary. Inside Capability the question does not arise.

ProviderTck.gateScenario() goes with it: ProviderSteps already receives the
harness, so it calls Capability.requireDeclared(uri, tags, harness().
capabilities()) directly. ProviderTck is now three constants plus startIfNeeded()
and stop().

Deliberately a static on an enum rather than a default method on
ProviderTckHarness, which would read better and would be wrong: an interface
cannot have a final method, so the adopter -- the party these rules police --
could override it away, and two of the four fail the run rather than skipping.
That is recorded at the fold, because moving the gate back out is a plausible
suggestion that would quietly reintroduce the public accessor.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
…erves

ProviderTck is now three public constants. startIfNeeded(), stop(),
isRunning(), harness() and backendControl() are a package-private
TckLifecycle in internal.steps, the package both callers already live
in: ProviderSteps' @BeforeAll/@afterall, and the object factory, which
moved there for the same reason and stays public only because
ServiceLoader requires it.

Not driven from the ObjectFactory instead: start()/stop() are per
scenario, so it could own construction but not suite teardown, and it
cannot tell "suite not started yet" from "no suite wanted" -- a launcher
selecting only the extension glue would start Docker and fail
discovery.

OBJECT_FACTORY and META-INF/services name the new FQN. That constant is
package-private and only the TCK's own suite classes read it, so no
adopter recompilation is implied.

extensions/README.md documented ProviderTck.harness() and
backendControl(), which went package-private one commit ago; it now
documents the constructor injection that replaced them.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
@aepfli
aepfli force-pushed the chore/tck-object-factory branch from 3e3f88b to b41ddc0 Compare October 16, 2026 23:00
tag(), reserved(), inexpressible() and fromTag() are package-private.
Their only callers are KnownDeviation and the tests, both in this
package. Across the four adoptions the single method anything outside
calls is declarableExcept(); the README documents that and declarable().

Capability now exposes four members instead of eight: declarable() and
declarableExcept() for harnesses, and requireDeclarable() plus
requireDeclared(), which are public only because internal.steps is a
different package.

This is what the gate fold buys, and the reason not to move the gate
out: a gate in internal.steps would need fromTag(), reserved(), tag(),
inexpressible() and inexpressibleBecause() public -- five accessors
rather than one entry point.

Free only before release: the artifact is not on Maven Central yet
(404, while a released sibling in the same group resolves). Promoting a
member later is source-compatible; demoting one is not.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
The listener calls harnessClassOf() only from its own two hook methods;
it was public for one line in ExtensionPointTest, which used it to find
the suite node in a discovered plan. The test knows which class it
discovered, so it now says so itself and the method is private.

Leaves TckSuiteListener with currentSuite(), which TckLifecycle reads
across the internal.steps boundary, and the two TestExecutionListener
overrides the SPI requires.

Not done to ControlApi.wireValue(), which the same sweep flags as having
no caller but Jackson's @jsonvalue: the report module holds the enum and
never calls it, so demoting rests on Jackson honouring the annotation on
a non-public method -- provable on feat/provider-tck-report, where
ConformanceReportPluginTest exists, not here.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
@aepfli
aepfli force-pushed the chore/tck-object-factory branch 2 times, most recently from 4585ae1 to 35f2949 Compare October 17, 2026 06:38
ControlApi named the thing by the API that drives it rather than by what
it is: either a real backend reached over the control API, or a provider
with no backend driven inside this process. That is a backend type, so
the enum is BackendType and controlApi() is backendType().

Public, because BackendControl can be implemented elsewhere and such an
implementation has to say which kind it is. The enum being closed is what
makes that safe -- an implementor picks one of two values rather than
inventing a third that no report can validate -- so nothing is left for a
sealed type to close.

The enum is not redundant with InProcessControl, which means in-process
*and* supplies the provider: ControllableBackendControl in this module is
in-process without being one, so deriving the value from the type would
report "http" for an in-process run.

The report's JSON key is unaffected, coming from the field name on
ConformanceReport.Backend, which this does not touch.
feat/provider-tck-report carries its own copy of the enum and a call site
in TckRuntime, and picks the rename up there.

Two javadoc comments still sent adopters to HttpBackendControl, package-
private since an earlier commit; they point at BackendControl#overHttp now.

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
@aepfli
aepfli force-pushed the chore/tck-object-factory branch from cc2fd49 to 84dcf4a Compare October 17, 2026 06:46
@aepfli
aepfli marked this pull request as ready for review October 17, 2026 09:16
@aepfli
aepfli requested a review from a team as a code owner October 17, 2026 09:16
@aepfli
aepfli merged commit ff87500 into chore/tck-seams Oct 17, 2026
5 checks passed
@aepfli
aepfli deleted the chore/tck-object-factory branch October 17, 2026 09:16
aepfli added a commit that referenced this pull request Oct 17, 2026
…r own ObjectFactory (#1926)

Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant