Repository navigation
feat: allow wildcard API and workspace sub-filters, and filtering API tag descriptions - #338
Open
Chris Dayne (cdayne) wants to merge 2 commits into
Open
Chris Dayne (cdayne) wants to merge 2 commits into
Chris Dayne (cdayne) wants to merge 2 commits into
Conversation
… tag descriptions
A nested API filter entry applied only to the API it named, so excluding
operations or schemas from every API meant one entry per API. The key of
a nested entry can now be a wildcard pattern: it selects the matching
APIs, like a plain pattern, and its sub-filter applies to each of them:
apis:
- '*':
operations: []
schemas: []
An entry keyed by the API's name takes precedence over a pattern. One
lookup (findApiSubFilter) is now used by filtering, the specification
operation filter and API publishing.
A nested workspaces entry keyed by a pattern selected the matching
workspaces, but its sub-filter was looked up by exact name only, so it
never applied: those workspaces were extracted and published with no
filter at all. Workspace sub-filters now use the same lookup. The check
that decides what --remove-stale manages uses it too, so it follows a
pattern-keyed sub-filter. A sub-filter attached to an exclusion ('!name')
is ignored with a warning, for APIs and workspaces, instead of being kept
where it could never apply.
API sub-filters also accept tagDescriptions, to choose an API's tag
descriptions by name (as stored under apis/<api>/tagDescriptions/). The
rest of Azure#307 is already handled: tag description artifacts keep the
source instance's tag ID, but publish drops that read-only property, so
they publish to another instance (confirmed by the filtering integration
tests).
The filtering guide and the extract command reference describe wildcard
keys, tagDescriptions and the exclusion warning.
Closes Azure#316
Closes Azure#307
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Empty exact-name overrides are discarded during loading, allowing wildcard exclusions to incorrectly prevail for APIs and workspaces; preserve these entries and add loader-backed regression tests.
1 open issue
What changed in this PR
This PR extends resource filtering with wildcard API and workspace sub-filters and adds API tag-description filtering, addressing #316 and the remaining filtering work in #307.
Changes:
- Shares sub-filter lookup across filtering, extraction, publishing, and stale-resource reconciliation, with exact-name precedence.
- Adds
tagDescriptionsfiltering and warnings for sub-filters attached to exclusions. - Updates schema descriptions, documentation, and unit tests.
| File | Description |
|---|---|
| tests/unit/services/workspace-extractor.test.ts | Tests wildcard workspace lookup and name precedence. |
| tests/unit/services/filter-service.test.ts | Tests wildcard filtering, reconciliation, and tag descriptions. |
| tests/unit/lib/config-loader.test.ts | Tests loading wildcard filters and exclusion warnings. |
| src/services/filter-service.ts | Centralizes wildcard lookup and filters tag descriptions. |
| src/services/api-publisher.ts | Uses shared API sub-filter lookup when planning publication. |
| src/services/api-extractor.ts | Applies wildcard operation filters to exported specifications. |
| src/models/config.ts | Adds tagDescriptions to API sub-filters. |
| src/lib/config-loader.ts | Loads tag-description filters and warns on exclusion sub-filters. |
| scripts/generate-schemas.mjs | Updates generated descriptions for wildcard keys. |
| schemas/v1/extractor-config.schema.json | Adds tag-description support and wildcard descriptions. |
| docs/guides/filtering-resources.md | Explains wildcard precedence, tag descriptions, and exclusions. |
| docs/commands/extract.md | Documents wildcard keys and tag-description filtering. |
Review effort: Balanced
…attern
The loader dropped a nested entry with no sub-resource lists (`- orders:
{}` or `- orders:`), so with `- '*': {operations: []}` beside it the
wildcard's sub-filter still applied to orders, against the rule that an
entry keyed by the name takes precedence. The same held for workspaces.
Such an entry is now kept as an empty sub-filter, which includes
everything.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Lets a nested API or workspace filter entry be keyed by a wildcard pattern, so one sub-filter applies to every matching API or workspace, and adds a
tagDescriptionsAPI sub-filter.Changes
apisentry can be a pattern (- '*': { operations: [], schemas: [] }). It selects the matching APIs, like a plain pattern, and its sub-filter applies to each of them. An entry keyed by an API's name takes precedence over a pattern, even an empty one (- orders: {}or- orders:), which includes everything; otherwise the first matching pattern applies. One lookup (findApiSubFilter) is used by filtering, the specification operation filter and API publishing.main, aworkspacesentry keyed by a pattern selected the matching workspaces, but its sub-filter was looked up by exact name only, so it never applied and those workspaces were extracted and published unfiltered. They now use the same lookup. The check that decides what--remove-stalemanages uses it too.tagDescriptionsAPI sub-filter (API tagDescriptions are extracted with hardcoded source APIM resource IDs. #307): chooses an API's tag descriptions by name, as stored underapis/<api>/tagDescriptions/. The rest of API tagDescriptions are extracted with hardcoded source APIM resource IDs. #307 is already handled onmain: publish drops the source instance's tag ID from tag descriptions (fix: strip source-service tagId from ApiTagDescription PUT payloads #313).- '!name': {...}) are ignored with a warning, for APIs and workspaces, instead of being kept where they could never apply.tagDescriptionsand the exclusion warning; the extractor config schema describes pattern keys andtagDescriptions.Behaviour changes users may notice
Validation
npm test: all pass except the 2 environment-relatedpackage-buildintegration tests that also fail onmain.npm run lintandnpm run buildare clean.apis: [{'*': {operations: [], schemas: [], tagDescriptions: []}}]: every API extracted with no operation, schema or tag description folders, and no operations left in the specifications.apis: [{orders: {operations: ['*']}}, {'*': {operations: []}}]: orders keeps its operations; billing has none.workspaces: [{'team-*': {apis: [ws-orders], tags: []}}]: the sub-filter applies to team-a and team-b.workspaces: ['*', {'!team-b': {apis: []}}]: "Sub-filter for exclusion '!team-b' … has no effect"; team-b not extracted.Related Issue(s)
Closes #316
Closes #307