[bpulluta]

Summary

Reworks the natural gas pipelines one-shot extraction plugin so its schema captures decomposed, multi-axis requirement rows instead of the previous flat 5-field outputs.

This is an initial cut to continue developing and track changes.

Schema (natural_gas_pipelines_schema.json)

• Each outputs row is now decomposed on four axes:
  • WHAT — feature, specific_subject, applies_to, requirement_description
  • HOW MUCH — value, value_category, value_interpretation, units, range_low/range_high
  • HOW BINDING — obligation
  • WHEN/WHERE — condition
  • Evidence — source_verbatim, summary, section, notes, explanation
• Feature enum expanded to 16: added roads / schools / hospitals / community-structures distances and permit_approval; folded day/night into a single noise feature + condition, and rear/side yard into property lines distance + specific_subject.
• Added $qualitative_features (zoning districts, permit_approval) — previously missing, so every feature routed as quantitative.
• Added $scope, $core_principles, grouped $definitions, $instructions, $examples, and $qa_checklist following the freshest sibling (oil_gas_wells) house style.

Config (plugin_config.yaml)

• Lowercased heuristic_keywords keys (house style).
• Added mixed oil-and-gas / gas-well chapter recall queries and keyword-score tuning for discovery.
• Rewrote extraction_system_prompt to teach the decomposition model (axes, one-row-per-requirement, most-restrictive rule, siting-focus exclusions).

Validation

• Live 1-jurisdiction run (Rehoboth, MA, gpt-5.4, full search→download→collect→extract→CSV): produces the 25-column decomposed output; correctly extracted noise 55 dBA (fully decomposed, with notes on the absence of separate day/night limits) and two permit_approval rows (special permit, site plan approval), routed to the quant/qual CSVs correctly. Compared to the old flat output it dropped both false positives (a site-plan filing radius mislabeled as a setback; an exhaust muffler mislabeled as an enclosure).

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.65%. Comparing base (373283e) to head (bc1528e).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #546   +/-   ##
=======================================
  Coverage   71.65%   71.65%           
=======================================
  Files          82       82           
  Lines        8507     8507           
  Branches      922      922           
=======================================
  Hits         6096     6096           
  Misses       2237     2237           
  Partials      174      174           

Flag	Coverage Δ
unittests	71.65% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

This PR reworks the natural gas pipelines one-shot extraction plugin to use a decomposed, multi-axis “one row per requirement” schema, expanding the output model to capture richer requirement context while keeping compatibility with COMPASS’s schema-driven one-shot pipeline.

Changes:

• Updated plugin_config.yaml discovery queries/keywords and rewrote the extraction system prompt to teach the decomposition model and stricter scope rules.
• Replaced the prior flat outputs schema with a decomposed row schema (WHAT / HOW MUCH / HOW BINDING / WHEN-WHERE + evidence), expanding features and adding richer evidence fields.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
compass/extraction/natural_gas_pipelines/plugin_config.yaml	Broadens discovery inputs and updates the extraction system prompt to align LLM behavior with the decomposed row model and scope rules.
compass/extraction/natural_gas_pipelines/natural_gas_pipelines_schema.json	Introduces the decomposed multi-axis output schema, expanded feature enum, and richer evidence/normalization guidance.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[ppinchuk]

Cool ideas here! Could you please update to the latest COMPASS version and make sure your schema aligns with the new "ordinance_text" field that Rajendra added recently?

[ppinchuk]

Is this ready for review?

[bpulluta]

Is this ready for review?

@ppinchuk sorry for the delay, yes it is now ready for review

[ppinchuk]

Couple of minor comments but I think we can move this forward if you are seeing good results on your end @bpulluta

[bpulluta]

Couple of minor comments but I think we can move this forward if you are seeing good results on your end @bpulluta

@ppinchuk this is ready to merge. I can't merge myself since it requires 2 reviews