Skip to content

feat(presets): support regex selectors for command declarations - #4776

Open
lmtyy wants to merge 1 commit into
github:mainfrom
lmtyy:feat/4659-regex-preset-selectors
Open

lmtyy wants to merge 1 commit into
github:mainfrom
lmtyy:feat/4659-regex-preset-selectors

Conversation

@lmtyy

@lmtyy lmtyy commented Sep 28, 2026 •

Copy link
Copy Markdown

Closes #4659.

This replaces the previous (much larger) version of this PR with the focused change the
maintainer asked for: regex:<pattern> support for preset command declarations only.
Everything else that had accumulated here is split out into subsequent targeted PRs; the
full earlier work is preserved on the branch feat/4659-regex-preset-selectors-full
(2aa5e892) as the source for those follow-ups.

What this does

A preset may use a bounded selector in a provides.templates entry of type command:

provides:
  templates:
    - type: command
      name: 'regex:^speckit\\.(plan|tasks|implement)$'
      file: commands/workflow-guidance.md
      strategy: append
  • The selector is expanded, in declaration order, into the concrete lower-layer commands it
    matches. Those concrete names then flow through the existing command resolution,
    registration, composition and removal paths.
  • Matching is against the full logical command name only — never a path, file name or
    substring — and only against commands that actually exist in the layers below (enabled
    presets, enabled extensions, core, including built-in core). The declaring preset itself,
    higher-priority presets and project-local overrides are not used as evidence of existence.
  • A literal selector string is never used as an output path, a .composed file name or a
    registry key: the registry only ever sees concrete names.
  • A regex: name on a non-command declaration is a validation error, not a silent
    no-op.
  • Entries without the prefix keep exact-name matching and are completely unaffected.
  • Diagnostics: preset info reports the commands a selector currently matches;
    preset resolve <name> names the declaration that matched when a layer came from a
    selector.
  • Matching is deterministic (declaration order preserved, overlapping declarations are not
    de-duplicated away) and uses the bounded matcher with its existing limits.

Scope: what is deliberately not here

Split into follow-up PRs, each based on its own point in upstream/main:

  • template and script selectors, and selector support on the extension side;
  • automatic reconciliation after provider/integration changes (see the known limitation);
  • historical recovery sweeps and the ownership/ledger redesign beyond the minimal guard;
  • the install-transaction rework, snapshot/journal machinery and CLI boundary refactors
    (design/cli.md, design/shared.md).

Kept deliberately, because the feature materializes command/skill files:

  • the minimal ownership-based guard on the command/skill write paths, so that a preset
    install cannot overwrite artifacts the user has edited. This is a prerequisite for the
    feature being safe, not a re-introduction of the wider ownership work.

Known limitations (please review with these in mind)

  1. No automatic broadcast on provider change. A selector is expanded at install/enable
    time and the existing reconciliation of concrete names still runs, but there is no new
    mechanism that re-expands selectors when a new provider/integration appears. This is the
    main user-visible limitation of the reduced scope and the reason the change is small.
  2. Bounded regex subset. The matcher intentionally implements a bounded subset rather
    than full Python regex compatibility; the limits and the rejection behaviour are covered
    by tests.

Tests / evidence

  • Selector surface: tests/specify_cli/presets/test_regex_selectors.py,
    test_selector_safety.py, test_selector_resolution_contract.py,
    test_resolver_candidates.py, test_regex_selector_diagnostics.py — 146 passed.
  • Whole preset suite: 854 passed.
  • uvx ruff@0.15.0 check src tests: All checks passed; compileall clean;
    git diff --check clean.
  • No tests were skipped, weakened or marked expected-failure to reach these numbers.
  • Against the same base, tests/specify_cli/test_file_policy.py shows the same 10
    pre-existing failures / 177 passed
    before and after this change (verified on a pristine
    checkout of the base) — i.e. this PR adds none.
  • Merged against the current main (which has moved past this branch's base) the full suite
    is 12423 passed / 98 skipped / 0 failed — the change merges cleanly with the latest
    base and does not regress it.
  • CI note: the workflow runs for a fork PR are in action_required state, i.e. they need a
    maintainer to approve them before they execute. No CI result should be assumed for this
    revision yet.
  • WINDOWS-VERIFIED: NO — the fork's Windows CI cannot be triggered from here, so the
    Windows job must be treated as unverified for this revision. The earlier branch's Windows
    fixes are not part of this PR (they belong with the files this PR does not touch).

AI disclosure

Authored with AI assistance (Hermes Agent); reviewed and verified against the test suite by
a human before submission.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Command resolution, skill registration, diagnostics, warnings, and lifecycle reconciliation have unresolved functional gaps.

Review effort: Balanced
Findings: 3 High severity · 2 Medium severity

Open (5)
What changed in this PR

Adds regex: resource selectors to preset resolution and command materialization.

Changes:

  • Validates and full-matches regex selectors.
  • Expands command selectors and adds lifecycle reconciliation.
  • Adds resolver and selector coverage.
File Description
tests/​specify_cli/​presets/​test_regex_selectors.py Tests selector matching and resolution.
src/​specify_cli/​presets/​command_set_priority.py Reconciles commands after priority changes.
src/​specify_cli/​presets/​command_info.py Displays selector matches.
src/​specify_cli/​presets/​command_enable.py Reconciles enabled preset commands.
src/​specify_cli/​presets/​_selectors.py Implements selector helpers.
src/​specify_cli/​presets/​_resolver.py Resolves regex template and script layers.
src/​specify_cli/​presets/​_manifest.py Validates regex expressions.
src/​specify_cli/​presets/​_manager.py Integrates expansion into lifecycle handling.
src/​specify_cli/​presets/​_manager_commands.py Expands command selectors.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/specify_cli/presets/_resolver.py Outdated
Comment thread src/specify_cli/presets/_resolver.py Outdated
Comment thread src/specify_cli/presets/command_info.py Outdated
Comment thread src/specify_cli/presets/_manager.py Outdated
Comment thread src/specify_cli/presets/_manager_commands.py Outdated
@mnriem

mnriem commented Sep 28, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

@lmtyy

lmtyy commented Sep 28, 2026

Copy link
Copy Markdown
Author

Addressed all five Copilot review findings:

  • Added command regex selectors to normal layer resolution and composition.
  • Preserved all matching regex declarations in manifest order.
  • Fixed the invalid _selectors imports used by diagnostics.
  • Reused selector expansion for AI-skill registration.
  • Added selector-aware reconciliation for relevant preset/extension lifecycle changes.

Also added regression coverage for overlapping selectors, command composition strategies, diagnostics, AI-skill registration, and lifecycle state changes.

Validation:

  • 8624 passed, 17 skipped
  • 183 targeted regression tests passed
  • Ruff passed
  • Python compilation passed
  • git diff --check passed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@mnriem mnriem added the triage-can-wait Verdict: valid and in-scope but deprioritized; held behind the evidence gate label Sep 28, 2026
@lmtyy

lmtyy commented Sep 29, 2026

Copy link
Copy Markdown
Author

All previous Copilot review findings have been addressed and the updated test suite is passing. The latest Copilot re-review appears to have failed due to a review error, so the PR is ready for another review when convenient.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Selector lifecycle paths can leave stale commands or skills and several acceptance criteria remain incomplete.

Review effort: Balanced
Findings: 6 Medium severity

Open (6)

Comment thread src/specify_cli/presets/_manager.py Outdated
Comment thread src/specify_cli/presets/_manager.py Outdated
Comment thread src/specify_cli/presets/_manager_skills.py
Comment thread src/specify_cli/presets/command_info.py Outdated
Comment thread src/specify_cli/presets/command_set_priority.py Outdated
Comment thread tests/specify_cli/presets/test_regex_selector_lifecycle.py Outdated
@mnriem

mnriem commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Selector lifecycle transitions can leave stale or untracked command and skill artifacts.

Review effort: Balanced
Findings: 3 Medium severity · 2 Low severity

Open (5)
Resolved since last review (6)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Stale artifacts remain when selectors lose all matches

src/​specify_cli/​extensions/​_commands.py:121

This refresh runs only after the extension state has changed, so a selector whose sole lower layer was disabled or removed now expands to nothing. register_enabled_presets_for_agent() never sees the previously materialized concrete name, and its reconciler also skips names with no remaining layers, leaving the old command/SKILL artifact active. Capture the pre-mutation matches (or persisted registrations), reconcile the union with post-mutation matches, and explicitly unregister names that no longer resolve.

Comment thread src/specify_cli/presets/_manager_commands.py Outdated
Comment thread src/specify_cli/presets/_manager_commands.py Outdated
Comment thread src/specify_cli/presets/_manager_commands.py Outdated
Comment thread src/specify_cli/presets/command_info.py
Comment thread tests/specify_cli/presets/test_regex_selector_lifecycle.py Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Selector cleanup, disabled-extension resolution, and installation rollback still have correctness gaps.

Review effort: Balanced
Findings: 1 High severity · 3 Medium severity

Open (4)
Resolved since last review (5)
Previously missed (2)

In code that hasn't changed since last review

Medium severity Selector expansion failure leaves partial installation

src/​specify_cli/​presets/​_manager.py:456

Selector diagnostics and expansion run after the preset directory and registry entry are created but before the existing rollback guard. _expand_command_selectors() deliberately raises when artifact enumeration fails, so such a failure leaves an enabled, partially installed preset behind. Move both calls into the guarded installation phase so the established cleanup executes.

Medium severity Re-enumeration causes false failure after registration

src/​specify_cli/​presets/​_manager.py:507

This repeats artifact enumeration outside the transactional registration block. If inventory changes or becomes unreadable after registration, the command reports installation failure even though the preset and artifacts are already committed. Reuse the concrete command_templates computed for registration.

Comment thread src/specify_cli/presets/_resolver.py Outdated
Comment thread src/specify_cli/integrations/_command_upgrade_layout.py Outdated
Comment thread src/specify_cli/presets/_manager_commands.py
Comment thread src/specify_cli/presets/command_disable.py Outdated
@mnriem

mnriem commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

@lmtyy

lmtyy commented Sep 30, 2026

Copy link
Copy Markdown
Author

@mnriem I've addressed the latest Copilot feedback in commit 1ce59c1, including the disabled-extension fallback, include_disabled handling, stale selector tracking/cleanup, and install rollback coverage.

When convenient, could you please re-request the Copilot review? Thanks!

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Selector rollback, disabled-state handling, and extension-layer deduplication contain unresolved correctness issues.

Review effort: Balanced
Findings: 1 High severity · 5 Medium severity

Open (6)
Resolved since last review (3)

Comment thread src/specify_cli/extensions/__init__.py Outdated
Comment thread src/specify_cli/integrations/_command_upgrade_layout.py Outdated
Comment thread src/specify_cli/presets/_manager.py Outdated
Comment thread src/specify_cli/presets/_resolver.py Outdated
Comment thread src/specify_cli/presets/command_disable.py Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread src/specify_cli/extensions/__init__.py Outdated
Comment thread src/specify_cli/presets/_resolver.py Outdated
Comment thread src/specify_cli/presets/command_info.py Outdated
Comment thread src/specify_cli/presets/command_set_priority.py Outdated
@mnriem
mnriem requested a balanced review from Copilot September 30, 2026 16:52
@mnriem

mnriem commented Oct 6, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback and resolve conflicts

@lmtyy

lmtyy commented Oct 7, 2026

Copy link
Copy Markdown
Author

@mnriem Thanks for your patience. I’ve addressed the latest Copilot feedback, resolved the upstream merge conflict, and pushed the updated changes.

The full test suite is passing locally, and the branch should now be ready for another review.

Thanks again for your time!

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread src/specify_cli/extensions/command_disable.py Outdated
Comment thread src/specify_cli/extensions/command_enable.py Outdated
Comment thread src/specify_cli/extensions/command_set_priority.py Outdated
@mnriem

mnriem commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback and resolve conflicts

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +350 to +351
# Command/skill winners were materialized atomically by the installer;
# a best-effort second pass would publish the same concrete outputs again.
Comment thread src/specify_cli/extensions/command_disable.py Outdated
Comment thread src/specify_cli/integrations/_command_upgrade_layout.py Outdated
Comment thread src/specify_cli/presets/_manager.py Outdated
@mnriem

mnriem commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback and fix test & lint errors

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Resolution diagnostics still omit the selector attribution required by the linked acceptance criteria.

1 open finding
3 resolved since last review
Previously missed (1)

In code that hasn't changed since last review

Medium severity Preserve selector attribution in preset resolve diagnostics

src/​specify_cli/​presets/​command_info.py:170

The linked acceptance criteria also require resolution diagnostics to show which selector matched each concrete resource, but this expansion is only rendered by preset info. preset resolve still receives layers containing only the preset source/path/strategy, so its composition chain cannot distinguish a regex match from an exact declaration or display the selector expression. Please carry selector attribution into resolved layers and render it in the resolution diagnostic as well.

🧠 Review effort: Balanced


Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

@mnriem

mnriem commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Legacy refresh paths can leave stale selector artifacts, and unrestricted backtracking regexes can hang CLI operations.

3 open findings

🧠 Review effort: Balanced

Comment thread src/specify_cli/presets/_selectors.py Outdated
Comment thread src/specify_cli/extensions/_commands.py Outdated
@mnriem

mnriem commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback and fix test & lint errors

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Extension disable can leave skill-only artifacts active under a previously selected integration.

3 open findings
2 resolved since last review

🧠 Review effort: Balanced

Comment thread src/specify_cli/extensions/command_disable.py Outdated
Comment thread src/specify_cli/presets/_selectors.py Outdated
@mnriem

mnriem commented Oct 12, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback and address test & lint errors

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +315 to +319
for directory in sorted(
(item for item in root.rglob("*") if item.is_dir() and not item.is_symlink()),
key=lambda item: len(item.parts),
reverse=True,
):
Comment thread src/specify_cli/presets/_resolver.py Outdated
Comment on lines +667 to +676
# ``templates/commands`` is a conventional command location in its own
# right: _conventional_command_names and _expand_command_selectors both
# enumerate it, for lower presets as well as for extensions. Searching it
# here — at the single shared lookup — is what keeps the preset-side
# eligibility check that calls this method directly in agreement with
# selector expansion; otherwise a matched resource is never offered as a
# layer and the selector materializes nothing.
subdirs = {
"template": ("templates", ""),
"command": ("commands", "templates/commands"),
Comment on lines +126 to +136
# Cleanup mutates several agents' artifacts, both registries, and the
# hook config, and it changes which lower layer preset selectors can
# match. A failure part-way through must not leave the extension
# enabled with the earlier agents already stripped, nor disabled with
# artifacts still composed from its layer, so snapshot every touched
# destination first, reconcile selectors strictly inside the same
# transaction, and restore everything as a unit on error.
import copy

# Disable hooks in extensions.yml
config = hook_executor.get_project_config()
if "hooks" in config:
for hook_name in config["hooks"]:
for hook in config["hooks"][hook_name]:
if hook.get("extension") == extension_id:
hook["enabled"] = False
hook_executor.save_project_config(config)

console.print(f"[green]✓[/green] Extension '{_escape_markup(str(display_name))}' disabled")
from ..presets import PresetManager
from ..presets._transaction import _ArtifactSnapshot, _capture_preset_artifacts
Comment on lines +63 to +68
# Enabling makes lower layers visible to selectors. Snapshot before
# publishing eligibility, including ownership of every refreshed winner.
import copy

from ..presets import PresetManager
from ..presets._transaction import _ArtifactSnapshot, _capture_preset_artifacts
Comment on lines +67 to +77
# Extension reordering can change the lower-layer candidates matched by
# enabled preset regex selectors, so the new priority is only committed
# together with the recomposed artifacts. Mirrors the preset priority
# transaction: snapshot the artifact trees and both registries, run the
# reconciliation strictly, and roll back to the previous priority on any
# failure instead of reporting a change that never reached the
# selector-backed outputs.
import copy

from ..presets import PresetManager
from ..presets._transaction import _ArtifactSnapshot, _capture_preset_artifacts
Comment on lines +53 to +57
from ._resolver import PresetResolver

resolver = PresetResolver(project_root)
affected = manager._collect_selector_command_names(resolver)
manifest = resolver._get_manifest(manager.presets_dir / preset_id)
Comment on lines +39 to +43
# Capture selector matches while the preset is disabled, then enable it and
# reconcile the newly active resolution stack.
from ._resolver import PresetResolver

resolver = PresetResolver(project_root)
Comment on lines +64 to +70
registry_before = copy.deepcopy(manager.registry.data)
snapshot = _ArtifactSnapshot()
captured = False
try:
# Capture before even the first scan: directory resolution may create
# skill roots, and rollback must not depend on the failing resolver.
_capture_preset_artifacts(manager, snapshot)
@mnriem

mnriem commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator

I am so sorry to ask for this and thank you for being so diligent in addressing the feedback. To keep the change focused and reviewable, can we please limit this PR to supporting regex:<pattern> for preset command declarations only, then address the remaining areas in subsequent targeted PRs? I am afraid we won't land this as it has snowballed and diverged from the original intent

A preset may declare `regex:<pattern>` in a `provides.templates` entry of
type `command`. At install/enable time the selector expands, in declaration
order, into the concrete lower-layer commands it matches; those names then
flow through the existing command resolution, registration, tracking and
removal behaviour. Matching is against the full logical command name only,
never a path, file name or substring, and a literal selector is never used
as an output path, composed file name or registry key.

Non-command declarations using a selector are rejected during validation.
Diagnostics: `preset info` reports the commands a selector currently matches
and `preset resolve <name>` attributes selector-contributed layers.

Kept from the earlier work: the self-contained selector engine, the bounded
matching limits with their tests, the minimal ownership-based guard on the
command/skill materialization write paths, and the command-only docs.

Deliberately out of scope, for subsequent targeted PRs: template/script
selectors, extension-side selectors, automatic reconciliation after provider
changes, historical recovery sweeps, and the install-transaction rework.

AI disclosure: this change was authored with AI assistance (Hermes Agent);
all behaviour was reviewed and verified against the test suite by a human.
@lmtyy
lmtyy force-pushed the feat/4659-regex-preset-selectors branch from 234c727 to 647c7c8 Compare October 14, 2026 04:34
@lmtyy lmtyy changed the title feat(presets): support regex resource selectors feat(presets): support regex selectors for command declarations Oct 14, 2026
@lmtyy

lmtyy commented Oct 14, 2026

Copy link
Copy Markdown
Author

Hi — thank you for the guidance, and sorry for letting this PR grow well past the original intent (it had reached 69 files).

I have now replaced it with a focused version: regex: support for preset command declarations only, expanding into concrete command names that then flow through the existing resolution and registration paths, with selectors rejected on non-command declarations. It is a single commit and 15 files (+3,446/−115). The remaining areas — template/script selectors, extension-side selectors, reconciliation after provider changes, and the lifecycle/transaction work — will come as separate targeted PRs, each based on its own point in main.

Locally the full suite is green on this revision (12,410 passed, 0 failed), and it also stays green after merging the current main (12,423 passed, 0 failed). The workflow runs are currently action_required, so there is no CI result yet — I would appreciate it if you could approve them.

Could you take another look when you have a chance?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

triage-can-wait Verdict: valid and in-scope but deprioritized; held behind the evidence gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support regex selectors for preset commands, templates, and scripts

3 participants