X Tutup
Skip to content

Feat/pygithub migration and announcements#1436

Open
lilfetz22 wants to merge 54 commits intopython-semantic-release:masterfrom
lilfetz22:feat/pygithub-migration-and-announcements
Open

Feat/pygithub migration and announcements#1436
lilfetz22 wants to merge 54 commits intopython-semantic-release:masterfrom
lilfetz22:feat/pygithub-migration-and-announcements

Conversation

@lilfetz22
Copy link
Contributor

Purpose

This PR delivers three capabilities:

  1. Release announcements — After a successful GitHub release is published, the version command now posts comments to the closed issues and merged PRs that are referenced in the commits included in the release. Two new default Jinja2 templates drive the comment body (.issue_resolution_announcement.md.j2 and .pr_publish_announcement.md.j2). A released label is also added to each announced issue/PR. Announcements are best-effort (failures are logged and skipped), deduplicated, noop-aware, and currently limited to the GitHub HVCS.

  2. PyPI reference link in release notes — The default release notes template can optionally render a link to the project's PyPI page. A new include_pypi_link boolean field is added to ReleaseNotesContext (default False) and propagated through generate_release_notes() so callers can opt in.

  3. Bug fixes across commit parsers, GitHub HVCS, and config generation — Issue footer references are now accepted both with and without a colon (e.g. Closes #123 and Closes: #123) across all four commit parsers. The deprecated changelog_file config option is no longer emitted by generate-config and no longer triggers a deprecation warning when the user has not explicitly set it. Various Windows cross-platform stability fixes were also included.

Rationale

Announcements: Contributors and issue reporters often have no visibility into which release resolves their issue or includes their PR. Automating a comment at release time closes that feedback loop without requiring any manual steps. Using the existing Jinja2 template system keeps the feature consistent with how changelogs and release notes work and lets users override the default message bodies.

PyPI link: Many projects want a "Download from PyPI" call to action at the top of their GitHub Release notes. Making the block conditional (opt-in via include_pypi_link=True) lets projects that publish to PyPI add the link without changing behaviour for projects that don't.

Issue footer without colon: GitHub and GitLab both accept Closes #123 (no colon) as a valid closing keyword in commit messages, but the parsers required the colon form Closes: #123. This was a source of user confusion because commits that worked fine on the hosting service were silently not being linked by the tool.

Deprecated config suppression: The generate-config command was printing the deprecated changelog_file key and triggering a deprecation warning on every invocation even for users who never set it. Suppressing both reduces noise and prevents a misleading onboarding experience.

How did you test?

  • Unit tests updated in test_github_pygithub.py covering post_comment(), check_issue_state(), and add_labels_to_issue() (success and exception paths).
  • Unit tests updated in test_release_notes.py to cover the new include_pypi_link field.
  • Unit tests updated in test_config.py to verify the deprecated changelog_file is absent from the generated config and that the deprecation warning is not emitted when the value is unset.
  • E2E tests updated in test_generate_config.py and conftest.py to align with the new generate-config output.
  • Manual testing on Windows to confirm cross-platform fixes (absolute path handling in TOML declarations tests, git subprocess discoverability when os.environ is partially cleared, avoiding a Git subprocess inside deep_copy_commit).
  • Edge cases considered: duplicate announcement prevention (deduplication set), noop mode (announcements skipped), non-GitHub HVCS (guard clause), HVCS errors during announcement (logged and continued), include_pypi_link defaulting to False (existing snapshot tests continue to pass).

How to Verify

  1. Check out this branch and install the development dependencies:
    pip install -e .[build,dev,test]
  2. Run the unit tests to verify the commit parser fixes, PyPI link context, and config deprecation suppression:
    pytest -m unit
  3. Run the e2e tests to verify generate-config no longer emits changelog_file and that release-notes rendering is unchanged by default:
    pytest -m e2e
  4. To exercise the announcement feature manually, point the tool at a repo with GitHub HVCS configured and commits that reference issues/PRs, then run semantic-release version --tag (or with --noop first to inspect output without posting).
  5. To verify the PyPI link opt-in, add include_pypi_link = true under [tool.semantic_release] and run semantic-release changelog — the rendered release notes should contain a PyPI badge/link block.
  6. To verify the issue footer fix, create a commit with Closes #1 (no colon) and confirm the linked issue is parsed correctly by all four parsers.

PR Completion Checklist

  • Reviewed & followed the Contributor Guidelines

  • Changes Implemented & Validation pipeline succeeds

  • Commits follow the Conventional Commits standard
    and are separated into the proper commit type and scope (recommended order: test, build, feat/fix, docs)

  • Appropriate Unit tests added/updated

  • Appropriate End-to-End tests added/updated

  • Appropriate Documentation added/updated and syntax validated for sphinx build (see Contributor Guidelines)

codejedi365 and others added 26 commits November 2, 2025 14:24
Removed test functions that tested requests implementation details
now handled by PyGithub:
- test_create_release_fails
- test_should_create_release_using_token_or_netrc
- test_edit_release_notes_succeeds
- test_edit_release_notes_fails
- test_get_release_id_by_tag
- test_asset_upload_url
- test_upload_release_asset_succeeds
- test_upload_release_asset_fails
- test_request_has_no_auth_header_if_no_token_or_netrc

These behaviors are now covered by test_github_pygithub.py.
Implemented post_comment(), check_issue_state(), and add_labels_to_issue()
methods using PyGithub Issue API. Added comprehensive test coverage with
8 test cases covering success and exception paths.

- post_comment(issue_id, body): Posts comment to issue/PR, returns comment ID
- check_issue_state(issue_id): Returns issue state ('open' or 'closed')
- add_labels_to_issue(issue_id, labels): Adds labels to issue/PR

All methods:
- Use PyGithub's repo.get_issue() and Issue methods
- Have proper error handling (GithubException  UnexpectedResponse)
- Include debug and info logging with @logged_function decorator
- Support both issues and pull requests (same API)

Test coverage in test_github_pygithub.py:
- TestGithubPostComment: 2 tests (success, exception)
- TestGithubCheckIssueState: 3 tests (open, closed, exception)
- TestGithubAddLabelsToIssue: 3 tests (multiple labels, single label, exception)

These methods enable release announcement functionality for closed issues
and merged PRs using the announcement templates in
src/semantic_release/data/templates/conventional/md/
- Add _post_release_announcements() helper to process linked issues/PRs
- Extract linked issues from ParsedCommit.linked_issues tuple
- Distinguish PRs using commit.linked_merge_request field
- Render .pr_publish_announcement.md.j2 for PRs
- Render .issue_resolution_announcement.md.j2 for issues
- Post comments via hvcs_client.post_comment()
- Add 'released' label to all announced issues/PRs
- Implement deduplication to prevent duplicate comments
- Use best-effort error handling (log warnings, continue processing)
- Respect noop mode and only run for Github HVCS
- Pass version and release_notes to announcement templates

Integration point: After successful hvcs_client.create_release() in version()
command, before final exception handling. This ensures announcements only
happen after the release is successfully created on the VCS.

Breaking Change: None - backward compatible, announcements are optional
Templates expect 'release.version' not just 'version' string.
Update _post_announcement_to_issue to pass release object
matching the template expectations in .pr_publish_announcement.md.j2
and .issue_resolution_announcement.md.j2
- Add type: ignore[misc, valid-type] for click.MultiCommand base class
- Change runtime parameter type from CliContextObj to RuntimeContext
- Change release parameter type from dict to Release TypedDict
- Add proper type checking imports for Release and RuntimeContext

All mypy checks now pass (58 source files checked).
…ncements

- Convert issue_id string to int before calling HVCS methods
- Use ReleaseNotesContext.bind_to_environment to inject required Jinja2 filters (create_release_url,
  etc.) into the announcement template context
- Prevent template rendering errors during release
- Add ValueError handling for invalid issue IDs
- Remove unused release_notes parameter from announcement functions
The markdown macros file defined a macro named 'format_link' but the
.release_notes.md.j2 and other templates were importing 'format_link_reference'.

Renamed 'format_link' to 'format_link_reference' in macros.md.j2 to match
the expected import name, aligning with the reStructuredText template pattern.
Updates to support the PyGithub client integration:

1. Enhanced post_mocker fixture in tests/e2e/conftest.py:
   - Added mock for GET requests to support PyGithub's lazy-loaded
     repository access when creating releases
   - Created PostOnlyMocker wrapper to filter call_count and last_request
     to POST-only requests, maintaining test compatibility

2. Updated test fixture in tests/fixtures/git_repo.py:
   - Added PyPI Registry section to generated release notes that matches
     the new template output
   - Properly handles line endings to match template structure

These changes allow the e2e test to work with the PyGithub client which
requires repository metadata from the GitHub API before creating releases.
…generated config

The generated config previously included both changelog.changelog_file (deprecated) and
changelog.default_templates.changelog_file, causing duplicate entries and spurious deprecation
warnings. Exclude changelog.changelog_file from RawConfig().model_dump output so generated defaults
only include default_templates.changelog_file. No runtime behavior change; only affects generated
config output.
… set

Avoid printing the deprecation warning during default configuration generation by returning early
when `changelog_file` is empty. The warning is still emitted when a user explicitly sets the
deprecated option.
…hangelog_file

Update the `raw_config_dict` fixture to exclude `changelog.changelog_file` when producing the
expected model dump. This mirrors the change in `generate-config` which omits the deprecated option
from generated defaults, preventing spurious deprecation warnings during test runs.
… env-clearing test

- Update default TOML test to exclude deprecated `changelog.changelog_file` from the expected model
  dump so it matches `generate-config` output changes.
- Make `test_git_remote_url_w_insteadof_alias` robust on Windows by preserving critical system
  environment variables (e.g., PATH, SYSTEMROOT, PATHEXT, TEMP, TMP) when clearing the rest of
  `os.environ`, so subprocess calls (like `git`) continue to be discoverable during the test.
…l debugging

Include the full test run log to aid local debugging and CI triage; intended for developer reference
only.
…d without colon)

Clarify that parsers accept both Git Trailer format with a colon (e.g., 'Closes: python-semantic-release#123') and the more
casual format without a colon (e.g., 'Closes python-semantic-release#123'). Added examples to illustrate both forms and
various list separators. This updates the user-facing docs to reduce confusion and match common VCS
behavior (GitHub/GitLab).
…e/PR announcements

Previously announcement rendering always bound to the user's template environment which failed when
an embedded default template (e.g., .issue_resolution_announcement.md.j2) was expected. Prefer the
user-provided template if present, otherwise fall back to the embedded default conventional markdown
templates. Add logic to resolve and bind the default template directory and use the existing
ReleaseNotesContext environment for rendering.
Make the issue footer regex accept both 'Closes python-semantic-release#123' and 'Closes: python-semantic-release#123' by making the colon
optional and allowing whitespace. This aligns Angular parser behavior with GitHub/GitLab conventions
and reduces confusing failures when users omit the colon.
Relax issue footer regex to accept both 'Closes python-semantic-release#123' and 'Closes: python-semantic-release#123'. Make the colon optional
and allow whitespace so the parser is more forgiving and consistent with common VCS behaviors.
Relax issue footer regex to accept both 'Closes python-semantic-release#123' and 'Closes: python-semantic-release#123' for the Emoji parser. This
brings Emoji parser behavior in line with other parsers and common VCS conventions.
Relax issue footer regex to accept both 'Closes python-semantic-release#123' and 'Closes: python-semantic-release#123' for the Scipy parser. Keep
behavior consistent across parsers and align with common VCS conventions.
…dling

- Merge provided env with current environment to preserve PATH and other vars
- Resolve shell executable using shutil.which() and log useful debug messages
- Use resolved shell_path when running subprocesses and pass merged env
- Handle FileNotFoundError in �uild_distributions and raise BuildDistributionsError
- Adjust default PATH to be None to better reflect missing values
- Reorder and tidy imports, and improve template env binding formatting
…uard response access

- Catch AssetUploadError alongside HTTPError to avoid uncaught exceptions
- Safely extract status_code only when err is an HTTPError with a response
- Improve error logging for failed uploads
@lilfetz22 lilfetz22 force-pushed the feat/pygithub-migration-and-announcements branch from e2b2d09 to f26ef7b Compare March 8, 2026 09:26
… compat

- Normalize CRLF/LF differences in assertions to avoid Windows failures
- Strip trailing whitespace where appropriate to reduce platform-dependent flakes
…ndows compat

- Replace absolute Path(__file__) usage with relative test path strings for cross-platform stability
lilfetz22 and others added 15 commits March 8, 2026 05:35
Introduce boolean flag that will drive rendering of a PyPI registry link
in the default release notes template. Default is False to preserve
existing behaviour.
…notes

Add new argument to the public helper and internal call site so callers
can request the PyPI link block. This matches the newly-added context field.
Default value remains False.
Tighten the linked issue regex to only match valid git footer syntax,
requiring a colon separator after the keyword. Bare patterns without a
colon were incorrectly recognized as linked issues, causing false positives.
Apply the same regex tightening as the conventional parser so scipy
no longer treats bare closure keywords as linked issues. This prevents
spurious matches and aligns behaviour across parsers.
…ws resource exhaustion

Refactor deep_copy_commit to skip lazy-loaded attributes like tree and gpgsig which previously
spawned git cat-file subprocesses. Add detailed comments explaining the WinError 1450/1455 issue and
suppress OSError alongside ValueError. This mitigates failures in long Windows test runs.
Wrap the previously unconditional PyPI registry block with a Jinja2 check on
include_pypi_link. This allows users to opt-in and keeps default templates
compatible with existing expectations.
Provides a default 'master' worker_id so repo-building fixtures don't fail in
single-process runs. pytest-xdist will override this when used.
@lilfetz22 lilfetz22 force-pushed the feat/pygithub-migration-and-announcements branch from f26ef7b to f66de84 Compare March 8, 2026 09:39
Use PYTEST_XDIST_WORKER for worker detection so the per-repo cache lock is
actually enabled during parallel test runs.

Persist cached repo metadata to shared JSON files in
.pytest_cache/d/psr-cached-repos/repo-data with atomic replace, allowing
workers to observe cache state consistently.

Hold the per-repo lock through cache copy operations and release it in a
finally block to prevent mid-copy delete/rebuild races that produced
FileNotFoundError and GitCommandError failures.
Implement _PostOnlyMocker.reset_mock() to clear the wrapper's internal
_post_list state in addition to delegating to requests-mock behavior.

Without clearing _post_list, call_count and last_request assertions can
observe stale POST requests from previous CLI invocations in the same test,
causing false failures.
Extend GenerateDefaultReleaseNotesFromDefFn and
generate_default_release_notes_from_def() with an include_pypi_link flag
defaulting to False.

Only append the PyPI artifacts section when include_pypi_link is enabled,
matching changelog context defaults and preventing release note expectation
mismatches in e2e assertions.
…ization

Add focused unit tests for cache-lock behavior and shared repo metadata files.
Verify locks stay held during cache copy operations to avoid mid-copy rebuild races.
Assert worker locking is enabled only when PYTEST_XDIST_WORKER is set.
@lilfetz22 lilfetz22 force-pushed the feat/pygithub-migration-and-announcements branch 2 times, most recently from 0b25f77 to 73eea58 Compare March 8, 2026 20:30
Addressed mypy type validation errors in version

command and repo test cache by adding null checks

and type ignores. Applied ruff formatting across

the test suite.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

X Tutup