X Tutup
Skip to content

[update_lib] fast date lookup for todo#7299

Merged
youknowone merged 3 commits intoRustPython:mainfrom
youknowone:update-lib
Mar 1, 2026
Merged

[update_lib] fast date lookup for todo#7299
youknowone merged 3 commits intoRustPython:mainfrom
youknowone:update-lib

Conversation

@youknowone
Copy link
Member

@youknowone youknowone commented Mar 1, 2026

Summary by CodeRabbit

  • Refactor

    • Optimized last-updated metadata lookups using batched, cached retrieval to reduce redundant queries and improve performance when aggregating update dates.
  • Bug Fixes

    • Test items now always display last-updated timestamps across all views (no longer gated by verbose mode).
  • Chores

    • Expanded test group mappings to include additional test sets for more comprehensive coverage.

@youknowone youknowone marked this pull request as ready for review March 1, 2026 14:39
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 1, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between aa1e793 and 44ab598.

📒 Files selected for processing (2)
  • scripts/update_lib/cmd_todo.py
  • scripts/update_lib/deps.py

📝 Walkthrough

Walkthrough

Adds a bulk git date-caching lookup in scripts/update_lib/deps.py and updates scripts/update_lib/cmd_todo.py to always assign last_updated metadata to displayed test items (no longer gated by verbose).

Changes

Cohort / File(s) Summary
Bulk git caching & lookup
scripts/update_lib/deps.py
Add _bulk_last_updated() to obtain per-file and directory last-commit dates in one git call; add helpers _lib_prefix_stripped() and _lookup_last_updated(); refactor get_module_last_updated() and get_test_last_updated() to use the cache-backed lookup instead of per-path git calls.
Todo metadata handling
scripts/update_lib/cmd_todo.py
Move last_updated computation for test items out of verbose-only branch so test["last_updated"] is assigned unconditionally for all displayed tests/test_by_lib and no_lib_tests.
Dependency/test mappings
scripts/update_lib/deps.py
Extended DEPENDENCIES: add curses test group, add mapping_tests.py to dict tests, add test_re.py and re_tests.py to re tests.

Sequence Diagram(s)

sequenceDiagram
    participant CLI as cmd_todo.py
    participant Deps as deps.py
    participant Cache as BulkCache
    participant Git as git CLI

    CLI->>Deps: request last_updated for module/test paths
    Deps->>Cache: check bulk cache for Lib-relative paths
    alt cache miss
        Deps->>Git: single bulk git query for all Lib/ paths
        Git-->>Deps: per-file commit dates + directory rollups
        Deps->>Cache: populate bulk cache (path -> date, dir aggregates)
    end
    Deps->>Cache: resolve most recent date for requested paths
    Cache-->>Deps: return resolved date or None
    Deps-->>CLI: return last_updated value
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested reviewers

  • moreal
  • ShaharNaveh

Poem

🐰
I hopped through commits and found each trail,
One bulk call saved the sniff and wail,
Timestamps stitched from root to leaf,
Tests now wear dates, neat and brief,
A rabbit’s hop, a tidy tale.

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main objective of the changeset: implementing a fast (cached bulk) date lookup mechanism for the todo update library functionality.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@scripts/update_lib/deps.py`:
- Around line 1164-1183: _normalize path strings before comparing to avoid false
negatives: in _lookup_last_updated(), call pathlib.Path(lib_prefix).as_posix()
and pass that normalized string into _lib_prefix_stripped (or normalize inside
_lib_prefix_stripped), and for each input path p convert it to a POSIX string
via pathlib.Path(p).as_posix() before using startswith() and slicing; keep using
_bulk_last_updated() and the existing key lookup but ensure both prefix and p
are normalized to POSIX so Windows backslashes and './' variants won't cause
cache misses.

ℹ️ Review info

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5eadae8 and f36614a.

📒 Files selected for processing (2)
  • scripts/update_lib/cmd_todo.py
  • scripts/update_lib/deps.py

@github-actions
Copy link
Contributor

github-actions bot commented Mar 1, 2026

Code has been automatically formatted

The code in this PR has been formatted using:

  • ruff format
    Please pull the latest changes before pushing again:
git pull origin update-lib

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
scripts/update_lib/deps.py (1)

1123-1129: Consider specifying cwd for the git command to improve robustness.

The git command runs without an explicit cwd, assuming the script is invoked from the repository root. If the script is run from a different working directory, the Lib/ path may not resolve correctly.

💡 Suggested improvement
+    # Run from repo root (parent of scripts/)
+    repo_root = pathlib.Path(__file__).parent.parent.parent
     try:
         result = subprocess.run(
             ["git", "log", "--format=%cd", "--date=short", "--name-only", "--", "Lib/"],
             capture_output=True,
             text=True,
             timeout=30,
+            cwd=repo_root,
         )
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@scripts/update_lib/deps.py` around lines 1123 - 1129, The subprocess.run call
that executes git (the invocation assigning to result) should set an explicit
cwd so the "Lib/" path is resolved relative to the repository root rather than
the caller's current directory; update the subprocess.run call to pass cwd
pointing to the repo root (resolve via the script's file location using
pathlib.Path(__file__).resolve().parents[...] or similar) and ensure pathlib/os
is imported if not already.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@scripts/update_lib/deps.py`:
- Around line 1123-1129: The subprocess.run call that executes git (the
invocation assigning to result) should set an explicit cwd so the "Lib/" path is
resolved relative to the repository root rather than the caller's current
directory; update the subprocess.run call to pass cwd pointing to the repo root
(resolve via the script's file location using
pathlib.Path(__file__).resolve().parents[...] or similar) and ensure pathlib/os
is imported if not already.

ℹ️ Review info

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f36614a and aa1e793.

📒 Files selected for processing (1)
  • scripts/update_lib/deps.py

@youknowone youknowone merged commit d0b5a5a into RustPython:main Mar 1, 2026
13 checks passed
@youknowone youknowone deleted the update-lib branch March 1, 2026 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

X Tutup