X Tutup
Skip to content

gh-129409: Fix Integer overflow - SEGV while writing data more than 2GB in CSV file#129413

Merged
sobolevn merged 7 commits intopython:mainfrom
srinivasreddy:gh-129409
Jan 29, 2025
Merged

gh-129409: Fix Integer overflow - SEGV while writing data more than 2GB in CSV file#129413
sobolevn merged 7 commits intopython:mainfrom
srinivasreddy:gh-129409

Conversation

@srinivasreddy
Copy link
Contributor

@srinivasreddy srinivasreddy commented Jan 28, 2025 •
edited by bedevere-app bot
Loading

@srinivasreddy srinivasreddy marked this pull request as ready for review January 28, 2025 17:05
@bedevere-app bedevere-app bot mentioned this pull request Jan 28, 2025
Closed
@srinivasreddy srinivasreddy marked this pull request as draft January 28, 2025 17:22
@srinivasreddy srinivasreddy changed the title gh-129409: Fix Integer overflow - aka SEGV while writing data more than 2GB in CSV file gh-129409: Fix Integer overflow - SEGV while writing data more than 2GB in CSV file Jan 28, 2025
tomasr8
tomasr8 reviewed Jan 28, 2025
View reviewed changes
Lib/test/test_csv.py Outdated
])

class HugeDataTest(unittest.TestCase):
def test_write_huge_data(self):
Copy link
Member

@tomasr8 tomasr8 Jan 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test should probably be guarded with support.requires("largefile", ...) so that it only runs when the resource is enabled via python -m test -u largefile

Copy link
Member

@sobolevn sobolevn Jan 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer not to test this, 2GB files are too much for this kind of change.

Copy link
Member

@tomasr8 tomasr8 Jan 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not disagreeing, but would you mind elaborating on why we shouldn't test this, just for future reference? 🙂

Copy link
Member

@sobolevn sobolevn Jan 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests are rather hard to reason about. I would put it this way: since it is a very slow and heavy test that tests an integer overflow. It is not specific to any arch / os / runtime / etc.

Copy link
Member

@tomasr8 tomasr8 Jan 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, thanks for the explanation :)

sobolevn
sobolevn requested changes Jan 28, 2025
View reviewed changes
Copy link
Member

@sobolevn sobolevn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I vote for removing this test.

@bedevere-app
Copy link

bedevere-app bot commented Jan 28, 2025

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@srinivasreddy srinivasreddy marked this pull request as ready for review January 29, 2025 05:58
sobolevn
sobolevn reviewed Jan 29, 2025
View reviewed changes
Copy link
Member

@sobolevn sobolevn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, add a NEWS entry, this is a user-facing bug.

tomasr8
tomasr8 reviewed Jan 29, 2025
View reviewed changes
Misc/NEWS.d/next/Library/2025-01-29-14-30-54.gh-issue-129409.JZbOE6.rst Outdated
Comment on lines +1 to +2
Fixed a segmentation fault in the :mod:`csv` module when writing data fields
larger than 2GB due to integer overflow.
Copy link
Member

@tomasr8 tomasr8 Jan 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit more concise:

Suggested change
Fixed a segmentation fault in the :mod:`csv` module when writing data fields
larger than 2GB due to integer overflow.
Fix an integer overflow in the :mod:`csv` module when writing data
larger than 2GB.

sobolevn
sobolevn approved these changes Jan 29, 2025
View reviewed changes
Copy link
Member

@sobolevn sobolevn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@sobolevn sobolevn added needs backport to 3.12 only security fixes needs backport to 3.13 bugs and security fixes labels Jan 29, 2025
@sobolevn sobolevn enabled auto-merge (squash) January 29, 2025 11:06
@sobolevn sobolevn merged commit 97b0ef0 into python:main Jan 29, 2025
49 checks passed
@miss-islington-app
Copy link

miss-islington-app bot commented Jan 29, 2025

Thanks @srinivasreddy for the PR, and @sobolevn for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jan 29, 2025
@srinivasreddy @miss-islington
pythongh-129409: Fix Integer overflow - SEGV while writing data more …
959b680 
…than 2GB in CSV file (pythonGH-129413)

(cherry picked from commit 97b0ef0)

Co-authored-by: Srinivas Reddy Thatiparthy (తాటిపర్తి శ్రీనివాస్  రెడ్డి) <thatiparthysreenivas@gmail.com>
@bedevere-app
Copy link

bedevere-app bot commented Jan 29, 2025

GH-129436 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Jan 29, 2025
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jan 29, 2025
@srinivasreddy @miss-islington
pythongh-129409: Fix Integer overflow - SEGV while writing data more …
46aa37a 
…than 2GB in CSV file (pythonGH-129413)

(cherry picked from commit 97b0ef0)

Co-authored-by: Srinivas Reddy Thatiparthy (తాటిపర్తి శ్రీనివాస్  రెడ్డి) <thatiparthysreenivas@gmail.com>
@bedevere-app
Copy link

bedevere-app bot commented Jan 29, 2025

GH-129437 is a backport of this pull request to the 3.12 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.12 only security fixes label Jan 29, 2025
sobolevn pushed a commit that referenced this pull request Jan 29, 2025
@miss-islington @srinivasreddy
[3.12] gh-129409: Fix Integer overflow - SEGV while writing data more…
cdaec31 
… than 2GB in CSV file (GH-129413) (#129437)

gh-129409: Fix Integer overflow -  SEGV while writing data more than 2GB in CSV file (GH-129413)
(cherry picked from commit 97b0ef0)

Co-authored-by: Srinivas Reddy Thatiparthy (తాటిపర్తి శ్రీనివాస్  రెడ్డి) <thatiparthysreenivas@gmail.com>
sobolevn pushed a commit that referenced this pull request Jan 29, 2025
@miss-islington @srinivasreddy
[3.13] gh-129409: Fix Integer overflow - SEGV while writing data more…
48b87d1 
… than 2GB in CSV file (GH-129413) (#129436)

gh-129409: Fix Integer overflow -  SEGV while writing data more than 2GB in CSV file (GH-129413)
(cherry picked from commit 97b0ef0)

Co-authored-by: Srinivas Reddy Thatiparthy (తాటిపర్తి శ్రీనివాస్  రెడ్డి) <thatiparthysreenivas@gmail.com>
@srinivasreddy srinivasreddy deleted the gh-129409 branch January 29, 2025 11:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tomasr8 tomasr8 tomasr8 left review comments

@sobolevn sobolevn sobolevn approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@srinivasreddy @sobolevn @tomasr8
X Tutup