X Tutup
The Wayback Machine - https://web.archive.org/web/20201123155711/https://github.com/shelljs/shelljs/issues/944
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shelljs.sort() changes the EOL '\r\n' (CRLF) into '\n' (LF) on a Windows PC #944

Open
alloutnow opened this issue Jun 10, 2019 · 3 comments
Open

Comments

@alloutnow
Copy link

@alloutnow alloutnow commented Jun 10, 2019

Node version (or tell us if you're using electron or some other framework):

Node v12.3.1

ShellJS version (the most recent version/Github branch you see the bug on):

shelljs@0.8.3

Operating system:

Microsoft Windows 10 Pro - v: 10.0.17763 Build 17763

Description of the bug:

Shelljs.sort() changes the EOL '\r\n' (CRLF) into '\n' (LF) on a Windows PC

Example ShellJS command to reproduce the error:

Create a file on a Windows PC containing the following text: 'xyz\r\nabc\r\n'. It has two lines of text each ending in the '\r\n' EOL

Let's call this file abc.txt.

Run the code shown below.

Examine the contents of the file, after sort, without overwriting it or changing in any way. If you inspect all characters you will find that, at least some of the EOL's will have changed. That is my conclusion after debugging a program I wrote.

On my system the file, after running the code below, would contain the string: 'abc\nxyz\r\n'

It's as if the last EOL is changed from CRLF to LF before sorting. That's strange behavior and causes problems, at least on a Windows system.

let shell = require('shelljs');
let file = './abc.txt';
// file contains the following text: 'xyz\r\nabc\r\n'

shell.sort(file).to(file);
// Now the file contains: 'abc\nxyz\r\n'
@nfischer
Copy link
Member

@nfischer nfischer commented Jul 8, 2019

Most shelljs commands assume that, when outputting as a string, the user wants cross-platform-consistent newlines. We chose \n for that.

Maybe .to() should convert \n to the OS's newline separator? This would corrupt binary data, although other methods corrupt data when they read it in off disk.

Happy to discuss better behavior here and to review a PR implementing the agreed-upon approach. This is low-priority and I will not work to fix this in the near future, so you should expect to write a PR yourself if you need this fixed.

@alloutnow
Copy link
Author

@alloutnow alloutnow commented Jul 10, 2019

Your answer seems a bit awkward to me, especially the end of the first paragraph: "We chose \n for that."

Why not just check os.EOL to decide if to use the POSIX or Windows end-of-line? That would be a real cross-platform way to handle the issue.

But, this is your decision, of course.

@alloutnow alloutnow closed this Jul 10, 2019
@alloutnow alloutnow reopened this Jul 10, 2019
@nfischer
Copy link
Member

@nfischer nfischer commented Jul 11, 2019

I think I may not have been clear. We convert \r\n to \n when we read files so that folks only have to check for one or the other in their javascript strings.

var output = shell.grep('foo', 'bar.txt');
// output might look like: "foo1\nfoo2\nfoo3\n"
// but dependent modules don't have to check for: "foo1\r\nfoo2\r\nfoo3\r\n"

I think we should better advertise this behavior, and think we need the inverse operation for .to() (otherwise we garble data on windows, as you've seen), but I still think there's value to converting to \n in JavaScript-land.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.
X Tutup