fs: refactoring declaratively with the Array.fromAsync function #54644
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nodejs-github-bot
added
fs
sonsurim
force-pushed
the
refactor-glob
branch
from
4dd3a9e
to
07d075a
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #54644 +/- ##
==========================================
+ Coverage 87.34% 87.89% +0.55%
==========================================
Files 649 651 +2
Lines 182524 183341 +817
Branches 35026 35713 +687
==========================================
+ Hits 159420 161155 +1735
+ Misses 16373 15464 -909
+ Partials 6731 6722 -9
|
RedYetiDev
reviewed
Aug 29, 2024
| for await (const entry of new Glob(pattern, options).glob()) { | ||
| ArrayPrototypePush(res, entry); | ||
| } | ||
| const res = await ArrayFromAsync(new Glob(pattern, options).glob()); |
There was a problem hiding this comment.
I'm not sure if there is a benchmark for this, but this should probably be benchmarked.
There was a problem hiding this comment.
@RedYetiDev Thanks for the review!
There are currently no benchmarks, so I'll add them!
There was a problem hiding this comment.
@RedYetiDev
Sorry for being late!
I added the fs glob benchmark f4025e5.
The benchmark results are as follows:
node-benchmark-compare compare-glob-result.csv
confidence improvement accuracy (*) (**) (***)
fs/bench-glob.js recursive='false' mode='async' pattern='*.js' dir='benchmark' n=1 0.68 % ±1.50% ±2.01% ±2.62%
fs/bench-glob.js recursive='false' mode='async' pattern='*.js' dir='benchmark' n=10 0.47 % ±1.59% ±2.11% ±2.75%
fs/bench-glob.js recursive='false' mode='async' pattern='*.js' dir='benchmark' n=100 * 0.68 % ±0.63% ±0.84% ±1.10%
fs/bench-glob.js recursive='false' mode='async' pattern='*.js' dir='lib' n=1 -0.40 % ±1.61% ±2.15% ±2.82%
fs/bench-glob.js recursive='false' mode='async' pattern='*.js' dir='lib' n=10 0.23 % ±1.31% ±1.74% ±2.27%
fs/bench-glob.js recursive='false' mode='async' pattern='*.js' dir='lib' n=100 1.67 % ±1.67% ±2.24% ±2.96%
fs/bench-glob.js recursive='false' mode='async' pattern='*.js' dir='test/parallel' n=1 0.77 % ±0.83% ±1.11% ±1.46%
fs/bench-glob.js recursive='false' mode='async' pattern='*.js' dir='test/parallel' n=10 *** 3.75 % ±1.90% ±2.55% ±3.37%
fs/bench-glob.js recursive='false' mode='async' pattern='*.js' dir='test/parallel' n=100 * 1.51 % ±1.16% ±1.54% ±2.01%
fs/bench-glob.js recursive='false' mode='async' pattern='**/*' dir='benchmark' n=1 0.13 % ±1.53% ±2.04% ±2.67%
fs/bench-glob.js recursive='false' mode='async' pattern='**/*' dir='benchmark' n=10 1.42 % ±1.64% ±2.20% ±2.91%
fs/bench-glob.js recursive='false' mode='async' pattern='**/*' dir='benchmark' n=100 0.27 % ±0.53% ±0.71% ±0.93%
fs/bench-glob.js recursive='false' mode='async' pattern='**/*' dir='lib' n=1 *** 1.39 % ±0.75% ±1.00% ±1.31%
fs/bench-glob.js recursive='false' mode='async' pattern='**/*' dir='lib' n=10 -3.22 % ±5.11% ±6.88% ±9.13%
fs/bench-glob.js recursive='false' mode='async' pattern='**/*' dir='lib' n=100 0.05 % ±0.78% ±1.04% ±1.35%
fs/bench-glob.js recursive='false' mode='async' pattern='**/*' dir='test/parallel' n=1 1.09 % ±1.37% ±1.82% ±2.38%
fs/bench-glob.js recursive='false' mode='async' pattern='**/*' dir='test/parallel' n=10 *** 5.67 % ±1.26% ±1.69% ±2.23%
fs/bench-glob.js recursive='false' mode='async' pattern='**/*' dir='test/parallel' n=100 1.22 % ±1.55% ±2.07% ±2.72%
fs/bench-glob.js recursive='false' mode='async' pattern='**/**.js' dir='benchmark' n=1 0.39 % ±3.01% ±4.03% ±5.27%
fs/bench-glob.js recursive='false' mode='async' pattern='**/**.js' dir='benchmark' n=10 0.72 % ±1.31% ±1.74% ±2.27%
fs/bench-glob.js recursive='false' mode='async' pattern='**/**.js' dir='benchmark' n=100 0.35 % ±1.11% ±1.49% ±1.97%
fs/bench-glob.js recursive='false' mode='async' pattern='**/**.js' dir='lib' n=1 1.28 % ±1.48% ±1.99% ±2.62%
fs/bench-glob.js recursive='false' mode='async' pattern='**/**.js' dir='lib' n=10 0.33 % ±1.02% ±1.35% ±1.76%
fs/bench-glob.js recursive='false' mode='async' pattern='**/**.js' dir='lib' n=100 0.03 % ±0.71% ±0.94% ±1.22%
fs/bench-glob.js recursive='false' mode='async' pattern='**/**.js' dir='test/parallel' n=1 * 1.49 % ±1.44% ±1.92% ±2.51%
fs/bench-glob.js recursive='false' mode='async' pattern='**/**.js' dir='test/parallel' n=10 *** 7.41 % ±2.81% ±3.79% ±5.03%
fs/bench-glob.js recursive='false' mode='async' pattern='**/**.js' dir='test/parallel' n=100 *** 1.56 % ±0.75% ±1.00% ±1.31%
fs/bench-glob.js recursive='false' mode='sync' pattern='*.js' dir='benchmark' n=1 -0.43 % ±1.26% ±1.68% ±2.19%
fs/bench-glob.js recursive='false' mode='sync' pattern='*.js' dir='benchmark' n=10 -0.12 % ±1.12% ±1.49% ±1.94%
fs/bench-glob.js recursive='false' mode='sync' pattern='*.js' dir='benchmark' n=100 -0.10 % ±0.91% ±1.21% ±1.58%
fs/bench-glob.js recursive='false' mode='sync' pattern='*.js' dir='lib' n=1 2.79 % ±4.15% ±5.59% ±7.39%
fs/bench-glob.js recursive='false' mode='sync' pattern='*.js' dir='lib' n=10 0.12 % ±2.12% ±2.83% ±3.70%
fs/bench-glob.js recursive='false' mode='sync' pattern='*.js' dir='lib' n=100 0.07 % ±3.03% ±4.04% ±5.26%
fs/bench-glob.js recursive='false' mode='sync' pattern='*.js' dir='test/parallel' n=1 0.41 % ±1.25% ±1.66% ±2.17%
fs/bench-glob.js recursive='false' mode='sync' pattern='*.js' dir='test/parallel' n=10 0.79 % ±2.11% ±2.81% ±3.66%
fs/bench-glob.js recursive='false' mode='sync' pattern='*.js' dir='test/parallel' n=100 0.08 % ±0.92% ±1.22% ±1.59%
fs/bench-glob.js recursive='false' mode='sync' pattern='**/*' dir='benchmark' n=1 -1.47 % ±4.43% ±5.96% ±7.90%
fs/bench-glob.js recursive='false' mode='sync' pattern='**/*' dir='benchmark' n=10 0.49 % ±0.51% ±0.68% ±0.89%
fs/bench-glob.js recursive='false' mode='sync' pattern='**/*' dir='benchmark' n=100 1.15 % ±1.23% ±1.66% ±2.19%
fs/bench-glob.js recursive='false' mode='sync' pattern='**/*' dir='lib' n=1 0.57 % ±0.91% ±1.21% ±1.58%
fs/bench-glob.js recursive='false' mode='sync' pattern='**/*' dir='lib' n=10 -1.87 % ±4.44% ±5.97% ±7.90%
fs/bench-glob.js recursive='false' mode='sync' pattern='**/*' dir='lib' n=100 2.01 % ±3.92% ±5.29% ±7.01%
fs/bench-glob.js recursive='false' mode='sync' pattern='**/*' dir='test/parallel' n=1 -0.10 % ±0.88% ±1.18% ±1.54%
fs/bench-glob.js recursive='false' mode='sync' pattern='**/*' dir='test/parallel' n=10 -0.12 % ±0.83% ±1.11% ±1.45%
fs/bench-glob.js recursive='false' mode='sync' pattern='**/*' dir='test/parallel' n=100 -0.17 % ±0.79% ±1.06% ±1.38%
fs/bench-glob.js recursive='false' mode='sync' pattern='**/**.js' dir='benchmark' n=1 * 1.12 % ±1.08% ±1.44% ±1.88%
fs/bench-glob.js recursive='false' mode='sync' pattern='**/**.js' dir='benchmark' n=10 1.85 % ±2.73% ±3.65% ±4.80%
fs/bench-glob.js recursive='false' mode='sync' pattern='**/**.js' dir='benchmark' n=100 0.16 % ±1.07% ±1.42% ±1.85%
fs/bench-glob.js recursive='false' mode='sync' pattern='**/**.js' dir='lib' n=1 0.54 % ±1.06% ±1.42% ±1.84%
fs/bench-glob.js recursive='false' mode='sync' pattern='**/**.js' dir='lib' n=10 -0.62 % ±1.21% ±1.62% ±2.14%
fs/bench-glob.js recursive='false' mode='sync' pattern='**/**.js' dir='lib' n=100 0.66 % ±1.14% ±1.52% ±2.00%
fs/bench-glob.js recursive='false' mode='sync' pattern='**/**.js' dir='test/parallel' n=1 -2.67 % ±5.08% ±6.85% ±9.09%
fs/bench-glob.js recursive='false' mode='sync' pattern='**/**.js' dir='test/parallel' n=10 -0.12 % ±0.73% ±0.97% ±1.26%
fs/bench-glob.js recursive='false' mode='sync' pattern='**/**.js' dir='test/parallel' n=100 0.55 % ±0.60% ±0.80% ±1.05%
fs/bench-glob.js recursive='true' mode='async' pattern='*.js' dir='benchmark' n=1 0.22 % ±1.42% ±1.90% ±2.49%
fs/bench-glob.js recursive='true' mode='async' pattern='*.js' dir='benchmark' n=10 0.08 % ±2.60% ±3.50% ±4.64%
fs/bench-glob.js recursive='true' mode='async' pattern='*.js' dir='benchmark' n=100 0.08 % ±0.78% ±1.03% ±1.35%
fs/bench-glob.js recursive='true' mode='async' pattern='*.js' dir='lib' n=1 -0.03 % ±1.26% ±1.68% ±2.20%
fs/bench-glob.js recursive='true' mode='async' pattern='*.js' dir='lib' n=10 0.38 % ±2.30% ±3.07% ±4.03%
fs/bench-glob.js recursive='true' mode='async' pattern='*.js' dir='lib' n=100 2.81 % ±3.46% ±4.65% ±6.16%
fs/bench-glob.js recursive='true' mode='async' pattern='*.js' dir='test/parallel' n=1 *** 1.77 % ±0.50% ±0.66% ±0.86%
fs/bench-glob.js recursive='true' mode='async' pattern='*.js' dir='test/parallel' n=10 *** 4.75 % ±1.45% ±1.93% ±2.52%
fs/bench-glob.js recursive='true' mode='async' pattern='*.js' dir='test/parallel' n=100 0.26 % ±1.28% ±1.72% ±2.27%
fs/bench-glob.js recursive='true' mode='async' pattern='**/*' dir='benchmark' n=1 0.45 % ±1.22% ±1.63% ±2.13%
fs/bench-glob.js recursive='true' mode='async' pattern='**/*' dir='benchmark' n=10 0.31 % ±0.81% ±1.09% ±1.42%
fs/bench-glob.js recursive='true' mode='async' pattern='**/*' dir='benchmark' n=100 -0.18 % ±0.84% ±1.13% ±1.47%
fs/bench-glob.js recursive='true' mode='async' pattern='**/*' dir='lib' n=1 0.21 % ±1.48% ±1.97% ±2.56%
fs/bench-glob.js recursive='true' mode='async' pattern='**/*' dir='lib' n=10 -0.23 % ±0.57% ±0.76% ±1.00%
fs/bench-glob.js recursive='true' mode='async' pattern='**/*' dir='lib' n=100 0.05 % ±1.43% ±1.92% ±2.52%
fs/bench-glob.js recursive='true' mode='async' pattern='**/*' dir='test/parallel' n=1 ** 2.56 % ±1.63% ±2.18% ±2.88%
fs/bench-glob.js recursive='true' mode='async' pattern='**/*' dir='test/parallel' n=10 *** 4.31 % ±1.28% ±1.71% ±2.23%
fs/bench-glob.js recursive='true' mode='async' pattern='**/*' dir='test/parallel' n=100 * 1.00 % ±0.76% ±1.02% ±1.34%
fs/bench-glob.js recursive='true' mode='async' pattern='**/**.js' dir='benchmark' n=1 2.96 % ±4.42% ±5.95% ±7.89%
fs/bench-glob.js recursive='true' mode='async' pattern='**/**.js' dir='benchmark' n=10 0.75 % ±1.01% ±1.35% ±1.78%
fs/bench-glob.js recursive='true' mode='async' pattern='**/**.js' dir='benchmark' n=100 0.04 % ±0.75% ±1.00% ±1.32%
fs/bench-glob.js recursive='true' mode='async' pattern='**/**.js' dir='lib' n=1 1.02 % ±1.31% ±1.74% ±2.27%
fs/bench-glob.js recursive='true' mode='async' pattern='**/**.js' dir='lib' n=10 -0.29 % ±1.13% ±1.52% ±1.99%
fs/bench-glob.js recursive='true' mode='async' pattern='**/**.js' dir='lib' n=100 0.26 % ±0.70% ±0.93% ±1.21%
fs/bench-glob.js recursive='true' mode='async' pattern='**/**.js' dir='test/parallel' n=1 *** 2.64 % ±1.26% ±1.69% ±2.23%
fs/bench-glob.js recursive='true' mode='async' pattern='**/**.js' dir='test/parallel' n=10 *** 5.59 % ±1.19% ±1.59% ±2.10%
fs/bench-glob.js recursive='true' mode='async' pattern='**/**.js' dir='test/parallel' n=100 ** 1.59 % ±1.12% ±1.49% ±1.96%
fs/bench-glob.js recursive='true' mode='sync' pattern='*.js' dir='benchmark' n=1 1.62 % ±2.70% ±3.61% ±4.72%
fs/bench-glob.js recursive='true' mode='sync' pattern='*.js' dir='benchmark' n=10 -0.09 % ±1.25% ±1.67% ±2.20%
fs/bench-glob.js recursive='true' mode='sync' pattern='*.js' dir='benchmark' n=100 0.03 % ±0.60% ±0.80% ±1.04%
fs/bench-glob.js recursive='true' mode='sync' pattern='*.js' dir='lib' n=1 0.87 % ±1.34% ±1.78% ±2.32%
fs/bench-glob.js recursive='true' mode='sync' pattern='*.js' dir='lib' n=10 0.81 % ±1.74% ±2.32% ±3.03%
fs/bench-glob.js recursive='true' mode='sync' pattern='*.js' dir='lib' n=100 0.71 % ±1.92% ±2.56% ±3.35%
fs/bench-glob.js recursive='true' mode='sync' pattern='*.js' dir='test/parallel' n=1 -0.14 % ±0.75% ±1.01% ±1.33%
fs/bench-glob.js recursive='true' mode='sync' pattern='*.js' dir='test/parallel' n=10 0.16 % ±0.61% ±0.81% ±1.06%
fs/bench-glob.js recursive='true' mode='sync' pattern='*.js' dir='test/parallel' n=100 0.09 % ±1.20% ±1.60% ±2.08%
fs/bench-glob.js recursive='true' mode='sync' pattern='**/*' dir='benchmark' n=1 0.35 % ±1.17% ±1.57% ±2.06%
fs/bench-glob.js recursive='true' mode='sync' pattern='**/*' dir='benchmark' n=10 0.77 % ±0.79% ±1.05% ±1.38%
fs/bench-glob.js recursive='true' mode='sync' pattern='**/*' dir='benchmark' n=100 1.09 % ±1.19% ±1.60% ±2.12%
fs/bench-glob.js recursive='true' mode='sync' pattern='**/*' dir='lib' n=1 0.57 % ±1.04% ±1.38% ±1.80%
fs/bench-glob.js recursive='true' mode='sync' pattern='**/*' dir='lib' n=10 -0.57 % ±2.63% ±3.52% ±4.65%
fs/bench-glob.js recursive='true' mode='sync' pattern='**/*' dir='lib' n=100 2.33 % ±3.84% ±5.17% ±6.86%
fs/bench-glob.js recursive='true' mode='sync' pattern='**/*' dir='test/parallel' n=1 -0.26 % ±0.60% ±0.80% ±1.04%
fs/bench-glob.js recursive='true' mode='sync' pattern='**/*' dir='test/parallel' n=10 0.28 % ±0.66% ±0.88% ±1.15%
fs/bench-glob.js recursive='true' mode='sync' pattern='**/*' dir='test/parallel' n=100 0.05 % ±0.93% ±1.24% ±1.61%
fs/bench-glob.js recursive='true' mode='sync' pattern='**/**.js' dir='benchmark' n=1 * 1.61 % ±1.44% ±1.94% ±2.55%
fs/bench-glob.js recursive='true' mode='sync' pattern='**/**.js' dir='benchmark' n=10 0.77 % ±3.60% ±4.82% ±6.33%
fs/bench-glob.js recursive='true' mode='sync' pattern='**/**.js' dir='benchmark' n=100 1.17 % ±1.52% ±2.05% ±2.71%
fs/bench-glob.js recursive='true' mode='sync' pattern='**/**.js' dir='lib' n=1 0.88 % ±0.93% ±1.23% ±1.60%
fs/bench-glob.js recursive='true' mode='sync' pattern='**/**.js' dir='lib' n=10 -0.21 % ±0.92% ±1.22% ±1.59%
fs/bench-glob.js recursive='true' mode='sync' pattern='**/**.js' dir='lib' n=100 0.86 % ±1.28% ±1.72% ±2.27%
fs/bench-glob.js recursive='true' mode='sync' pattern='**/**.js' dir='test/parallel' n=1 0.28 % ±0.50% ±0.66% ±0.87%
fs/bench-glob.js recursive='true' mode='sync' pattern='**/**.js' dir='test/parallel' n=10 1.55 % ±2.77% ±3.71% ±4.90%
fs/bench-glob.js recursive='true' mode='sync' pattern='**/**.js' dir='test/parallel' n=100 * 0.86 % ±0.86% ±1.15% ±1.52%
Be aware that when doing many comparisons the risk of a false-positive result increases.
In this case, there are 108 comparisons, you can thus expect the following amount of false-positive results:
5.40 false positives, when considering a 5% risk acceptance (*, **, ***),
1.08 false positives, when considering a 1% risk acceptance (**, ***),
0.11 false positives, when considering a 0.1% risk acceptance (***)Performance is significantly improved when asynchronous compared to synchronous.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The following applies to this PR:
for await...ofloop withArrayFromAsync().ArrayPrototypePush.ArrayFromAsynctype toprimordials.d.tsfileArray.fromAsyncwas defined relatively recently, but I used it because I knew it was officially adopted in Node.js v22. (MDN Document)