Add `-Delimiter` & `-NoTrailingDelimiter` parameters to `Set-Content/Add-Content` cmdlets #24648

ArmaanMcleod · 2024-12-07T04:37:54Z

PR Summary

Add -Delimiter & -NoTrailingDelimiter parameters to Set-Content/Add-Content cmdlets.

PR Context

I've added new parameters to Set-Content/Add-Content so a user can specify their own delimiters using -Delimiter when writing to files.

This included making changes to WriteContentCommandBase class to process all lines into a buffer in ProcessRecord() instead of writing one at a time, and do all the writing in EndProcessing() so we can append delimiters but also make sure the trailing delimiter can be suppressed using -NoTrailingDelimiter. This seemed much simpler to do since Value can be used with a pipeline which makes it difficult to detect the last line and suppress the final delimiter.

Update from above

When -NoTrailingDelimiter is specified, all delimiters are prepended to all lines except first line. Otherwise, we fall back to default behaviour and prepend delimiter after every line. This approach is more ideal than the previous buffering approach because streaming behaviour of input lines is maintained.

I also needed to make changes to the FileSystemProvider method IContentWriter GetContentWriter(string path) to handle new dynamic parameters in FileSystemContentWriterDynamicParameters for delimiters and also include a new overload for FileSystemContentReaderWriter. There is also some error handling if we use parameter combinations which are not supported.

Example 1

Write contents to file using delimiters:

> "a", "b", "c" | Set-Content -Path test.txt -Delimiter ","
> cat ./test.txt
a,b,c,

Example 2

Write contents to file using comma delimiter and remove trailing delimiter:

> "a", "b", "c" | Set-Content -Path test.txt -Delimiter "," -NoTrailingDelimiter
> cat ./test.txt
a,b,c

PR Checklist

…ontent cmdlets

ArmaanMcleod · 2024-12-07T13:49:52Z

@mklement0 Please review if you get a chance. I've added the delimiter parameters in, I will tackle the -NoTrailingNewLine parameter in another PR when we can get the other issue #5108 re-opened again 😄.

mklement0 · 2024-12-07T13:52:55Z

I'll leave the review to others, @ArmaanMcleod, but thank you for tackling this.

JunielKatarn · 2025-05-30T23:36:20Z

Does this also apply to Out-File?
If not, why is it excluded?

mklement0 · 2025-05-31T01:41:42Z

@JunielKatarn, please see #23906 (comment)

adityapatwardhan · 2025-12-03T18:06:34Z

Can the behavior of -Delimiter + -NoTrailingDelimiter be not achieved as follows

PS> "a", "b", "c" | Join-String -Separator "," | Set-Content test.txt
PS> cat test.txt
a,b,c

daxian-dbw · 2025-12-03T19:47:51Z

This included making changes to WriteContentCommandBase class to process all lines into a buffer in ProcessRecord() instead of writing one at a time, and do all the writing in EndProcessing() so we can append delimiters but also make sure the trailing delimiter can be suppressed using -NoTrailingDelimiter.

This breaks the streaming behavior. What if tons of strings are fed into the pipeline? Holding them up in a buffer could drastically increase the memory usage and might even cause an out-of-memory exception.

ArmaanMcleod · 2025-12-03T20:11:26Z

This included making changes to WriteContentCommandBase class to process all lines into a buffer in ProcessRecord() instead of writing one at a time, and do all the writing in EndProcessing() so we can append delimiters but also make sure the trailing delimiter can be suppressed using -NoTrailingDelimiter.

This breaks the streaming behavior. What if tons of strings are fed into the pipeline? Holding them up in a buffer could drastically increase the memory usage and might even cause an out-of-memory exception.

This effectively is what ConvertFrom-Json does as well.

daxian-dbw · 2025-12-03T20:32:09Z

ConvertFrom-Json is different. You need to wait for all the JSON content to be available to do the parsing, or otherwise it will fail on an in-complete JSON text.

But still, ConvertFrom-Json could be improved with a -Stream switch for the scenarios where it gets JSON Lines fed from the pipeline.

ArmaanMcleod · 2025-12-03T21:20:05Z

ConvertFrom-Json is different. You need to wait for all the JSON content to be available to do the parsing, or otherwise it will fail on an in-complete JSON text.

But still, ConvertFrom-Json could be improved with a -Stream switch for the scenarios where it gets JSON Lines fed from the pipeline.

Yeah its a similar issue with this feature as well since we need to detect last item which requires loading all lines into buffer. I wonder though if we can just keep streaming behaviour and only buffer if user wants to use these flags, unless I am missing something and this can all be achieved with streaming?

Might make command too complex for what its worth though, and still keeps performance issue around just to save some keystrokes and reduce pipelining.

Happy to close PR if this is not desired 🙂.

Copilot

Pull request overview

This PR adds new -Delimiter and -NoTrailingDelimiter parameters to the Set-Content and Add-Content cmdlets, allowing users to specify custom delimiters when writing content to files. The implementation includes a significant architectural change: content is now buffered in ProcessRecord() and written in EndProcessing() to enable proper handling of the trailing delimiter suppression. The PR includes comprehensive test coverage, parameter validation, and appropriate error messages for invalid parameter combinations.

Key Changes:

Added -Delimiter and -NoTrailingDelimiter parameters to enable custom delimiter control when writing content
Refactored content writing to use a buffering approach, accumulating content during pipeline processing and writing once in EndProcessing()
Added validation logic to prevent incompatible parameter combinations (Delimiter with NoNewLine/AsByteStream, NoTrailingDelimiter without Delimiter)

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
`test/powershell/Modules/Microsoft.PowerShell.Management/Set-Content.Tests.ps1`	Adds comprehensive test coverage for delimiter functionality including parameter validation, various delimiter types, and nested content handling
`test/powershell/Modules/Microsoft.PowerShell.Management/Add-Content.Tests.ps1`	Adds comprehensive test coverage for delimiter functionality with append operations, and removes trailing whitespace
`src/System.Management.Automation/resources/FileSystemProviderStrings.resx`	Adds error message strings for invalid parameter combinations
`src/System.Management.Automation/namespaces/FileSystemProvider.cs`	Adds dynamic parameters (Delimiter, NoTrailingDelimiter) and validation logic to GetContentWriter method
`src/System.Management.Automation/namespaces/FileSystemContentStream.cs`	Adds new constructor overload for delimiter support and refactors Write method to handle delimiter insertion and trailing delimiter suppression
`src/Microsoft.PowerShell.Commands.Management/commands/management/WriteContentCommandBase.cs`	Refactors ProcessRecord to buffer content and EndProcessing to perform actual writing, enabling proper last-item detection for delimiter suppression

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

test/powershell/Modules/Microsoft.PowerShell.Management/Set-Content.Tests.ps1

test/powershell/Modules/Microsoft.PowerShell.Management/Add-Content.Tests.ps1

Copilot · 2025-12-05T01:12:52Z

src/System.Management.Automation/namespaces/FileSystemProvider.cs

            string streamName = null;
            bool suppressNewline = false;

+            string delimiter = "\n";


The default delimiter value is hardcoded as "\n" here, but FileSystemContentReaderWriter uses a constant DefaultDelimiter = '\n'. Consider using a consistent approach, either by using the same constant or ensuring the default values match. This helps maintain consistency and makes future changes easier.

Suggested change

string delimiter = "\n";

string delimiter = FileSystemContentReaderWriter.DefaultDelimiter;

src/System.Management.Automation/resources/FileSystemProviderStrings.resx

src/System.Management.Automation/namespaces/FileSystemContentStream.cs

src/Microsoft.PowerShell.Commands.Management/commands/management/WriteContentCommandBase.cs

test/powershell/Modules/Microsoft.PowerShell.Management/Add-Content.Tests.ps1

src/System.Management.Automation/namespaces/FileSystemProvider.cs

…iter needs to be suppressed

…ntent.Tests.ps1 Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…trings.resx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…tream.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…ntent.Tests.ps1 Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

ArmaanMcleod · 2025-12-05T02:19:57Z

@daxian-dbw I think I found a good solution here.

Instead of buffering all lines to handle last line trailing delimiter, we can maintain streaming behaviour if we just prepend delimiter after the first line if last delimiter needs to be suppressed with -NoTrailingDelimiter. Otherwise, we just append delimiter normally.

I am not sure why I didn't do this before, but I think there should be no performance issue now 🙂.

iSazonov · 2025-12-05T04:21:41Z

src/System.Management.Automation/namespaces/FileSystemContentStream.cs

+                    // Always append delimter if we don't suppress last delimiter
+                    if (!_suppressLastDelimiter)
+                    {
+                        _writer.Write(_delimiter);
+                    }


We could assign _delimiter in Beginning as needed and write here unconditionally.

I kept same _usingDelimiter and _delimiter flags and set them in new FileSystemContentReaderWriter constructor. Seemed much simpler to manage it this way.

Logic is simpler in f6378d3 now. Default should be to append delimiter unless we need to prepend after first line.

Apologise above actually doesn't work, I have split the code paths using _suppressLastDelimiter in 6255eb3.

iSazonov · 2025-12-05T04:26:59Z

src/System.Management.Automation/namespaces/FileSystemProvider.cs

+        /// Gets the status of the delimiter parameter. Returns true
+        /// if the delimiter was explicitly specified by the user, false otherwise.
+        /// </summary>
+        public bool DelimiterSpecified


Do we really need this. It seems we could simply check for default value.

Existing dynamic parameters code checked for this flag, I just maintained it as well for this to keep it consistent. I think it can be simplified but perhaps in another PR when both FileSystemContentWriterDynamicParameters and FileSystemContentReaderDynamicParameters is consolidated.

We also should also centralise the default delimiter "\n" in a better place, it seems it is also hardcoded in many places.

This reverts commit f6378d3.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

ArmaanMcleod added 2 commits December 7, 2024 15:20

Add -Delimiter & -NoTrailingDelimiter parameters to Set-Content/Add-C…

95c8056

…ontent cmdlets

Fix trailing delimiter parameter description

22e3889

ArmaanMcleod requested review from adityapatwardhan, anmenaga and daxian-dbw as code owners December 7, 2024 04:37

Merge branch 'master' into set-content-delimeter

36a25b3

ArmaanMcleod mentioned this pull request Dec 7, 2024

Document -Delimiter & -NoTrailingDelimiter parameters for Set-Content/Add-Content cmdlets MicrosoftDocs/PowerShell-Docs#11567

Closed

2 tasks

microsoft-github-policy-service bot added the Review - Needed The PR is being reviewed label Dec 15, 2024

Merge branch 'master' into set-content-delimeter

1a627f9

Copilot AI review requested due to automatic review settings December 5, 2025 01:04

Copilot started reviewing on behalf of ArmaanMcleod December 5, 2025 01:05 View session

Copilot finished reviewing on behalf of ArmaanMcleod December 5, 2025 01:08

Copilot AI reviewed Dec 5, 2025

View reviewed changes

ArmaanMcleod and others added 8 commits December 5, 2025 12:56

Revert buffering changes on WriteContentCommandBase

977d55b

Add first line flag to make sure delimiter is prepended if last delim…

28657a5

…iter needs to be suppressed

Update test/powershell/Modules/Microsoft.PowerShell.Management/Set-Co…

e761d8b

…ntent.Tests.ps1 Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update test/powershell/Modules/Microsoft.PowerShell.Management/Add-Co…

45c18ae

…ntent.Tests.ps1 Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/System.Management.Automation/resources/FileSystemProviderS…

c12168d

…trings.resx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/System.Management.Automation/resources/FileSystemProviderS…

632945e

…trings.resx Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/System.Management.Automation/namespaces/FileSystemContentS…

1698c38

…tream.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/System.Management.Automation/namespaces/FileSystemProvider.cs

f31364f

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

ArmaanMcleod and others added 2 commits December 5, 2025 13:07

Update test/powershell/Modules/Microsoft.PowerShell.Management/Add-Co…

5272dd9

…ntent.Tests.ps1 Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Add comment for isFirstLine flag

55222a8

iSazonov reviewed Dec 5, 2025

View reviewed changes

Simplify delimiter condition to if/else

f6378d3

iSazonov added the CL-General Indicates that a PR should be marked as a general cmdlet change in the Change Log label Dec 5, 2025

ArmaanMcleod added 2 commits December 6, 2025 07:14

Revert "Simplify delimiter condition to if/else"

cfb8c43

This reverts commit f6378d3.

Minor fix to conditions to fix tests

6255eb3

iSazonov requested a review from Copilot December 6, 2025 05:13

Copilot started reviewing on behalf of iSazonov December 6, 2025 05:14 View session

Copilot AI reviewed Dec 6, 2025

View reviewed changes

iSazonov requested a review from Copilot December 11, 2025 17:58

Copilot started reviewing on behalf of iSazonov December 11, 2025 17:58 View session

Copilot AI reviewed Dec 11, 2025

View reviewed changes

Jan	FEB	Mar
	02
2025	2026	2027

	string delimiter = "\n";
	string delimiter = FileSystemContentReaderWriter.DefaultDelimiter;

Add -Delimiter & -NoTrailingDelimiter parameters to Set-Content/Add-Content cmdlets #24648

Are you sure you want to change the base?

Add -Delimiter & -NoTrailingDelimiter parameters to Set-Content/Add-Content cmdlets #24648

Uh oh!

Conversation

ArmaanMcleod commented Dec 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

PR Context

Example 1

Example 2

PR Checklist

Uh oh!

ArmaanMcleod commented Dec 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mklement0 commented Dec 7, 2024

Uh oh!

JunielKatarn commented May 30, 2025

Uh oh!

mklement0 commented May 31, 2025

Uh oh!

adityapatwardhan commented Dec 3, 2025

Uh oh!

daxian-dbw commented Dec 3, 2025

Uh oh!

ArmaanMcleod commented Dec 3, 2025

Uh oh!

daxian-dbw commented Dec 3, 2025

Uh oh!

ArmaanMcleod commented Dec 3, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ArmaanMcleod commented Dec 5, 2025

Uh oh!

iSazonov Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

ArmaanMcleod Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

ArmaanMcleod Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

iSazonov Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

ArmaanMcleod Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Add `-Delimiter` & `-NoTrailingDelimiter` parameters to `Set-Content/Add-Content` cmdlets #24648

Add `-Delimiter` & `-NoTrailingDelimiter` parameters to `Set-Content/Add-Content` cmdlets #24648

ArmaanMcleod commented Dec 7, 2024 •

edited

Loading

ArmaanMcleod commented Dec 7, 2024 •

edited

Loading