X Tutup
The Wayback Machine - https://web.archive.org/web/20231209122751/https://github.com/PowerShell/PowerShell/issues/14775
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

STDOUT/STDIN buffer size error when passing arguments into Windows PowerShell #14775

Closed
KUTlime opened this issue Feb 12, 2021 · 13 comments
Closed

STDOUT/STDIN buffer size error when passing arguments into Windows PowerShell #14775

KUTlime opened this issue Feb 12, 2021 · 13 comments
Labels
Needs-Triage The issue is new and needs to be triaged by a work group. Resolution-No Activity Issue has had no activity for 6 months or more WG-Interactive-Console the console experience

Comments

@KUTlime
Copy link

KUTlime commented Feb 12, 2021 •
edited

This issue is partially an issue of Windows PowerShell and PowerShell.

When I try pass a PSCustomObject loaded from JSON from PowerShell into Windows PowerShell, it will fail once JSON pass a certain size level.

PowerShell 7.1.1 and Windows PowerShell throw different errors and passing through of PSCustomObject argument doesn't work from properly. It depends on the argument size or complexity, see this screenshot.

2021-02-12 150531 l5lxRrDMSR

The problem disappears when I remove any two subkeys from root\WebApp in the configuration JSON.

Steps to reproduce

For a minimal, reproducible example, see this comment. When I debug the PS run, it pointed me to STDOUT/STDIN buffer size too.

$configuration = Get-Content .\configuration.json | ConvertFrom-Json
$block = {param([pscustomobject]$inputArgument) $PSVersionTable; Write-Host ('Test ' + $inputArgument.AxisUser.Name)}
powershell -nologo -noprofile $block -Args $configuration

Expected behavior

Windows PowerShell

Name                           Value
----                           -----
PSVersion                      5.1.18362.1171
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.18362.1171
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
Test String from json

PowerShell 7.1.1

Name                           Value
----                           -----
PSVersion                      5.1.18362.1171
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.18362.1171
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
Test String from json

Actual behavior

Windows PowerShell

Program 'powershell.exe' failed to run: Filename or Extension too longAt line:1
char:1
+ powershell -nologo -noprofile $block -Args $configuration
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.
At line:1 char:1
+ powershell -nologo -noprofile $block -Args $configuration
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ResourceUnavailable: (:) [], ApplicationFailedException
    + FullyQualifiedErrorId : NativeCommandFailed

PowerShell 7.1.1

ResourceUnavailable: Program 'powershell.exe' failed to run: The Process object must have the UseShellExecute property set to false in order to use environment variables.At line:1 char:1
+ powershell -nologo -noprofile $block -Args $configuration
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.

Environment data

Name                           Value
----                           -----
PSVersion                      7.1.1
PSEdition                      Core
GitCommitId                    7.1.1
OS                             Microsoft Windows 10.0.18363
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Configuration JSON

{
    "SupportedPSVersion": "7.1.0",
    "Localhost": true,
    "SourceCodeLocalPath": "%USERPROFILE%\\source\\CompanyName\\ProductName\\",
    "AzureDevOpsUser": {
        "Name": "",
        "Password": ""
    },
    "VPNUserName": "John.Doe",
    "ProductNameUser": {
        "Name": "String from json",
        "FullName": "ProductName DEV user",
        "Password": "123456",
        "Group": "Department"
    },
    "NetworkHost": "0.0.0.0",
    "AppPool": {
        "Name": "ProductName"
    },
    "Website": [
        {
            "Name": "ProductNameWeb",
            "Port": 1810,
            "ApplicationPool": "ProductName",
            "BindingInformation": "*:1810:",
            "Id": 2,
            "PhysicalPath": "%SystemDrive%\\inetpub\\wwwroot",
            "SelfSignedCertificatePassword": "WithGreatPowerComesGreatResponsibility"
        },
        {
            "Name": "ProductNameApp",
            "Port": 1985,
            "ApplicationPool": "ProductName",
            "BindingInformation": "*:1985:",
            "Id": 3,
            "PhysicalPath": "%SystemDrive%\\inetpub\\wwwroot"
        }
    ],
    "FrontendWebsiteId": 2,
    "BackendWebsiteId": 3,
    "FrontendEndpointToSubstitude": "ProductNamev2devweb1.ProductNamegraphics.tv",
    "BackendEndpointToSubstitude": "ProductNamev2devapp1.ProductNamegraphics.tv",
    "WebApp": {
        "Portal": {
            "FilePathToConfig": "%USERPROFILE%\\source\\CompanyName\\ProductName\\portal\\CompanyName.Portal.Web\\Root.Web.Release-Dev.config",
            "TargetAbsolutePathToConfig": "%SystemDrive%\\inetpub\\wwwroot\\Web.config",
            "ProjectsToPublish": [
                {
                    "RelativeProjectPath": "portal\\CompanyName.Portal.Web\\CompanyName.Portal.Web.csproj",
                    "RelativePublishProfilePath": "portal\\CompanyName.Portal.Web\\Properties\\PublishProfiles\\FolderProfile.pubxml"
                },
                {
                    "RelativeProjectPath": "portal\\CompanyName.Portal.API\\CompanyName.Portal.API.csproj",
                    "RelativePublishProfilePath": "portal\\CompanyName.Portal.API\\Properties\\PublishProfiles\\FolderProfile.pubxml"
                },
                {
                    "RelativeProjectPath": "portal\\CompanyName.Portal.Application.WebService\\CompanyName.Portal.Application.WebService.csproj",
                    "RelativePublishProfilePath": "portal\\CompanyName.Portal.Application.WebService\\Properties\\PublishProfiles\\FolderProfile.pubxml"
                }
            ],
            "SymbolicLinkToCreate": [],
            "AppToCreateOnFrontendWebsite": [
                {
                    "Name": "Portal",
                    "PhysicalPath": "%SystemDrive%\\inetpub\\wwwroot\\Portal"
                },
                {
                    "Name": "PortalApi",
                    "PhysicalPath": "%SystemDrive%\\inetpub\\wwwroot\\PortalApi"
                }
            ],
            "AppToCreateOnBackendWebsite": [
                {
                    "Name": "Portal",
                    "PhysicalPath": "%SystemDrive%\\inetpub\\wwwroot\\CompanyName.Portal.Application.WebService"
                }
            ],
            "HomePageFiles": {
                "TargetWebApplication": "Portal",
                "Name": "HomePageFiles",
                "PhysicalPath": "\\\\0.0.0.0\\pogo\\ProductNamev2\\dev\\Portal\\HomePageFiles"
            }
        },
        "Track": {
            "ProjectsToPublish": [
                {
                    "RelativeProjectPath": "track\\CompanyName.Track.Web\\CompanyName.Track.Web.csproj",
                    "RelativePublishProfilePath": "track\\CompanyName.Track.Web\\Properties\\PublishProfiles\\FolderProfile.pubxml"
                },
                {
                    "RelativeProjectPath": "track\\CompanyName.Track.Api\\CompanyName.Track.Api.csproj",
                    "RelativePublishProfilePath": "track\\CompanyName.Track.Api\\Properties\\PublishProfiles\\FolderProfile.pubxml"
                },
                {
                    "RelativeProjectPath": "track\\CompanyName.Track.WebService\\CompanyName.Track.WebService.csproj",
                    "RelativePublishProfilePath": "track\\CompanyName.Track.WebService\\Properties\\PublishProfiles\\FolderProfile.pubxml"
                },
                {
                    "RelativeProjectPath": "track\\CompanyName.Track.ConfigurationDomain\\CompanyName.Track.ConfigurationDomain.csproj",
                    "RelativePublishProfilePath": "track\\CompanyName.Track.ConfigurationDomain\\Properties\\PublishProfiles\\FolderProfile.pubxml"
                }
            ],
            "SymbolicLinkToCreate": [
                {
                    "RelativePublishProfilePath": "track\\CompanyName.Track.Web\\Properties\\PublishProfiles\\FolderProfile.pubxml",
                    "TargetRelativePath": "bin\\Shared",
                    "SourcePath": "%SystemDrive%\\inetpub\\wwwroot\\Portal\\bin"
                },
                {
                    "RelativePublishProfilePath": "track\\CompanyName.Track.Web\\Properties\\PublishProfiles\\FolderProfile.pubxml",
                    "TargetRelativePath": "Shared",
                    "SourcePath": "%SystemDrive%\\inetpub\\wwwroot\\Portal\\Shared"
                }
            ],
            "AppToCreateOnFrontendWebsite": [
                {
                    "Name": "Track",
                    "PhysicalPath": "%SystemDrive%\\inetpub\\wwwroot\\Track"
                },
                {
                    "Name": "TrackApi",
                    "PhysicalPath": "%SystemDrive%\\inetpub\\wwwroot\\CompanyName.Track.Api"
                },
                {
                    "Name": "ConfigurationDomain",
                    "PhysicalPath": "%SystemDrive%\\inetpub\\wwwroot\\CompanyName.Track.ConfigurationDomain"
                }
            ],
            "AppToCreateOnBackendWebsite": [
                {
                    "Name": "Track",
                    "PhysicalPath": "%SystemDrive%\\inetpub\\wwwroot\\CompanyName.Track.WebService"
                }
            ],
            "Cache": {
                "LocalFolderPath": "%SystemDrive%\\inetpub\\wwwroot\\TrackCache",
                "Name": "TrackCache\\Cache",
                "PhysicalPath": "\\\\1.1.1.1\\pogo\\ProductNamev2\\dev\\track"
            },
            "VueJS": {
                "RelativeSourcePath": "track\\CompanyName.Track.Web\\",
                "RelativeBuildPath": "track\\CompanyName.Track.Web\\src\\build",
                "Destination": "%SystemDrive%\\inetpub\\wwwroot\\Track\\src\\"
            }
        },
        "ProductNameAPI": {
            "ProjectsToPublish": [
                {
                    "RelativeProjectPath": "ProductNameapi\\ProductNameAPI\\ProductNameAPI.csproj",
                    "RelativePublishProfilePath": "ProductNameapi\\ProductNameAPI\\Properties\\PublishProfiles\\FolderProfile.pubxml"
                }
            ],
            "SymbolicLinkToCreate": [],
            "AppToCreateOnFrontendWebsite": [
                {
                    "Name": "ProductNameApi",
                    "PhysicalPath": "%SystemDrive%\\inetpub\\wwwroot\\ProductNameApi"
                }
            ],
            "AppToCreateOnBackendWebsite": []
        },
        "Compose": {
            "ProjectsToPublish": [
                {
                    "RelativeProjectPath": "compose\\CompanyName.Compose.Web\\CompanyName.Compose.Web.csproj",
                    "RelativePublishProfilePath": "compose\\CompanyName.Compose.Web\\Properties\\PublishProfiles\\FolderProfile.pubxml"
                }
            ],
            "SymbolicLinkToCreate": [
                {
                    "RelativePublishProfilePath": "compose\\CompanyName.Compose.Web\\Properties\\PublishProfiles\\FolderProfile.pubxml",
                    "TargetRelativePath": "bin\\Shared",
                    "SourcePath": "%SystemDrive%\\inetpub\\wwwroot\\Portal\\bin"
                }
            ],
            "AppToCreateOnFrontendWebsite": [
                {
                    "Name": "Compose",
                    "PhysicalPath": "%SystemDrive%\\inetpub\\wwwroot\\Compose"
                }
            ],
            "AppToCreateOnBackendWebsite": []
        },
        "Charts": {
            "ProjectsToPublish": [
                {
                    "RelativeProjectPath": "charts\\CompanyName.Charts.Web\\CompanyName.Charts.Web.csproj",
                    "RelativePublishProfilePath": "charts\\CompanyName.Charts.Web\\Properties\\PublishProfiles\\FolderProfile.pubxml"
                }
            ],
            "SymbolicLinkToCreate": [
                {
                    "RelativePublishProfilePath": "charts\\CompanyName.Charts.Web\\Properties\\PublishProfiles\\FolderProfile.pubxml",
                    "TargetRelativePath": "bin\\Shared",
                    "SourcePath": "%SystemDrive%\\inetpub\\wwwroot\\Portal\\bin"
                }
            ],
            "AppToCreateOnFrontendWebsite": [
                {
                    "Name": "Charts",
                    "PhysicalPath": "%SystemDrive%\\inetpub\\wwwroot\\Charts"
                }
            ],
            "AppToCreateOnBackendWebsite": [],
            "VueJS": {
                "RelativeSourcePath": "charts\\CompanyName.Charts.Web\\",
                "RelativeBuildPath": "charts\\CompanyName.Charts.Web\\src\\build",
                "Destination": "%SystemDrive%\\inetpub\\wwwroot\\Charts\\src\\"
            }
        },
        "Order": {
            "ProjectsToPublish": [
                {
                    "RelativeProjectPath": "order\\CompanyName.Order.Web\\CompanyName.Order.Web.csproj",
                    "RelativePublishProfilePath": "order\\CompanyName.Order.Web\\Properties\\PublishProfiles\\FolderProfile.pubxml"
                }
            ],
            "SymbolicLinkToCreate": [
                {
                    "RelativePublishProfilePath": "order\\CompanyName.Order.Web\\Properties\\PublishProfiles\\FolderProfile.pubxml",
                    "TargetRelativePath": "bin\\Shared",
                    "SourcePath": "%SystemDrive%\\inetpub\\wwwroot\\Portal\\bin"
                }
            ],
            "AppToCreateOnFrontendWebsite": [
                {
                    "Name": "Order",
                    "PhysicalPath": "%SystemDrive%\\inetpub\\wwwroot\\Order"
                }
            ],
            "AppToCreateOnBackendWebsite": [],
            "MediaFormatImages": {
                "LocalFolderPath": "%SystemDrive%\\inetpub\\wwwroot\\MediaFormatImages",
                "Name": "MediaFormatImages",
                "PhysicalPath": "\\\\1.1.1.1\\pogo\\ProductNamev2\\dev\\Order\\MediaFormatImages"
            }
        },
        "Typefaces": {
            "ProjectsToPublish": [
                {
                    "RelativeProjectPath": "typefaces\\CompanyName.Typefaces.Web\\CompanyName.Typefaces.Web.csproj",
                    "RelativePublishProfilePath": "typefaces\\CompanyName.Typefaces.Web\\Properties\\PublishProfiles\\FolderProfile.pubxml"
                }
            ],
            "SymbolicLinkToCreate": [
                {
                    "RelativePublishProfilePath": "typefaces\\CompanyName.Typefaces.Web\\Properties\\PublishProfiles\\FolderProfile.pubxml",
                    "TargetRelativePath": "bin\\Shared",
                    "SourcePath": "%SystemDrive%\\inetpub\\wwwroot\\Portal\\bin"
                }
            ],
            "AppToCreateOnFrontendWebsite": [
                {
                    "Name": "Typefaces",
                    "PhysicalPath": "%SystemDrive%\\inetpub\\wwwroot\\Typefaces"
                }
            ],
            "AppToCreateOnBackendWebsite": []
        },
        "Transport": {
            "ProjectsToPublish": [
                {
                    "RelativeProjectPath": "transport\\CompanyName.Transport.Web\\CompanyName.Transport.Web.csproj",
                    "RelativePublishProfilePath": "transport\\CompanyName.Transport.Web\\Properties\\PublishProfiles\\FolderProfile.pubxml"
                }
            ],
            "SymbolicLinkToCreate": [
                {
                    "RelativePublishProfilePath": "transport\\CompanyName.Transport.Web\\Properties\\PublishProfiles\\FolderProfile.pubxml",
                    "TargetRelativePath": "bin\\Shared",
                    "SourcePath": "%SystemDrive%\\inetpub\\wwwroot\\Portal\\bin"
                }
            ],
            "AppToCreateOnFrontendWebsite": [
                {
                    "Name": "Transport",
                    "PhysicalPath": "%SystemDrive%\\inetpub\\wwwroot\\Transport"
                }
            ],
            "AppToCreateOnBackendWebsite": []
        }
    },
    "NuGet": {
        "RestoreProjectForAzureCredentialManager": "%USERPROFILE%\\source\\CompanyName\\ProductName\\portal\\CompanyName.Portal.Web\\CompanyName.Portal.Web.csproj"
    }
}
@KUTlime KUTlime added the Needs-Triage The issue is new and needs to be triaged by a work group. label Feb 12, 2021
@iRon7
Copy link

iRon7 commented Feb 13, 2021 •
edited

Unlike the CMD prompt, PowerShell doesn't automatically Expand Environmental Variables which leaves an invalid json file. This means that you will need to explicitly expand the $configuration file before passing it as an argument.

In other works, your long/complex json configuration works fine for me when it is first expanded:

$configuration = Get-Content .\configuration.json | ConvertFrom-Json
$configuration = [System.Environment]::ExpandEnvironmentVariables($configuration)
$block = {param([pscustomobject]$inputArgument) $PSVersionTable; Write-Host ('Test ' + $inputArgument.AxisUser.Name)}
powershell -nologo -noprofile $block -Args $configuration

@KUTlime
Copy link
Author

KUTlime commented Feb 13, 2021

@iRon7 Thx for explanation! This will solve my main problem.

The question is if PowerShell should or should not have the same behaviour here.

@iRon7
Copy link

iRon7 commented Feb 13, 2021

The question is if PowerShell should or should not have the same behaviour here.

Since Windows PowerShell, PowerShell Core has undergone quiet some changes along with the NewtonSoft JSON engine (see e.g.: PowerShell 7 Changes to JSON Cmdlets, this shows in minor discrepancies along with the error descriptions.

@KUTlime
Copy link
Author

KUTlime commented Feb 13, 2021

Oh, I see... Make sense.

The only thing which is a mystery to me is why PowerShell does not protest about the configuration JSON file until the JSON reaches some particular level (size).

@iRon7
Copy link

iRon7 commented Feb 13, 2021 •
edited

I need to correct my initial answer a bit as you (obviously) will need to expand the environment variables prior the ConvertTo-Json Conversion (it appears to work because the $Configuration holds nothing more then a Json String):

$configuration = Get-Content .\configuration.json
$configuration = [System.Environment]::ExpandEnvironmentVariables($configuration) | ConvertFrom-Json
$block = {param([pscustomobject]$inputArgument) $PSVersionTable; Write-Host 'Test ' $inputArgument.AxisUser.Name}
powershell -nologo -noprofile $block -Args $configuration

But as you will see this will fail at the second statement, with:

ConvertFrom-Json: Conversion from JSON failed with error:
Bad JSON escape sequence: \U. Path 'SourceCodeLocalPath', line 1, position 92.

And this is because %USERPROFILE% expands to something like C:\Users\Username which is not a valid Json value as the backslashes (escapes characters) should be escaped themselves, thus: C:\\Users\\Username

@iRon7
Copy link

iRon7 commented Feb 15, 2021 •
edited

Besides, why using the pwsh command-line interface which requires to pass arguments (-args) in a cumbersome <arg-array> (knowing that ConvertFrom-Json converts your Json string to a complex object).
It would be much easier to just use a cmdlet like Invoke-Command which accepts objects:

$configuration = Get-Content .\configuration.json | ConvertFrom-Json
$block = {$PSVersionTable; Write-Host 'Test' $Input.ProductNameUser.Name}
# powershell -nologo -noprofile $block -Args $configuration
Invoke-Command $Block -InputObject $configuration

Name                           Value
----                           -----
PSVersion                      7.1.1
PSEdition                      Core
GitCommitId                    7.1.1
OS                             Microsoft Windows 10.0.19042
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0
Test String from json

@KUTlime
Copy link
Author

KUTlime commented Feb 15, 2021

I have to redirect to Windows PowerShell. Invoke-Command doesn't support this.

In addition, even after I expand the environmental variables in the JSON string, the problem with redirection to Windows PowerShell remains.

@KUTlime
Copy link
Author

KUTlime commented Feb 16, 2021

It looks like this problem has nothing to do with the environmental variables in the JSON. I just used a JSON with no environmental variables with the same results.

ResourceUnavailable: Program 'powershell.exe' failed to run: The Process object must have the UseShellExecute property set to false in order to use environment variables.At line:1 char:1
+ powershell -nologo -noprofile $block -args $configuration

@iRon7
Copy link

iRon7 commented Feb 16, 2021 •
edited

Agree, I guess it has something to do with the STDOUT/STDIN buffer size and leave it to the PowerShell guys to answer...

A Minimal, Reproducible Example of this issue would be:

powershell {''} -Args ('x' * 100000)

Results in an error:

ResourceUnavailable: Program 'powershell.exe' failed to run: The Process object must have the UseShellExecute property set to false in order to use environment variables.At line:1 char:1

(Where the error doesn't appear with smaller strings, e.g.: powershell {''} -Args ('x' * 10000)

@KUTlime KUTlime changed the title Inconsitend behaviour with Windows PowerShell when passing arguments into Windows PowerShell STDOUT/STDIN buffer size error when passing arguments into Windows PowerShell Feb 16, 2021
@daxian-dbw daxian-dbw added the WG-Interactive-Console the console experience label Feb 19, 2021
@microsoft-github-policy-service Microsoft GitHub Policy Service
Copy link
Contributor

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

2 similar comments
@microsoft-github-policy-service Microsoft GitHub Policy Service
Copy link
Contributor

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

@microsoft-github-policy-service Microsoft GitHub Policy Service
Copy link
Contributor

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

@microsoft-github-policy-service microsoft-github-policy-service bot added Resolution-No Activity Issue has had no activity for 6 months or more labels Nov 16, 2023
@microsoft-github-policy-service Microsoft GitHub Policy Service
Copy link
Contributor

This issue has been marked as "No Activity" as there has been no activity for 6 months. It has been closed for housekeeping purposes.

@KUTlime KUTlime mentioned this issue Nov 24, 2023
Open
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Needs-Triage The issue is new and needs to be triaged by a work group. Resolution-No Activity Issue has had no activity for 6 months or more WG-Interactive-Console the console experience
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@daxian-dbw @iRon7 @KUTlime
X Tutup