X Tutup
The Wayback Machine - https://web.archive.org/web/20210926083112/https://github.com/PowerShell/PowerShell/issues/4568
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need straightforward way to honor the caller's preference-variable values / inherited common parameters in functions defined in script modules #4568

Open
mklement0 opened this issue Aug 14, 2017 · 38 comments

Comments

@mklement0
Copy link
Contributor

@mklement0 mklement0 commented Aug 14, 2017

This is a longstanding issue that @dlwyatt explained in detail in this 2014 blog post, and he has even published module PreferenceVariables with advanced function Get-CallerPreference to ease the pain, via a hard-coded list of preference variable names.

In short: A script module's functions do not see the preference-variable values set in the caller's context (except if that context happens to be the global one), which means that the caller's preferences are not honored.

Update: Since implicitly setting preference variables is PowerShell's method for propagating (inheriting) common parameters such as -WhatIf, such parameters are ultimately not honored in calls to script-module functions when passed via an advanced function - see #3106, #6556, and #6342. In short: the common-parameter inheritance mechanism is fundamentally broken for advanced functions across module scopes, which also affects standard modules such as NetSecurity and Microsoft.PowerShell.Archive .

  • From a user's perspective this is (a) surprising and (b), once understood, inconvenient.
    Additionally, given that compiled cmdlets do not have the same problem, it is not easy to tell in advance which cmdlets / advanced functions are affected. In a similar vein, compiled cmdlets proxied via implicitly remoting modules also do not honor the caller's preferences.

  • From a developer's perspective, it is (a) not easy to keep the problem in mind, and (b) addressing the problem requires a workaround that is currently quite cumbersome, exacerbated by currently not having a programmatic way identify all preference variables in order to copy their values to the callee's namespace - see #4394.

A simple demonstration of the problem:

# Define an in-memory module with a single function Foo() that provokes
# a non-terminating error.
$null = New-Module {
  function Foo {
    [CmdletBinding()] param()
    # Provoke a non-terminating error.
    Get-Item /Nosuch
  }
}

# Set $ErrorActionPreference in the *script* context:
# Request that errors be silenced.
# (Note: Only if you defined this in the *global* context would the module see it.)
$ErrorActionPreference = 'SilentlyContinue'

# Because the module in which Foo() is defined doesn't see this script's
# variables, it cannot honor the $ErrorActionPreference setting, and 
# the error message still prints.
Foo

Desired behavior

No output.

Current behavior

Get-Item : Cannot find path '/Nosuch' because it does not exist.
...

Environment data

PowerShell Core v6.0.0-beta.5 on macOS 10.12.6
PowerShell Core v6.0.0-beta.5 on Ubuntu 16.04.3 LTS
PowerShell Core v6.0.0-beta.5 on Microsoft Windows 10 Pro (64-bit; v10.0.15063)
Windows PowerShell v5.1.15063.483 on Microsoft Windows 10 Pro (64-bit; v10.0.15063)
@iSazonov
Copy link
Collaborator

@iSazonov iSazonov commented Aug 14, 2017

Module has its own context. It may be preferable to set the preference variable value for the module
Set-Variable -Module moduleName

@mklement0
Copy link
Contributor Author

@mklement0 mklement0 commented Aug 14, 2017

Yes, modules have their own scope, which is generally a good thing.

However, this special case calls for a concise, PS-supported way to copy all the preference-variable values from the caller's scope.
(The current workaround of using $PSCmdlet.SessionState.PSVariable based on a hard-coded list of preference variable names is neither future-proof nor reasonable.)

Consider the user's perspective:

If $ErrorActionPreference = 'Ignore'; Get-Item /NoSuch works,
$ErrorActionPreference = 'Ignore'; Get-MyItem /NoSuch not working, just because the Get-MyItem command happens to be defined as an advanced function in a script module, is highly obscure and confusing.

Giving module developers an easy, built-in way to opt into the caller's preferences would mitigate that problem.

@iSazonov
Copy link
Collaborator

@iSazonov iSazonov commented Aug 15, 2017

Main module's function is to isolate code that expose public API and hide implementation. We can get unpredictable behavior if we just copy any variables (and maybe indirectly functions) to a module context.
Also internally there are difference of global and local preference variables.
Preference variables is not protected and can be used other ways:

Remove-Variable ErrorActionPreference
$ErrorActionPreference =@()

We may have problems if these variables are simply copied in a module context.

What scenarios do we need to? It seems it is only debug. If so we could resolve this otherwise, for example, by loading a module in debug mode. If we want to manage the preference variables in a module, we must do this in a predictable way (We can load tens modules in session!):

Import-Module testModule -SetErrorActionPreference Stop

@iSazonov
Copy link
Collaborator

@iSazonov iSazonov commented Aug 15, 2017

Also we have an Issue with the request to share modules between runspaces. If we want implement this we can not simply copy any variables - no single parent context exists.

@mklement0
Copy link
Contributor Author

@mklement0 mklement0 commented Aug 15, 2017

Preference variables is not protected and can be used other ways:

Indeed, but that is (a) incidental to the issue at hand and (b) should in itself be considered a bug: see #3483

@mklement0
Copy link
Contributor Author

@mklement0 mklement0 commented Aug 15, 2017

We can get unpredictable behavior if we just copy any variables (and maybe indirectly functions) to a module context.

We're not talking about any variables. We're talking about the well-defined set of preference variables - even though, as stated, there's currently no programmatic way to enumerate them.

@mklement0
Copy link
Contributor Author

@mklement0 mklement0 commented Aug 15, 2017

What scenarios do we need to? It seems it is only debug.

  • Please read the blog post that the original post links to.
  • Then consider my Get-Item vs. Get-MyItem example.

This is very much a production issue, unrelated to debugging:

Currently, if you create a script module, that module's functions will by default ignore a caller's preferences variables (unless you're calling from the global scope), which to me is self-evidently problematic:

For instance, if a caller sets $ErrorActionPreference to Stop, they have a reasonable expectation that cmdlets / advanced functions invoked subsequently honor that setting - without having to know whether a given command happens to be defined in a script module that doesn't see that setting by default.

@lzybkr
Copy link
Member

@lzybkr lzybkr commented Aug 15, 2017

Honoring the callers preference might cause serious problems by introducing exceptions (e.g. $errorActionPreference = 'Stop' in places where none were expected.

Some examples - Dispose might not get called and resources leak, or the system may be left in an inconsistent state because some side effects do not execute.

If the goal is to silence a noisy function - one can work with the function author to provide a better experience or else explicitly pass -ErrorActionPrefernce SilentlyContinue.

@dlwyatt
Copy link
Contributor

@dlwyatt dlwyatt commented Aug 15, 2017

I think the focus is too much on the implementation here and not on the problem: a user shouldn't have to know or care how a particular PowerShell command is implemented. Whether it's a cmdlet, function, cdxml, imported from a remote session, whatever: they should have a consistent experience with regard to common parameters and preference variables. If I put $VerbosePreference = 'Continue' in my script, I should expect to see all the verbose output that I asked for. If I say $ErrorActionPreference = 'Stop', then anything that bubbles up through the Error stream to my code should cause a terminating error.

Honoring the callers preference might cause serious problems by introducing exceptions (e.g. $errorActionPreference = 'Stop' in places where none were expected.

I don't see this as a problem, since that's exactly what happens if the caller specifies -ErrorAction Stop today. (Common parameters cause the equivalent preference variables to be set in the advanced function's scope.)

@mklement0
Copy link
Contributor Author

@mklement0 mklement0 commented Aug 15, 2017

@dlwyatt: Amen to that. I was in the middle of composing the following:

The point is that if compiled cmdlets respect preference variables set in the caller's scope, it's reasonable to expect advanced functions to respect them too - the user shouldn't have to worry about module scopes or how a given command happens to be implemented.

With respect to $ErrorActionPreference = 'Stop': It sounds like using Stop (whether via the pref. variable or the common parameter) is inherently problematic, which is a separate conversation (and I personally wasn't aware that it's problematic - probably worth documenting).

Let's take a more innocuous example: setting $VerbosePreference = 'Continue' in an effort to make all cmdlet invocations in the current script produce verbose output:

The following example defines Get-Foo1 as a compiled cmdlet, and functionally equivalent Get-Foo2 as an advanced function in a module:

Get-Foo1 respects the verbose preference, Get-Foo2 does not.

# Define compiled cmdlet Get-Foo1 (via an in-memory assembly and module).
Add-Type -TypeDefinition @'
    using System;
    using System.Management.Automation;
    [Cmdlet("Get", "Foo1")]
    public class GetFoo1 : PSCmdlet {

        protected override void EndProcessing() {
            WriteVerbose("foo1");
        }
    }
'@ -PassThru | % Assembly | Import-Module

# Define analogous advanced function Get-Foo2 via an in-memory module.
$null = New-Module {
    Function Get-Foo2 {
        [cmdletbinding()]
        param()

        End {
            Write-Verbose("foo2");
        }
    }
}

$VerbosePreference = 'Continue'

# Compiled cmdlet respects $VerbosePreference.
Get-Foo1
# Verbose: foo1

# Advanced function in script module does NOT, because it doesn't see the caller's
# $VerbosePreference variable.
Get-Foo2
# (no output)

Expecting the user to even anticipate a distinction here and to then know which commands are affected based on how they happen to be implemented strikes me as highly obscure.

A scenario in which the behavior is even more obscure is with implicit remoting modules that proxy compiled cmdlets that execute remotely. In that case, the remotely executing cmdlets, even though they normally respect the caller's preference, do not. (On a related note: even -ErrorAction Stop does not work in that scenario, because the parameter is applied remotely, and terminating errors that occur remotely are by design converted to non-terminating ones.)


I don't know what the right solution is, but if we can agree that there is a problem, we can tackle it.

@lzybkr
Copy link
Member

@lzybkr lzybkr commented Aug 15, 2017

Keep in mind one big difference between binary cmdlets and functions - binary cmdlets are rarely implemented in terms of PowerShell functions or other cmdlets.

The error or verbose output is likely carefully crafted for a binary cmdlet, whereas it's a bit more random for functions.

Also note that the equivalence of -ErrorAction Stop and setting the preference variable could be thought of as an implementation detail and is considered a bug by some people for some of the reasons mentioned above.

It's worth pointing out that extra verbosity is not always desirable - and this proposal could turn some useful verbose output into noisy useless output.

@mklement0
Copy link
Contributor Author

@mklement0 mklement0 commented Aug 15, 2017

Keep in mind one big difference between binary cmdlets and functions - binary cmdlets are rarely implemented in terms of PowerShell functions or other cmdlets.

Users needing to be aware of how a given command happens to be implemented is an unreasonable expectation.
Aside from that, I don't understand your comment.

The error or verbose output is likely carefully crafted for a binary cmdlet, whereas it's a bit more random for functions.

PowerShell has evolved toward making the PowerShell language a first-class citizen with respect to creating cmdlets (advanced functions).
Someone skilled enough to create advanced functions as part of a module should be assumed to exercise the same care as someone creating a compiled cmdlet.

It's worth pointing out that extra verbosity is not always desirable - and this proposal could turn some useful verbose output into noisy useless output.

If I, as a user, set $VerbosePreference = 'Continue' scope-wide, I expect all commands invoked subsequently to honor that settings; if I didn't want that, I would use -Verbose on a per-command basis.

Note that backward compatibility is a separate discussion - opting in to copying the caller's preference variables is a viable option in that context, perhaps via a new [System.Management.Automation.CmdletBindingAttribute] property such as [CmdletBinding(UseCallerPreferences)].

the equivalence of -ErrorAction Stop and setting the preference variable

Again I don't fully understand your comment; at the risk of going off on a tangent:

They are not equivalent: -ErrorAction Stop only affects non-terminating errors (escalates them to script-terminating errors); $ErrorActionPreference = 'Stop' affects both non-terminating and statement-terminating errors - see Our Error Handling, Ourselves - time to fully understand and properly document PowerShell's error handling.

@lzybkr
Copy link
Member

@lzybkr lzybkr commented Aug 16, 2017

Users needing to be aware

I think you're emphasizing my point. Users of a command aren't aware of the implementation and do have the expectation of reasonable output.

Command authors are a different story - they need to be aware of the implications of preference variables and parameters like -Verbose in a way that differs from the user.

In some cases, the command author and user are the same, and I do wonder if that's where some of this feedback is coming from - because the command author sometimes wants more help debugging during development.

@mklement0
Copy link
Contributor Author

@mklement0 mklement0 commented Aug 16, 2017

Users of a command aren't aware of the implementation and do have the expectation of reasonable output.

No argument there. And let's not forget predictable behavior, which brings us to the next point:

I do wonder if that's where some of this feedback is coming from

I can't speak for @dlwyatt, but to me this is all about the user perspective:

  • I set a preference variable to modify the behavior scope-wide.
  • I expect commands called from that scope to honor them, just as the standard cmdlets do.

If I can't predict which of the commands I'll invoke will actually honor the preferences, I might as well do without preferences altogether and only use common parameters.

We've covered $VerbosePreference, but let's consider $WhatIfPreference and $ConfirmPreference:

If I set these with the expectations that all commands invoked from the same scope will honor them, I may be in for nasty surprises.

Similarly, to return to $ErrorActionPreference: An approach I've personally used it to set $ErrorActionPreference = 'Stop' at the start of a script as fallback and then handle errors that I anticipate on a per-command basis. That way, unanticipated errors cause the script to abort, as a fail-safe - or so I'd hoped.

@mklement0
Copy link
Contributor Author

@mklement0 mklement0 commented Aug 18, 2017

Also note that the equivalence of -ErrorAction Stop and setting the preference variable could be thought of as an implementation detail and is considered a bug by some people for some of the reasons mentioned above.

I think I now understand what you're saying, and I hope I'm not wasting my breath based on a misinterpretation:

It sounds like you're conceiving of preference variables as being limited to the current scope only, without affecting its descendant scopes or the potentially completely separate scopes of commands invoked.

  • By itself, this conception runs counter to how variables work in PowerShell, at least with respect to descendant scopes.
    To be consistent with how PS variables work, you'd have to define a variable as $private: in order to limit it to the current scope only.

  • Certainly, given that core cmdlets currently do honor the preference variables, changing that behavior - selectively, for preference variables only - would be a serious breaking change and would require a focused effort to reframe the purpose of preference variables.


Leaving the syntax and backward compatibilities issues aside, I can see how some preference variables can be helpful as limited to the current scope:

  • $DebugPreference and $VerbosePreference and $ErrorActionPreference only applied to Write-Debug and Write-Verbose and Write-Error commands issued directly in the current scope could help with debugging only a given script, without being distracted by debug/verbose commands / errors from commands invoked from the current scope.
    As an aside: As stated, against documented behavior, $ErrorActionPreference currently also takes effect for terminating errors.

    • Again: In the realm of PowerShell variables, that would call for something like $private:VerbosePreference = 'Continue'. On a side note: that doesn't actually work - descendant scopes still see the value, which means that any non-module-defined functions currently do respect these preferences either way.
  • And, again, just as I and @dlwyatt have, I assume that many people are currently relying on $ErrorActionPreference applying to errors reported by commands called from the current scope too - or at least have had that expectation, which currently only holds true for compiled cmdlets by default, a distinction that brings us back to the original problem.

@StingyJack
Copy link
Contributor

@StingyJack StingyJack commented Feb 8, 2018

I found that specifying $global: addresses this for progress bar issues.

$global:ProgressPreference = 'silentlycontinue' at the top of a script suppresses the performance killing progress indicator when downloading things.

It's probably easier to do that with VerbosePreference than adding -Verbose:($PsCmdlet.MyInvocation.BoundParameters['verbose'] -eq $true) to every single command my script calls.

@alx9r
Copy link

@alx9r alx9r commented Feb 28, 2018

It seems to me that when considering code inside a script module there are two distinct kinds of code to which preference variables (eg. $ErrorActionPreference, etc) and common parameters (eg. -ErrorAction, etc) might apply:

  • business logic implemented within the module
    • The cause of errors arising from such code are bugs (ie. "boneheaded" errors per Eric Lippert's taxonomy). Accordingly, such code should be run with $ErrorActionPreference='Stop'.
    • Deciding the conditions for and quantity of verbose messages output from the business logic requires the good judgment of the module author. This probably involves careful consideration of what should happen when the module is called with the different values of $VerbosePreference and -Verbose at the call site of module entry points.
    • $DebugPreference, $WarningPreference, and $InformationPreference for such code should probably be the PowerShell defaults regardless of what the caller's values of those variable are.
  • calls made to outside the module
    • Such code is subject to all sorts of untidy unpredictable external realities that cause errors (Eric Lippert calls these "exogenous"). The user of the module should be the party deciding what to do when such an error arises. Accordingly,
    • the value of common parameters and preference variables for such calls should be the same as they were at the call site of the entry point to the module.

The above points aren't hard and fast, but I think they're a reasonable starting point for most modules involving a mix of business logic and calls made to outside the module. PowerShell, of course, does not behave this way by default. To implement a module that is consistent with the above points requires two things to happen as follows:

  1. Capture preference variables and common parameters at the call site of the entry point to the module.
  2. Apply those preference variables and common parameters to the calls made to outside the module.

I suspect that these tasks can be handled reasonably well by a utility module. I wrote an annotated proof-of-concept of such a module. The entry points to a user module that uses the utility module would look, for example, like this:

function Get-MyItem {
    param(
        [Parameter(Position=1)] $Path
    )
    HonorCallerPrefs {
        Get-MyItemImpl $Path
    }
}

function New-MyItem {
    param(
        [Parameter(Position=1)] $Path,
        [switch]                $WhatIf,
        [switch]                $Confirm
    )
    HonorCallerPrefs {
        New-MyItemImpl $Path
    }
}

The corresponding sites that call out of the user module would look like this:

InvokeWithCallerPrefs {
    Get-Item $path @CallerCommonArgs
}

InvokeWithCallerPrefs {
    New-Item $path @CallerCommonArgs
}

HonorCallerPrefs{} and InvokeWithCallerPrefs{} are implemented by the utility module and capture and apply, respectively, the preference variables and common parameters. The concept should work even if the call stack between HonorCallerPrefs{} and InvokeWithCallerPrefs{} is deep and complicated, as long as the calls to HonorCallerPrefs{} and InvokeWithCallerPrefs{} are in the same module.

@mklement0
Copy link
Contributor Author

@mklement0 mklement0 commented Apr 4, 2018

The recent #6556 is a more pernicious manifestation of the problem discussed here:

Because CDXML-based cmdlets are seemingly advanced functions rather than binary cmdlets, the functions in the NetSecurity module such as Enable-NetFirewallRule situationally do not honor the -WhatIf switch, namely if invoked via an advanced function that itself supports -WhatIf or via an explicitly set $WhatIfPreference value, as discussed.

Revisiting @lzybkr's comment:

Also note that the equivalence of -ErrorAction Stop and setting the preference variable could be thought of as an implementation detail

From what I gather, PowerShell implicitly translating a common parameter such as -WhatIf into the equivalent locally scoped preference variable such as $WhatIfPreference is the mechanism for automatically propagating common parameters to calls to other cmdlets inside an advanced function.
So, yes, you can conceive of it as an implementation detail, but even as such it is broken.

@alx9r: As commendable as starting scripts / functions with $ErrorActionPreference = 'Stop' is (it is what I usually do in my own code), it actually interferes with that mechanism (I have yet to take a look at your proof of concept).

@mklement0
Copy link
Contributor Author

@mklement0 mklement0 commented Apr 4, 2018

#6342 is a similar manifestation of the problem, in the context of Expand-Archive.

@mklement0 mklement0 changed the title Need straightforward way to honor the caller's preference-variable values in functions defined in script modules Need straightforward way to honor the caller's preference-variable values / relayed common parameters in functions defined in script modules Apr 4, 2018
@KirkMunro
Copy link
Contributor

@KirkMunro KirkMunro commented Apr 16, 2019

This issue seems stalled, but it's such an important one, and with the upcoming PowerShell 7.0 milestone it would be really, really great if people put their heads together and come up with a model that makes sense for the caller, has the right default behavior, and that gives the developer the flexibility they need to override the default behavior if necessary. @SteveL-MSFT

@vexx32
Copy link
Collaborator

@vexx32 vexx32 commented Apr 16, 2019

Agreed. If we can come to a conclusion on this it would be great to get a more consistent model in place.

@iSazonov
Copy link
Collaborator

@iSazonov iSazonov commented Apr 16, 2019

We could move forward if we created examples of the desired behavior in the form of Pester tests.

@StingyJack
Copy link
Contributor

@StingyJack StingyJack commented Apr 16, 2019

The expectation from my POV is that caller preferences that do not affect the script or module execution paths are replicated out to every script or module that gets involved, as a default, without having to opt-in (other than [CmdletBinding()] ofc).

This would be things like Verbose, Debug, Information, and Progress, but not something like ErrorActionPreference, PSEmailServer, or ModuleAutoLoadingPreference.

@KirkMunro
Copy link
Contributor

@KirkMunro KirkMunro commented May 9, 2019

I just read through this discussion again, taking some more time this time around to catch all of the details. Before I share what I believe may be a solution, I think it's worth summing up some of the key points, as follows:

  1. To an end user who is not a command author, a function, advanced function, filter, cmdlet, and alias, and even a script discoverable via $env:Path that is invoked by name, are all just commands.
  2. End users should never have to care what type of command they invoke, and any scripts they write should function the same regardless of the type of command they invoke.
  3. If a command author decides to convert a cmdlet into an advanced function, or vice versa, as long as that author can perform a perfect conversion of the functionality within the command, they should be able to do so without having to worry about existing scripts that directly or indirectly call that command behaving differently just because they changed the way that command was implemented.

With this issue as it exists today, only the first item in that list is true, and that is a problem that leads to non-intuitive behavior at best and bugs that cannot be easily worked around without duplicating a lot of code at worst.

When it comes to invocation preferences, I strongly feel that all preferences (not necessarily preference variables, read on to see what I mean) that can be influenced by common parameters (Error, Warning, Information, Verbose, Debug, Progress, Confirm, and WhatIf) need to be carried through the entire invocation of a command, end to end, regardless of what other commands are invoked internally as part of that process. That means a few things:

  1. End users get a declarative syntax for every command they invoke. If they invoke with -Verbose, they should get all verbose messages that the command author intended them to get. Ditto with -Debug, but for debug messages. And if they use any of the -*Action common parameters, that action needs to apply to everything done internally in that command and in any commands invoked internally within that command according to how the author intentionally wrote the command.

  2. Command authors need features that allow them to control the volume of output sent to various streams, as well as features that allow them to silence or ignore certain messages or errors. They have this already. The behavior of -Debug has very recently changed such that users are no longer prompted when messages are sent to the debug stream, so now command authors have two levels of output for troubleshooting-type messages. This gives command authors more flexibility over the volume of output to specific message streams (they no longer just have the verbose stream to realistically work with). Similarly they have common parameters and preference variables to control other message streams and error handling in the commands that they invoke.

Part of the discussion here suggests that it may be detrimental to carry through these preferences -- that users may receive too much verbose output, for example. What is/is not too much output from a command, regardless of which stream it is sent to, is a decision that must be made by the command author, intentionally, not incidentally. Command authors can turn verbose/debug output off for other commands that they invoke, if they feel that information is not helpful/relevant to the caller. They can also silence or ignore errors, warnings, or information messages in commands that they invoke, regardless of how their command is being invoked. The decision on what to allow to pass through or not needs to be a conscious one, not accidental, made by the command author, and it should respect the way the command was invoked by default (common parameters). It should never have been something that a caller should concern themselves with by having to pay attention to the nuances associated with type of command they are invoking and the way that command was loaded.

It is also worth pointing out how we got here, and some relevant changes, because that information may lead to a solution to this problem. We've had common parameters and preference variables in PowerShell since v1 was released in 2006. When PowerShell v2 came out in 2009, the PowerShell Team added modules (along with the scoping they have today that contributes to this problem because of how all variables, including preference variables, are only accessible in nested scopes), as well as the addition of the largely underused $PSDefaultParameterValues feature, which allows users to assign default values to be used for parameters on commands that are invoked. While the globally visible $PSDefaultParameterValues variable was initially accessible from modules (which caused problems because $PSDefaultParameterValues is meant to be a user convenience for commands users invoke ad hoc or in scripts, but its use was also impacting commands used within module internals), in PowerShell v5, modules received their own $PSDefaultParameterValues so that module commands could run as they were designed without this user convenience getting in the way.

What's most interesting about $PSDefaultParameterValues with respect to this issue is that the values it contains impact the parameters (not variables) used in the invocation of all commands in its scope and any child scope. What if $*Preference variables continued to function as they do today (current scope plus child scopes unless they are redefined), but when a command is invoked using common parameters to control the various streams and how errors should be handled would not only assign the locally-scoped $*Preference variable, but would also result in assignment of a locally-scoped dictionary (maybe as a property of $PSCmdlet?) that works like $PSDefaultParameterValues but with higher precedence (so $PSDefaultParameterValues would still function if it was already in use), such that any command invoked within that command would automatically be invoked with the same common parameter values used in the invocation if those parameters were not specifically used.

For example, consider this script:

# First, create two modules, where a command in one module invokes a command in the other
nmo -name test1 {
    function Test-1 {
        [cmdletBinding()]
        param()
        Test-2
    }
} | ipmo
nmo -name test2 {
    function Test-2 {
        [cmdletBinding()]
        param()
        Write-Verbose 'Verbose output'
    }
} | ipmo

# The next command shows verbose output, as expected
Test-2 -Verbose

# Output: VERBOSE: Verbose output

# This command does not show verbose output, because of the design flaw identified in this GitHub issue
Test-1 -Verbose

# Output: none

# Now let's redefine the test1 module, with some extra logic to assign a locally-scoped
# PSDefaultParameterValues hashtable based on the Verbose common parameter (this would be a
# property of PSCmdlet, but I'm just using PSDefaultParameterValues to show how it could work).
nmo -name test1 {
    function Test-1 {
        [cmdletBinding()]
        param()
        $PSDefaultParameterValues = @{'*:Verbose' = ($VerbosePreference -eq 'Continue')} # The magic
        Test-2
    }
} | ipmo

# Now this works, because of PSDefaultParameterValues impact on common parameters
Test-1 -Verbose 

# Output: VERBOSE: Verbose output

As you can see, if we maintain the current scope visibility on preference variables, but apply a locally-scoped collection based on common parameters, then we get commands whose internal behavior is end-to-end as a default, resolving this issue. Command/script authors who are assigning preference variables that are not globally scoped in hopes that they will get this behavior will be provided with a solution that allows them to get this more desirable functionality, and rules can be created for PowerShell Script Analyzer to catch these variable assignments and recommend how users should work with the locally scoped collection in PSCmdlet instead.

Simple, right? 😄

The only detail I find missing from this is related to how this can work with ShouldProcess common parameters in the following scenario: If within a single module or script file you have one advanced function that supports ShouldProcess, and therefore has a -WhatIf common parameter, and if internally you invoke another advanced function in the same file that does not support -WhatIf, the $WhatIfPreference variable set in the first advanced function will influence behavior in the second advanced function. That always felt wrong to me, because the second advanced function doesn't support ShouldProcess (i.e. if it's properly designed, it's deemed "safe"). It's generally not much of a problem, unless you invoke something like Set-Variable where you may not want to use -WhatIf. This is also a case where the way the command was written (advanced function in PowerShell vs cmdlet in C#) results in different behavior, even if the proposed solution is implemented. I'm going to sleep on it and see if I can come up with something that works for that scenario.

@KirkMunro
Copy link
Contributor

@KirkMunro KirkMunro commented May 9, 2019

Thinking about the last paragraph in my previous comment a little more, I'm inclined to push for the following to address this (and yes, I'll log this as a separate issue):

Any ShouldProcess command that works with PowerShell entities in the current scope (e.g. any of the following commands invoked without -Scope: Clear-Variable, New-Variable, Set-Variable, Remove-Variable, Import-Alias, New-Alias, Set-Alias, along with any of their corresponding New-Item, Set-Item, or Remove-Item commands) should run without checking $PSCmdlet.ShouldProcess. i.e. If -Scope is not used, or if -Scope is 0, just do it regardless of WhatIf or Confirm preferences. Local scope operations for these commands should be considered safe, because they have no bearing on any parent scopes whatsoever, and are therefore non-destructive. This model should be followed for any command that supports ShouldProcess for both actions that may be destructive as well as actions that are non-destructive -- just because ShouldProcess is supported in the command, doesn't mean it should be the gatekeeper to all actions taken by that command.

This is an exception, but one easily explained and worth having because some commands should only cause -WhatIf or -Confirm to change how they run some of the time. For example, there are cases where you want to create a set of variables with dynamic names in a loop without ShouldProcess preferences getting in the way of invocation, and while you can do this using APIs, it should be possible to do this using cmdlets since that's what they are for. Command authors shouldn't have to learn by trial and error which one they should be using -- the cmdlets should just do the right thing.

If that gets accepted, then the last paragraph in my previous post in this discussion becomes a non-issue.

@KirkMunro
Copy link
Contributor

@KirkMunro KirkMunro commented Jun 10, 2019

FYI, I opened a set of RFCs to address error handling issues in PowerShell, this one included. If you're keen and you don't want to wait for the RFCs to reach the draft stage, you can find the PR with those RFCs here.

@nacitar
Copy link

@nacitar nacitar commented Oct 29, 2019

This is still a problem and still needs to be addressed. Just adding another voice to the pile here... I ran into difficulties from this, figured out the origin of my difficulties and ended up here after some web searches.

@iSazonov
Copy link
Collaborator

@iSazonov iSazonov commented Oct 29, 2019

@nacitar You can leave a feedback in RFC PowerShell/PowerShell-RFC#221

@KirkMunro
Copy link
Contributor

@KirkMunro KirkMunro commented Oct 29, 2019

For anyone who has tripped on this issue and is following this discussion, please do share your feedback in PowerShell/PowerShell-RFC#221.

That RFC, the way it is currently written, talks about adopting optional features in PowerShell (described in another RFC: PowerShell/PowerShell-RFC#220). There is some pushback on the optional feature idea because it requires time/resources to properly manage, and it may be too risky to be worth it to the PowerShell Team (see discussion in optional features RFC), in which case you need to ask yourself: is the current problem with how execution preferences do not propagate beyond module or script scope worth fixing even if it may result in breaking changes?

Breaking changes may occur because you would be changing how execution preferences influence existing code without specifically testing that code to make sure that it still works as intended.

From my perspective, I think it is worth it, because the current behavior is the root cause of many issues that show up in different modules. If folks agree, and if the optional feature idea is officially rejected, I can at least set up an experimental feature (these are different from optional features) to allow folks to try it out with their modules and make sure things still work the way they intended with execution preferences properly propagating beyond module or script scope.

@JustinGrote
Copy link
Contributor

@JustinGrote JustinGrote commented Mar 18, 2020

EDIT: Comment Moved to #12148

@vexx32
Copy link
Collaborator

@vexx32 vexx32 commented Mar 18, 2020

Couldn't say off the top of my head, but given that you're setting $VerbosePreference immediately before it, I would be inclined to say that it's likely a separate issue, something to do with how WhatIf only see verbose set by the switch and not an in-session variable.

@MatejKafka
Copy link

@MatejKafka MatejKafka commented Nov 25, 2020

Sorry if this has already been mentioned, but this thread is quite long so I mostly just skimmed it.

I believe there isn't a single "right answer" here:

For $ErrorActionPreference, I personally use

Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"

as a header in every module I write, because I want to fail hard when an unhandled error occurs, and my code is written with this in mind. So just blindly copying the variable from caller scope is imo not a good approach, because scripts that do not declare the variable themselves will work differently in different sessions.

For output level preference variables ($VerbosePreference, $InformationPreference,...), I think the current situation where the output is controlled in each scope separately (so the effective output is the minimum of all set preference variables in all scopes the output passes through) is quite nice and I'd like to keep it both as a module writer and a user.

EDIT: I just tested my assumption from the last paragraph, and surprisingly, this is NOT the current behavior. When $VerbosePreference is set to SilentlyContinue in module A, and it calls function from module B and passes -Verbose, the message is printed, which is imo wrong - it would seem more intuitive to evaluate each message stream on each scope it passes through separately, not just at the source (so if any scope along the way disables verbose output, no output is printed, even if a nested scope runs with verbose output enabled), and inheriting value from parent scope when not explicitly set.

@mklement0
Copy link
Contributor Author

@mklement0 mklement0 commented Nov 25, 2020

@MatejKafka: There are two separate aspects, which are independent of one another:

  • Designing your module so that it reports the appropriate type of error, based on the official guidelines, summarized here (by me). The same applies to using the other output streams, such as the one for verbose output.

  • Respecting the caller's preference variables so as to modify the module's default behavior on demand. For that, there's no justification for distinguishing between (a) functions (and scripts) outside a module scope, (b) functions inside a module, and (c) cmdlets.

    • (a) and (c) properly respect the caller's preferences, whereas (b) doesn't. This distinction doesn't make sense, and no user should ever have to think about the implementation details of a given command, i.e. whether it happens to be implemented as (a), (b), or (c).

@MatejKafka
Copy link

@MatejKafka MatejKafka commented Nov 25, 2020

@mklement0 I agree that the behavior being inconsistent is an issue, but it seems we partially disagree in which direction they should be unified.

Ad 1) - I'm more of a programmer than sysadmin - the concept of a non-terminating error seems like something terrifying to me - an error occurred, meaning that something bad happened and I did not correctly anticipate it and handle it, and the program is in an undefined state. Continuing ahead instead of panicking is imo irresponsible, as I cannot further safely reason about the behavior of my own program. Non-terminating error should imo either be a terminating error, or a warning, not a weird mix of both.

I also edited my previous comment, as I found out that my assumption in the last paragraph isn't correct.

@wisemoth
Copy link

@wisemoth wisemoth commented Nov 26, 2020

the concept of a non-terminating error seems like something terrifying to me

It shouldn't do. There's a difference between errors and exceptions. Whether an error is "exceptional" depends on the context. If you've correctly functionally decomposed your code into a series of small pipeline functions each function by itself can't determine if an error is an exception, because it can't own that policy because it doesn't know why it's being called. Errors from that function should bubble through the pipeline to the creator of the pipeline who can, in that context, determine whether the error is exceptional in that context/pipeline.

@vexx32
Copy link
Collaborator

@vexx32 vexx32 commented Nov 26, 2020

As a general rule, non-terminating errors tend to be something that comes from an individual object input in the pipeline. If one bad piece of data is received, you don't necessarily want to cancel processing the next 10,000 bits of input. On the other hand, if the error comes from bad / unusable input as the result of a fixed (non-pipeline) parameter input, you can deduce that processing all 10,000 bits of input will yield with the same result (say, for example, the cmdlet requires authentication, and the authentication provider you're using indicates that the credentials specified are invalid) -- then you can throw a terminating error to avoid wasting time trying the known-bad credentials 10,000 times.

@jazzdelightsme
Copy link
Contributor

@jazzdelightsme jazzdelightsme commented Nov 26, 2020

I would prefer we do not use this Issue to discuss/debate PowerShell's error handling models. There are existing Issues you can pile onto, or start a new one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
X Tutup