Skip null-element check for collections with a value-type element type #5432

Merged
merged 7 commits into from Nov 30, 2017

Conversation

Projects
None yet
5 participants
Member

daxian-dbw commented Nov 13, 2017

Fix #5417

  • For Mandatory parameter, ValidateNotNull and ValidateNotNullOrEmpty attributes, skip null-element check if the collection's element type is value type.
  • For ValidateNotNull and ValidateNotNullOrEmpty, they are changed to do null/empty check only on collections (as defined in ParameterCollectionTypeInformation) and hashtable. They currently enumerate elements of any object that implements IEnumerable and also enumerate elements if the argument is an enumerator. I think this behavior is not right for the following reasons:
    • The argument may be an object that talks to Azure to retrieve elements back. It doesn't seem right to have the ValidateNotNull/ValidateNotNullOrEmpty attributes wait on that.
    • On Windows, COM object may be enumerable, but the GetEnumerator() method is not supported in .NET Core on COM object. (See the COM example below)
    • When the argument is an enumerator, the enumerator may not be useful after the validate attribute walks through all its elements. (See the enumerator example below)

COM Example

function bar {
   param ([ValidateNotNull()] $object)
}
> $shell = New-Object -ComObject Shell.Application
> $folder = $shell.NameSpace("F:\tmp\dd\")
> bar -object ($folder.Items())
bar : Cannot validate argument on parameter 'object'. Could not load type 'System.Runtime.InteropServices.ComTypes.IEnumerable' from assembly 'System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e'.
At line:1 char:13
+ bar -object ($folder.Items())
+             ~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidData: (:) [bar], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationError,bar

Enumerator Example

function bar {
   param ([ValidateNotNull()] $object)
   "MoveNext:  $($object.MoveNext())"
}

> bar -object @(1,2,3).GetEnumerator()
MoveNext:  False

daxian-dbw added some commits Nov 13, 2017

@daxian-dbw daxian-dbw requested review from BrucePay and TravisEz13 as code owners Nov 13, 2017

@daxian-dbw daxian-dbw removed request for BrucePay and TravisEz13 Nov 13, 2017

build.psm1
@@ -568,7 +568,7 @@ Fix steps:
if ($ReleaseTagToUse) {
$ReleaseVersion = $ReleaseTagToUse
} else {
- $ReleaseVersion = (Get-PSCommitId) -replace '^v'
+ $ReleaseVersion = (Get-PSCommitId -WarningAction SilentlyContinue) -replace '^v'
@adityapatwardhan

adityapatwardhan Nov 13, 2017

Member

This change does not seem related to this PR.

@daxian-dbw

daxian-dbw Nov 14, 2017

Member

Yes, I will revert it.

+ param(
+ [ValidateNotNullOrEmpty()]
+ $Value,
+ [string] $TestType
@adityapatwardhan

adityapatwardhan Nov 13, 2017

Member

Can you add a parameter like: [System.Nullable``1[[System.Int32]]] $nullableInt

@daxian-dbw

daxian-dbw Nov 14, 2017

Member

A [System.Nullable[int]] parameter is not useful in this case because the type is not a collection. Hence ValidateNotNullOrEmpty attribute won't validate it as a collection.

+ $expected = $baseline + 20
+
+ if ($IsWindows) {
+ $null = New-Item -Path $TESTDRIVE/file1 -ItemType File
@adityapatwardhan

adityapatwardhan Nov 13, 2017

Member

-ItemType File is not necessary anymore. The default itemtype is file

@daxian-dbw

daxian-dbw Nov 14, 2017

Member

OK. I will remove it.

@@ -1975,12 +1990,18 @@ protected override void Validate(object arguments, EngineIntrinsics engineIntrin
Metadata.ValidateNotNullOrEmptyFailure);
}
}
- else if ((ienum = arguments as IEnumerable) != null)
+ else if (IsArgumentCollection(arguments.GetType(), out bool isElementValueType))
@iSazonov

iSazonov Nov 14, 2017

Collaborator

Is in the context isEmpty = !isElementValueType ?

@daxian-dbw

daxian-dbw Nov 14, 2017

Member

No, the check of element type doesn't involve inspecting the elements of the collection. You can see that the type of arguments is passed in instead of the arguments itself.
Please see the constructor of ParameterCollectionTypeInformation for details.

+ isEmpty = false;
+ // If the element of the collection is of value type, then no need to check for null
+ // because a value-type value cannot be null.
+ if (isElementValueType) { break; }
@iSazonov

iSazonov Nov 14, 2017

Collaborator

It seems we can move this before while line 1997 - we check first element in ParameterCollectionTypeInformation in IsArgumentCollection.

@daxian-dbw

daxian-dbw Nov 14, 2017

Member

We don't check the elements of arguments in IsArgumentCollection. The argument passed in is the type of arguments, not arguments itself.

@iSazonov

iSazonov Nov 14, 2017

Collaborator

Yes, I see that no element check is in ParameterCollectionTypeInformation.

But the code is not clear. Maybe:

# Check if the collection is empty.
if (ienum.MoveNext()) isEmpty = false;

# Check every element in the collection only if the element type of the collection is NOT a value type. 
if (!isElementValueType && !isEmpty)
{
    do
    {
        ...
        object element = ienum.Current; 
        ...
    } while(ienum.MoveNext()))
}
@daxian-dbw

daxian-dbw Nov 14, 2017

Member

I think the logic is clear -- if the element type of the collection is a value type, then skip checking each element. The isEmpty check is needed in either case.

@iSazonov

iSazonov Nov 15, 2017

Collaborator

The logic is right but the code is slightly tricky for me. I added your comments in my sample above. Also we do two actions that can be carried before the cycle - if we are thinking about performance it is important - I think Measure-Object show a delay starting with 10000 elements.

@daxian-dbw

daxian-dbw Nov 15, 2017

Member

I get your point. I will update as you suggested. Thanks!

isEmpty = false;
+ // If the element of the collection is of value type, then no need to check for null
+ // because a value-type value cannot be null.
+ if (isElementValueType) { break; }
@iSazonov

iSazonov Nov 14, 2017

Collaborator

Why if we calculate this before the cycle?

@daxian-dbw

daxian-dbw Nov 14, 2017

Member

Same reason as above. The element type is resolved using the collection type.

Member

daxian-dbw commented Nov 20, 2017

@iSazonov You comments have been addressed. Please take another look when you have time. Thanks!

LGTM with two minor comments.

@@ -1865,11 +1865,40 @@ public ValidateUserDriveAttribute()
#endregion
#region NULL validation attributes
+
+ /// <summary>
+ /// Base type of Null Validation attributes
@iSazonov

iSazonov Nov 21, 2017

Collaborator

Could you please add final dot in public comments?

+ Context "ValidateNotNull, ValidateNotNullOrEmpty and Not-Null-Or-Empty check for Mandatory parameter" {
+
+ BeforeAll {
+ function MandType {
@iSazonov

iSazonov Nov 21, 2017

Collaborator

Could you please use more nice name? MandatoryType?

@SteveL-MSFT SteveL-MSFT added this to the 6.0.0-GA milestone Nov 29, 2017

Owner

SteveL-MSFT commented Nov 29, 2017

@PowerShell/powershell-committee reviewed this and approves with this change

Member

adityapatwardhan commented Nov 29, 2017

@daxian-dbw restarted AppVeyor build. Failed due to #5567

@adityapatwardhan adityapatwardhan merged commit b5f84c2 into PowerShell:master Nov 30, 2017

3 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
license/cla All CLA requirements met.
Details

@daxian-dbw daxian-dbw deleted the daxian-dbw:perf branch Nov 30, 2017

@TravisEz13 TravisEz13 modified the milestones: 6.0.0-GA, 6.0.0-RC.2 Dec 2, 2017

TravisEz13 added a commit to TravisEz13/PowerShell that referenced this pull request Dec 2, 2017

Skip null-element check for collections with a value-type element type (
#5432)

* Fix NotNullOrEmpty check logic

* Fix a test issue on Unix

TravisEz13 added a commit that referenced this pull request Dec 2, 2017

Skip null-element check for collections with a value-type element type (
#5432)

* Fix NotNullOrEmpty check logic

* Fix a test issue on Unix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment