Checking Out The Where and ForEach Operators in PowerShell V4

#Requires –Version 4.0

With the release of PowerShell V4, we received the Desired State Configuration (DSC) module as the main piece as well as some other things, some known and some not known that well. One of those things is the use of Where() and ForEach() operators (both added to support DSC) on an object, similar to the example below:

$services = Get-Service
$services.where()
$services.ForEach()

Ok, that really doesn’t show you the power that is in these two items (especially Where()).  So lets step into using Where() and try to do some filtering on some services.

$services = Get-Service
$services.where({$_.Name -match '^w'})

image

Ok, nothing we haven’t seen before. Instead of piping the objects into Where-Object, we use a dot notation to call Where and supply a script block to it and we get the same results. So what is the big deal? Well, there is another parameter that you can set after the scriptblock to get some very interesting results. 

The documentation for this is well, lacking. So in order to really what it is looking for, you can force an error to occur and it will show you the parameters that are expected on this.

$services.Where()

image

Here we see the following syntax is expected:

.Where({ expression } [, mode [, numberToReturn]])

Basically, you have to supply a System.Management.Automation.WhereOperatorSelectionMode enum and another optional parameter after that. What are the possible values that I can use? Lets find out!

[enum]::GetNames('System.Management.Automation.WhereOperatorSelectionMode')

image

Default

Default filtering like using Where-Object

First

Display the First item in the filtered collection; allows optional integer to specify more items to show (First 3)

Last

Display the Last item in the filtered collection; allows optional integer to specify more items to show (Last 3)

SkipUntil

Display the filtered results while only showing the n number of objects after the initial object being filtered (this can include non-filtered items) only after it finds a match

Until

Display the objects in the collection until the expected results of the filter

Split

Splits results filtered and with the remaining results in separate collections; an optional parameter can specifiy an integer that will split out the first n number of filtered results with the rest of the filtered results being dropped into the second collection regardless if they match the criteria or not.

Now we have defined all of these options, so now lets put them to some use and get a better idea as to what we are looking at. Note: All of these will use the $services=Get-Service object to keep everything consistent unless otherwise noted.

Default

This is your standard filtering of data. It will only show you what was filtered without anything else special happening.

$services.where({$_.Name -match '^r'})
# OR
$services.where({$_.Name -match '^r'},'Default')

image

This verifies that we have essentially duplicated what Where-Object already does. So lets now move on the next item and see what we can do.

First

Now we start to get into the fun stuff! Just by calling First after the script block, it will return 1 item from the collection by default.

$services.where({$_.Name -match '^r'},'First')

image

Now lets find the first 3 items in my filtered match.

$services.where({$_.Name -match '^r'},'First',3)

image

Last

Similar to using First, I will show you how to get the last item in the collection that is filtered using the Where() operator.

$services.where({$_.Name -match '^r'},'Last')

image

Here you see that the last item in the collection is displayed. Just like using First with an additional count of just how many to display, we can do the same with Last.

$services.where({$_.Name -match '^r'},'Last',3)

image

SkipUntil

This is where it starts getting interesting. SkipUntil brings something different to the game when used with the Where() operator. I am also going to switch up and use a range of integers (1..20) to better show this one and Until. Basically, what happens is that when you use SkipUntil with your filter, it will display the results of the filter up until the number you specify. If you specify nothing for the 3rd parameter, then it will display everything including anything that would not have normally been matched via the filter. First lets look at this using a number to determine returned results.

@(1..10).Where({$_ -eq 2},"skipuntil",2)

 

image

Here you see that we returned the match of the filter, 2 as well as another value that is after the matched value in the collection.

If you just want that match, then do this.

@(1..10).Where({$_ -eq 2},"skipuntil",1)

image

Now with nothing specified for the 3rd parameter.

@(1..10).Where({$_ -eq 2},"skipuntil")

image

You can see that it not only displays 2, but the rest of the values afterwards. So what happens if no match is found based on the filter, will it display everything or nothing?

@(1..10).Where({$_ -eq 14},"skipuntil")

image

If you guessed nothing, then you are correct! This only works if there is a match found on the filter, otherwise nothing else will happen.

Until

Until works in the opposite direction of SkipUntil. By that I mean that it will return values up until the matched object in the collection and will not include that match in the returned results.

@(1..10).Where({$_ -eq 5},"until")

image

As you can see, everything up until the 5 is displayed based on the match. If I want, I can choose to only display the first 2 values.

@(1..10).Where({$_ -eq 5},"until",2)

image

Split

I really saved the most interesting, in my opinion, for last. Using Split will actually return 2 separate collections of objects: the first containing everything that matched the filter and the second containing everything that was filtered out if used without any parameter to specify a number of objects to return.

$services = Get-Service
$results = $services.where({$_.Name -match '^r'},'split')

Ok, I have my $results variable filled with the collections. Now lets see the first item in that collection.

$results[0]

image

Here is everything that matched my filter. So if that is in this collection, then that must mean everything else is in the other collection…

$results[1]

image

image

Everything before the match and after the match was stored in the other collection. Pretty cool stuff!

We can also specify a number of those matches to be in the first collection and then discard everything else into the other collection with the excluded objects.

$results = $services.where({$_.Name -match '^r'},'split',2)
$results[0]
$results[1]

image

The first two items of that match in the collection and…

image

everything else is in the second collection.

Performance

There are performance gains from using the Where() operator over piping everything to Where-Object as well. This is due to you having everything loaded up into memory vs. running it through the pipeline. The following examples show various uses of Where against similar Where-Object examples:

$where_operator = (1..20 | ForEach {
    Measure-Command {
         (1..1E4).Where({$_ -gt 777})
    }
} | Measure-Object -Property TotalMilliseconds -Average).Average

$whereobject = (1..20 | ForEach {
    Measure-Command {
         (1..1E4) | Where {$_ -gt 777}
    }
} | Measure-Object -Property TotalMilliseconds -Average).Average

$where_operator_first = (1..20 | ForEach {
    Measure-Command {
         (1..1E4).Where({$_ -gt 777},'first')
    }
} | Measure-Object -Property TotalMilliseconds -Average).Average

$whereobject_first = (1..20 | ForEach {
    Measure-Command {
         (1..1E4) | Where {$_ -gt 777} | Select -First 1
    }
} | Measure-Object -Property TotalMilliseconds -Average).Average

$where_operator_last = (1..20 | ForEach {
    Measure-Command {
         (1..1E4).Where({$_ -gt 777},'last')
    }
} | Measure-Object -Property TotalMilliseconds -Average).Average


$whereobject_last = (1..20 | ForEach {
    Measure-Command {
         (1..1E4) | Where {$_ -gt 777} | Select -Last 1
    }
} | Measure-Object -Property TotalMilliseconds -Average).Average

Write-Verbose "Time measured in milliseconds" -Verbose
[pscustomobject]@{
    where_operator = $where_operator
    whereobject = $whereobject
    where_operator_first = $where_operator_first
    whereobject_first = $whereobject_first
    where_operator_last= $where_operator_last
    whereobject_last = $whereobject_last
}

image

You can see here that using the Where operator does have the edge in each of the examples.

ForEach

The ForEach() operator is also new to PowerShell V4 and like Where(), does have performance perks vs. its pipeline counterpart yet slower than using ForEach () {}.

$foreach_operator = (measure-command {@(1..1E6).ForEach({$_})}).TotalMilliseconds
$foreach_pipeline = (measure-command {@(1..1E6) | ForEach{$_}}).TotalMilliseconds
$foreachobject_pipeline = (measure-command {@(1..1E6) | ForEach-object{$_}}).TotalMilliseconds
$foreach_ = (measure-command {ForEach ($i in @(1..1E6)) {$i}}).TotalMilliseconds

Write-Verbose 'Results in milliseconds' -Verbose

[pscustomobject]@{
    foreach_operator = $foreach_operator
    foreach_pipeline = $foreach_pipeline
    foreachobject_pipeline = $foreachobject_pipeline
    foreach_ = $foreach_
} | Format-List

image

As with Where(), you can find out what it is expecting in parameters by doing the following;

@(1..10).ForEach()

image

Unlike the where operator, I have had little success in getting the expression along with the arguments to work properly. As I explore this some more, I will try to find some good examples as to how you can use these.

One thing that you can do is cast the object like this:

@(1..10).ForEach([double])

image

We can verify that this actually happened by first seeing this done without the casting…

Get-Member -input @(1..10).ForEach({$_}) | Select TypeName -first 1

image

… with one that does the casting.

Get-Member -input @(1..10).ForEach([double]) | Select TypeName -first 1

image

Pretty neat stuff!

About Boe Prox

Microsoft PowerShell MVP working as a Senior Systems Administrator
This entry was posted in powershell, V4 and tagged , , , , . Bookmark the permalink.

15 Responses to Checking Out The Where and ForEach Operators in PowerShell V4

  1. rkeithhill says:

    With these two methods, if you don’t care about the extra options you can skip the parens and just use the curly braces – at least on PowerShell V5 e.g.:

    (gsv).Where{$_.Name -match ‘w3’}

  2. Justin Marshall says:

    i’m having a very wierd issue with foreach, i know this is an older blog but maybe someone can help. My script works perfectly fine in powershell 2 but after uprading to powershell 4 it stopped working. What it does it retrieves the registry for a given path on a remote server and stores that to a variable then iterates on each property to compare. in powershell 4 the iteration variable becomes null yet the ISE tooltip shows there is data.

           $_Path = (Invoke-Command -cn $ReferenceServerName -cred $cred {param($KeyPath); Get-Item $KeyPath | get-childitem -recurse} -Args $KeyPath)
    
        # initialize result arrays
        $ReferenceServer = @()
        $TargetServer= @()
        $results = @()
    
        ForEach ($key in $_Path)
        {
    
    
            $key | Get-Member
            ForEach ( $Property in $key )
            {           
    

    …………………………….

    the $key | Get-Member returns nothing, the foreach never enters the loop on $Property as it believeles $key is null yet tooltip shows it has data… i’m completely stumped.

    • Justin Marshall says:

      just noticed a typoe above the foreach loop should be ($Property in $key.Property) this is what fails, interestingly if it is $key only the loop enters and it partialy works (although the remainder of the logic fails).

  3. Kirk Munro says:

    Hi Boe,

    Readers of this article might also like to know about a similar article I posted up on PowerShell Magazine about the foreach and where methods. I think my article answers a few questions you had in yours. Here’s the link: http://www.powershellmagazine.com/2014/10/22/foreach-and-where-magic-methods/.

    Kirk out.

  4. Pingback: Generic Collections and Arrays revisited | The Powershell Workbench

  5. Pingback: More on generic collections in Powershell V4, and a gotcha. | The Powershell Workbench

  6. Pingback: Where() method in powershell 4 breaks my objects! – Simon Wåhlin

  7. glenn says:

    Nice. Thanks Boe.

  8. Lance says:

    Everything is timing. I had just downloaded v4 to play around and see what it was about after spending most all of my time on v2 (skipped v3). Hmmm, must be a fairly beefy package because everything sloowwwed way down. I admit I’m a huge fan of the new “commands” window.

  9. mjolinor says:

    Very nice article,

    One quibble. You should also include filters in that performance test. In situations where you can’t fit everything into memory and have to rely on the pipeline they add substantially less overhead than the Foreach pipeline constructs:

    $foreach_operator = (measure-command {@(1..1E6).ForEach({$_})}).TotalMilliseconds
    $foreach_pipeline = (measure-command {@(1..1E6) | ForEach{$_}}).TotalMilliseconds
    $foreachobject_pipeline = (measure-command {@(1..1E6) | ForEach-object{$_}}).TotalMilliseconds
    $foreach_ = (measure-command {ForEach ($i in @(1..1E6)) {$i}}).TotalMilliseconds
    filter test {$_}
    $filter_ = (measure-command {@(1..1E6) | test}).TotalMilliseconds

    Write-Verbose ‘Results in milliseconds’ -Verbose

    [pscustomobject]@{
    foreach_operator = $foreach_operator
    foreach_pipeline = $foreach_pipeline
    foreachobject_pipeline = $foreachobject_pipeline
    foreach_ = $foreach_
    filter_ = $filter_
    } | Format-List

  10. This is an excellent write-up.

  11. Chris Duck says:

    This works for me:

    @(1..10).foreach({$_ * $args[0]}, 2)

    It also works with a param statement in there:

    @(1..10).foreach({param($x) $_ * $x}, 2)

    If you want to check out the implementation it looks like the extensions are defined in the System.Management.Automation assembly. You can use ILSpy or similar and check out the Where and ForEach methods of the System.Management.Automation.EnumerableOps class.

    • Boe Prox says:

      Excellent! Thanks, Chris for identifying more about ForEach(). I knew that there were other ways to use it, but was just not getting it on my own. Thanks for the top on ILSpy, too. I really need to dive into that one of these days.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s