Latest Updates to PoshRSJob

Posted on April 19, 2015 by Boe Prox

I felt the need to make a follow up to address some new updates to my module, PoshRSJob. I had my initial post which covered most of the basics of this module and covered the majority of the functions which are available to use.

Better Pipeline Support for Variables in Scriptblock

Previously, If you wanted to send the piped data into the scriptblock (which is a big thing for this project), you had to specify a Param() block in the scriptblock and specify a variable that would serve as the data coming from the pipeline into Start-RSJob like this:

1..10 | Start-RSjob -ScriptBlock {
    Param($Number)
    [pscustomobject] @{
        Number = $Number
        DoubleIt = ($Number * 2)
    }
}

This way is no longer supported (if enough people want it back, then I will look at adding it back in) and has now been replaced by the removal of the Param() block and just using $_ in your scriptblock.

1..10 | Start-RSjob -ScriptBlock {
    [pscustomobject] @{
        Number = $_
        DoubleIt = ($_ * 2)
    }
}

image

We now have a much cleaner approach of handling the data coming from the pipeline which means less code to write. Don’t worry if you use another $_ in a ForEach block as it will handled appropriately.

The Param() block will still exist if you have data coming from the –ArgumentList parameter and also keep in mind that $Using: is still available as well in PowerShell V2 and above.

$Test = 42
$AnotherTest = 7
$String = 'SomeString'
$ProcName = 'powershell_ise'
$ScriptBlock = {
    Param($y,$z)
    [pscustomobject] @{
        Test = $y
        Proc = (Get-Process -Name $Using:ProcName)
        String = $Using:String
        AnotherTest = ($z+$_)
        PipedObject = $_
    }
}

1..5|Start-RSJob $ScriptBlock -ArgumentList $test, $anothertest

Get-RSJob | Receive-RSJob

image

A New Function

The second thing that I am going to highlight is a new function called Wait-RSJob, which as it sounds, will wait for the RSJobs to finish up before allowing access to the console again. It will also output the RSJob type by default which is useful if you wanted to chain some commands together. There is also a timeout which means that it will stop waiting for the RSJob to finish and allow access to the console again. If you have ever used Wait-Job, then you are already an expert at using this function.

A quick example is right here:

1..10 | Start-RSJob -ScriptBlock {
    Start-Sleep -Seconds (Get-Random (5..10))
    $_
} |Wait-RSJob

image

Chaining the commands together:

1..10 | Start-RSJob -ScriptBlock {
    Start-Sleep -Seconds (Get-Random (5..10))
    $_
} |Wait-RSJob | Receive-RSJob

image

Lastly, using the timeout parameter for 4 seconds:

1..10 | Start-RSJob -ScriptBlock {
    Start-Sleep -Seconds (Get-Random (5..10))
    $_
} |Wait-RSJob -Timeout 4

image

This function comes courtesy of Ryan Bushe who was nice enough to put this together.

Pester Testing

Thanks goes out to Warren, AKA psCookieMonster for taking the time to set up some Pester and AppVeyor testing out on Github. Pester is something that I want to work more with and is definitely on my TODO list.

Bugs and Other Things

Besides those additions, the rest of the things that have been added were mostly bug fixes in various parts of code as well as some behind the scenes tweaks.

Hopefully those of you who are using this module have been enjoying it and find it useful in your day to day activities! As always, if you run into issues, have requests for improvement, just let me know or you can work on it yourself and submit a Pull Request out on Github.

PoshRSJob Site

Posted in powershell | Tagged , | 24 Comments

Quick Hits: Finding Exception Types with PowerShell

Posted on April 9, 2015 by Boe Prox

I had a suggestion recently after the posting of my previous article on using Try/Catch for error handling with PowerShell that mentioned that it would be useful to see how to find exceptions that could be caught explicitly vs. using a Catch {} to grab all exceptions.

So in this post I will show you a couple of ways to find an exception that you can then use with Try/Catch to be sure to handle that particular error.

The first approach is actually found in the error exception record itself. Let’s assume that we want to try and get a file or view the items using Get-ChildItem. If we are not careful, we may come across something like this:

SNAGHTMLf8f8a8f

Ok, we can see that it is an ItemNotFoundException, but just adding that as a type won’t work too well.

image

What we can do is dig into the record and pull the full name of this exception so that we can see if it is publicly available and then to use it in a Catch block. We can expand out the object by using Select –Property * to see all of the properties and hopefully make a good determination of what the exception is.

$Error[4] | Select –Property *

SNAGHTMLf948df2

We can also (and more easily) see the exception by looking at the type as well.

$Error[4].exception.GetType().fullname

image

We can see now that the full name of the exception is System.Management.Automation.ItemNotFoundException and if we verify it in the console, we can see that it doesn’t throw any errors.

image

Now let’s use that in our command again and see what happens.

Try {
    Get-ChildItem \\prox-hyperv -ErrorAction Stop
} Catch [System.Management.Automation.ItemNotFoundException] {
    Write-Verbose "Path $($_.TargetObject) not found!" -Verbose
} Catch {
    Write-Verbose "Catch all" -Verbose
}

image

As you can see, that exception is now being handled differently than if it was just another exception.

The last way to find an error exception is by iterating through all of the currently loaded exceptions in PowerShell by getting all of the exported types of each assembly. We can do this by running the following command:

[appdomain]::CurrentDomain.GetAssemblies() | ForEach {
    Try {
        $_.GetExportedTypes() | Where {
            $_.Fullname -match 'Exception'
        }
    } Catch {}
} | Select FullName

image

This is just a handful of the 325 exception types that I found on my system. There could be more or less depending on what you have loaded in your session. But with this, you can hopefully find an exception that you need for your Catch block.

There you go! A quick look at locating exceptions that can be handled with Catch individually to better isolate various errors in your code.

Posted in powershell | Tagged , , , , | 2 Comments

A Look at Try/Catch in PowerShell

Posted on April 4, 2015 by Boe Prox

My article here wraps up PowerShell Blogging Week (#PSBlogWeek), which has been a series of articles on Windows PowerShell Advanced Functions. We hope that you have enjoyed this series and hope you will supply us with more ideas so we can continue to provide fun and useful material for you!

When working with errors and trying to get a handle on them, you need to use what is available to not only catch them, but also to determine what the next course of action is after you have caught them. In this article, I am focusing solely on Try/Catch (along with Finally) and how you can use these to perform some essential error handing in your scripts.

Let’s break down each of these statements and what they accomplish.

Try

Try is where you are going to place your code block in that you want to watch for errors that will be handled later on in the script. Know that wherever the error occurs, it will immediately stop at that point and move onto the Catch keyword (assuming that the error is a terminating error). During this you need to ensure that you either have $ErrorActionPreference=’Stop’ so every error that occurs is considered a terminating error (useful when working with outside commands that do not throw terminating errors by default) or you can make use of –ErrorAction ‘Stop’ in a cmdlet to force the cmdlet to throw a terminating error.

Note: Did you know that using the Throw keyword says that the error being thrown is a terminating error? Try it in a Try/Catch and see what happens!

Here is an action of what be in a Try statement:

Try {
    Write-Verbose "[TRY] Checking for OS" -Verbose
    $OS = Get-WmiObject -ComputerName RemoteServer -Class Win32_OperatingSystem -ErrorAction Stop
    Write-Verbose "[TRY] No issues found" -Verbose
}

Assuming that I can connect to this remote system, I should see the Verbose output afterwards, otherwise based on the ErrorAction of Stop, it will send the execution down to the Catch statement.

Catch

Here is where the execution of code continues after an error occurs within the Try statement. One of the neat things about using Catch is that you can have multiple Catch statements based on the error type that you want to handle. If you want to take a different action on an Access Denied error, you can specify the System.UnauthorizedAccessException type and anytime the error record relates to an unauthorized access exception, it will be handled in that statement while all other errors will be handles by the Catch statement with no exception type defined. Think of this one as the ‘catch all’ Catch block. Here is an example of such a Catch statement.

Catch [System.UnauthorizedAccessException] {
    Write-Warning "[CATCH] You do not have the proper access to this system!"
    BREAK
}
Catch [System.Runtime.InteropServices.COMException] {
    Write-Warning "[CATCH] Communications Exception occurred!"
    BREAK
}
Catch {
    Write-Warning "[CATCH] Errors found during attempt:`n$_"
    BREAK
}

Finally

This keyword performs an action regardless of what happens in the Try/Catch statements. This allows you to perform a sort of cleanup of resources or anything else that you may want to do. Even if we specify BREAK in our Catch block, this will still run any code that happens to be in the Finally block. In fact, even adding Exit to my Catch block will not prevent anything in the Finally block from running before the session is closed.

Finally {
    Write-Verbose "[FINALLY] Performing cleanup actions." -Verbose
}
Write-Verbose "Doing something outside of Try/Catch/Finally" -Verbose

Note that I added some verbose output to run outside of the Finally block. If something throws a terminating error, you will see the Verbose output in the Finally block, but nothing beyond that.

Putting it all together

Putting this all together, we can see how these all come together to work just like we want them to. First let’s see a good attempt with our code:

clip_image002

Now let’s try this with a known issue happening:

clip_image004

Notice that we did not see the verbose output that exists outside of the Try/Catch/Finally blocks. You can also notice that it caught the System.Runtime.InteropServices.ComException exception rather than the ‘catch all’ block. Had I run into an issue where I was getting access denied, the System.UnauthorizedAccessException would have caught the error, otherwise my last Catch block will get the error.

With that, you can see how to use Try/Catch/Finally to control the flow of your code in the event of errors during execution. Of course, this wasn’t an exhaustive look at error handling but is still helpful to get your going!

Hungry for more information about Try/Catch and error handling? Check out the following links for more information about this topic:

#PSBlogWeek Articles:

Posted in powershell | Tagged , , , , | 11 Comments

Introducing PoshRSJob as an Alternative to PowerShell Jobs

Posted on March 31, 2015 by Boe Prox

Maybe you have heard me blogging, speaking or tweeting off and on about using runspaces (or runspacepools) as a means to run tasks in the background vs. using PowerShell Jobs (PSJobs) (Start-Job, Get-Job, etc…).

While I have seen a lot of folks make some great scripts/functions that utilize this technique to run multiple concurrent operations in the background using throttling as well as being speedier than using PSJobs, the one thing that I thought was missing was a similar PSJobs like architecture that would send these runspaces to their own runspacepool that would then be tracked and disposed of when no longer needed. We would also have an easier way to view the state of each runspace job just like we would using PSJobs with Get-Job.

Enter PoshRSJob.

This is something that I have worked on for the past several months off and on to bring in a PSJobs like approach that instead uses runspacepools and runspaces to do what you could normally, but now with throttling, better performance and just because it was something fun to work on. This is still very much a Beta project and doesn’t cover everything that you find in PSjobs, but I am confident that it will work with most of what you want to do with it.

So how do I get this brand new module? You can download this currently from GitHub here. Eventually when I present a version 1.0 release, it will find its way to the PowerShell Gallery and perhaps CodePlex.

Once it is downloaded and placed in your Modules folder of choice.

SNAGHTML2fb318c5

In my case, I am placing this under my Documents folder so it will be available with my profile.

Once I have done that, I can verify that the module is showing up.

image

Now I will go ahead and import the module so it is available in my session using the following command;

Import-Module –Name PoshRSJob

image

And now we are ready to start using this!

I intentionally made the functions as similar as I could to what you would find with *-Job as well as similar parameters so there would be that familiarity that we are already used to.

image

Currently these are the only functions that I have available and some of the parameters may be limited, but I do have plans to expand out with a Wait-RSJob and some extra parameters for certain functions. With that out of the way, let’s get started with looking at how these work.

Start-RSJob

Arguable the most important function here. This is the function that will kick off various runspace jobs that we can then send to the background so we have the console free to do whatever we want. This is really the only function that I will go into depth with because the other functions behave very much like their PSJobs counterparts and share the same parameters.

Let’s look at the parameters of this function.

image

  • InputObject
    • This is where you would pipe data into the function (or provide a collection of items) that will then take that object and use it as the first Parameter in your scriptblock as long as you add a Param() block in it.
  • Name
    • This is the name of your job. You can also take the input being piped to this function and use a scriptblock to create a more useful name. –Name {“Test_$($_)”} for instance.
  • ArgumentList
    • This is similar to using Start-Job and supplying extra variables that will be referenced based on what you have in your Param() block in the scriptblock (and in the same order).
  • Throttle
    • This is one of the cool things about this module. We can now throttle how many concurrently running jobs are going at a given time. The default value is 5, but you can make this whatever you want. No more worries about finding a way to do throttling on your own!
  • ModulesToImport
    • This lets us specify various modules that we want to import in each runspace job to run various commands.
  • FunctionsToLoad
    • Similar to the modules parameter, but this is something new with PoshRSJob that you won’t find in the PSJobs cmdlets.

Those are the parameters, but we aren’t done yet with some cool features. As you know, $Using: was introduced in V3 that allows us to bring in variables from our parent PowerShell process into the PSJob’s scriptblock like magic! Well, after some digging around and some tears, I have brought this excellent feature to PoshRSJob to use as well! But it doesn’t stop there because I back ported this capability to work in PowerShell V2 as well!

Now let’s see some of this in action. Notice that I do not use ForEach when using Start-RSJob. This takes input directly from the pipeline to work properly and fully utilize the throttling capabilities.

$Test = 42
1..5|Start-RSJob -Name {"TEST_$($_)"} -ScriptBlock {
    Param($Object)
    $DebugPreference = 'Continue'
    $PSBoundParameters.GetEnumerator() | ForEach {
        Write-Debug $_
    }
    Write-Verbose "Creating object" -Verbose
    New-Object PSObject -Property @{
        Object=$Object
        Test=$Using:Test
    }
}

image

This is a familiar view. We see our job id, name as well as the state of the job and if we have errors/data available. You can see how we set up our –Name to be whatever the input object is that is being passed via the pipeline.

We can use Get-RSJob and filter for various states to see how our jobs are doing. In the case of above, everything has already finished up. What is also worth noting is that this took pretty much no time at all to set up and run. We can compare that with something similar to PSJobs and see the performance.

Write-Verbose "RSJobs Test" -Verbose
$Test = 42
(Measure-Command {
    1..5|Start-RSJob -Name {"TEST_$($_)"} -ScriptBlock {
        Param($Object)
        $DebugPreference = 'Continue'
        $PSBoundParameters.GetEnumerator() | ForEach {
            Write-Debug $_
        }
        Write-Verbose "Creating object" -Verbose
        New-Object PSObject -Property @{
            Object=$Object
            Test=$Using:Test
        }
    }
}).TotalSeconds

Write-Verbose "PSJobs Test" -Verbose
$Test = 42
(Measure-Command {
    1..5 | ForEach {
        Start-Job -Name "TEST_$($_)" -ScriptBlock {
            Param($Object)
            $DebugPreference = 'Continue'
            $PSBoundParameters.GetEnumerator() | ForEach {
                Write-Debug $_
            }
            Write-Verbose "Creating object" -Verbose
            New-Object PSObject -Property @{
                Object=$Object
                Test=$Using:Test
            }
        } -ArgumentList $_
    }
}).TotalSeconds

image

The test here is about a second faster with just 5 jobs. Given this was a simple job example, but depending on what you are using and how many jobs you plan to run, there will be an advantage with using PoshRSJob.

So I mentioned that $Using works great on PowerShell V2, so let me back up that claim with this:

UsingVariable

You can see that I am using V2 and have complete use of $Using to keep my variables in my parent scope into the RSJob.

You can also see my use of Receive-RSJob to grab the data that is available from the job.

If you want streams support that you find in PSJobs, they are here as well in the RSJob. Checking out the RSJob object, we can see the Verbose and Debug stream.

image

If we had any errors, we would see that the HasErrors property would be True and we could check out the Error stream as well to see what happened.

Stopping a RSJob and Removing a RSJob behaves just like you would find with the PSJobs cmdlets. You can pipe the objects from Get-RSJob into each function or specify filtering based on various parameters.

image

One last example this time letting me run a query against multiple systems.

$Cred = Get-Credential
@($Env:Computername,'prox-hyperv','dc1')|Start-RSJob -Name {"TEST_$($_)"} -ScriptBlock {
    Param($Computername)
    $DebugPreference = 'Continue'
    $PSBoundParameters.GetEnumerator() | ForEach {
        Write-Debug $_
    }
    Write-Verbose "Connecting to $Computername" -Verbose
    If ($Computername -eq $Env:Computername) {
        Get-WMIObject -class Win32_Computersystem
    } Else {
        Get-WMIObject -class Win32_ComputerSystem -ComputerName $Computername -Credential $Using:Cred
    }
}

Now we will grab the data that is waiting inside each RSjob object.

image

So with that, feel free to give it a download and let me know what works, what doesn’t work and what you think should be in it. Remember, this is still a beta and has more work to do, but in my own limited testing, things seemed to work pretty well with minimal issues (that I have hopefully fixed up to this current release). I also hope to add a Wait-RSJob and a few more parameters to keep these as close as I can to their PSJob equivalents for ease of use.

Download PoshRSJob

https://github.com/proxb/PoshRSJob

Posted in powershell | Tagged , , , , | 44 Comments

Checking For Disconnected Connections with TCPListener Using PowerShell

Posted on March 29, 2015 by Boe Prox

Working on my PoshChat project, one of the things that I was dealing with was handling connections that was disconnected through different means other than just the client disconnecting on its own such as a client being rebooted or the network just dropping the connection.

After some research and testing, I found an approach that seems to handle dropped connections and alert the Listener when the client has disconnected the connection. The approach is done by using a Poll() method from the Client object (more on this in my example). By doing this, we can test to see if the remote end connection is still available and if it isn’t available, we can process that as a closed connection and dispose of the connection objects.

Set up the Listener

Let’s get started with this. First thing is that I am going to create the the listener object and start it up so I can begin accepting TCP connections.

$Listener = [System.Net.Sockets.TcpListener]15600
$Listener.Start()

SNAGHTML2571d6ca

Now that we have our listener up and going, we can start monitoring for incoming connections using an approach I talked about a while back.

If ($Listener.Pending()) { 
    ...
}

Client connection

While we are waiting for the connection, we can now initiate that connection from a different console.

$Server = 'Boe-PC'

#Connect to server
$Endpoint = new-object System.Net.IPEndpoint ([ipaddress]::any,$SourcePort)
$Client = [Net.Sockets.TCPClient]$endpoint  
$Client.Connect($Server,15600)

And now we will have a pending connection on the server. This means we get to accept the client connection.

$ServerClient = $Listener.AcceptTcpClient()           

Write-Verbose ("[$(Get-Date)] New Connection from {0} <{1}>!" -f $ServerClient.Client.RemoteEndPoint.Address, 
$ServerClient.Client.RemoteEndPoint.Port) –Verbose

image

We can verify that the connection is established.

SNAGHTML257b614c

Now note the Connected property here on the server side client. Currently it shows as being connected when I view its properties.

image

Now let’s view it again.

SNAGHTML257c506e

That is weird. What in the world does this mean? The connection still shows as being good when I view the TCP connections, but I am seeing the client showing it as no longer connected. Well, this is because this value updates based on the last time an I/O operation has occurred. So we had the initial data transfer when I viewed it initially, but now that nothing has occurred prior to this viewing, it simply shows as being false.

This means that we should not use just this approach as a reliable way of testing a connection between to endpoints. Enter the Poll method that I mentioned earlier.

image

This takes two parameters: the number in milliseconds that it will take to poll and the type of SelectMode to test for. I am using SelectRead because I want to know if the connection has been reset, terminated or closed. I am also going to be combining this with checking the Available property as well.

A quick test will show that the connection is still available even though the Connected property is showing False.

$ServerClient.Client.Poll(1,[System.Net.Sockets.SelectMode]::SelectRead)

image

Currently shows False, which means that we do not have any connection issues, but also keep in mind that this also means that we do not have any data available, which is why we also need to throw in the DataAvailable check as well.

While (-NOT ($ServerClient.Client.Poll(1,[System.Net.Sockets.SelectMode]::SelectRead) -AND 
$ServerClient.Client.Available -eq 0)) {
    Start-Sleep -Milliseconds 100
}

Let’s assume that we have a network issue (in my case I am closing the PowerShell console). Once the poll runs, it will see that the return from the Poll is True and since there is no data available, it sees this as a closed connection and processes it accordingly.

Write-Verbose "[$(Get-Date)] Connection has closed remotely" -Verbose
#Shutdown the connection
Write-Verbose "[$(Get-Date)] Closing socket" -Verbose
$ServerClient.Client.Shutdown([System.Net.Sockets.SocketShutdown]::Both)
$ServerClient.Client.Close()

image

And now we are back to just the listener showing up in our TCP connections.

image

In my testing, I have found this to be a good approach to tracking multiple connections to a single listener and in handling their sudden disconnects. In my case I am using runspaces to handle the listener, the subsequent connections, performing Poll checks as well as data send and receives. This helps in the case where I have some data that has been sent to the server prior to a disconnect happening in that it will be processed in a different thread and resets the DataAvailable back to 0 which will cause the Poll thread to see that the client has disconnected on its next check. Otherwise if you are not handling the incoming data receives, you will have a harder time of determining that a disconnection has occurred.

While I do not doubt that this has a small audience who may use this, it will hopefully help out those who are in a similar situation as me when it came to handling these disconnects gracefully.

Posted in powershell | Tagged , , , | Leave a comment