Latest Updates to PoshRSJob

I felt the need to make a follow up to address some new updates to my module, PoshRSJob. I had my initial post which covered most of the basics of this module and covered the majority of the functions which are available to use.

Better Pipeline Support for Variables in Scriptblock

Previously, If you wanted to send the piped data into the scriptblock (which is a big thing for this project), you had to specify a Param() block in the scriptblock and specify a variable that would serve as the data coming from the pipeline into Start-RSJob like this:

1..10 | Start-RSjob -ScriptBlock {
    Param($Number)
    [pscustomobject] @{
        Number = $Number
        DoubleIt = ($Number * 2)
    }
}

This way is no longer supported (if enough people want it back, then I will look at adding it back in) and has now been replaced by the removal of the Param() block and just using $_ in your scriptblock.

1..10 | Start-RSjob -ScriptBlock {
    [pscustomobject] @{
        Number = $_
        DoubleIt = ($_ * 2)
    }
}

image

We now have a much cleaner approach of handling the data coming from the pipeline which means less code to write. Don’t worry if you use another $_ in a ForEach block as it will handled appropriately.

The Param() block will still exist if you have data coming from the –ArgumentList parameter and also keep in mind that $Using: is still available as well in PowerShell V2 and above.

$Test = 42
$AnotherTest = 7
$String = 'SomeString'
$ProcName = 'powershell_ise'
$ScriptBlock = {
    Param($y,$z)
    [pscustomobject] @{
        Test = $y
        Proc = (Get-Process -Name $Using:ProcName)
        String = $Using:String
        AnotherTest = ($z+$_)
        PipedObject = $_
    }
}

1..5|Start-RSJob $ScriptBlock -ArgumentList $test, $anothertest

Get-RSJob | Receive-RSJob

image

A New Function

The second thing that I am going to highlight is a new function called Wait-RSJob, which as it sounds, will wait for the RSJobs to finish up before allowing access to the console again. It will also output the RSJob type by default which is useful if you wanted to chain some commands together. There is also a timeout which means that it will stop waiting for the RSJob to finish and allow access to the console again. If you have ever used Wait-Job, then you are already an expert at using this function.

A quick example is right here:

1..10 | Start-RSJob -ScriptBlock {
    Start-Sleep -Seconds (Get-Random (5..10))
    $_
} |Wait-RSJob

image

Chaining the commands together:

1..10 | Start-RSJob -ScriptBlock {
    Start-Sleep -Seconds (Get-Random (5..10))
    $_
} |Wait-RSJob | Receive-RSJob

image

Lastly, using the timeout parameter for 4 seconds:

1..10 | Start-RSJob -ScriptBlock {
    Start-Sleep -Seconds (Get-Random (5..10))
    $_
} |Wait-RSJob -Timeout 4

image

This function comes courtesy of Ryan Bushe who was nice enough to put this together.

Pester Testing

Thanks goes out to Warren, AKA psCookieMonster for taking the time to set up some Pester and AppVeyor testing out on Github. Pester is something that I want to work more with and is definitely on my TODO list.

Bugs and Other Things

Besides those additions, the rest of the things that have been added were mostly bug fixes in various parts of code as well as some behind the scenes tweaks.

Hopefully those of you who are using this module have been enjoying it and find it useful in your day to day activities! As always, if you run into issues, have requests for improvement, just let me know or you can work on it yourself and submit a Pull Request out on Github.

PoshRSJob Site

About Boe Prox

Microsoft Cloud and Datacenter MVP working as a SQL DBA.
This entry was posted in powershell and tagged , . Bookmark the permalink.

21 Responses to Latest Updates to PoshRSJob

  1. Haris Buchal says:

    How do I throttle jobs here?
    I am using following code:

    Get-RSJob -State Completed | Remove-RSJob
    while (@(Get-RSJob -State Running).Count -ge $JOB_THROTTLE_LIMIT) { }

    Is this correct? It works fine for PSJobs

    Also can I use mutex in each job? It seems to be working.

    $lock = New-Object ‘Threading.Mutex’ $false, “LOGFILE”
    $lock.WaitOne() | Out-Null
    “output result” >> $logfile
    $lock.ReleaseMutex()

    i am doing a test with RSJobs and PSJobs by running 400 jobs at once.
    Yes I know… but I am trying to process over 10K+ machines so I want to span as many threads as possible. I have 8 core machine with 16GB ram.

    I noticed in proc explorer that RSJobs only spans one powershell.exe process with many threads while PSJobs run multiple processes with multiple threads in each. Actually initially PSJobs takes up a lot of RAM before reaching 400 jobs (which i think it’s normal and thus it’s slow) and then as completed jobs are removed memory is released. I noticed that for a while all PSJobs powershell.exe processes run on single core but later I can see that all 8 cores are being utilized and the memory usage is well down.

  2. Summer says:

    Great job Boe!
    Would appreciate to have examples of how passing $script_block or .ps1 files parameters to -ScriptBlock as jbruns2015 asked.
    Tried $files | Start-RSJob $script_block -ArgumentList $_ | Wait-RSJob | Receive-RSJob
    $script_block = [scriptblock] {
    Param($file)
    [pscustomobject] @{
    PipedObject = $_
    }
    operations…
    }

    Got this Error:
    You cannot call a method on a null-valued expression.
    At C:\Program Files\WindowsPowerShell\Modules\PoshRSJob\1.7.2.9\Public\Start-RSJob.ps1:492 char:70
    + … Write-Verbose “Adding Argument: $($) <$($.GetType().Fullname)>”

    • Boe Prox says:

      You will not need to use -ArgumentList in your example as $_ is already being used as the $files input that you are passing through the pipeline. You also won’t need the param() block either in this case.

  3. Phillip Marshall says:

    First thanks so much for this module. it has made my life infinitely easier.

    Now for my question : does Start-RSjob support a credential parameter in any way? i can’t seem to find a way to pass that to these functions.

  4. Cody says:

    I tried this on the current version and it works, so, I suspect using parameters is now back in because people wanted it?

    $jobs = @(“ABC”) | Start-RSJob -ScriptBlock { param ($Moo) $Moo }
    $jobs | Receive-RSJob

    Moo

  5. Cody says:

    I tried recommending this to someone today but they could not use it because of the DLL. What’s the DLL for and where is the source code?

  6. jbruns2015 says:

    Got any examples of using poshRSjob with ps1 files and passing arguments to that ps1?

  7. brendan62269 says:

    Hey Boe!
    Thanks for all your work on this module. I have a database user inventory I’ve just added this functionality to, with some thread tuning it’s taken the whole process from over 66 minutes down to 28 minutes. I’m processing about 45k user objects in all, on my inventory database I’m peaking at 450 ops/sec, up from 80-90 before PoshRSJob. My aim is to peg my 4 CPU VM at 95% CPU and maintain the 450 ops/sec, and I’m getting there but having issues getting it to sustain either.
    Not really looking for help yet, I just wanted to give you a practical example of how your script is making my enterprise more efficient. I’m sure this is a similar story in 10,000 other cases (but all you ever hear is “great mod/script/etc” and/or issues people are having with stuff).
    Great module and documentation! Thanks so much for all your effort!

    Brendan62269

    • Boe Prox says:

      Hi Brendan!

      This is great to hear about how PoshRSJob is helping out in your environment! While I do enjoy the “great job” and the bug/feature requests, it is nice to hear how it is being used. Glad that it is helping to reduce the time spent running the commands.

  8. JB Lewis says:

    I’m liking it! Would you like help with the help?

    • Boe Prox says:

      Thank you! If you want to assist with writing some more help examples (or just updating the help in general), feel free to fork the repo and to a PR and I will review and accept it!

  9. Mike Gallo says:

    Great job and thanks for adding the Progress parameter to the Wait-RSJob script.

    I would though suggest a “Start-Sleep -Milliseconds 100” in the Do/While loop.

  10. Jarko says:

    Hi Boe, I am trying to start a WPF GUI in it’s own runspace with Start-RSJob but I run into trouble. Could you please give us a Quick example how that can be accomplished 🙂

  11. Awesome work Boe ! Will give it a try soon

  12. Manuel Henke says:

    Great work!
    Please create a new Release on Github. Thx!

  13. Jaap Brasser says:

    Cool I like the work you are doing on this, it’s nice to have a speedier alternative to the Start-Job cmdlet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s