I felt the need to make a follow up to address some new updates to my module, PoshRSJob. I had my initial post which covered most of the basics of this module and covered the majority of the functions which are available to use.
Better Pipeline Support for Variables in Scriptblock
Previously, If you wanted to send the piped data into the scriptblock (which is a big thing for this project), you had to specify a Param() block in the scriptblock and specify a variable that would serve as the data coming from the pipeline into Start-RSJob like this:
1..10 | Start-RSjob -ScriptBlock {
Param($Number)
[pscustomobject] @{
Number = $Number
DoubleIt = ($Number * 2)
}
}
This way is no longer supported (if enough people want it back, then I will look at adding it back in) and has now been replaced by the removal of the Param() block and just using $_ in your scriptblock.
1..10 | Start-RSjob -ScriptBlock {
[pscustomobject] @{
Number = $_
DoubleIt = ($_ * 2)
}
}
We now have a much cleaner approach of handling the data coming from the pipeline which means less code to write. Don’t worry if you use another $_ in a ForEach block as it will handled appropriately.
The Param() block will still exist if you have data coming from the –ArgumentList parameter and also keep in mind that $Using: is still available as well in PowerShell V2 and above.
$Test = 42
$AnotherTest = 7
$String = 'SomeString'
$ProcName = 'powershell_ise'
$ScriptBlock = {
Param($y,$z)
[pscustomobject] @{
Test = $y
Proc = (Get-Process -Name $Using:ProcName)
String = $Using:String
AnotherTest = ($z+$_)
PipedObject = $_
}
}
1..5|Start-RSJob $ScriptBlock -ArgumentList $test, $anothertest
Get-RSJob | Receive-RSJob
A New Function
The second thing that I am going to highlight is a new function called Wait-RSJob, which as it sounds, will wait for the RSJobs to finish up before allowing access to the console again. It will also output the RSJob type by default which is useful if you wanted to chain some commands together. There is also a timeout which means that it will stop waiting for the RSJob to finish and allow access to the console again. If you have ever used Wait-Job, then you are already an expert at using this function.
A quick example is right here:
1..10 | Start-RSJob -ScriptBlock {
Start-Sleep -Seconds (Get-Random (5..10))
$_
} |Wait-RSJob
Chaining the commands together:
1..10 | Start-RSJob -ScriptBlock {
Start-Sleep -Seconds (Get-Random (5..10))
$_
} |Wait-RSJob | Receive-RSJob
Lastly, using the timeout parameter for 4 seconds:
1..10 | Start-RSJob -ScriptBlock {
Start-Sleep -Seconds (Get-Random (5..10))
$_
} |Wait-RSJob -Timeout 4
This function comes courtesy of Ryan Bushe who was nice enough to put this together.
Pester Testing
Thanks goes out to Warren, AKA psCookieMonster for taking the time to set up some Pester and AppVeyor testing out on Github. Pester is something that I want to work more with and is definitely on my TODO list.
Bugs and Other Things
Besides those additions, the rest of the things that have been added were mostly bug fixes in various parts of code as well as some behind the scenes tweaks.
Hopefully those of you who are using this module have been enjoying it and find it useful in your day to day activities! As always, if you run into issues, have requests for improvement, just let me know or you can work on it yourself and submit a Pull Request out on Github.
Learn Powershell | Achieve More 
How do I throttle jobs here?
I am using following code:
Get-RSJob -State Completed | Remove-RSJob
while (@(Get-RSJob -State Running).Count -ge $JOB_THROTTLE_LIMIT) { }
Is this correct? It works fine for PSJobs
Also can I use mutex in each job? It seems to be working.
$lock = New-Object ‘Threading.Mutex’ $false, “LOGFILE”
$lock.WaitOne() | Out-Null
“output result” >> $logfile
$lock.ReleaseMutex()
i am doing a test with RSJobs and PSJobs by running 400 jobs at once.
Yes I know… but I am trying to process over 10K+ machines so I want to span as many threads as possible. I have 8 core machine with 16GB ram.
I noticed in proc explorer that RSJobs only spans one powershell.exe process with many threads while PSJobs run multiple processes with multiple threads in each. Actually initially PSJobs takes up a lot of RAM before reaching 400 jobs (which i think it’s normal and thus it’s slow) and then as completed jobs are removed memory is released. I noticed that for a while all PSJobs powershell.exe processes run on single core but later I can see that all 8 cores are being utilized and the memory usage is well down.
Great job Boe!
Would appreciate to have examples of how passing $script_block or .ps1 files parameters to -ScriptBlock as jbruns2015 asked.
Tried $files | Start-RSJob $script_block -ArgumentList $_ | Wait-RSJob | Receive-RSJob
$script_block = [scriptblock] {
Param($file)
[pscustomobject] @{
PipedObject = $_
}
operations…
}
Got this Error:
You cannot call a method on a null-valued expression.
At C:\Program Files\WindowsPowerShell\Modules\PoshRSJob\1.7.2.9\Public\Start-RSJob.ps1:492 char:70
+ … Write-Verbose “Adding Argument: $($) <$($.GetType().Fullname)>”
You will not need to use -ArgumentList in your example as $_ is already being used as the $files input that you are passing through the pipeline. You also won’t need the param() block either in this case.
First thanks so much for this module. it has made my life infinitely easier.
Now for my question : does Start-RSjob support a credential parameter in any way? i can’t seem to find a way to pass that to these functions.
I tried this on the current version and it works, so, I suspect using parameters is now back in because people wanted it?
$jobs = @(“ABC”) | Start-RSJob -ScriptBlock { param ($Moo) $Moo }
$jobs | Receive-RSJob
Yea, there were some folks who wanted to have it working for backwards compatibility.
Hey Boe, great stuff with this module and others. I’m not sure I’m using this module correctly though. Here is my script: http://pastebin.com/ewMk3H41
The csv file has 2 fields, VMName & OSFullName.
The errors I’m getting are below:
Attempt by method ‘DynamicClass.CallSite.Target(System.Runtime.CompilerServices.Closure,
System.Runtime.CompilerServices.CallSite, System.Object[])’ to access type
‘System.Management.Automation.Language.PSGetMemberBinder’ failed.
At C:\Program Files\WindowsPowerShell\Modules\PoshRSJob\1.5.7.7\Public\Start-RSJob.ps1:227 char:19
+ If (@($ArgumentList).count -match ‘0|1’) {
+ ~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], TypeAccessException
+ FullyQualifiedErrorId : System.TypeAccessException
Attempt by method ‘DynamicClass.CallSite.Target(System.Runtime.CompilerServices.Closure,
System.Runtime.CompilerServices.CallSite, System.Object[])’ to access field
‘System.Management.Automation.Language.PSGetMemberBinder._version’ failed.
At C:\Program Files\WindowsPowerShell\Modules\PoshRSJob\1.5.7.7\Public\Start-RSJob.ps1:268 char:27
+ … $SBParamCount = @(GetParamVariable -ScriptBlock $ScriptBlock).Count
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], FieldAccessException
+ FullyQualifiedErrorId : System.FieldAccessException
Attempt by method ‘DynamicClass.CallSite.Target(System.Runtime.CompilerServices.Closure,
System.Runtime.CompilerServices.CallSite, System.Object)’ to access field
‘System.Management.Automation.Language.PSGetMemberBinder._version’ failed.
At C:\Program Files\WindowsPowerShell\Modules\PoshRSJob\1.5.7.7\Public\Start-RSJob.ps1:269 char:30
+ $ArgumentCount = If ($ArgumentList.count -eq $Null) {
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], FieldAccessException
+ FullyQualifiedErrorId : System.FieldAccessException
Start-RSJob : Attempt by method ‘DynamicClass.CallSite.Target(System.Runtime.CompilerServices.Closure,
System.Runtime.CompilerServices.CallSite, System.Object)’ to access field
‘System.Management.Automation.Language.PSGetMemberBinder.version’ failed.
At \amon\shared\Information Systems\Technology Support\Powershell\Tim\Scripts\VSphere\get-psversion.ps1:31 char:30
+ … rList | % { Start-RSJob -ScriptBlock $Scriptblock -ArgumentList $ -t …
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [Start-RSJob], FieldAccessException
+ FullyQualifiedErrorId : System.FieldAccessException,Start-RSJob
Hey Boe, great stuff with this module and others. I’m not sure I’m using this module correctly though. Here is my script: http://pastebin.com/ewMk3H41
The csv file has 2 fields, VMName & OSFullName.
The errors I’m getting are below:
Attempt by method ‘DynamicClass.CallSite.Target(System.Runtime.CompilerServices.Closure,
System.Runtime.CompilerServices.CallSite, System.Object[])’ to access type
‘System.Management.Automation.Language.PSGetMemberBinder’ failed.
At C:\Program Files\WindowsPowerShell\Modules\PoshRSJob\1.5.7.7\Public\Start-RSJob.ps1:227 char:19
+ If (@($ArgumentList).count -match ‘0|1’) {
+ ~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], TypeAccessException
+ FullyQualifiedErrorId : System.TypeAccessException
Attempt by method ‘DynamicClass.CallSite.Target(System.Runtime.CompilerServices.Closure,
System.Runtime.CompilerServices.CallSite, System.Object[])’ to access field
‘System.Management.Automation.Language.PSGetMemberBinder._version’ failed.
At C:\Program Files\WindowsPowerShell\Modules\PoshRSJob\1.5.7.7\Public\Start-RSJob.ps1:268 char:27
+ … $SBParamCount = @(GetParamVariable -ScriptBlock $ScriptBlock).Count
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], FieldAccessException
+ FullyQualifiedErrorId : System.FieldAccessException
Attempt by method ‘DynamicClass.CallSite.Target(System.Runtime.CompilerServices.Closure,
System.Runtime.CompilerServices.CallSite, System.Object)’ to access field
‘System.Management.Automation.Language.PSGetMemberBinder._version’ failed.
At C:\Program Files\WindowsPowerShell\Modules\PoshRSJob\1.5.7.7\Public\Start-RSJob.ps1:269 char:30
+ $ArgumentCount = If ($ArgumentList.count -eq $Null) {
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], FieldAccessException
+ FullyQualifiedErrorId : System.FieldAccessException
Start-RSJob : Attempt by method ‘DynamicClass.CallSite.Target(System.Runtime.CompilerServices.Closure,
System.Runtime.CompilerServices.CallSite, System.Object)’ to access field
‘System.Management.Automation.Language.PSGetMemberBinder.version’ failed.
At \amon\shared\Information Systems\Technology Support\Powershell\Tim\Scripts\VSphere\get-psversion.ps1:31 char:30
+ … rList | % { Start-RSJob -ScriptBlock $Scriptblock -ArgumentList $ -t …
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [Start-RSJob], FieldAccessException
+ FullyQualifiedErrorId : System.FieldAccessException,Start-RSJob
Hi Boe,
I’ve just started looking at the potential of powershell in order to automate and speed up some of my jobs and your POSHRSJob looks great:)
I’m trying to use PoshRSJob to run a console application recursively on a few tens of thousand of folders.
I can get the list of directories using: Get-ChildItem -Recurse | ?{ $_.PSIsContainer } | Select-Object FullName
The application needs the directory path as a parameter.
I did a few attempts to create a script that would use PoshRSJob but I am definietly doing something wrong.
Could you please refer to a sample script that would put me in the right direction?
Thanks,
Haran
I tried recommending this to someone today but they could not use it because of the DLL. What’s the DLL for and where is the source code?
Thanks for reminding me to do that! I don’t make a lot of .dlls so I completely forgot to add the source code along with it! All the dll does is it creates the necessary types for PoshRSJob to use and was added to provide support for Nano server. You can now see the source here: https://github.com/proxb/PoshRSJob/blob/master/PoshRSJob/bin/PoshRS.PowerShell.cs
Thanks for the quick action.
Got any examples of using poshRSjob with ps1 files and passing arguments to that ps1?
Hey Boe!
Thanks for all your work on this module. I have a database user inventory I’ve just added this functionality to, with some thread tuning it’s taken the whole process from over 66 minutes down to 28 minutes. I’m processing about 45k user objects in all, on my inventory database I’m peaking at 450 ops/sec, up from 80-90 before PoshRSJob. My aim is to peg my 4 CPU VM at 95% CPU and maintain the 450 ops/sec, and I’m getting there but having issues getting it to sustain either.
Not really looking for help yet, I just wanted to give you a practical example of how your script is making my enterprise more efficient. I’m sure this is a similar story in 10,000 other cases (but all you ever hear is “great mod/script/etc” and/or issues people are having with stuff).
Great module and documentation! Thanks so much for all your effort!
Brendan62269
Hi Brendan!
This is great to hear about how PoshRSJob is helping out in your environment! While I do enjoy the “great job” and the bug/feature requests, it is nice to hear how it is being used. Glad that it is helping to reduce the time spent running the commands.
I’m liking it! Would you like help with the help?
Thank you! If you want to assist with writing some more help examples (or just updating the help in general), feel free to fork the repo and to a PR and I will review and accept it!
Great job and thanks for adding the Progress parameter to the Wait-RSJob script.
I would though suggest a “Start-Sleep -Milliseconds 100” in the Do/While loop.
Thank you! That would be a good suggestion to implement to ensure the CPU doesn’t ramp up.
Hi Boe, I am trying to start a WPF GUI in it’s own runspace with Start-RSJob but I run into trouble. Could you please give us a Quick example how that can be accomplished 🙂
Awesome work Boe ! Will give it a try soon
Great work!
Please create a new Release on Github. Thx!
Thanks! I just published a new release.
Cool I like the work you are doing on this, it’s nice to have a speedier alternative to the Start-Job cmdlet.