I’ve had requests for a while to talk about sharing data between runspaces and I was also asked about this during my recent interview on the Powerscripting Podcast last week. I admit that I have had this article in a draft status for close to a year now and actually forgot about it. So without further ado, here is my take on how to share variables and live data between runspaces in PowerShell.
I’ve shown in some articles various examples of using runspaces vs. PSJobs in your scripts and functions to achieve better performance and to make your UIs more fluid. What I haven’t really gotten into is another great feature of doing this which is to share your data between those runspaces or inject more data into a currently running runspace.
I would also like to leave a small disclaimer stating that this is an advanced technique and for anyone who is just getting into PowerShell, please do not get concerned if you do not understand what is going on or if it confusing. Just take it one command at a time and you will have it figured out in no time!
Synchronized Collections
The key component to all of this is done via a synchronized (thread safe) collection. There are multiple types of synchronized collections that include a hash table, arraylist, queue, etc… I typically use a synchronized hash table just because it is easier to call the variables or live objects by a name ($hash.name) versus using an arraylist or something similar and filtering for what I need using a Where-Object statement (($array | Where {$_.name –eq ‘Name’}).Value). Of course there may be cases when one might be better than another and in that case, always use the one that will help you out the most.
Creating a synchronized collection is pretty simply. This example will show how to create a synchronized hash table to use.
$hash = [hashtable]::Synchronized(@{})
All I had to do with use the [hashtable] type accelerator with the Synchronized() method and supply an empty hash table. Now I have a hash table that is synchronized and will work across runspaces regardless of the depth of the runspace.
Example of a PSJob with a synchronized hash table
First I want to show an example of using a synchronized hash table with a PSjob and see the outcome of the action.
Write-Host "PSJobs with a synchronized collection" -ForegroundColor Yellow -BackgroundColor Black
$hash = [hashtable]::Synchronized(@{})
$hash.One = 1
Write-host ('Value of $Hash.One before PSjob is {0}' -f $hash.one) -ForegroundColor Green -BackgroundColor Black
Start-Job -Name TestSync -ScriptBlock {
Param ($hash)
$hash.One++
} -ArgumentList $hash | Out-Null
Get-Job -Name TestSync | Wait-Job | Out-Null
Write-host ('Value of $Hash.One after PSjob is {0}' -f $hash.one) -ForegroundColor Green -BackgroundColor Black
Get-Job | Remove-Job
As expected, after supplying the synchronized hash table to the PSJob via the –ArgumentList parameter, instead of seeing the expected value of 2 from the hash table, it is still only a 1. This is because of rolling another runspace in the background of the current session, a new PowerShell process is spawned and performs the work before returning any data back when using Receive-Job.
Now we will take a look at using a synched hash table with a single runspace and with multiple runspaces using a runspace pool because each of these approaches has a slightly different way of injecting the synched collection into a runspace.
Synchronized hash table with a single runspace
The first thing we are going to do is create the synched hash table and add a value of 1 to it.
$hash = [hashtable]::Synchronized(@{})
$hash.One = 1
Next I am going to create the runspace using the [runspacefactory] accelerator and using the CreateRunspace() method. Before I can add anything into the runspace, it must first be opened using the Open() method.
Write-host ('Value of $Hash.One before background runspace is {0}' -f $hash.one) -ForegroundColor Green -BackgroundColor Black
$runspace = [runspacefactory]::CreateRunspace()
$runspace.Open()
Now comes the moment where we will add the synched hash table into the runspace so that I can be accessed and used. Using the SetVariable() method of the $runspace.sessionstateproxy object, we add the synched hash table. We first give the a name of the variable and then add the actual synched collection.
$runspace.SessionStateProxy.SetVariable('Hash',$hash)
With that done, I then create the PowerShell instance and add my runspace object into it.
$powershell = [powershell]::Create() $powershell.Runspace = $runspace
Using the AddScript() method, I add the code that will actually happen within my background runspace when it is invoked later on. As I did with my PSJob example, I am only going to increment the hash table by one.
$powershell.AddScript({
$hash.one++
}) | Out-Null
The Out-Null at the end is used to prevent the output of the object that occurs.
Next, I actually kick off the runspace using BeginInvoke(). It is very important to save the output of this to a variable so you have a way to end the runspace when it has completed, especially when you are expecting to output some sort of object or other types of output. I also use my $handle variable to better track the state of the runspace so I can tell when it has finished.
$handle = $powershell.BeginInvoke()
While (-Not $handle.IsCompleted) {
Start-Sleep -Milliseconds 100
}
Once the runspace has completed, I then perform the necessary cleanup tasks. Note where I use my $handle variable with EndInvoke() to end the PowerShell instance.
$powershell.EndInvoke($handle) $runspace.Close() $powershell.Dispose()
Lastly, we can now see that the value did indeed increment in the runspace.
Write-host ('Value of $Hash.One after background runspace is {0}' -f $hash.one) -ForegroundColor Green -BackgroundColor Black
Pretty cool, right?
How about injecting more data or even viewing the state of the synched hash table while it is running? This can easily be done as well with the following example. I set up a loop within my background runspace that will run until I inject a boolean value to stop the loop. But before I do that, I will play with the data inside by seeing the value and then resetting it back to something else.
$hash = [hashtable]::Synchronized(@{})
$hash.value = 1
$hash.Flag = $True
$hash.Host = $host
Write-host ('Value of $Hash.value before background runspace is {0}' -f $hash.value) -ForegroundColor Green -BackgroundColor Black
$runspace = [runspacefactory]::CreateRunspace()
$runspace.Open()
$runspace.SessionStateProxy.SetVariable('Hash',$hash)
$powershell = [powershell]::Create()
$powershell.Runspace = $runspace
$powershell.AddScript({
While ($hash.Flag) {
$hash.value++
$hash.Services = Get-Service
$hash.host.ui.WriteVerboseLine($hash.value)
Start-Sleep -Seconds 5
}
}) | Out-Null
$handle = $powershell.BeginInvoke()
I also added the parent host runspace into the child runspace so I can use the WriteVerboseLine() method to write the values to the parent session. I’ll sleep for 5 seconds to give me some time to inject values and read the values if needed. Also added is a call to Get-Service so I can pull live objects as well.
As you can see, the values continue to rise and I can even call $hash.value from my parent runspace to see the value or data. This can even include live objects if needed as seen below.
And now I inject some new data into the synched hash table to reset the counter.
Ok, enough fun here, time to close down the loop.
Don’t worry about the weird placement of the characters above. As long as you continue with the typing and hit return, the command will execute like you want it to.
That is really all with working with a single runspace. You can in fact go many many levels deep with background runspaces and use the same synched collection. The important thing to remember is that you must add the synched collection into each new runspace using the SetVariable() method that I talked about earlier.
Working with a runspace pool
I mentioned that working with synched collections is a little different with runspace pools than with a single runspace and now I will show you why. First I am going to set up my runspace pools with a helper function to handle the runspaces and also a scriptblock that will be added into the runspace pool.
Function Get-RunspaceData {
[cmdletbinding()]
param(
[switch]$Wait
)
Do {
$more = $false
Foreach($runspace in $runspaces) {
If ($runspace.Runspace.isCompleted) {
$runspace.powershell.EndInvoke($runspace.Runspace)
$runspace.powershell.dispose()
$runspace.Runspace = $null
$runspace.powershell = $null
} ElseIf ($runspace.Runspace -ne $null) {
$more = $true
}
}
If ($more -AND $PSBoundParameters['Wait']) {
Start-Sleep -Milliseconds 100
}
#Clean out unused runspace jobs
$temphash = $runspaces.clone()
$temphash | Where {
$_.runspace -eq $Null
} | ForEach {
Write-Verbose ("Removing {0}" -f $_.computer)
$Runspaces.remove($_)
}
[console]::Title = ("Remaining Runspace Jobs: {0}" -f ((@($runspaces | Where {$_.Runspace -ne $Null}).Count)))
} while ($more -AND $PSBoundParameters['Wait'])
}
$ScriptBlock = {
Param ($computer,$hash)
$hash[$Computer]=([int]$computer*10)
}
$Script:runspaces = New-Object System.Collections.ArrayList
$Computername = 1,2,3,4,5
$hash = [hashtable]::Synchronized(@{})
$sessionstate = [system.management.automation.runspaces.initialsessionstate]::CreateDefault()
$runspacepool = [runspacefactory]::CreateRunspacePool(1, 10, $sessionstate, $Host)
$runspacepool.Open()
I only created an empty synched hash table because I will be adding data into it for each runspace in the runspace pool. With the runspace pool, I set the max concurrent runspaces running to 10 which will easily handle the 5 items that I want to add for each runspace.
Now I will iterate through each of the items and add them into the runspace pool.
ForEach ($Computer in $Computername) {
#Create the powershell instance and supply the scriptblock with the other parameters
$powershell = [powershell]::Create().AddScript($scriptBlock).AddArgument($computer).AddArgument($hash)
#Add the runspace into the powershell instance
$powershell.RunspacePool = $runspacepool
#Create a temporary collection for each runspace
$temp = "" | Select-Object PowerShell,Runspace,Computer
$Temp.Computer = $Computer
$temp.PowerShell = $powershell
#Save the handle output when calling BeginInvoke() that will be used later to end the runspace
$temp.Runspace = $powershell.BeginInvoke()
Write-Verbose ("Adding {0} collection" -f $temp.Computer)
$runspaces.Add($temp) | Out-Null
}
As you can see, I use the AddArgument() parameter to add the synched hash table into the runspace. This works hand in hand with the Param() statement that I had in the scriptblock that was also added into the PowerShell instance using AddScript().
Calling the $hash.GetEnumerator() to show all of the values, we can see that I can view all of the data of each runspace (if it has written data to it yet).
$hash.GetEnumerator()
As expected again, I see the values from each runspace in the pool. Now I need to clean up after myself using my helper function.
Get-RunspaceData -Wait
Just like my other examples using the single runspace, I could inject values into the synched collection and monitor the synched collection. These are all simple examples that I have shown you but hopefully it is enough to take what I have shown you and expand out with more complex scripts/functions.
As always, I am interested to hear your thoughts and also any examples of what you have done using some of my techniques/examples that have been presented here.
Learn Powershell | Achieve More 
Pingback: Practical PowerShell for IT Security, Part IV: Security Scripting Platform (SSP)
Great Article! Helped me a bunch. I was not aware of the ability to use Synchronized hash tables in runspaces and it has greatly helped me with my project. I do have a question though.
I’m actually working on a program to synch data and relocate file shares from one server to another. Since this is a common occurrence, i’m using a GUI. I’m not writing the GUI myself.. we have a license to use Sapien’s PowerShell Studio 2015.
I have a listview grid showing my list of shares and their current status, limited to 3 robocopy runspaces at a time. I can add any number of shares to the list and one by one the status gets updated as they finish. The problem is… in order to not ‘freeze’ the GUI, I submit all of those tasks to the runspace pool and then have a Timer ‘tick’ that does the reading of the hash table every 4 seconds refreshing the status column. I do not know how to let that timer tick event also do the EndInvoke on the runspaces as the object(s) must be outside the bounds of the tick function. I actually do not NEED to end the runspaces because the script block can (and will) actually add the results all by itself into the hash table once i’m done, but I’d like to know how to track the runspaces in the pool and end them once their status in the hash table is marked ‘finished’.
Any thoughts on that?
GREAT!
It’s awesome.
Thanks 😀
Pingback: Supporting Synchronized Collections in PoshRSJob | Learn Powershell | Achieve More
Hello Boe,
I have made this tool using your article which can fetch a machines basic details, it uses a single runspace for each machine it queries. [Don’t worry about the export-csv button, I am yet to figure that out]
My question is, as each runspace is getting disposed once the machine is queried and output is written in data grid would it be fine to run this script on hundreds of computers at a time ?
——————————————————————————————————–
Function Get-MachineInfo {
$servers = $inputbox.Lines
$hash = [hashtable]::Synchronized(@{})
$hash.Host = $Host
foreach ($server in $servers) {
$hash.Server = $server
if (Test-Connection $hash.server -count 2) {
$runspace = [RunspaceFactory]::CreateRunspace()
$runspace.Open()
$runspace.SessionStateProxy.SetVariable(‘Hash’,$hash)
$powershell = [powershell]::Create()
$powershell.Runspace = $runspace
$powershell.AddScript({
$OS = Get-WmiObject win32_operatingsystem -computername $hash.Server
$Machine = Get-WmiObject win32_computersystem -ComputerName $hash.Server
$Bios = Get-WmiObject win32_bios -ComputerName $hash.Server
$LBtime=$os.ConvertToDateTime($os.lastbootuptime)
[TimeSpan]$uptimes=New-Timespan $LBtime $(get-date)
$uptime=”$($uptimes.days) Days, $($uptimes.hours) Hours, $($uptimes.minutes) Minutes, $($uptimes.seconds) Seconds”
$hash.Machine = $OS.Csname
$hash.OperatingSystem = $OS.Caption
$hash.ServicePack = $OS.csdversion
$hash.Architecture = $OS.OSArchitecture
$hash.Domain = $Machine.domain
$hash.PhysicalMemory = ($machine.totalphysicalmemory/1GB)
$hash.Manufacturer = $bios.manufacturer
$hash.Model = $machine.model
$hash.Version = $bios.version
$hash.Uptime = $uptime
#$hash.host.ui.writeverboseline(“$($hash.Machine), $($hash.OperatingSystem), $($hash.ServicePack), $($hash.FQDN), $($hash.Domain), $($hash.PhysicalMemory), $($hash.Manufacturer), $($hash.Model), $($hash.Version), $($hash.Uptime)”)
}) | Out-Null
$handle = $powershell.BeginInvoke()
while (-not $handle.IsCompleted) {Start-Sleep -Milliseconds 100}
$DataGrid.Rows.Add(“$($hash.Machine)”, “$($hash.OperatingSystem)”, “$($hash.ServicePack)”, “$($hash.Architecture)”, “$($hash.Domain)”, “$($hash.PhysicalMemory) GB”, “$($hash.Manufacturer)”, “$($hash.Model)”, “$($hash.Version)”, “$($hash.Uptime)”)
$powershell.EndInvoke($handle)
$runspace.Close()
$powershell.Dispose()
} else {$DataGrid.Rows.Add(“$($hash.Server): Unreachable” ); clear-host}
}
}
#………………..This is the GUI Section……………………………………………………….#
[void] [System.Reflection.Assembly]::LoadWithPartialName(“System.Windows.Forms”)
[void] [System.Reflection.Assembly]::LoadWithPartialName(“System.Drawing”)
$form = New-Object System.Windows.Forms.Form
$form.Size = New-Object System.Drawing.Size(1035,459)
$form.FormBorderStyle = “FixedDialog”
$form.MaximizeBox = $false
$form.StartPosition = ‘centerscreen’
$form.BackColor = ‘Chocolate’
$inputbox = New-Object System.Windows.Forms.RichTextBox
$inputbox.Location = New-Object System.Drawing.Size(12,29)
$inputbox.Size = New-Object System.Drawing.Size(269,339)
$inputbox.BorderStyle = ‘NONE’
$inputbox.MultiLine = $true
#$inputbox.WordWrap = $true
$inputbox.Font = New-Object System.Drawing.Font(“segoe UI”,9)
$Inlabel = New-Object System.Windows.Forms.Label
$Inlabel.Location = New-Object System.Drawing.Size(9,9)
$Inlabel.Size = New-Object System.Drawing.Size(193,17)
$Inlabel.Font = New-Object System.Drawing.Font(“segoe UI”,9.75,[System.Drawing.FontStyle]::Underline)
$Inlabel.Text = ‘Please input server names.’
$Inlabel.ForeColor = ‘Control’
$Outlabel = New-Object System.Windows.Forms.Label
$Outlabel.Location = New-Object System.Drawing.Size(294,9)
$Outlabel.Size = New-Object System.Drawing.Size(141,17)
$Outlabel.Font = New-Object System.Drawing.Font(“segoe UI”,9.75,[System.Drawing.FontStyle]::Underline)
$Outlabel.Text = ‘Please see output here.’
$Outlabel.ForeColor = ‘Control’
$Script:DataGrid = New-Object System.Windows.Forms.DataGridView
$DataGrid.Location = New-Object System.Drawing.Size(297,29)
$DataGrid.Size = New-Object System.Drawing.Size(719,339)
$DataGrid.BorderStyle = ‘NONE’
$DataGrid.ColumnHeadersDefaultCellStyle.Font = New-Object System.Drawing.Font(“segoe UI”,9.25)
$DataGrid.DefaultCellStyle.Font = New-Object System.Drawing.Font(“segoe UI”,9.25)
$DataGrid.AllowUserToAddRows = $false
$DataGrid.RowHeadersVisible = $false
$DataGrid.ColumnCount = 10
$DataGrid.Columns[0].Name = ‘Machine’
$DataGrid.Columns[1].Name = ‘OperatingSystem’
$DataGrid.Columns[2].Name = ‘ServicePack’
$DataGrid.Columns[3].Name = ‘Architecture’
$DataGrid.Columns[4].Name = ‘Domain’
$DataGrid.Columns[5].Name = ‘PhysicalMemory’
$DataGrid.Columns[6].Name = ‘Manufacturer’
$DataGrid.Columns[7].Name = ‘Model’
$DataGrid.Columns[8].Name = ‘Version’
$DataGrid.Columns[9].Name = ‘Uptime’
$DataGrid.AutoResizeColumns()
#$DataGrid.BackgroundColor = ‘Window’
$Okbutton = New-Object System.Windows.Forms.Button
$Okbutton.Location = New-Object System.Drawing.Size(12,386)
$Okbutton.Size = New-Object System.Drawing.Size(75,23)
$Okbutton.Text = “OK”
$Okbutton.BackColor = ‘LightGray’
$Okbutton.UseVisualStyleBackColor = $true
$Okbutton.Font = New-Object System.Drawing.Font(“segoe UI”,9)
$Okbutton.Add_Click({Get-MachineInfo})
$Clearbutton = New-Object System.Windows.Forms.Button
$Clearbutton.Location = New-Object System.Drawing.Size(159,386)
$Clearbutton.Size = New-Object System.Drawing.Size(75,23)
$Clearbutton.Text = “Clear”
$Clearbutton.BackColor = ‘LightGray’
$Clearbutton.UseVisualStyleBackColor = $true
$Clearbutton.Font = New-Object System.Drawing.Font(“segoe UI”,9)
$Clearbutton.Add_Click({$inputbox.Clear()})
$Savebutton = New-Object System.Windows.Forms.Button
$Savebutton.Location = New-Object System.Drawing.Size(467,386)
$Savebutton.Size = New-Object System.Drawing.Size(75,23)
$Savebutton.Text = ‘Export-CSV’
$Savebutton.BackColor = ‘LightGray’
$Savebutton.UseVisualStyleBackColor = $true
$Savebutton.Font = New-Object System.Drawing.Font(“segoe UI”,9)
$Exitbutton = New-Object System.Windows.Forms.Button
$Exitbutton.Location = New-Object System.Drawing.Size(716,386)
$Exitbutton.Size = New-Object System.Drawing.Size(75,23)
$Exitbutton.Text = ‘Exit’
$Exitbutton.BackColor = ‘LightGray’
$Exitbutton.UseVisualStyleBackColor = $true
$Exitbutton.Font = New-Object System.Drawing.Font(“segoe UI”,9)
$Exitbutton.add_click({$form.Close()})
#…………………………………………………………………………..#
$form.Controls.Add($inputbox)
$form.Controls.Add($Inlabel)
$form.Controls.Add($Outlabel)
$form.Controls.Add($DataGrid)
$form.Controls.Add($Okbutton)
$form.Controls.Add($Clearbutton)
$form.Controls.Add($Savebutton)
$form.Controls.Add($Exitbutton)
$form.ShowDialog()
#…………………………….End of Script………………………………#
———————————————————————————————————————
Hello ,i really need to use this but i fail to make it work.
When i copy/paste exemple with verboseline writing, runspace is launched but the $hash is $null when i type it in command line….
What did i miss ?
Ok i just find the problem this exemple as it is can’t work in a script file…my bad
Hi Boe!
like to invite you to my discussion: Invoke-Parallel need help to clone the current Runspace
http://powershell.org/wp/forums/topic/invpke-parallel-need-help-to-clone-the-current-runspace/
Greets Peter Kriegel
Founder member of the European, German speaking, Windows PowerShell Community
http://www.PowerShell-Group.eu
Pingback: More on PowerShell multithreading via runspace pools | Dave Wyatt's Blog
Hi Boe!
Again a very good Article! The knowledge to use the synchronized Types with runspaces is not wide spread.
Here is a tip of mine for your Articles:
Use the assignment to $Null instead the pipe to Out-Null
I am doing User support in the German TechNet Forum.
There I see different Arts of coding. So I know 4 ways to suppress output with PowerShell.
The first possibility to suppress an output (echo) is to pipe the output to the Cmdlet Out-Null:
“Not hello to World!” | Out-Null
The second one is similar to the good old CMD behavior to redirect the output simply to $Null:
“Not hello to World!” > $Null
The third one is to cast (convert) the output into a Type of Void, by use of the [Void] Type accelerator:
[Void]”Not hello to World!”
And the fourth one is to assign then output simply to $Null
$Null = “Not hello to World!”
But which one is the fastest Method? Which one to use?
Let’s measure it!
PowerShell has the very needful Measure-Command Cmdlet. This is to measure the time consumption a command needs to process.
These are the results on my system (yours may differ):
Measure-Command -Expression {
for($I = 0 ;$I -lt 10000;$I++) {
“Not hello to World!” | Out-Null
}
}
Result for Pipe to Out-Null cmdlet:
Seconds : 1
Milliseconds : 506
Measure-Command -Expression {
for($I = 0 ;$I -lt 10000;$I++) {
“Not hello to World!” > $Null
}
}
Result for redirect to $Null:
Seconds : 0
Milliseconds : 552
Measure-Command -Expression {
for($I = 0 ;$I -lt 10000;$I++) {
[Void]”Not hello to World!”
}
}
Result for [Void]:
Seconds : 0
Milliseconds : 19
Measure-Command -Expression {
for($I = 0 ;$I -lt 10000;$I++) {
$Null = “Not hello to World!”
}
}
Result for assignment to $Null:
Seconds : 0
Milliseconds : 14
You see the simple assignment to $Null is the fastest. Up to 107 times faster than to pipe the output to the Out-Null cmdlet.
This is because the overhead to create the Pipeline and to call the Out-Host Cmdlet.
With a simple assignment to $Null there is no such!
In the future I first try to use this one! 😉
Cheers Peter Kriegel
http://www.admin-source.de
You can’t rely on static data when making these measurements 🙂
Measure-Command -Expression {
for($I = 0 ;$I -lt 100000;$I++) {
Get-Random | Out-Null
}
}
TotalMilliseconds : 6191,4108
Measure-Command -Expression {
for($I = 0 ;$I -lt 100000;$I++) {
Get-Random > $Null
}
}
TotalMilliseconds : 2859,912
Measure-Command -Expression {
for($I = 0 ;$I -lt 100000;$I++) {
[Void](Get-Random)
}
}
TotalMilliseconds : 2864,1401
Measure-Command -Expression {
for($I = 0 ;$I -lt 100000;$I++) {
$Null = Get-Random
}
}
TotalMilliseconds : 2863,1909
vs
Measure-Command -Expression {
for($I = 0 ;$I -lt 100000;$I++) {
“Get-Random” | Out-Null
}
}
TotalMilliseconds : 4131,9042
Measure-Command -Expression {
for($I = 0 ;$I -lt 100000;$I++) {
“Get-Random” > $Null
}
}
TotalMilliseconds : 201,7418
Measure-Command -Expression {
for($I = 0 ;$I -lt 100000;$I++) {
[Void](“Get-Random”)
}
}
TotalMilliseconds : 155,3484
Measure-Command -Expression {
for($I = 0 ;$I -lt 100000;$I++) {
$Null = “Get-Random”
}
}
TotalMilliseconds : 156,821
Great article Boe.
There also another solution when using runspace pools – adding variable to initial session state before calling CreateRunspacePool:
$sessionstate.Variables.Add((New-Object -TypeName System.Management.Automation.Runspaces.SessionStateVariableEntry -ArgumentList ‘hash’, $hash, ”))
Nice! I didn’t know of that approach. Thanks for sharing it!