Some Updates to Out-SquarifiedTreemap

It’s been close to a year since I published my Out-SquarifiedTreemap function (and blogged about it) after spending a number of weeks prior working on it to ensure that everything was accurate when used and also trying to figure out a good algorithm to handling the sizes of each square as well as the heatmap on it.

Fast forward those 11 months and I haven’t really done anything with it other than adding some code examples for those who might have been interested on the GitHub page. I use the code in the real world but I admit that the output is rather hard to read unless you use the tooltip when you hover over each square to see what the data actually is. Fortunately another individual by the name of Aaron Nelson (Blog | Twitter) who is a Data Platform MVP happened to see this and hit me up to ask about adding some features to it. Naturally I couldn’t resist the ideas for improvement and proceeded to implement a couple of I feel were much needed updates to the function.

Label for Each Square

Honestly, this one was way overdue to implement. Having the squares with their various sizes and colors for the heatmap gave a great view into how things shaped up, not having an ‘at a view’ glance due to no labels made the UI a little painful to read unless you hover over each square for its respective tooltip.

It took a little bit of time to figure out how I wanted to do this. I wanted something that would scale appropriately depending on the size of the square. That meant that I didn’t want to worry about the font size at all and didn’t want it to look like it was constrained to its own text box. The Label control jumped out at me because doesn’t show up as though it is in a box and looking like it was just typed in.  This works nicely until you try to use this in a Canvas layout. My testing was done using a Grid layout control which makes scaling of the text simple whereas a Canvas is the lowest form of a layout as the user has complete control of where the child controls can be placed and sized with pretty much no constraints at all.

This was a problem. I didn’t want a label that didn’t scale properly…or at all for that matter. Enter the ViewBox control which allows me to place the label inside of it and lets me scale treat the Label as though it was in the Grid by scaling the text based on the size of each square it is in. Perfect! Using these controls,  I can then create the control and then just size it as the same size as my rectangle that shows off each item and place it within the same coordinates as the rectangle so it appears like it is the same thing. The last thing that I needed to do was to turn off the hit detection on the Viewbox and Label so only the Rectangle would be detected when clicked on (which plays a major role in the –PassThru support). The code that handles the creation of the Label and Viewbox is here:

Function New-ViewBox {
    Param(
        $Width,
        $Height,
        $Text
    )
    $Label = new-object System.Windows.Controls.Label
    $Viewbox = new-object System.Windows.Controls.Viewbox
    $DropShadow = New-Object System.Windows.Media.Effects.DropShadowEffect
    $DropShadow.Opacity = 5
    $DropShadow.BlurRadius = 5
    $DropShadow.Color = 'Black'
    $DropShadow.ShadowDepth = 0
    $Viewbox.Stretch = 'Uniform'
    $Viewbox.Width = $Width
    $Viewbox.Height = $Height
    $Label.IsHitTestVisible = $False
    $Label.FontFamily = 'Calibri'
    $Label.FontWeight = 'Bold'
    $Label.Foreground = 'White'
    $Label.Content = $Text
    $Label.Effect = $DropShadow
    $Viewbox.AddChild($Label)
    Return $Viewbox
}  

I can call it in the same way that I called my New-Rectangle function and place it in the same location on the canvas.

If ($ShowLabel) {                
    $Viewbox = New-ViewBox -Width $_.Width -Height $_.Height -Text $_.$ShowLabel
    [void]$Canvas.Children.Add($Viewbox) 
    [System.Windows.Controls.Canvas]::SetLeft($Viewbox,$_.Coordinate.X)
    [System.Windows.Controls.Canvas]::SetTop($Viewbox,$_.Coordinate.Y)
}

Ah yes, the –ShowLabel parameter is what will make up whether the label is actually shown as well as what type of label will be displayed on the UI. Currently, there are 3 possible values that you can use: LabelProperty, DataProperty and HeatmapProperty. You might notice that these are named after the same parameters that already exist in the function. They are directly related and will display the respective label on the UI when used.

$Tooltip = {
@"
Process Name <PID>:   $($This.LabelProperty) <$($This.ObjectData.Id)>     
WorkingSet Memory(MB): $([math]::Round(($This.DataProperty/1MB),2))
"@
}
Get-Process | Sort-Object -prop WS -Descending | Select -First 8 | 
Out-SquarifiedTreeMap -Tooltip $Tooltip -LabelProperty ProcessName -DataProperty WS -HeatmapProperty WS -Width 600 -Height 400 -ShowLabel LabelProperty

image

 

Looks like Chrome is definitely a memory hog based on the UI being displayed. As you can see, the text does some nice scaling based on the size of each rectangle. Naturally,the smaller the box the smaller the text, so at that point you will want to rely on the tooltip for those cases.

Providing PassThru Support

Next up is providing PassThru support so that if you wanted to do more than just view the pretty UI, you can specify the –PassThru parameter and then pick an item that you want to do more with so that it will pass through the original object that was sent to the UI. Pretty handy if you wanted to stop the process with the most memory utilization or needed to do more with some files or folders or anything else that you can think of! Setting this up meant that I had to specify a command to run when one of the rectangles has been clicked on. One problem that I ran into was that I am already tracking when I click on the mouse button to drag the UI around with the mouse. This gets thrown out of the window when using –PassThru and if you did want to move the window around, you need to be holding down on one of the CTRL buttons before attempting to click and move it.

Some of the code that sets all of this up is below:

#region TabControl event handler
[System.Windows.RoutedEventHandler]$Global:RectangleKeyDownChangeHandler = { 
    Write-Verbose "[KeyDwnhandler-CTRLKeyDown] $($Script:KeyDown)"
    If ($Script:KeyDown) {
        Try {
            Write-Verbose "[KEYUP] DragMove"
            $Window.DragMove()
        }
        Catch {Write-Warning $_}
    }
}
$Window.AddHandler([System.Windows.Shapes.Rectangle]::MouseLeftButtonUpEvent, $RectangleKeyDownChangeHandler)

[System.Windows.RoutedEventHandler]$Global:RectangleKeyUpChangeHandler = { 
    If ($Script:IsPassThru -AND -NOT $Script:KeyDown) {               
        If ($_.OriginalSource -is [System.Windows.Shapes.Rectangle]) { 
            $Source = $_.OriginalSource           
            $Script:Result = $DataHash.TreeMapData | Where {
                $_.Tag -eq $Source.Tag
            } | Select-Object -ExpandProperty ObjectData
            $Window.Close()
        }
    }
}
$Window.AddHandler([System.Windows.Shapes.Rectangle]::MouseLeftButtonUpEvent, $RectangleKeyUpChangeHandler)
#endregion TabControl event handler

This code sets up the event handlers for each rectangle by making use of the bubble up eventing so that regardless of which rectangle is clicked, the handler will fire and proceed with the appropriate actions.

To handle the output when you click the UI, I have a piece of code at the end that runs after the UI itself is closed. This ensures that if there was a selection picked, that it would actually send the object onto the next command.

#Show UI
Write-Verbose "[END] Show UI"
[void]$Window.ShowDialog()
Write-Verbose "[END] UI Close"
If ($IsPassThru) {
    Write-Verbose "Output Object"
    $Result
}

I’ll make use of the same code earlier but now I will add the –PassThru piece so I could attempt to stop a process that was appearing to be hogging up my much needed memory.

$Tooltip = {
@"
Process Name <PID>:   $($This.LabelProperty) <$($This.ObjectData.Id)>     
WorkingSet Memory(MB): $([math]::Round(($This.DataProperty/1MB),2))
"@
}
Get-Process | Sort-Object -prop WS -Descending | Select -First 8 | 
Out-SquarifiedTreeMap -Tooltip $Tooltip -LabelProperty ProcessName -DataProperty WS -HeatmapProperty WS -Width 600 -Height 400 `
-PassThru -ShowLabel LabelProperty | 
Stop-Process –WhatIf

SqTreemapPassThruDemo

So with that, there are a few new things that have been added to my function that will hopefully make using this UI a little more useful than previously.

As stated earlier, you can find this function over at GitHub: https://github.com/proxb/SquarifiedTreemap

And as always, feel free to let me know of new features that you would like to see or go ahead and submit a Pull Request and let me see what you have in mind!

Posted in powershell | Tagged , , , | Leave a comment

Setting up Local Administrator Password Solution (LAPS)

I decided to spend some time implementing LAPS in my lab as it is Microsoft’s solution to local administrator account password management. Why would I want something like this in my environment? Great question! Most organizations probably use the same password (maybe a slightly modified password based on each client…maybe) that ensures that the people who help manage the workstations have a way to log into the system should the computer lose its network configuration or some other issue where the only way to troubleshoot might he to log into the workstation using the local administrator account.

This is great until you someone such as an insider threat manages to get control of the password to the administrator account and now has a way to laterally move around the network from host to host until they can elevate their account to a domain admin. And at that point we know it is game over.

You might be saying “Boe,WTF?!? You are a PowerShell guy so why are you talking about something like this?”. Well, we are all responsible for securing our environment and this provides a great way to do so. Plus, there are PowerShell commands available to manage this service which makes it doubly awesome! Smile

Download the LAPS Install File and Review

You can find the files available to download from the following site: https://www.microsoft.com/en-us/download/details.aspx?id=46899

Included are some great documents that show how to setup and configure LAPS such as using Group Policy as well as extending the schema for the two (2) new attributes that are required for LAPS. In one of these attributes (ms-Mcs-AdmPwd) on each computer object you will find the password (!) for the local administrator account. Before you become too alarmed, these are called “Confidential Attributes” meaning that the attributes are protected by ACLs which are only accessible by the Domain Admins group and any other group that you specify. Standard users (who have Read access to AD) will not be able to view the attributes data and will only see blank information. Only the computer account can write to this location as well as to the other attribute (ms-Mcs-AdmPwdExpirationTime) which is used to determine if the password has expired and needs to be changed. With this in mind, any group that you provide access to the attributes should be monitored to ensure only those that need access, have access.

Installing LAPS Management

I am going to get things started by installing the management tools on one of my servers which includes the following items:

  • Client UI
  • PowerShell Module
  • GPO Templates

The process for the installation is pretty simple; all I do is double click on the msi file (LAPSx64.msi in this case) and run through the wizard, selecting all of the components for installation.

image

Once that is done, we can then move on to updating the schema by adding the two attributes that we talked about earlier using the AdmPwd.PS module that we just installed. But before we update the schema, now is a great time to review the module!

Reviewing the AdmPwd.PS Module

As mentioned, we can do a lot of our management via the AdmPwd.PS module for PowerShell. Yea, the name of the module probably could have been called LAPS in my own opinion, but that is all right Winking smile

Let’s take a quick look at the available cmdlets on this module.

Get-Command –Module AdmPwd.PS

image

Here we see that there our cmdlet to update the schema is called Update-AdmPwdSchema as well as other cmdlets to locate the password for a particular system, resetting a password and setting some permissions to for a variety of things.

Updating the Schema using PowerShell

Now that we know what commands are available to use, we should move forward in our configuration and get the schema updated so our computer account objects have the required attributes. How do we use this cmdlet? Time to check out the help file to learn what needs to be done.

Get-Help Update-AdmPwdADSchema

image

Well…there really isn’t much here to do at all. We simply just run the command and it should “just work”. Nothing wrong with that at all. The one thing that is a little odd is the lack of a –WhatIf parameter to simulate what would happen if we ran this. How do I know this?

image

Not really a major issue by any means, but typically if a command performs any change then it should use –WhatIf as part of its parameters.

I’ll go ahead and run this while using –Verbose just to see if we happen to have any sort of output.

Update-AdmPwdADSchema –Verbose

image

Whoops! Remember folks, if you are updating the Schema, you must be a member of Schema Admins first! Simply add your account to the group, log out and log back in and make the update.

image

image

No verbose output here, but we do get object output showing that the update was successful! Now we remove our account from Schema Admins and log out and back in and then move on to our next part.

image

Watching out for Extended Rights

The documentation mentions that you should be aware of any group or user that has the All Extended Rights access under an OU which has a computer account as they will be able to read the confidential attribute value meaning that they have access to the password. The accounts/groups that have this permission and do not need them should have it removed to prevent unauthorized access.

We can quickly create a report on what accounts and groups have this by using the Find-AdmPwdExtendedRights cmdlet. To make this even more useful, we can combine the output of Get-ADOrganizationUnit from the ActiveDirectory module and then massage the object that is outputted to see all of the accounts and groups with this permission.

Get-ADOrganizationalUnit -Filter *|Find-AdmPwdExtendedRights -PipelineVariable OU |ForEach{
$_.ExtendedRightHolders|ForEach{
[pscustomobject]@{
OU=$Ou.ObjectDN
Object = $_
}
}
}

image

Optionally, you could filter out System and Domain Admins if you wanted to make locating potential unauthorized accounts and groups easier to locate. In this case, I have nothing that concerns me at the moment but it is something that would be a good scheduled task to run daily to ensure nothing suspicious might be happening.

Installing the Client Side Extensions

Before we go too much deeper into setting up the permissions, we should look at installing the client side extensions on each client. In a larger environment you would probably use something like SCCM to install this on all of the systems. In my case, I am going the group policy route and will configure a GPO that will install the LAPSx64.msi file onto each system.

image

I really didn’t do anything special here other than just point this deployment package to a UNC path that was reachable by everyone on the network.

Once that has been completed and I assigned the GPO to my workstations OU. I proceeded to reboot the client and see what happens in hope that it installs correctly. Of course, that didn’t happen as I had package errors with error codes of %%2 and %%1274

image

Turns out the fix for this was to increase the GPO startup processing time to 30 seconds by going to the Computer Configuration>Administrative Templates>System>Group Policy and locating the Specify startup policy processing wait time and setting the value for this to 30 (seconds). Reboot the client and we now have a successful installation!

image

image

All ready to go now! As of right now, we haven’t actually enabled LAPS to begin setting the local administrator passwords just yet. We just need to work on some extra permissions and configure the group policy to enable and configure how LAPS will set the passwords.

Optional Installation

If you wanted a command line approach to installing the client side extension, you can use one of the following approaches to install it.

x64 machines

msiexec /q /i \\server\share\LAPS.x64.msi

x86 machines

msiexec /q /i \\server\share\LAPS.x86.msi

Deploy and Create an optional local administrator account

msiexec /q /i \\server\share\LAPS.x86.msi CUSTOMADMINNAME=NewLocalAdmin

Configure Permissions for the Computer to Update its Attributes

Next up is to ensure that the systems which will be managed by LAPS will be able to update the new attributes on their active directory computer account object. To make this process as easy as we can, we will make use of the Set-AdmPwdComputerSelfPermission cmdlet which we can assign to the OU where the computer accounts reside at. Using this cmdlet is simple: simply point it towards an OU and it will assign the necessary permissions.

Set-AdmPwdComputerSelfPermission –Identity ManagedWorkstations –Verbose

image

This will ensure that this OU and any sub-OU will assign SELF the ability to update the new attributes that we have added to the computer object.

Unfortunately, there isn’t a Remove-AdmPwdComputerSelfPermission cmdlet that can remove this permission from an OU. So if you need to perform this action, you will have to do it another way.

Granting Rights to User or Groups to Read and Reset the Password

With this completed, out next step is to configure the rights for a user (or members of a group) to have the rights to read the password from the confidential attributes as well as writing to the password expiration date attribute. Note that there are two (2) cmdlets that will set this up:

  • Set-AdmPwdReadPasswordPermission
  • Set-AdmPwdResetPasswordPermission

These cmdlets will allow you to set the permissions on a given OU (or OUs) to delegate the necessary permissions to either view the password or write to the ms-Mcs-AdmPwdExpirationTime attribute which will allow for the resetting of the system’s password.

I am going to grant my WorkstationPwdAdmins the ability to read and reset the password on the computers under the ManagedWorkstation OU.

Set-AdmPwdResetPasswordPermission –Identity ManagedWorkstations –AllowedPrincipals rivendell.local\WorkstationPwdAdmins –Verbose

Set-AdmPwdReadPasswordPermission –Identity ManagedWorkstations –AllowedPrincipals rivendell.local\WorkstationPwdAdmins –Verbose

image

At this point we are pretty much ready to go! All that is left to do is enable LAPS via our GPO and make a couple more configurations to set the password policy and we will be able to give this a spin and see if it works!

Configuring the GPO to Enable and Set Password Policy

First off, we need to ensure that whatever system we are going to be updating the policy on has the AdmPwd.admx file located under C:\Windows\PolicyDefinitions and the AdmPwd.adml under C:\Windows\PolicyDefinitions\en-us folder. That way when we open up the GPO, we can find LAPS under Computer configuration>Administrative Templates.

image

From here I will set the Password Settings first.

image

I’ve opted to keep password complexity at the default of using Upper and Lower case lettings, numbers and special characters. For the Password length and Password age, you should check your domain password policy to ensure that you are not stepping out of bounds. Setting a password length that is shorter than what is allowed will cause the process to fail.

image

Since I do not have a second administrator account that is locally on my systems, I am going to leave this in an non-configured state. By default, LAPS will reset the password on the Administrators account by looking at the well known SID but we do have the option of being able to reset a second administrator account if we choose to do so.

image

Here I want to ensure that if a system has gone longer than the password expiration setting then it will be changed immediately.

image

At this point, I am telling LAPS to begin management of the local administrator account passwords. Once this is set, the next time that group policy refreshes on the local systems, their password will be reset.

Validating that the Password is being Managed

So far, we can see that the password is not being managed by looking for the two attributes from my account which has the necessary rights.

image

I’ll initiate a group policy refresh on the client and then we should see the attributes show up with values.

Invoke-GPUpdate –Computer Client1A –Verbose

After a short wait, we will re-run our ADComputer cmdlet again and see what shows up now.

SNAGHTML1f165a51

My clients are now being managed by LAPS and I can view the password as well with my admin account. So what happens if an account that doesn’t have the rights to view the password tries the same command? Let’s find out!

SNAGHTML1f18e736

While the account can tell when the password expires next, they cannot read the password as this particular account doesn’t have access to the confidential attribute.

So what about that datestamp for the expiration time of the password? How in the world can we read this thing? Simple! We make use of the FromFileTime() method of [DateTime] and supply that string as the parameter and we will  instantly know when the password will expired and be reset.

$Computer = (Get-ADComputer –Identity client1a -prop ms-Mcs-AdmPwd,ms-Mcs-AdmPwdExpirationTime)
[datetime]::FromFileTime($Computer."ms-Mcs-AdmPwdExpirationTime")

image

Of course, there is always an easier way to accomplish things like this in PowerShell and the way here comes courtesy of the Get-AdmPwdPassword cmdlet. The nice thing here is that the password is also converted for us to a more human readable time.

Get-AdmPwdPassword –Computername Client1A

image

 

Forcing a Password Reset

What if something happens and we need to force a password reset on a local system? Yes, there is the fat client that you can use, but we know that every is better in the PowerShell console, right? With the ActiveDirectory module and Set-ADComputer cmdlet,  we can quickly knock out a password reset on 1 or more systems.

The key to this is being able to change the data in the ms-Mcs-AdmPwdExpirationTime attribute. Remember that the value translates to a date down the road that the machine will read upon its group policy refresh and determine if it is time to reset the admin account password. With that in mind,we can just set that value to 0 which will then initiate the process at the next policy refresh.

Set-ADComputer –Identity Client1A –Replace @{"ms-Mcs-AdmPwdExpirationTime"=0}

A quick check of the object shows that it has been set to 0.

image

We aren’t done yet. As I have mentioned a couple of times, this only sets the attribute but doesn’t actually initiate anything until the group policy is refreshed. So with that I will initiate the refresh using Invoke-GPUpdate against the client and we should then see the attribute update with a new expiration time.

image

And now we can see that the password was reset. We can further verify this by comparing it against the previous password that I had shown earlier.

image

Now this is a great approach when you are on a different system where you might not have the AdmPwd.PS module available, but if you do happen to have it, well then you get the benefit of using Reset-AdmPwdPassword cmdlet at your disposal! You can supply a computername and the date when the password should expire (called WhenEffective) and you will have the same results that we had with the previous example with the expectation that you will force a policy refresh after.

Reset-AdmPwdPassword –Computer Client1A –WhenEffective (Get-Date).AddDays(-1)

By specifying yesterdays date, the computer will see that the time has expired and will then begin the process of resetting the password.

You can definitely perform the same actions by using the client tool as we have done in PowerShell but we will still have the same results of needing the policy to be refreshed manually.

image

Personally, I thing that both the UI and the cmdlet in the AdmPwd.PS module should have a way to force policy to refresh (maybe as an additional parameter) and the UI could use a checkbox or something similar. An issue with the UI is that it only works with a single machine at a time, so scaling that out to your enterprise may not work too well.

So that wraps up my post on installing and configuring LAPS in your environment and using PowerShell as a means to accomplish most of that configuration. If haven’t considered a means to keep your local administrator passwords different from one another and need an automated approach, then I would recommend that you give LAPS a try.

References

https://adsecurity.org/?p=1790

https://www.microsoft.com/en-us/download/details.aspx?id=46899

https://technet.microsoft.com/en-us/library/security/3062591.aspx?f=255&MSPPError=-2147217396

Posted in powershell | Tagged , , | 12 Comments

Renewed as a Cloud and Datacenter Management MVP!

image

I have been grateful once again to be renewed as a Microsoft MVP in Cloud and Datacenter Management for 2016. This is an honor that is never guaranteed and I am always humbled to receive it.

This is something that doesn’t come easily as we are evaluated each year with the same folks who are trying to get into the program. The work is always fun and I know that this award would not have been possible if not for the community itself, both in viewing and using the things that I have posted as well as providing great feedback to me so I can improve on not only things that I write about, but also the tools that I present for others to use.

It has definitely been an amazing year for the PowerShell community with PowerShell going open source and being made available on other operating systems such as Linux and MacOS. Seeing such an increase on the use of PowerShell in the InfoSec community has also been amazing as well as the many cool and unique things that others have been working on and showing off just shows that PowerShell is not going away at all. It is only becoming bigger and more important to learn for those in our field.

So with that I say Thank You to the community and I cannot wait to see what the next year brings!

Posted in powershell | Tagged , , , | 2 Comments

Building a Chart Using PowerShell and Chart Controls

Reports are one of those things that everyone has done at some point in time using a variety of methods to include, Excel, XML, CSV, HTML, etc.… A lot of times, while the data is one thing to present, there are times when we need to just present a snapshot of information, usually in the form of a chart of some kind. This is where we usually will dig into Excel and its COM object to take the data and create a quick chart to show the data. But doing this can be slow as working with COM objects are notoriously slow (they are much faster with PowerShell V5), so another approach might be better instead. Enter the .Net Chart Controls which gives us a way to take the data that we have and quickly create a chart such as a pie chart or a line chart and present the data with little to no effort on the system.

What Kind of Charts are Available?

There are a little over 30 possible charts at your disposal ranging from a pie chart to a bar chart or even a bubble chart if you wanted one.

image

Some of these charts like the Pie and Doughnut have a lot in common and you can use the same type of approach with the data to quickly present information while working with a Line chart might require a little more work to ensure the data is presented accurately and also provides support for multiple “Series” of data which allows for you to provide a comparison between different points of time for your data (useful in column or bar charts). For a better look at each chart type and the expectations associate with each chart (such as number of series allowed), the following link has the information to look at: https://msdn.microsoft.com/en-us/library/dd489233.aspx

What is a Series of Data?

A series of data can be looked at as different captures of data that will be applied to a chart. One example is that you can track the current capacity (Series1) of a hard drive as well as its current drive usage (Series2) over the course of several months and see how the current drive usage changes during the course of the time. Given, the capacity may not change at all if it is a physical drive, but may change if the drive is a virtual drive or SAN attached. Something like this would make for a good line chart.

Another example would be to track the memory (or CPU) utilization of several processes. Here you would take a reading at the beginning (Series1) and then wait maybe a minute or so and take another reading (Series2). From these two samples, you can then display the results as a Bar chart or a Column chart to get an idea of the differences in values, if there happen to be differences.

Where do I begin?

Glad you asked. If you are running PowerShell V3+ then you are good to go and have everything already installed, but if you happen to be running PowerShell 2.0, then odds are you might need to download and install the required bits for the Microsoft Chart Controls for Microsoft .NET Framework 3.5 here.

Let’s Build a Pie Chart!

Building a Pie chart is pretty simply as we only require a single series of data which will consist of a label for the data and its value. In this case we are going to chart out our processes by their WorkingSet (WS) property to see what our top 10 memory hogs are.

$Processes = Get-Process |
Sort-Object WS -Descending |
Select-Object -First 10

Now we need to do a few other things before we start diving into the world of chart controls. First off I am going to define a couple of helper functions that will assist in some areas.

Edit (10/02/2016) Using a hashtable originally was probably a bad idea when you consider multiple same named processes (or anything that is used more than once with the same name) as it one will just overwrite the next meaning the last one will win. I have since taken out the ability to create the hashtable and used a different approach.

#region Helper Functions

Function Invoke-SaveDialog {
    $FileTypes = [enum]::GetNames('System.Windows.Forms.DataVisualization.Charting.ChartImageFormat')| ForEach {
        $_.Insert(0,'*.')
    }
    $SaveFileDlg = New-Object System.Windows.Forms.SaveFileDialog
    $SaveFileDlg.DefaultExt='PNG'
    $SaveFileDlg.Filter="Image Files ($($FileTypes))|$($FileTypes)|All Files (*.*)|*.*"
    $return = $SaveFileDlg.ShowDialog()
    If ($Return -eq 'OK') {
        [pscustomobject]@{
            FileName = $SaveFileDlg.FileName
            Extension = $SaveFileDlg.FileName -replace '.*\.(.*)','$1'
        }

    }
}
#endregion Helper Functions

Next up is loading the required types to work with the chart controls as well as the windows forms.

 

Add-Type -AssemblyName System.Windows.Forms
Add-Type -AssemblyName System.Windows.Forms.DataVisualization

If you are still using the old way ([void][Reflection.Assembly]::LoadWithPartialName(“System.Windows.Forms.DataVisualization”)) then you should look at using Add-Type instead.

Next up is to set create our Chart, ChartArea and Series objects as well as making it easier to find all of our available charts by saving the Enum to a variable.

$Chart = New-object System.Windows.Forms.DataVisualization.Charting.Chart
$ChartArea = New-Object System.Windows.Forms.DataVisualization.Charting.ChartArea
$Series = New-Object -TypeName System.Windows.Forms.DataVisualization.Charting.Series
$ChartTypes = [System.Windows.Forms.DataVisualization.Charting.SeriesChartType]

Now picking a chart is as simple as using $ChartTypes:

$Series.ChartType = $ChartTypes::Pie

With this, we have defined our series as being a Pie chart (default is a line chart). What is interesting here is that we are not defining our chart type where you would have expected it to be at (in the $Chart) but instead define it within the Series object. Now we do end up placing the Series object within the chart and then the Chart within the ChartArea. Note:I don’t actually need a ChartArea with a Pie chart, but am including this for the sake of covering all of the pieces of the chart build.

$Chart.Series.Add($Series)
$Chart.ChartAreas.Add($ChartArea)

You can almost visually see how these all stack within one another.

image

I want to ensure that the Name (X axis) is the Name of the process and that the WS (Y axis) is the WS property which holds the data value. This way, when we apply it to the series, the pie chart control will understand how to present the data.

Note that I am using the DataBindXY method to load my data. The first item in the method parameter has to be the X value which is my label and the Y axis is the corresponding data. Because I am using PowerShell V4+, I can get away with just specifying the property name and it will automatically unroll the values of each property.

image

$Chart.Series['Series1'].Points.DataBindXY($Process.Name, $Process.WS)

The ‘Series1’ is a default name for the series (you can name it something else if you wish) and any subsequent series added will be Series2,3,4 and so forth if left at the default names.

With the data added for our pie chart, I can now work to make some adjustments to the size of the chart as well as its position and background color.

$Chart.Width = 700
$Chart.Height = 400
$Chart.Left = 10
$Chart.Top = 10
$Chart.BackColor = [System.Drawing.Color]::White
$Chart.BorderColor = 'Black'
$Chart.BorderDashStyle = 'Solid'

All good charts should have a title, right? How else would we know what the chart might be about if a title is not there to tell us what is going on. With that in mind, we will add a title that gives a brief description about what is being displayed.

$ChartTitle = New-Object System.Windows.Forms.DataVisualization.Charting.Title
$ChartTitle.Text = 'Top 5 Processes by Working Set Memory'
$Font = New-Object System.Drawing.Font @('Microsoft Sans Serif','12', [System.Drawing.FontStyle]::Bold)
$ChartTitle.Font =$Font
$Chart.Titles.Add($ChartTitle)

Typically, if I want to add a legend along with a pie chart, I will avoid having anything on the actual chart itself and leave the description for each piece to be in the legend. This is just a personal preference, but if you want, you can certainly have both. With that in mind, I will show two alternative approaches for the chart display with and without the legend.

Using a Legend

As I am using a legend here, I want to avoid any data from being displayed on the chart itself, so I will make sure to disable the pie chart styles.

$Chart.Series[‘Series1’][‘PieLabelStyle’] = ‘Disabled’

The next step is to set up my legend so it displays useful information.

$Legend = New-Object System.Windows.Forms.DataVisualization.Charting.Legend
$Legend.IsEquallySpacedItems = $True
$Legend.BorderColor = 'Black'
$Chart.Legends.Add($Legend)
$chart.Series["Series1"].LegendText = "#VALX (#VALY)"

And now I have my configurations completed for including a legend with my chart. Note that the VALX will display the values of the X axis while the VALY displays the Y value. So in this case I will have the Process name as VALX and the Working Set (WS) memory as VALY in the parentheses.

Avoiding a Legend

Ok, so  adding a legend wasn’t really in the cards and we just want to show the chart, but at the still time have the items labeled so we know what the pieces of the pie mean. Simple enough, we will just add some more configurations to add the data point labels.

$Chart.Series['Series1']['PieLineColor'] = 'Black'
$Chart.Series['Series1']['PieLabelStyle'] = 'Outside'
$Chart.Series['Series1'].Label = "#VALX (#VALY)"

Now we are set! All that is really left to do is display the results of our work. But before we do  that, we need to define a WinForm object that will host the chart object and properly display our work.

#region Windows Form to Display Chart
$AnchorAll = [System.Windows.Forms.AnchorStyles]::Bottom -bor [System.Windows.Forms.AnchorStyles]::Right -bor
    [System.Windows.Forms.AnchorStyles]::Top -bor [System.Windows.Forms.AnchorStyles]::Left
$Form = New-Object Windows.Forms.Form
$Form.Width = 740
$Form.Height = 490
$Form.controls.add($Chart)
$Chart.Anchor = $AnchorAll

# add a save button
$SaveButton = New-Object Windows.Forms.Button
$SaveButton.Text = "Save"
$SaveButton.Top = 420
$SaveButton.Left = 600
$SaveButton.Anchor = [System.Windows.Forms.AnchorStyles]::Bottom -bor [System.Windows.Forms.AnchorStyles]::Right
# [enum]::GetNames('System.Windows.Forms.DataVisualization.Charting.ChartImageFormat')
$SaveButton.add_click({
    $Result = Invoke-SaveDialog
    If ($Result) {
        $Chart.SaveImage($Result.FileName, $Result.Extension)
    }
})

$Form.controls.add($SaveButton)
$Form.Add_Shown({$Form.Activate()})
[void]$Form.ShowDialog()
#endregion Windows Form to Display Chart

The result is a chart that we can display to people with the added bonus of being able to save it via a save button.

With Legend

pie-legend

Without Legend

pie

A 3D Touch

If you want to give this a little better look by making the chart 3D, then you can add the following code to your chart configuration to make it a little more eye popping. And yes, we finally managed to sneak in some use of the $ChartArea in this demo.

$ChartArea.Area3DStyle.Enable3D=$True
$ChartArea.Area3DStyle.Inclination = 50

3d-pie

And just like that, instant 3D chart!

Saving a File

But what if I wanted to save a file instead? That’s fine, we can completely skip the process of creating the WinForm and instead make use of the builtin SaveImage method and supplying the file name as well as the extension of the file to save the image as a specific file type.

image

We can find the supported values here:

[enum]::GetValues([System.Windows.Forms.DataVisualization.Charting.ChartImageFormat])

image

Now we can save the chart using the code below:

$Chart.SaveImage('C:\temp\chart.jpeg', 'jpeg')

Where is My Cool Function at?

Yea, so about that function. I decided instead of just building a function to display a pie chart, that I would instead work on and build a module that would allow you to use a variety of charts instead! Stay tuned to https://github.com/proxb/PoshCharts (look at the Dev branch) and you will soon see a working module that not only does pie charts like shown today, but others such as a bar or line chart! Being that this is still in development, I don’t really have any help put together…yet. But as soon as this is more polished I will be updating this blog post (and posting another blog) so you can check it out! And as always, if anyone wants to dive in and help with this, then fork the repo and submit some Pull Requests and I will work to get them merged.

Posted in powershell | Tagged , , , | 4 Comments

What’s New in PoshRSJob

I’ve been hard at work on PoshRSJob to address some some bugs as well as adding some new features that have been reported in the Issues page on the GitHub site for the module. Some of the feature requests on the site didn’t just come from me as other folks had some great ideas and some of those said folks took it a step further by submitting a Pull Request with the needed update to add a particular feature (or fixing a bug). For those of you who submitted a PR,  I definitely thank you as that allows me more time to work on other things, such as ensuring that PoshRSJob works with PowerShell Core so it can run on Linux and MacOS.

In the last 5 months, PoshRSJob has had around 19 version updates ranging from simple bug fixes to more complex things such as accounting for when a user is using Start-RSJob within a ForEach loop. If you have been working with this module, you know that doing this was considered a bad idea because it would create a runspacepool for each iteration of the ForEach loop. Well, that has been resolved in my series of updates. Being that there have been 19 updates, I won’t be covering all of these in great detail, if at all but I will make sure to highlight what I think were noteworthy.

Support for Use on PowerShell Core (Linux and MacOS)

As soon as the announcement was made about PowerShell going open source and being made available on Linux and MacOS,  I made it my goal to ensure that PoshRSJob would work on those systems. Turns out the process wasn’t as bad as what I was seeing it to be initially.

Starting out, I had errors right away during the module import process where I am attempting to load a Type file into a runspace for my RSJob monitor code.

image

The problem here is that there are a couple of types which are now marked as private, meaning that my usual means of accessing them will end in failure and errors. The types which are no longer playing nice are:

  • System.Management.Automation.Runspaces.TypeConfigurationEntry
  • System.Management.Automation.Runspaces.RunspaceConfiguration

This didn’t mean that I was at a stopping point because they could no longer  be found. It just meant that I had to dig into some more advanced methods called reflection to dig into the types and pull out what I needed. Getting the RunspaceConfiguration property was pretty simple because I simply had to get the value of the property as shown below:

$Runspace = [runspacefactory]::CreateRunspace()
#Use the necessary flags to locate the property
$Flags = 'nonpublic','static','instance'
$_RunspaceConfig = $Runspace.GetType().GetProperty('RunspaceConfiguration',$Flags)
#Pull the RunspaceConfiguration property
$RunspaceConfig = $_RunspaceConfig.GetValue($Runspace,$Null)

Specifying the proper flags allows me to locate the private properties and with that, I have the first step of this puzzle completed. The tougher (at least for me) part was now figuring out how I can create the TypeEntry object that is required for me to load the object for my monitor code. After a little bit of testing, I was able to figure out how to create the constructor and use it to create the object that I needed.

#Locate the constructor of the TypeConfigurationEntry object 
$ctor = $RunspaceConfig.Types[0].GetType().GetConstructor([string])

#Invoke the constructor with the type file to build the object
$TypeConfigEntry = $ctor.Invoke($TypeFile)

Finally I add that into the RunspaceConfiguration.Types property.

#Load the type files into the RunspaceConfiguration
$RunspaceConfig.Types.Append($TypeConfigEntry)

What I have is a working approach that covers PowerShell V2+ as well as working in PowerShell Core. Naturally I wanted to see if this could be a public api instead of a private one so I went to the GitHub issues on PowerShell and filed an issue. After talking with PowerShell Team member Jason Shirk, he mentioned that the types that I was targeting were looking to be deprecated and that I should look at the more modern (and very much public) apis to accomplish what I was looking for.

image

After some testing, they worked like a champ and I began work to replace the private apis with the public ones.

$InitialSessionState = [System.Management.Automation.Runspaces.InitialSessionState]::CreateDefault()

#Create Type Collection so the object will work properly
$Types = Get-ChildItem "$($PSScriptRoot)\TypeData" -Filter *Types* | Select -ExpandProperty Fullname
ForEach ($Type in $Types) {
    $TypeConfigEntry = New-Object System.Management.Automation.Runspaces.SessionStateTypeEntry -ArgumentList $Type
    $InitialSessionState.Types.Add($TypeConfigEntry)
}
$PoshRS_RunspacePoolCleanup.Runspace =[runspacefactory]::CreateRunspace($InitialSessionState)

After this change was made, I brought down my module into my Ubuntu VM and it now works like I was hoping it would!

PoshRSJob

Remove the need for a DLL File to build classes

This was a workaround originally to provide not only support for Nano server, but also to keep its existing support for down-level support for PowerShell V2. The problem that this brought wasn’t really anything technical, but people who wanted to use this module had reservations to use compiled code that didn’t at least have the source code along with it. Of course, this is perfectly understandable because you don’t know what you are getting (even if the “source code” is provided). I finally went with an approach that would help to mitigate this issue but either using the Class keyword for PowerShell Core and Add-Type for everything else to create the same types needed by the module.

This did not come without its own set of issues as the parser very quickly lets you know if the class keyword is not allowed, even if you have it within an If statement. Kind of annoying but still not a show stopper as my solution for this was to create a here-string that contained the class code and invoked that code in the If statement using Invoke-Expression.

If ($PSVersionTable.PSEdition -eq 'Core') {
#PowerShell V4 and below will throw a parser error even if I never use classes
@'
    class V2UsingVariable {
        [string]$Name
        [string]$NewName
        [object]$Value
        [string]$NewVarName
    }

    class RSRunspacePool{
        [System.Management.Automation.Runspaces.RunspacePool]$RunspacePool
        [System.Management.Automation.Runspaces.RunspacePoolState]$State
        [int]$AvailableJobs
        [int]$MaxJobs
        [DateTime]$LastActivity = [DateTime]::MinValue
        [String]$RunspacePoolID
        [bool]$CanDispose = $False
    }

    class RSJob {
        [string]$Name
        [int]$ID
        [System.Management.Automation.PSInvocationState]$State
        [string]$InstanceID
        [object]$Handle
        [object]$Runspace
        [System.Management.Automation.PowerShell]$InnerJob
        [System.Threading.ManualResetEvent]$Finished
        [string]$Command
        [System.Management.Automation.PSDataCollection[System.Management.Automation.ErrorRecord]]$Error
        [System.Management.Automation.PSDataCollection[System.Management.Automation.VerboseRecord]]$Verbose
        [System.Management.Automation.PSDataCollection[System.Management.Automation.DebugRecord]]$Debug
        [System.Management.Automation.PSDataCollection[System.Management.Automation.WarningRecord]]$Warning
        [System.Management.Automation.PSDataCollection[System.Management.Automation.ProgressRecord]]$Progress
        [bool]$HasMoreData
        [bool]$HasErrors
        [object]$Output
        [string]$RunspacePoolID
        [bool]$Completed = $False
        [string]$Batch
    }
'@ | Invoke-Expression
}
Else {
    Add-Type @"
    using System;
    using System.Collections.Generic;
    using System.Text;
    using System.Management.Automation;

    public class V2UsingVariable
    {
        public string Name;
        public string NewName;
        public object Value;
        public string NewVarName;
    }

    public class RSRunspacePool
    {
        public System.Management.Automation.Runspaces.RunspacePool RunspacePool;
        public System.Management.Automation.Runspaces.RunspacePoolState State;
        public int AvailableJobs;
        public int MaxJobs;
        public DateTime LastActivity = DateTime.MinValue;
        public string RunspacePoolID;
        public bool CanDispose = false;
    }

    public class RSJob
    {
        public string Name;
        public int ID;
        public System.Management.Automation.PSInvocationState State;
        public string InstanceID;
        public object Handle;
        public object Runspace;
        public System.Management.Automation.PowerShell InnerJob;
        public System.Threading.ManualResetEvent Finished;
        public string Command;
        public System.Management.Automation.PSDataCollection<System.Management.Automation.ErrorRecord> Error;
        public System.Management.Automation.PSDataCollection<System.Management.Automation.VerboseRecord> Verbose;
        public System.Management.Automation.PSDataCollection<System.Management.Automation.DebugRecord> Debug;
        public System.Management.Automation.PSDataCollection<System.Management.Automation.WarningRecord> Warning;
        public System.Management.Automation.PSDataCollection<System.Management.Automation.ProgressRecord> Progress;
        public bool HasMoreData;
        public bool HasErrors;
        public object Output;
        public string RunspacePoolID;
        public bool Completed = false;
        public string Batch;
    }
"@
}

FunctionsToImport now Imports Aliases with Function

This was something that I found out while using the module along with another custom function. I cheated in the scriptblock of the RSJob and just tried to use the custom alias that I had given my function, but quickly found out via a nice error message that the alias didn’t exist in the context of the RSJob. Fortunately it didn’t take too long or provide a challenge to quickly add the alias for each imported function into the RSJob.

#Check for an alias and add it as well
If ($Alias = Get-Alias -Definition $Function) {
    $AliasEntry = New-Object System.Management.Automation.Runspaces.SessionStateAliasEntry -ArgumentList $Alias.Name,$Alias.Definition
    $InitialSessionState.Commands.Add($AliasEntry)
}

 

Using the same runspacepool with ForEach Loops

This was a fairly popular request as folks familiar with PSJobs would use a ForEach loop to create a new job for each thing that they wanted to run. This didn’t work out too well with RSJobs as it would create a new runspacepool for each iteration, meaning there was no throttling happening and could potentially bog the system down as everything would be running at the same time. The way that Start-RSJob would work is that you pipe directly into the function and it would handle the work behind the scenes. Well, the requests that I had wanted a way to provide support for both approaches while still delivering the same result of throttled jobs. After some work I was able to handle this by taking the Batch parameter that I had implemented a while back to group sets of jobs and merging that with the runspacepoolID in my RSRunspacePool type (now a string property instead of a GUID) and creating a default value for both that they shared (in the form of a GUID) and also allowed a user defined entry as well. Now during the RSJob creation process, the command will look for existing runspacepools and use that if the user hasn’t defined one.

#region RunspacePool Creation        
If (-NOT $PSBoundParameters.ContainsKey('Batch')) {
    If ($PoshRS_RunspacePools.Count -gt 0) {
        [System.Threading.Monitor]::Enter($PoshRS_RunspacePools.syncroot) 
        $RSPObject = $PoshRS_RunspacePools[0]
        $Batch = $RSPObject.RunspacePoolID
        Write-Verbose "Using current runspacepool <$($__RSPObject.RunspacePoolID)>"
        #Update LastActivity so it doesn't get removed prematurely; runspacepool cleanup will update once it detects running jobs
        $RSPObject.LastActivity = $RSPObject.LastActivity.AddMinutes(5)
        [System.Threading.Monitor]::Exit($PoshRS_RunspacePools.syncroot)
    }
    Else {
        $Batch = $RunspacePoolID = [guid]::NewGuid().ToString()
        Write-Verbose "Creating new runspacepool <$Batch>"
        $RunspacePool = [runspacefactory]::CreateRunspacePool($InitialSessionState)
        Try {
            #ApartmentState doesn't exist in Nano Server
            $RunspacePool.ApartmentState = 'STA'
        } 
        Catch {}
        [void]$RunspacePool.SetMaxRunspaces($Throttle)
        If ($PSVersionTable.PSVersion.Major -gt 2) {
            $RunspacePool.CleanupInterval = [timespan]::FromMinutes(5)    
        }
        $RunspacePool.Open()
        $RSPObject = New-Object RSRunspacePool -Property @{
            RunspacePool = $RunspacePool
            MaxJobs = $RunspacePool.GetMaxRunspaces()
            RunspacePoolID = $RunspacePoolID
        }
        [System.Threading.Monitor]::Enter($PoshRS_RunspacePools.syncroot) 
        [void]$PoshRS_RunspacePools.Add($RSPObject)
        [System.Threading.Monitor]::Exit($PoshRS_RunspacePools.syncroot)
    }
}
Else {            
    [System.Threading.Monitor]::Enter($PoshRS_RunspacePools.syncroot) 
    $__RSPObject = $PoshRS_RunspacePools | Where {
        $_.RunspacePoolID -eq $Batch
    }
    If ($__RSPObject) {
        Write-Verbose "Using current runspacepool <$($__RSPObject.RunspacePoolID)>"
        $RSPObject = $__RSPObject
        $RSPObject.LastActivity = $RSPObject.LastActivity.AddMinutes(5)
    }
    Else {
        Write-Verbose "Creating new runspacepool <$Batch>"
        $RunspacePoolID = $Batch
        $RunspacePool = [runspacefactory]::CreateRunspacePool($InitialSessionState)
        Try {
            #ApartmentState doesn't exist in Nano Server
            $RunspacePool.ApartmentState = 'STA'
        } 
        Catch {}
        [void]$RunspacePool.SetMaxRunspaces($Throttle)
        If ($PSVersionTable.PSVersion.Major -gt 2) {
            $RunspacePool.CleanupInterval = [timespan]::FromMinutes(2)    
        }
        $RunspacePool.Open()
        $RSPObject = New-Object RSRunspacePool -Property @{
            RunspacePool = $RunspacePool
            MaxJobs = $RunspacePool.GetMaxRunspaces()
            RunspacePoolID = $RunspacePoolID
        }
        [void]$PoshRS_RunspacePools.Add($RSPObject)
    }
    [System.Threading.Monitor]::Exit($PoshRS_RunspacePools.syncroot)            
}
#endregion RunspacePool Creation

That’s all for today. This was just a handful of the things that I have worked on with this module to try and make it as useful as possible. There are still many more things to do with it and I am always open to new suggestions so be sure to hit up the Issues page on the GitHub site and let me know what you want to see, or fork the repo and submit a Pull Request with the change and I will look  to get it into the main build.

You can find the rest of the issues that I had fixed (or had help via a PR to fix) below.

Release Notes For PoshRSJob


|1.7.2.4|

* Fixed Issue #92 (Cannot load module in PS4 due to “class” keyword)

|1.7.2.3|

  • Fixed Issue #87 (Stop-RSJob gives an error if it has no input)

|1.7.2.2|

  • Fixed Issue #59 (Receive-RSJob doesn’t clear a job’s HasMoreData state)

|1.7.2.1|

* Fixed Issue #83 (FunctionsToImport should include the function’s Alias where applicable)

|1.7.1.0|

  • Replaced private apis with public apis (#85 Update RunspaceConfiguration apis to use InitialSessionState instead)

|1.7.0.0|

  • Remove need for DLL file for building out the classes. Using pure PowerShell (mostly) via means of here-strings and Add-Type for PowerShell V2-4 and the new Classes keywords for PowerShell V5 which includes PowerShell Core/Nano.
  • Remove the prefixes for custom objects so they no longer start with PoshRS.PowerShell.. Now they are V2UsingVariable, RSJob and RSRunspacePool.

|1.6.2.1|

  • Add support for PowerShell Core on Linux/MacOS (this still needs more work but should load types within a runspace now!)

|1.6.1.0|

  • Fixed Issue #75 (Feature Request: Add RunspaceID handling to Start-RSJob for better throttling support)
  • Fixed Issue #82 (Exception setting “RunspacePool” in 1.6.0.0 build)

|1.5.7.7|

  • Fixed Issue #69 (Module produces error if imported more than once (PS v2 only))
  • Fixed Issue #64 (HadErrors in PoshRS.PowerShell.RSJob throws errors in PowerShell V2)
  • Fixed Issue #67 (Converted Add-Type code for C# classes to be created via Reflection for Nano server support) <- Created custom dll
  • Fixed Issue #61 (Receive-RSJob not allowing -Job parameter input)
  • Fixed Issue #63 (Replaced Global variables with Script scope)
  • Fixed Issue #66 (Parameters don’t work with PowerShell V2)
  • Fixed Issue #65 (Bug with v2 variable substitution – single-quoted strings get $var references replaced)
  • Fixed Issue #68 (ApartmentState Does Not Exist in Nano)
  • Fixed Issue #76 (Jobs don’t have output when using ADSI WinNT provider (Receive-RSJob))
Posted in powershell | Tagged , , , , | 2 Comments