PowerShell Open Sourced and Available on Linux and MacOS

No joke here. It was announced today at 11:14am (fun fact: this is when PowerShell V1 was released which was on November 14th,  2006) that PowerShell is not only being open sourced, but also is now available to install on Linux and MacOS. This is huge as being able to run PowerShell on anything other than Windows seemed like a pipe dream. But here it is! I won’t go crazy here listing out everything that has already been said before and those links are located at the bottom of this article. I just wanted to highlight some awesome things about this announcement.

You can head out to Github (https://github.com/PowerShell/PowerShell) right now and take a look at it in its open source glory! You will also notice a flurry of activity on its Issues page and also add your own as well!

image

Now, I mentioned that Linux is now an OS that supports running PowerShell. Currently there are a few distributions that it seems to support in Ubuntu and CentOS, but I imagine that more will be added as time progresses. Also MacOS is included in the list that you can install it on.

image

With this being an Alpha release, there will be bugs…lots of them…but that is OK because in its current state, it is pretty awesome! And that only means things will continue to be more and more better as the team takes in all of the issues (both internal and external) and fixes those as well as other things that they want to work on.

Installing on Ubuntu 14.04

Let’s go ahead and perform an install on a Linux system. In my case, I went ahead and installed Ubuntu 14.04 on my virtual environment so I could try this out!

The download package for this is at https://github.com/PowerShell/PowerShell/releases/download/v6.0.0-alpha.9/powershell_6.0.0-alpha.9-1ubuntu1.14.04.1_amd64.deb which I placed in the downloads folder under my home share.

I then navigate to that location and run the following commands:

sudo apt-get install libunwind8 libicu52
sudo dpkg -i powershell_6.0.0-alpha.9-1ubuntu1.14.04.1_amd64.deb

Once that has completed, I simply run PowerShell to load the shell.

image

The first thing I notice when running $PSVersionTable is that this is the alpha for version 6.0.0. Also note that the PSEdition is Core, meaning that this is running on .Net Core and probably missing some features that we are used to on the Windows side.

Looking at the variables, I see 4 that really stand out to that help to determine whether this system is running CoreCLR, is either Linux, Windows or OSX.

image

Some cmdlets are missing, such as Get-EventLog and Get-Service. Also noticeably missing are the CIM cmdlets.

image

Some things such as Desired State Configuration and Pester are available to use.

Aliases are also not as they seem in this version of PowerShell.

image

The Linux commands for ls and ps are not treated like PowerShell aliases currently, so you definitely want to keep that in mind. Also, case sensitivity, while not worried about in Windows should be respected here not for the commands, but for the file/folder structure.

image

Community Support

So with that announcement, when are we going to see some community involvement with porting their modules over to support Linux? Well, that is happening now! For instance, I am working on my PoshRSJob module to bring it into the Linux fold.

ubuntu_runspace

Needless to say that it is not there yet. I am seeing issues with what are usually public properties now being made private.

ubuntu_runspaceconfig

Will that change? I don’t know but I am working to work around that so I can have my module provide cross platform support to the masses Smile.

So with that, go ahead and give PowerShell a try on Linux or OSX and let everyone know how you like it. Got a bug or issue, drop over to GitHub and let the team know so they can prioritize it and make it better!

Reference Links

http://www.powershellmagazine.com/2016/08/18/open-source-powershell-on-windows-linux-and-osx/

https://blogs.msdn.microsoft.com/powershell/2016/08/18/powershell-on-linux-and-open-source-2/

https://azure.microsoft.com/en-us/blog/powershell-is-open-sourced-and-is-available-on-linux/

Posted in powershell | Tagged , , , , , | Leave a comment

Viewing Net Sessions using PowerShell and PInvoke

We are all used to view net sessions on a local or remote system by calling net session to see what clients are connected to another system such as a file server or even a domain controller. This provides a nice way to determine where a user might be logged in at assuming that they have an active session or one that is just idle. The problem with net session is that it requires admin rights on the system that you are running the command against in order for it to provide the data you need.

image

If you are a server admin, then this really isn’t that much of a deal as you can just use PowerShell remoting to query a bunch of  systems and get the information that you need. I wanted a way to  do this without worrying about admin rights or being able to remote into a system to get the information so I decided to look beyond the usual .Net and native PowerShell approaches to see if anything was available. The next logical step was to look lower in the stack and the Win32API and see what functions were available for me to use. I’ve done a fair amount of work with PInvoke and wasn’t afraid to see what kind of fun would await me if I happened to find something that might fit what I need.

After a decent amount of research,  I came across the NetSessionEnum function which provided me exactly the type of data that I am looking for. As with many of these functions, it isn’t enough to create the method to use but also to set up the Structs and/or Enums that the functions require to support the marshaling of data in and out of managed memory. In this case,  I have to make use of the SESSION_INFO_10 structure as I am looking to only return back data on the session which provides the client, username, active time and idle time. The image below highlights the structure I picked and also the other available Structs if I wanted different types of data.

image

The last thing that I need to look at is adding NetApiBufferFree so I can be sure to free up the buffer and avoid any unnecessary memory leaks.

Now that we have this out of the way, we can look to put these pieces together and make a function that we can use repeatedly.

If you want to see more about using pinvoke with PowerShell and a better look at what you need to do, then check out this article I did a while back. For this article, I am going to just point out one part that  I had to do in order to allow the parameters within the NetSessionEnum to properly work with the method.

Looking at the pinvoke signature for this function out on http://www.pinvoke.net/default.aspx/netapi32/NetSessionEnum.html, you can see that the first 3 parameters are a little more unique than what I typically see in that they unmanaged type of LPWStr which is a 32 bit pointer.

SNAGHTML31a8e4

That means that my usual approach to dynamically building the method has to account for this, otherwise just adding a string type will result in the method throwing an error when used.

I will build the method just like I normally would and give it all of the proper parameter and return value types that it needs. Instead of proceeding by adding some custom attributes to the method, I am going to focus first on defining the parameters that have special requirements by building custom attributes for those first.

 

#region Custom Attribute Builder
$ctor = [System.Runtime.InteropServices.MarshalAsAttribute].GetConstructor(@([System.Runtime.InteropServices.UnmanagedType]))
$CustomAttribute = [System.Runtime.InteropServices.UnmanagedType]::LPWStr
$CustomAttributeBuilder = New-Object System.Reflection.Emit.CustomAttributeBuilder -ArgumentList $ctor, $CustomAttribute
#endregion Custom Attribute Builder

 

Notice here that I am defining the LPWStr unmanaged type within the custom attribute. This will be important to add to my parameters that need it. Fortunately, these happen to be the first 3 parameters of the method (Servername, UserClientName and Username).  I need to make use of the DefineParameter method that is within the method object that I created earlier. All I need is the position of the parameter, a parameter attribute and supply $Null for the last parameter. After that I use SetCustomAttribute and give the parameter the extra attribute that it needs and there will be no problems with these parameters causing issues on the method.

 

#Define first three parameters with custom attributes
1..3 | ForEach {
    $Parameter = $PInvokeMethod.DefineParameter(
        $_,
        [System.Reflection.ParameterAttributes]::In,
        $Null
    )
    $Parameter.SetCustomAttribute(
        $CustomAttributeBuilder
    )
}

My function, Get-NetSession will get around that limitation that we saw earlier when trying to view the sessions on a domain controller. By design, I exclude sessions that this command creates on the remote system by looking at the current user and computer and performing the exclusion. If the user happens to be on a different system also, then it will be displayed.

Get-NetSession –Computername vdc1

image

I can use the –IncludeSelf parameter to include my own sessions create by the command.

Get-NetSession –Computername vdc1 –IncludeSelf

image

I also have a UserName parameter that accepts a single string to filter for a particular user if to avoid looking at a large dump of data.

You can find the script to download below as well as the source code from my GitHub repo at the bottom.

Download Script

https://gallery.technet.microsoft.com/scriptcenter/View-Net-Sessions-locally-d6eb2ba0

Source Code

https://github.com/proxb/PInvoke/blob/master/Get-NetSession.ps1

Posted in powershell | Tagged , , , | 1 Comment

Speaking at Omaha PowerShell User Group on PowerShell Runspaces

For those of you around the Omaha/Lincoln area, you can catch me tomorrow night speaking at the Omaha PowerShell User group where I will be talking about PowerShell runspaces and how you can use them for multithreading with your commands as well as demoing my module, PoshRSJob. There is still time to sign up if you want to hear it! Unfortunately, this will not be recorded as I will probably be walking around and that doesn’t bode too well for audio recordings.

Sign Up Here: https://www.eventbrite.com/e/july-omaha-powershell-user-group-tickets-26412769304

Posted in powershell | Tagged , , , , | Leave a comment

Scraping the Web for Water Levels using PowerShell

I like to take my bike out to ride it on trails whenever I get the time. It’s fun to see how far I can go and a lot of times the scenery is always great. The problem is when it rains, the trail that I typically use sits next to a creek that can and will flood over if we get too much rain. Fortunately for me, there is a web site that the United States Geological Survey uses to monitor and report on the water levels of streams, rivers, etc… that I can bring up to see. The default display is a nice graph that shows the history for the last couple of days of the water level so you can see where it is at and also where it is trending toward now. The link that I use for my travels is this one: http://waterdata.usgs.gov/nwis/uv?site_no=06610795

image

Now, graphs are great and all, but I want to see something with actual data because to me,that is more interesting to see because I can then look at using PowerShell to scrape it from the web. Luckily, there are other options to display the data that happens to reside near the top of the web page for me that includes a table display with the raw data to look at.

image

 

image

This is just a quick snapshot of some of the data that is available to me. I can see the exact height of the water as well as its discharge rate. Now that I know where this resides, I can grab the link and see what I can do to use it.

image

The key pieces of data here are the code for the location that I want to pull data from which is available a couple of different ways (1)From the site that I pull the data from or (2) from the following link which provides you a map that you can then locate the nearest data point and find the code: http://maps.waterdata.usgs.gov/mapper/index.html.

image

My warning here is that the way the data is collected and displayed does not follow a consistent approach. So my way of parsing the data here will be hit or miss depending on the location that you are looking at. My recommendation is that you look at the table data and see if it matches up with what is available here…if it even has this type of data (I did find some areas that just flat out didn’t have anything useful).

I have a little function that works in the instance that I want to use it for in my area, so your use may vary. The first thing that I am going to do is set up my RegEx so it will grab all of the necessary data that I need. RegEx is pretty important in the world of web scraping because it can be difficult to pull the data unless you have some other tool to do so.

[OutputType('Web.USGS.Data')]
[cmdletbinding()]
Param (
    [string]$Location = '06610795', #Ft Crook Rd in Bellevue, NE
    [datetime]$StartDate = (Get-Date).AddDays(-1),
    [datetime]$EndDate = (Get-Date)
)

If ($PSBoundParameters.ContainsKey('Debug')) {
    $DebugPreference = 'Continue'
}
If ($StartDate.Date -eq (Get-Date).Date) {
    $StartDate = $StartDate.AddDays(-1)
}

$__StartDate = $StartDate.ToString('yyyy-MM-dd')
$__EndDate = $EndDate.ToString('yyyy-MM-dd')
$URI = "http://waterdata.usgs.gov/ne/nwis/uv?cb_00065=on&cb_00060=on&format=html&site_no=$($Location)&period=&begin_date=$($__StartDate)&end_date=$($__EndDate)"
$RegEx = [regex]"^(?<DateTime>(?:\d{2}/){2}\d{4}\s\d{2}:\d{2})\s(?<TimeZone>[a-zA-Z]{1,})(?<Height>\d{1,}\.\d{2})(?:A|P)\s{2}(?<Discharge>(?:\d{1,},)*\d{1,})(?:A|P)"

 

As you can see, I am looking for the DateTime, Height and Discharge from the web page. Note that I fill in the blanks on the web page url based on the location and the starting date and time. If I do not use a starting or ending datetime, I just use default values so that way the query will not throw errors. Once that is done, we get to send out the query using Invoke-WebRequest.

 

Try {
    $Data = Invoke-WebRequest -Uri $URI
}
Catch {
    Write-Warning $_
    BREAK
}

If ($Data.ParsedHtml.body.innertext -match 'Redirecting') {
    Write-Verbose "Requesting data older or longer than 120 days, performing redirection"
    $Data = Invoke-WebRequest -Uri $Data.links.href
}

 

The query will go out and bring back all of the web page data. Sometimes, depending on what I am sending it, it may have to redirect me to a different site to get the data. I want to be sure to recognize it and handle that change so I get the data that I need by looking at the ParsedHtml.Body.InnerText for the word ‘Redirecting’ and then look at the link provided.

From there, we will take the data and begin parsing each line to get what we needed.

$Title = ((@($Data.ParsedHtml.getElementsByTagName('Title'))[0].Text -replace '.*USGS(.*)','$1').Trim() -replace ',|\.') -replace ' ','-'
Write-Verbose "[$($Title)]"
@($Data.ParsedHtml.getElementsByTagName('Table'))[3].InnerText -split '\r\n' | ForEach {
    If ($_ -match $RegEx) {
        $Object = [pscustomobject]@{
            Location = $Title
            DateTime = [datetime]$Matches.DateTime
            Height_FT = [decimal]$Matches.Height
            Discharge = [decimal]$Matches.Discharge -replace ','
        }
        $Object.pstypenames.insert(0,'Web.USGS.Data')
        $Object
    }
    Else {
        Write-Debug "[$($_)] No match found!"
    }
}

The end result looks like this:

image

 

We now have our data in object form about the current water levels on a stream, in this case the stream where I happen to ride my bike by most days.

The function that I wrote to help me easily view this is available at https://github.com/proxb/PowerShell_Scripts/blob/master/Get-USGSWaterData.ps1

Using the map provided, you will get limited success in pulling water data depending on the data source that you use. Of course, you can update the code to make it work with other areas.

Posted in powershell | Tagged , , | Leave a comment

Quick Hits: Writing to a Read-Only Property

This post is less on doing something useful and more on just proving that I can do something even though PowerShell is trying to tell me that I cannot do it. That little thing is updating a property on an object that is set as Read-Only.

My example to show you how you can do something like this is by looking at $PSVersionTable which as we all know gives us some great information about what version of PowerShell that we are using.

$PSVersionTable

image

I am also using this as this was an example used in our previous user group meeting in which we were shown that the properties here could not be updated.

From this object, I am going to look more at the PSVersion property which of course is an object within this object.

$PSVersionTable.PSVersion

image

Looking at the properties of this object, we can see that each of the properties are shown as Read-Only by only having the Get method available to use.

$PSVersionTable.PSVersion | Get-Member

image

Furthermore, I am greeted with a nice error stating that my attempts to modify this have been foolish and that I should just give up.

$PSVersionTable.PSVersion.Major = 10

image

Yea, I guess I could just throw in the towel and move onto something more useful like updating some bugs on a module, but I think that I can do better here! And by using some reflection magic, I can make this happen!

To do this, I need to look at the hidden fields for the PSVersion object.

$PSVersionTable.PSVersion.GetType().GetFields('static,nonpublic,instance').Name

image

Here we can see that the field that we need to look at updating is _Major.  Now we need to figure out how we can set this.

$Field = $PSVersionTable.PSVersion.GetType().GetField('_Major','static,nonpublic,instance')
$Field | Get-Member

SNAGHTML32a4c8

Looking at the methods, I see a SetValue that accepts a object and a value for the object. This seems pretty simple,right? Just call the method and update the value.

$Field.SetValue($PSVersionTable.PSVersion,10)
$PSVersionTable.PSVersion

SNAGHTML59e81d

Perfect! We have now updated this “Read-Only” property to give us PowerShell V10! Now this will go away once we close and re-open PowerShell but it still a fun thing to do to show off how you can update these kinds of values.

Posted in powershell | Tagged , , , | 3 Comments