Building a PowerShell Analysis Tool in PowerShell — Read Scripts with the AST, Not Regular Expressions

· Updated: · · PowerShell, AST, Static Analysis, Developer Tools

Suppose you want to survey a set of PowerShell scripts and list every place that calls Write-Host. Searching for the word is easy, but all three of the following lines match.

$source = @'
# Write-Host displays text on screen
$message = 'Write-Host'
Write-Host 'Hello'
'@

Only line 3 is what you are after. Line 1 is a comment and line 2 is a string assigned to a variable, so neither of them calls Write-Host.

PowerShell itself tells these apart as it runs the code. You can also take the result of that reading out for yourself. This article starts by parsing the code and examining the objects that come out of it. No additional modules are used.1

The @' ... '@ above is a single-quoted here-string. It puts those three lines into $source as a string that is not executed for now and in which $message is not expanded.2

1. Display What the Parser Returns

First, pass $source to ParseInput.

$tokens = $null
$parseErrors = $null
$ast = [System.Management.Automation.Language.Parser]::ParseInput(
    $source, [ref] $tokens, [ref] $parseErrors)

if ($parseErrors.Count -gt 0) {
    throw $parseErrors[0].Message
}

$ast.GetType().Name
ScriptBlockAst

What landed in $ast is neither a string nor the result of running the script: it is an object of type ScriptBlockAst. It represents the whole of the code you supplied. Besides returning this object, ParseInput returns the tokens and the parse errors through the variables passed with [ref].1

AST is short for “abstract syntax tree.” The name may suggest some special data structure, but from PowerShell’s point of view it is first of all an object with properties and methods. Follow those properties and you reach other objects that represent assignments and command calls.3

In the diagram a solid line marks a relation that always holds and a dashed line marks a conditional one (the conditions are given per relation on the detail page). The full list of relations (5 in total, with evidence and certainty) and the definitions of the main concepts are collected on the knowledge map detail page (in Japanese). Data: JSON-LD / Turtle

2. What Objects Did Those Three Lines Become?

In code like this one, which does not spell out begin, process, and end, ordinary statements live in EndBlock.Statements. Let us line up their types next to the original code.4

$statements = $ast.EndBlock.Statements
$statements | ForEach-Object {
    [pscustomobject]@{
        Type = $_.GetType().Name
        Text = $_.Extent.Text
    }
}
Type                   Text
----                   ----
AssignmentStatementAst $message = 'Write-Host'
PipelineAst            Write-Host 'Hello'

Two entries. The comment line does not appear in this list of statements. If you need the comments themselves, you can get them from the $tokens you received a moment ago.3

Line 2 became an AssignmentStatementAst, which represents an assignment. Line 3 is a PipelineAst: even without writing a |, it is represented as a pipeline with a single command. Extent.Text is the original code that corresponds to the object. Whenever the type name alone leaves you unsure, this tells you which part of the source is under discussion.56

How the original code maps onto syntax objectsThe EndBlock of the whole script holds the assignment on line 2 and the pipeline on line 3, and the latter contains the command call.ScriptBlockAstEndBlockAssignment: line 2Pipeline: line 3CommandAst

Figure 1: The call on line 3 sits inside the pipeline.

Array indexes start at 0, so $statements[1] is the pipeline from line 3. From there, take the first element.

$pipeline = $statements[1]
$call = $pipeline.PipelineElements[0]

$call.GetType().Name
$call.Extent.Text
$call.GetCommandName()
CommandAst
Write-Host 'Hello'
Write-Host

There is the CommandAst. Extent.Text, which returns the original code, and GetCommandName(), which returns the name of the call, are both used against that same object.7

The breakdown including the arguments is in CommandElements.

$call.CommandElements | ForEach-Object {
    [pscustomobject]@{
        Type = $_.GetType().Name
        Text = $_.Extent.Text
    }
}
Type                        Text
----                        ----
StringConstantExpressionAst Write-Host
StringConstantExpressionAst 'Hello'

The name and the argument are both nodes that represent strings. Even so, what GetCommandName() returns is Write-Host. It is not choosing by the spelling of the string alone: it is looking at the element that serves as the name within a command call. The 'Write-Host' on line 2 is on the right-hand side of an assignment and does not belong to this call at all.78

This is the difference between a text search and parsing. Even for the same word, examining the structure it sits in lets you tell its roles apart.

3. Search for CommandAst Instead of Walking by Index

We have confirmed the contents. In a real script, though, you cannot hard-code “the first element of the second statement.” Calls appear inside functions, inside if blocks, and partway through a pipeline.

The search method for that is FindAll.

$commands = @($ast.FindAll({
    param($node)
    $node -is [System.Management.Automation.Language.CommandAst]
}, $true))

$commands.Count
$commands[0].GetType().Name
$commands[0].Extent.Text
1
CommandAst
Write-Host 'Hello'

FindAll walks the syntax tree and hands each node to the test in { ... }. Check the type of the $node you are given with -is, and return $true when it is a CommandAst. That node stays in the results. The trailing $true tells it to search inside nested functions and script blocks as well.9

With function Show-Message { Write-Host 'hello' }, for instance, it descends into the function body and picks up the call. There is no need to run the function.

Searching by type down into a functionThe search descends into the body of a function contained in the script and includes the call matching CommandAst in the results.ScriptFunction definitionFunction bodyCommandAstType matches: kept in the results

Figure 2: You can obtain the nodes matching the search condition without counting the nesting depth yourself.

In the three lines from the beginning, it found the same single call we reached by index. Extract the name and the location, and you have something you can use as a search result.

$commands | ForEach-Object {
    [pscustomobject]@{
        Name   = $_.GetCommandName()
        Line   = $_.Extent.StartLineNumber
        Column = $_.Extent.StartColumnNumber
    }
}
Name       Line Column
----       ---- ------
Write-Host    3      1

Lines and columns are 1-based. Because the same node carries both the name and the location, you do not have to go back and hunt for the line with a separate text search.6

4. How a Call Through a Variable Looks

Let us change the target of the analysis slightly. In addition to writing the name directly, include cases that pass a string or a variable to the call operator &.

$source = @'
Write-Host 'direct'
& 'Write-Host' 'quoted'
$command = 'Write-Host'
& $command 'variable'
'@

$ast = [System.Management.Automation.Language.Parser]::ParseInput(
    $source, [ref] $tokens, [ref] $parseErrors)
if ($parseErrors.Count -gt 0) { throw $parseErrors[0].Message }

$commands = @($ast.FindAll({
    param($node)
    $node -is [System.Management.Automation.Language.CommandAst]
}, $true))

$commands | ForEach-Object {
    [pscustomobject]@{
        Line = $_.Extent.StartLineNumber
        Name = $_.GetCommandName()
        Text = $_.Extent.Text
    }
}
Line Name       Text
---- ----       ----
   1 Write-Host Write-Host 'direct'
   2 Write-Host & 'Write-Host' 'quoted'
   4            & $command 'variable'

Line 4 was found as a CommandAst too. The return value of GetCommandName(), however, is $null.

A person reading these four lines can infer the value of $command from the assignment just above. This method does not go back through assignments to a variable and compute its value. Unlike the case where the name is written directly in the code, this API alone cannot extract the name.7

The same call node, different results for obtaining the nameFor a call whose name is a string, Write-Host can be obtained; for a call through a variable the name is null, but both carry the location of the call.CommandAstName element is a stringName element is a variableName: Write-HostName: null

Figure 3: Even with the name blank, you can still tell that a call is written on line 4.

In a tool that surveys files, put this difference into a NameKind column. Lines whose name string could be obtained are Static; lines where it could not are Unresolved. Keeping the nameless lines means you do not lose track of the places a person should check.

5. Extract the Locations of Write-Host from a File

When you read a .ps1 instead of a string, use ParseFile in place of ParseInput. The way you read the AST does not change.10

Get-ScriptCommand, which gathers everything so far, is provided in the complete code at the end of this article and in the sample download. Save the complete code as Get-ScriptCommand.ps1, and save the four lines inside the here-string from section 4 as demo.ps1 in the same folder. The sample download contains both files.

Run it in that folder.

. .\Get-ScriptCommand.ps1
$calls = @(Get-ScriptCommand -LiteralPath .\demo.ps1)

$calls |
    Where-Object { $_.NameKind -eq 'Static' -and $_.Name -eq 'Write-Host' } |
    Format-Table Line, Column, NameKind, Name -AutoSize
Line Column NameKind Name
---- ------ -------- ----
   1      1 Static   Write-Host
   2      1 Static   Write-Host

That gives you the locations of the calls whose name is Write-Host. The assignment on line 3 is not included. It avoids the problem we started with, where comments and ordinary strings end up mixed into the search results.

Line 4, on the other hand, falls outside this filter. Not because there is no call there, but because the name is undetermined. Check the undetermined lines separately.

$calls |
    Where-Object NameKind -eq 'Unresolved' |
    Format-Table Line, Column, NameKind, Name -AutoSize
Line Column NameKind   Name
---- ------ --------   ----
   4      1 Unresolved

Static is a mark meaning “the name string could be obtained.” It is not a guarantee that the command exists or that you know which implementation will be invoked. echo, for example, comes back as echo, and Microsoft.PowerShell.Utility\Write-Host comes back as the qualified name, so neither is included in the exact-match search above. If aliases and qualified calls are also in scope for your survey, the search condition has to match that.11

Note that what was dot-sourced is the analysis tool. demo.ps1 is only read with ParseFile; it is never started.

6. The Search Results Are Not an Execution History

This search examines how something is written in the code. Calls are written inside functions that are never used and inside if ($false) { Write-Host ... } as well, so they appear in the list. It tells you nothing about the order of execution or how many times something runs.

Strings differ in how they are handled, too: 'Write-Host' and "Today: $(Get-Date)" are not the same. The latter has an expression embedded in it, so the Get-Date inside it is found. Code sitting inside an ordinary string, by contrast, is never re-parsed as a separate script.2

A .NET method call such as [Console]::WriteLine(...) is a different kind of node from CommandAst. This list neither covers every operation nor proves anything about safety. It is a tool for surveying scripts you manage yourself. Confirming runtime aliases or same-named functions also requires the environment in which the script runs.1211

At the center of what we used here are the objects you confirm with GetType() and Extent.Text. Whenever you want to examine another piece of syntax, you can start the same way: pass a short piece of code to ParseInput and put those two side by side. When your goal is inspecting quality against existing rules, PSScriptAnalyzer is the better choice.

The Complete Tool for Searching Files

Below is the whole of Get-ScriptCommand used in section 5. The part that reads the syntax is the same as in section 3; around it are file retrieval, parse-error handling, and input of multiple files.

function Get-ScriptCommand {
    [CmdletBinding()]
    [OutputType([pscustomobject])]
    param(
        [Parameter(Mandatory, ValueFromPipelineByPropertyName)]
        [Alias('FullName')]
        [ValidateNotNullOrEmpty()]
        [string[]] $LiteralPath
    )

    process {
        foreach ($path in $LiteralPath) {
            $file = Get-Item -LiteralPath $path -Force -ErrorAction Stop
            if ($file -isnot [System.IO.FileInfo]) {
                throw "A file is required: $path"
            }

            $tokens = $null
            $parseErrors = $null
            $ast = [System.Management.Automation.Language.Parser]::ParseFile(
                $file.FullName, [ref] $tokens, [ref] $parseErrors)
            if ($parseErrors.Count -gt 0) {
                $first = $parseErrors[0]
                throw ('Parse error: {0}:{1}:{2} ({3})' -f $file.FullName,
                    $first.Extent.StartLineNumber,
                    $first.Extent.StartColumnNumber, $first.ErrorId)
            }

            $commands = $ast.FindAll({
                    param($node)
                    $node -is [System.Management.Automation.Language.CommandAst]
                }, $true)
            foreach ($command in ($commands | Sort-Object { $_.Extent.StartOffset })) {
                $name = $command.GetCommandName()
                $kind = if ($null -eq $name) { 'Unresolved' } else { 'Static' }
                [pscustomobject]@{
                    Path     = $file.FullName
                    Line     = $command.Extent.StartLineNumber
                    Column   = $command.Extent.StartColumnNumber
                    NameKind = $kind
                    Name     = $name
                }
            }
        }
    }
}

It obtains the actual file with Get-Item -LiteralPath and then passes FullName to ParseFile. A name such as draft[1].ps1 is not treated as a wildcard. Directories and files that do not exist raise an error.

If there is a parse error, it stops before emitting any results for that file. A partially returned AST does not count as a successful analysis. The point is to keep “read correctly and found zero” separate from “could not be read.”

The return value is not formatted into a table: it is an object with Path, Line, Column, NameKind, and Name. Besides filtering as in section 5, you can save the results for multiple files to CSV. Because it also accepts input through a property named FullName, you can pass the FileInfo objects returned by Get-ChildItem straight through.

Get-ChildItem -LiteralPath .\scripts -Filter *.ps1 -File -Recurse |
    Get-ScriptCommand |
    Export-Csv -LiteralPath .\commands.csv -NoTypeInformation -Encoding UTF8 -NoClobber

-NoClobber prevents overwriting an existing CSV. If a file partway through fails, the results before it may already have gone into the CSV. Do not take the existence of the file as a sign that every entry succeeded — check the errors as well.

The grammar used for parsing follows the version of PowerShell running the tool. Parsing successfully under 7.x does not mean the script will run under 5.1. If you also read files containing Japanese under 5.1, take character encoding into account, such as using UTF-8 with a BOM.13

Samples and Verification

The analysis tool, samples, and tests (ZIP) contains the finished function, examples to analyze, and Pester tests. The function itself is the same as the code shown here; the distributed version adds help comments. If you cannot obtain the ZIP, you can still save and use the complete code above.

The finished function and the existing 26 Pester cases are unchanged from the previous revision. For this revision, the 12 PowerShell code blocks extracted from the body were run under Windows PowerShell 5.1 and PowerShell 7.x, and the type names, original code, call locations, and filtered results were checked against each other. The exact versions and the scope of verification are documented in the README of the sample download.

References

  1. Microsoft Learn, Parser.ParseInput Method. On the API that returns an AST from a string and returns tokens and parse errors through output arguments.  2

  2. Microsoft Learn, about_Quoting_Rules. On single-quoted here-strings and subexpressions in expandable strings.  2

  3. Microsoft PowerShell Team, Using abstract syntax trees (ASTs) with ISE to make scripting more productive. On reaching the syntax tree from PowerShell and searching for nodes such as function definitions.  2

  4. Microsoft Learn, NamedBlockAst Class. On blocks whose name is not spelled out and on Statements, which holds the statements. 

  5. Microsoft Learn, PipelineAst.PipelineElements Property. On the elements that make up a pipeline. 

  6. Microsoft Learn, IScriptExtent Interface. On the extent in the source, the start position, and lines and columns being 1-based.  2

  7. Microsoft Learn, CommandAst.GetCommandName Method. On null being returned for calls whose name cannot be obtained statically.  2 3

  8. Microsoft Learn, CommandAst.CommandElements Property. On syntax elements such as the call name and the arguments. 

  9. Microsoft Learn, Ast.FindAll Method. On walking the nodes that match a condition and on the option to search nested functions and script blocks. 

  10. Microsoft Learn, Parser.ParseFile Method. On the API that parses a file and yields the AST, the tokens, and the parse errors. 

  11. Microsoft Learn, about_Command_Precedence. On the runtime precedence of same-named commands, aliases, functions, and the like.  2

  12. Microsoft Learn, InvokeMemberExpressionAst Constructors. On the nodes that represent instance and static method calls. 

  13. Microsoft Learn, about_Character_Encoding. On how Windows PowerShell reads scripts and on the handling of the UTF-8 BOM. 

Recent articles sharing the same tags. Deepen your understanding with closely related topics.

These topic pages place the article in a broader service and decision context.

Frequently Asked Questions

Common questions about the topic of this article.

What is the PowerShell AST?
It is an abstract syntax tree that represents code as nodes for each syntactic construct, such as an assignment, a command call, or a function definition. It lets you tell apart the same characters written inside a comment or a string from the same characters written as the name of a call.
Do I have to run the .ps1 files I am analyzing?
The tool in this article reads them with Parser.ParseFile; it neither launches nor dot-sources the target. It is not a sandbox that guarantees safety, however — it is meant for taking inventory of scripts you manage yourself. Finding no call is also not a proof of safety.
Can I get the name of a command that is called through a variable?
When GetCommandName cannot obtain the name statically, it returns null. The tool in this article does not discard that line: it keeps it with a NameKind of Unresolved. It does not trace variable assignments, and it does not evaluate expressions to guess the name.
Does this work on Windows PowerShell 5.1?
The tool presented here targets 5.1 and the 7.x line. The grammar used for parsing, however, is that of the PowerShell running the tool. Parsing successfully under 7.x is not a proof of compatibility with 5.1. If you also read files containing Japanese under 5.1, take character encoding into account as well, such as using UTF-8 with a BOM.
How does this compare with PSScriptAnalyzer?
The tool in this article exists to list the names and locations of calls. When you want to inspect quality against existing rules and manage the warnings, use PSScriptAnalyzer. The way of reading the AST shown here is an entry point to understanding static analysis results and how custom rules work.

Author Profile

Profile page for the article author.

Go Komura

Representative of KomuraSoft LLC

Focused on Windows software development, technical consulting, and investigations into failures that are difficult to reproduce.

Back to the Blog