Jeffrey Snover and the Making of PowerShell

Host here, thanks for sharing.

PowerShell faced extreme opposition at Microsoft, and its creator Jeffrey Snover was demoted for pursuing it.

Jeffrey was originally brought into Microsoft to help MS learn how to compete in the data center, but culturally they were so tied to the personal computer model of the world, that they fought him every step of the way.

Edit: Another interesting thing, is how Powershell exists because Windows isn't file based. Jeffrey's goal was server administration, but on Windows you can't just edit files to administer things, you need to call various APIs and get structured data back forth. The rich object model fell out of that. It was the only way.

( Also, apologies if the transcript has errors. I've gone from professional transcriptions to Descript and then a pass of GPT4 trying to find the right punctation breaks and then me doing a quick read through. I don't think its coming out as high quality as I'd like. )

I've always been curious to know. I am an experienced Bash developer, and when PowerShell was released, I was very excited. Finally, we would have a cool shell on Windows for development. However, since then, I've never managed to grok PowerShell and continue to use my good old Bash even on Windows.

What is the experience of other developers who are experts in both shells to compare them? Did PowerShell really fulfill the promise of being a more efficient and modern shell? Or people just use it because it is already installed and better than CMD?

Powershell wasn't awful when I wrote it but I never understood why arrays of length 1 removed the array and become the contained type. This caused a huge # of bugs as you suddenly had to care how many items could potentially be in the array and check each time you modified vs. handling them generically. Does anyone know WHY they did this?

Unless I'm interacting with some Windows subsystem, and need the specific Powershell commands, I just think "why the fuck am I not using Python?"

It's also way too verbose and slow for 90% of the stuff I'd use Bash for (or would have used Perl in another life)

I often wonder why Microsoft didn't base it on Python, Node, or something else. I can't remember when PS was first released so I'm not sure what would have been ideal at the time.

At $dayjob I am blessed with the task of wrangling a 20+ year old codebase of SQL Server stored procedures. It's around 300k lines of monkey tested, business-critical code that:

* wasn't source controlled

* was never tuned (properly) for performance

* deployed into environments by editing/executing sql in SSMS.

* of course, no automated tests, etc...

It's a windows shop, I am developing on a Mac, and we do linux on Github Actions.

I selected tools like PowerShell Core, sqlcmd, docker for running Windows SQL Server instances, RedGate SQL Compare for extracting existing schema and code from the legacy servers, tSQLt for unit testing, TSqlLint for code compliance, SQLFluff for style compliance, and Flyway for deployments.

We quickly discovered PowerShell Core was the most interoperable cross-platform scripting shell when Windows had to be one of the platforms.

It wasn't pleasant to code in. The regex engine comes from .Net, which has bad catastrophic backtracking problems, and the array situation was goofy. Launching executables with any sort of control over the launch and capturing the output stream was hit or miss - I would often have to launch a process, redirect its output to a temp file, and then read the temp file after the child exited. So piping around with child's stdout into a string variable was always more trouble than it should have been.

BUT, PowerShell Core executes fast (nice job M$FT, if there's one thing you do well, it's micro-optimizing!) It has nice tools for interacting with the user, like ascii art list pickers and easy input prompt generators. And, most of the strange quirks in dealing with the Windows file system are papered over if one avoids folders with locked files.

Anything you want to achieve can probably be done if you search hard enough. Recommended!

I'll agree with some of the other sentiments here that whenever my career brought me close to Windows administration I absolutely loathed the experience.

PowerShell however was actually pretty great to use, despite everything else on Windows being extremely clunky. It always felt very thoughtfully designed.

Linux is great, and I'll always use it as my daily driver for work, but using bash is absolutely horrible. But because it's always everywhere, it's something that everyone reaches for first, and so we'll probably still be dealing with bash scripts in 2100, warts and all.

Powershell was truly a product of Microsoft monopolistic confidence. To create a language that allows absolutely no syntactical carryover from any other langauge is wild. You couldn't guess or assume any command, parameter or flag. Even with Microsoft's ambition they should have realized that legions of admins and programmers had to learn and maintain scripts in both Powershell and bash for a few decades at least.

The extreme verbosity of the syntax may look good for a presentation to the committee, but dealing with it regularly collides with well-researched and understood limits of the human brain, where information of a certain size, and delays of a certain length break the state of flow and requires concentration, explicit memorization and double-takes. Even with practice one could never quickly execute the common shell incantation to turn a thought into reality, instead one would have to wrestle with syntax, even if it's just waiting for autocomplete to appear and deciding to accept the next word of a multipart command.

Let's try the start menu and search for "pow..", chose between 4 amazing options, Powershell or Powershell ISE, both in regular and x86 flavour. Either would take a while to load, breaking flow. The ISE shows a little splash screen that jumps to a second location. Another dialog shows up and informs you that you've closed the last session without saving the unnamed script files. But it opens them anyway, as you would expect. So why scold me for this? Because, you know damn well that saving the textfile is trouble: I can type or copy, and then execute any kind of evil code imaginable, but saving the file and then running it as a .ps Script triggers the ridiculous execution-policy song and dance. Probably trauma from Microsoft's bad security reputation of early Internet Explorers and Windows versions.

I tried to love it anyway but one day my script encountered filenames with square brackets. Powershell implicitly interpreted those [1] and [2] as iterators somehow (https://stackoverflow.com/questions/21008180/copy-file-with-...). Sorry but dealing with files is the one job a scripting language has, filenames are beyond the control of script authors, and the space of valid filenames on Windows should be known. This gave me some long lasting trust issues with that language.

(The Azure team aparently had enough power within Microsoft to create their own, sane and readable syntax: "az find vm", "az account show".)

Seems so weird in retrospect that Microsoft wouldn’t have seen the value of an easily composable and programmatic way to configure anything in Windows and their other important enterprise applications like Active Directory and Exchange. The idea that connecting with Remote Desktop and clicking around with a mouse was suggested as an alternative strikes me as absurd. Especially because automating that sort of thing (at least based on my experience using autohotkey and window spy) is horrendously difficult and annoying.

I have never, and I mean never, been a Windows user, even though I've been using computers since 1982. During the rise of Wintel in the early 1990s, I followed the rise of Linux and 386BSD. When Win95 and NT ruled the business desktop in the late 1990s, I sought refuge in SPARCStations, Linux, and discontinued NeXT hardware. After the turn of the century, I adopted the newly-POSIX-compliant Mac OS. All this to say, avoidance of Microsoft products has been a cornerstone of my computing policy for nearly half a century (with the notable exception of Applesoft BASIC).

But PowerShell? PowerShell's nice.

Something that is under-appreciated is that if you need to write your own command-line tool with a "proper" programming language such as C/C++ or whatever, then there is a vast difference in productivity between writing for traditional shells or PowerShell.

I've never been able to make a generally useful CLI tool in under a few thousand lines of messy code. You typically have to deal with: pipeline inputs, optional parameters, parameters with values, defaults with overrides, "dry run" mode, and the various output formatting requirements, and so on. You end up with 90% fluff and 10% action.

With PowerShell, a C# module is basically 20 lines or so of overhead, and the rest is all action. It's mindblowing how productive this is! You get parameter validation, parameter name tab-complete, pipeline input, pipeline output, formatting, strong typing, globbing, etc... all for free.

So much was so right about PowerShell. But it failed to attract a wider audience, and in their quest to woo Linux devs Microsoft has been undermining PowerShell lately. Knowing what PowerShell offers, falling back to bash CLI tools feels like two steps back.

Just some of the stuff PowerShell did right:

- PowerShell cmdlets are self-describing and rich in information. Rather than each command doing its own parsing of parameters, cmdlets describe parameters and delegates the actual parsing to the shell. The shell understands data types, parsing rules, e.g. how to parse a UUID or a date. Not only does this ensure a consistency that was never in *sh shells, but it also enables cool stuff like e.g. autocomplete, predictive input, help instructions etc. almost for free.

- "Simulation" mode (-Confirm and -WhatIf) where a cmdlet can describe the action it is about to take, and the mode of the shell may decline everything (effectively a "simulation mode") or may actually ask the user for permission (-Conform) for each action.

But, alas, PowerShell never caught on outside Windows, and now MS is leaving it to wither in their quest to not upset a wider non-Windows community.

PowerShell is very comfortable in interactive mode.

For scripting, I don't get it. It's not designed to be a simple text-based glue like the bourne shell is, so it feels weird in many places (quoting, escaping, even more than sh is). It's very good for glueing Windows stuff though, like .NET libraries and so on.

I once had 2 microsoft engineers call me for help with qmail because I was active on the qmail mailing list. They couldn't wrap their heads around how svc and daemontools worked.. the good old days! I never asked why 2 microsoft engineers were working with qmail. I think I was 20 at the time and I was just happy to be on the phone with people from Microsoft even though I was very much a Linux guy.

I love corecursive! Keep up the good work @adam!!

The Monad Manifesto mentioned in the podcast:

https://jsnover.com/Docs/MonadManifesto.pdf

I used to be a hardcore Windows dev but never figured out powershell. I actually ended up writing scripts in C# and had a utility to load and run it.

I think PowerShell is a bit scary, for example I could never get curl to work in it, say a simple POST command: curl -X "POST" google.com should return Error 411 (Length required) from google (as it does in CMD.EXE)

When I try it in PowerShell I get: Invoke-WebRequest : A parameter cannot be found that matches parameter name 'X'. and some more error messages

even curl -X "GET" doesn't work :-(

This is a great story of finding a way to be productive inside a giant, impersonal machine like Microsoft.

That said, it did nothing to convince me to invest more mental energy in PowerShell. Every time I use it, I have the same “meh” reaction: The learning curve is too steep and the syntax is too ugly. It doesn’t ever “stick” with me, so I end up starting from zero every time I encounter it again.

I think the fundamental problem is that PWSH inhabits a dark valley between quick-and-dirty scripting and I’m-serious-about-this programming. It really needed to pick one side or the other to win people over, but never did.

I found the most interesting part about this article how it kind of makes a direct line from UNIX’s place as a place for programmer system admins versus professional services-driven servers.

These people at Microsoft knew that they’d wipe the floor with the professional services folks by getting that professional services cost out of the picture by providing the small UNIX-like toolbox to do sysadmin work via automations.

Of course, PowerShell being API-based instead of file-based was what came out of that.

It’s interesting because what has survived has essentially been Windows and UNIX (via Linux being able to drop all the proprietary baggage of UNIX solutions). Everything else that was built to sell professional services is dead.

Big fan of PowerShell. Such a shame that being in Microsoft, you aren't allowed to do great work. That is why hackers shouldn't join large companies. You can create much more value working for yourself or joining a small startup.

Besides the whole interesting story background, yet another confirmation of the anti-.NET bias by WinDev during the Longhorn efforts.

Instead of uniting and having everyone collaborating into a common like Google with Android making it happen no matter what, or how Bell Labs tried with Inferno / Limbo, the active fight against .NET, and anything related.

To what ended up being the WinRT failure.

Ironically, WinDev is now shipping JavaScript and Webview2 all over the place on Windows 11.

Would have loved to hear why he made the move to Google considering that he became a Technical Fellow and very popular in the community

PowerShell, is still the best way to scale your workload, administering a Windows server environment on premise. For Azure, there were features that were present in the Graph API that were not in PowerShell.

I haven't checked back in a while, but I think most new features in PowerShell are just pointing back to the Graph API.

PowerShell helped me a lot in my earlier career, bash always felt like banging rocks together in comparison.

It says a lot about the uphill battle that he faced when the first two sentences Jeffrey says in the podcast are "By the way, is it okay to swear?" and "You know, I had executives say, ‘Jeffrey, exactly which part of fucking Windows is confusing you, Jeffrey?’".

Favorite line:

"I’ve never seen anybody use a GUI in a clever way. Ever. There’s no cleverness to it. No, like, Oh my God, you should see the way Adam clicked that mouse. Oh my God. Guys, guys, guys, guys, come on, check it out. Adam’s going to click the button. Oh my God. That’s amazing. It just doesn’t happen."

Thanks for this. Just seeing the word "PowerShell" gets me excited every time... been using it for hours a day for years and it's so much fun. It's how I first learned to code, professionally. Kudos and Cheers!

I am not a fan of PowerShell, but I am a fan of Jeffrey--he toiled and talked sense at a time way before Azure was a thing, but everything since he shipped has proven he was right.

I wonder how many engineers use AWS's Powershell cmdlets?

They are incredibly well built and feel first class/more polished than a lot of AWS tools.

Honestly I love Powershell and run it on Linux or OS X. Being able to access fields instead of having to play with `cut` to get the data I want is priceless. Just feels cleaner and more maintainable.

Why wouldn’t they just replicate bash or some other UNIX shell, along with the basic UNIX tools like cp and find with matching APIs? Huge mistake there imo, even if they did add a few bells and whistles with powershell

I love this podcast!

PowerShell is MIT licensed, cross-platform Windows, Linux, macOS compatible, and you can download it in various installers and packages here: https://github.com/PowerShell/PowerShell/releases

for anyone who wants to try the newer version on Windows, get either Windows Terminal (from Microsoft Store or Github releases: https://learn.microsoft.com/en-us/windows/terminal/install) or Visual Studio Code. The classic Windows' command prompt console host engine just can't do Unicode and fonts and colours and Unix shell escape sequences.

After that, find something which will immediately trigger you to froth at the mouth, hurry onto some Microsoft forum and post about how Microsoft is the devil. Here's some popular choices, many of them valid complaints: ('curl' and 'wget' on Windows override the real programs with M$ imposters). (gci doesn't support the parameters of either dir or ls). (aliases work differently to Unix shells). (almost everything works differently to Bash). (gci -recurse is frustratingly slow). (it doesn't have a CLI text editor like nano). (you don't understand that GNU and Unix utilities aren't "Bash"). (PowerShell remoting with enter-pssession and invoke-command aren't SSH). (there isn't any sudo because Windows isn't Linux). (Execution policies are annoying). (it doesn't use UTF8 everywhere always). (backslash, the one true Unix escape character isn't PowerShell's escape character). (Line endings aren't Unix line endings). (having to use sigils to disambiguate between shell and code is worse than Python for coding and worse than Bash/cmd for shelling). (You want > to be both numeric comparison and/or Unix shell IO redirect and it's not). (you hate Verb-Noun and the one true way is Noun-Verb). (byte streams don't pipeline well or quickly). (It's verbose which you hate, but the elastic syntax one-liners are unreadable which you hate, it should have exactly the right amount of verbosity which coincidentally is exactly the amount you are comfortable with).

Moving on from there, avoid falling for the tempting usermode filesystem equivalent, which is abandoned and only exists for backwards compatibility making everything slower. Avoid falling for the declarative host config system Desired State Configuration (DSC) which is semi-abandoned and only hangs around for backwards compatibility. Control your enthusiasm about .NET/C# LINQ in a shell, because nope. Prepare yourself for the weirdness of a programming language which has shell style dynamic scoping instead of lexical scoping, shell style output handling where all output goes to the pipeline, pipeline obsessed array unrolling which spills the contents of containers all over the floor if you aren't paying attention, having to learn that there's more to output than just stdout and stderr and that the host and pipeline are different outputs, and that there's a lot of non-powershelly .NET and Windows stuff poking through everywhere. Prepare your armoured-toe boots for a large number of footguns and bugs in what is an intricate and complex shell/scripting language mashup.

Moving on from there, it's a REPL:

    $x = 5
    $x + 3

Numeric literals in hex and binary:

    0xff
    0b1010

Strings in single quotes are literal:

    'foo $bar'

Strings in double quotes are not literal:

    $bar = 'hello'
    "foo $bar $( 4 + 3 )"

It's a shell:

    ping google.com

It's introspective:

    gcm ping    # get-command details of ping
    gcm p*      # commands starting with pi
    help gcm

Function calls don't use () or "return" because shell-style usage and behaviour, variable names and function calls aren't case sensitive:

    PS C:\> function test($Left, $Right) { $left + $right }
    PS C:\> test 4 6
    10

    PS C:\> test -Left 4 -Right 6  # named parameters
    PS C:\> test -L<ctrl+space>    # autocomplete

Reach for .NET libraries:

    PS C:\> [System.Ma<ctrl+space>  # autocomplete to explore and find System.Math
    PS C:\> [System.Math]::<ctrl+space>  # autocomplete to find e.g. [System.Math]::Pow(4, 5)

Basic data types:

    $array = 1,2,3
    $array | foreach { $_ }

    $hash = @{KeyA = 4; KeyB = "b"}
    $hash.keyC = 5
    $hash['keyD'] = get-childitem

    $x = 'KeyA'
    $hash.$x   # oooh

on Windows:

    gci | ogv    # out-gridview

Text filter ("grep ish"):

    gc words.txt | sls '[iou]'    # get-content and Select-String

Objects:

    ls | select Las<tab>  # tab complete will pickup the available properties
                          # on the objects coming through the pipeline, where possible

    ls | select LastWriteTime, FullName    # two properties, kept separate, no text parsing

    ls | % {$_.LastAccessTime.DayOfWeek}   # the access time is a .NET System.DateTime,
                                           # not a timestamp or text string.

    ipcsv data.csv | select col1, col3     # import-csv, select-object

    $hash = @{KeyA = 1; KeyB = "foo"}
    [pscustomobject] $hash                 # casting

(Objects are used as containers for multiple properties and keeping them separate; it's not a full "object oriented programming with inheritance and interfaces" kind of shell/language, although it has some nods to that).

I met Snover at a DevOps Days forever ago. We were talking about PowerShell, and I didn't know he was at our table. Super nice dude.