I have been doing development in some form or another for a long time. I learned Perl around 1997. I toyed with Basic and Pascal in high school and college, but it was nothing serious. Since that time, I have been fluent with C++, Perl, C#, and PowerShell while toying with languages like Javascript, Java, and a handful of others. Most recently I’ve tasked myself with learning Python for some new Linux scripts I have to write for work. When sitting down to learn a new language, I realize that there are topics that you must learn that go beyond the core syntax in order to feel comfortable tackling a development task (especially if you are more of an IT Pro than a developer). There are plenty of books and tutorials that can teach you the syntax of a language, but that’s really only the tip of the iceberg. I believe that these are very basic recipes that you need to learn about and store in your utility belt as quickly as possible in order to become efficient with a language.

Because, I’m seriously learning Python now, I thought I would document this as I go. And hey, since most people come to my blog to learn PowerShell and since I happen to know most of this stuff in Perl, I’ll explain how to do it in all three languages. I think it will also give people a fair assessment of the strengths, weaknesses, and overall quirks in the languages to help us have a more informed discussion about language in the wild. So without further ado, Part I of learning a new programming language.

Learning a New Programming Language – Perl, Python, and PowerShell – Part I – Executing a System Command

Generally the libraries that come with most languages these days can do everything you could ever dream of, but there are often occasions when the libraries fall short of a native command in Linux or Windows or there is a binary that does something better than you could ever code. For example, to rewrite robocopy.exe or rsync in any language is a waste of time. No offense intended to those who are writing libraries that duplicate their functionality – if they are robust and do what I need, I’ll be happy to use them :)

The following will break down the different ways to call external commands in each language while trying to satisfy the following three separate use cases that I generally have for system commands, i.e., execute and wait to get the exit code, execute and continue, execute and retrieve the output so that the text from STDOUT can be parsed or stored for later use. If you are not familiar with the term STDOUT it stands for standard output; this is the normal text output from a non-windowed command.

For the sake of this article, I am assuming that we are talking about external commands within a Windows operating system.

Perl

Perl has a few ways that you can interact with the system depending on what you expect to do with the external command. When it comes to simplicity, Perl gets it right.

Execute and wait to get the exit code – System

The following will run an ipconfig command and return an exit code to Perl:

$Exitcode = System("ipconfig");

If you want to suppress the output since you only care about the return code, you can redirect the command to NULL:

$Exitcode = System("iconfig >nul");

Execute and continue – Exec

Exec executes an external command and continues. You receive no indication about how the command ran.

Exec("robocopy d:\\folder1 d:\\folder2");

Again, if you want to suppress the output, you can redirect to NULL:

Exec("robocopy d:\\folder1 d:\\folder2 >nul");

Execute and retrieve the output – Backticks

If you want to capture the output of the command, you need to use backticks. For example, the following will populate the $ipinfo variable with the returned output from ipconfig. The print will display it to the screen.

$ipinfo = `ipconfig`;
Print $ipinfo;

PowerShell

PowerShell has many ways to launch system commands as well, but it’s mainly due to the fact that system commands can be run natively in PowerShell. Just type an executable and it will run. Because of this, you can spawn a background PowerShell job with start-job or execute a command on a remote computer with invoke-command. Mind you, I’m not showing all of the ways that you can execute a command, only the ways that I personally use over and over.

Execute and continue – Start-Process

Start-Process is very versatile. I personally like to use the alias start whenever I use this cmdlet. If you want to launch a command and not wait for the output, you can do the following:

start ipconfig

However, if you want to launch a command with command line arguments, you’ll need to use the ArgumentList parameter. This can be feel a bit awkward, but it’s not difficult. The following illustrates how to use robocopy with arguments in Start-Process:

start robocopy -ArgumentList 'D:\Folder1', 'D:\Folder2'

Execute and wait to get the exit code – -Wait & .Exitcode

You can wait for a command to finish by using the -Wait parameter

start robocopy -ArgumentList 'D:\Folder1', 'D:\Folder2' -Wait

There is also a passthru command that will give you the process object for the process you are starting. Here is my personal technique for getting the exitcode (yes, there is more than one way to get exit codes, this is just my preferred method):

(start ipconfig -wait -passthru).exitcode

Execute and retrieve the output

Generally if I need to parse the return text from a command I’ll just execute the command inline. For example to set the variable $ipconfig to the output of ipconfig.exe, you can do the following:

$ipconfig = ipconfig

In the above, $ipconfig is actually a collection of strings. Each line of the output is an element in the collection. If you expect it to return a single string, you’ll need to join the elements together with “`r`n”.

$ipconfig = $ipconfig -join "`r`n"

Finally, if you need to use spaces in your command, you’ll need to use the call operator (&). It won’t hurt anything to use the following for all of the commands you need from STDOUT, but just remember that it takes a collection of strings. For example, the following will work (note that d:\folder1 isn’t quoted, but it still works because it will attempt to convert anything it sees into a string):

& 'c:\path with spaces\robocopy.exe' d:\folder1 'd:\folder2'

However, this will fail:

& 'c:\path with spaces\robocopy.exe d:\folder1 d:\folder2'

To do the above, you would need to use Invoke-Expression, but I very rarely use that function as it is considered bad practice. The honest truth is that I care very little about injection attacks in most of my code. If I did care, I’m sure this article would be littered with it for discussion with every language.

For more information about all of the ways to call system commands from PowerShell, check out the following article.

Python

Python also has a few ways of calling external programs. According to the documentation, it appears that all use cases should now use the subprocess module. To launch an external command and return the exit code, you can simply use the call() method.

Execute and wait to get the exit code – Subprocess.Call()

Import subprocess
Exitcode = subprocess.call("ipconfig")

One thing to note, if you are using a function that exists in cmd.exe, the above won’t work. You must use the shell=True argument as well. For example, to use dir you would do the following:

Import subprocess
Exitcode = subprocess.call(["dir", "/ad", "c:\\"], shell=True)

You can do the above with a single string as well:

Exitcode = subprocess.call("dir /ad c:\\", shell=True)

Execute and continue – subprocess.Popen()

In order to execute without waiting for the exit code, you need to use the Popen() method of subprocess. Everything with this feels a bit more developery than it does with Perl and Python. However, the entire subprocess module is packed with a ton of things – useful things? I’m unsure as of yet, but here is what I know I need.

subprocess.Popen("ipconfig")

This technique shares STDOUT and partially STDIN. This gets really confusing when you do this:

t=subprocess.Popen("powershell")

The above is a fun party trick that will alternate between the PowerShell and Python prompts and interactive interpreters. It’s probably not a good idea to run interactive commands this way.

If you want to suppress the output from displaying, you can use the redirect to NULL like we saw earlier in this article when discussing Perl. However, there are some neat little tricks with the subprocess module that I think are worth learning about. You can set STDOUT to NULL when invoking Popen(). You can also set other streams such as STDERR this way. For example, the below will set STDOUT to NULL and then STDERR to STDOUT.

subprocess.Popen("ipconfig", stdout=(open(os.devnull, 'w')),stderr=subprocess.STDOUT)

Execute and retrieve the output – suprocess.PIPE

Similarly how we set stdout to null, we can also set stdout to equal PIPE so that we can later inspect the stdout property of the object returned by Popen(). The following illustrates this:

p = subprocess.Popen("ipconfig", stdout=subprocess.PIPE)
output=p.stdout.read()

Summary

Scripting languages by their nature are designed to be extremely versatile. There are sometimes just too many ways to do the same thing. This sort of sprawl can make things both extremely easy to use and extremely difficult to figure out at the same time. Because of this, it is really important to track the techniques that work for you and build up a library for your common use cases. Personally, I think the three use cases that I outlined in this article are more than sufficient: execute and wait to get the exit code, execute and continue, execute and retrieve the output. I think in the case of all three languages there are fair ways to accomplish these tasks.

The Python technique stands out because it is a bit cumbersome. To be fair, the Python subprocess module appears to be filled with tons o’ fun so it is understandable that it is a bit heavy to wield. To be further fair to Python, there are other ways of doing the above in Python using methods in modules and libraries that the documentation recommends you no longer use. Running commands in PowerShell at the command line and with the call (&) operator feels easy, but part of the reason for that is because PowerShell is intended to be a native shell to Windows. It almost seems unfair – like comparing how you run commands in ksh versus bash versus cmd. Actually, when you compare bash to PowerShell or even cmd to PowerShell, PowerShell has such quirks that it would probably lose that battle. As I think we’ll see in the upcoming articles, there are pros, cons, quirks, and angles to all 3 sides of this coin.