Getting started with Expect scripting
I’ve been playing with Expect lately. Expect is an extension of the TCL scripting language developed in the 1990s. It main purpose in life is to automate terminal interactions and it does that job very well.
I spend most of my day in a shell and automate as much as humanly possible so that I can be as lazy as humanly possible. Using tools like ssh and scp it’s very easy to automate simple commands and simple file transfers. But when these tasks become complex enough that they need to respond to terminal prompts, or provide arbitrary changing input, those tools fall apart.
My particular use case was a need to grep through logs on multiple Linux servers looking for PAN (credit card) data as part of a PCI compliance exercise. This would be a trivial task to achieve using plain old ssh except for the fact that I use a Yubi key to log on to the servers and I have to go through a bastion host, so every login happens twice. I need to interactively provide the PIN for my Yubi at each login. The same problem exists for encrypted public keys. For a while I just copied my Yubi PIN and pasted it at every prompt, but that became a pain pretty quickly so I started casting around for other options.
There’s a few solutions that specifically address the ssh password problem such as sshpass and ssh-agent, but I wanted something more generic that I could continue to use once logged in to do complex things. And I live for portability. Enter Expect.
Expect can handle some very complex interactions, but at its core is a very simple game of catch. Tell your script to “expect” some output and then “send” the appropriate input. Here’s a very simple example how to SSH into a server with a password.
#!/usr/bin/expect -f set host [lindex $argv 1] set password “password” spawn ssh “$host” expect “password: “ send “$password\r” interact
You can chmod this script to make it executable
chmod +x script and run it using the format
interact command is important. It tells Expect to release the terminal so you can type things and do things. Without the interact command, you won’t be able to…well…interact. If you’re just planning on executing a lot of commands without any interaction that is fine.
This little snippet is meant as an example and as such it stores the password in the script. Obviously, that’s crap and you should not do that, but what options are there? One option is to store the credential in a separate file that does not comprise the codebase. Expect can read external files using this format (assuming a file named PASS with just one single line it, your password):
set fp [open “PASS” r] set PASS [read $fp]
You can then replace the
send "$password\r" with
Another useful feature is Expect’s logging function. In my case, I am running a bunch of greps on log files and I want to save the output locally to my machine so I can review it later. The Expect command
log_file works perfectly for this.
Using the command
log_file by itself will write a log file into the current directory on your local machine. You can also specify a log file using the format
To start logging, you simply put the
log_file command in the script where you’d like the logging to start. If you want to end logging at some point, put
log_file by itself again (regardless of whether you specified at log file name or not when you began logging). Expect is smart enough to know it already has a log file open and knows to close it when it encounters a single
log_file all by itself.
Let’s put this all together to log into a machine, grep the Apache log file for something, and then save the log for review.
#!/usr/bin/expect -f set host [lindex $argv 1] set fp [open “PASS” r] set PASS [read $fp] spawn ssh “$host” expect “password: “ send “$PASS\r” expect “$ “ log_file website_logins.log send “grep login /var/log/httpd/access.log\r” log_file
./script host.com and sit back. In a few seconds you should have a file named website_logins.log in the script directory that contains the output of your grep.
There’s a lot more to learn, but these basic building blocks will get you started.