Saturday, February 4, 2012

Windows Update Error Code 0x80073712

I have several Windows 7 VMs which I use for various dev testing.  For the past 9 months all of them having been failing during Windows Update.  Most updates apply, but Service Pack 1 consistently errors during installation with the unknown error code 0x80073712.

Microsoft has a well written KB explaining steps you can take when this issue occurs: 

http://support.microsoft.com/kb/957310

I went through each step, running the checksur.exe and sfc.exe tools and downloading the stand alone Service Pack 1 installer.  All failed.  Next, I tried step 4 attempting to upgrade with the original Windows 7 CD.  Note, that the CD image must be the exact same used to install the original OS.  Same release flavor, Enterprise, in my case, and same SP level.  I thought I'd want to upgrade using the ISO image with SP1 built in, but no that refuses to even attempt an upgrade.

Finally, with the correct ISO image I got the clue that no previous error message had given:

  • There is not enough free space on partition (C:). A total of 16372 megabytes (MB) of free disk space is required. Try Disk Cleanup, or moving files to an external location such as a CD, DVD, or external hard drive, and then restart the installation.

I didn't have enough free space on the drive to install the Service Pack.  At first I was surprised because I had 10GB free, but in fact Win7 SP1 requires about 16GB to install.  Next, I was annoyed that this simple error wasn't raised by Windows Update in the first place.

I keep my drive space pretty sparse on my test VMs because they don't have many applications or data stored on them.  But a 40GB primary drive size is not enough for Windows 7.  I expanded it to 80GB using VMWare Workstation 8's built-in "Expand Disk" tools then used sysresccd and gparted to expand the NTFS partition across the whole disk.

Success!  Windows Update installs Service Pack 1 with no problems.  I hope this helps you or someone else avoid a half-day of IT induced profanity.

Friday, January 20, 2012

Using shquote() to Combat Command Injection

I've been incredibly impressed with FreeBSD's shquote(3) function as a singular way to prevent command injection in a server process.  I'm currently working on a RESTful API (aren't we all), backed in part by a server daemon written in C.  The largest threat to any networked API is user input.  A server developer must assume that all input is malicious until proven valid.  There are many types of injection attacks against web services; I've been focused on Unix command injection today.

Let's say you're building a service which takes a directory path as input and creates this directory path on the server's file system.  In C we can just call:

1 void mkdir_recursive(char* path) {
2      char* cmd;
3      asprintf(&cmd, "mkdir -p %s", path);
4      system(cmd);
5 }

Now wait!  Using system() is bad!  It's prone to command injection attacks and there's usually always a programmable API to do the same thing in a safer, cleaner way.

I agree, but I am interested in exploring command injection and mkdir -p is so convenient for recursive directory creation.  There's no easy libc alternative and writing it myself would take 10x as much code.  Yes, I'm complaining about 3 lines vs 30, but Larry Wall says laziness is one of a programmers greatest virtues.

So normally to protect this simple code from command injection we'd validate the input for shell meta-characters.  We must check for ; & ( ) ` | > < $.  These could allow an attacker to append an arbitrary command onto our simple mkdir, read or write files, and access environment variables.  For instance we could receive a path which would make the above command look like this:

mkdir -p a/new/dir ; cat /etc/passwd > /usr/local/apache/htdocs/passwd

Which unfiltered would cause the server's local user list to now be published in a publicly accessible directory.  To further cause issues, in my particular use case, all of the above characters are valid for Unix directory paths.  So simply removing them from the user's input would be incorrect behavior.

In comes shquote().  It protects against all of these command injections and allows these characters to be used in path names, because it puts everything in strong quotes.  With a refactor of our above code snippet:

1 void mkdir_recursive(char* path) {
2      char *cmd;
3      int len = shquote(path, NULL, 0) + 1;
4      char *safe_path = malloc(len);
5      shquote(path, &safe_path, len);
6      asprintf(&cmd, "mkdir -p %s", path);
7      system(cmd);
8 }

The execute command now looks like this:

mkdir -p 'a/new/dir ; cat /etc/passwd > /usr/local/apache/htdocs/passwd'

Which will safely create 10 new directories, instead of calling 2 commands.

# while [ "`ls`" ]; do ls -F && cd "`ls`"; done
a/
new/
dir ; cat /
etc/
passwd > /
usr/
local/
apache/
htdocs/
passwd/

Surely we can still trick shquote() with embedded \ or ' characters right?  Let's try adding intermediate quotes to break up the two commands:

a/new/dir' ; 'cat /etc/passwd > /usr/local/apache/htdocs/passwd

becomes

mkdir -p 'a/new/dir'\'' ; '\''cat /etc/passwd > /usr/local/apache/htdocs/passwd'

# while [ "`ls`" ]; do ls -F && cd "`ls`"; done
a/
new/
dir' ; 'cat /
etc/
...

That didn't give us an exploit.  So let's try escaping those intermediate quotes:

a/new/dir\' ; \'cat /etc/passwd > /usr/local/apache/htdocs/passwd

becomes

mkdir -p 'a/new/dir\'\'' ; \'\''cat /etc/passwd > /usr/local/apache/htdocs/passwd'

# while [ "`ls`" ]; do ls -F && cd "`ls`"; done
a/
new/
dir\' ; \'cat /
etc/
...

My simple function handles them both correctly with successive strong quote wrappings.  It still isn't protected from other attacks like path traversal trickery using embedded .. but shquote() goes a long way in santizing input.  I couldn't think of a way to break it.  Please tell me if you can.

Monday, January 2, 2012

64-bit Stack Padding

I tracked down an interesting bug recently worth noting due to the lessons I learned about 64-bit computing.  The bug was in some unit test code, essentially testing different types of a custom object. Here's a simplified example of the bug in a standalone program:
 1 #include <stdio.h>
 2 #include <stdlib.h>
 3 
 4 enum type {
 5         T0 = 0,
 6         T1,
 7         T2,
 8         T3,
 9         T4,
10         T5,
11         TMAX = T5
12 };
13 
14 struct typed_item {
15         enum type t;
16         void *data;
17 };
18 
19 void test1(void)
20 {
21         struct typed_item *list[TMAX];
22         char *success_string = "Success";
23         int i = 0;
24 
25         for (i = T0; i <= TMAX; i++) {
26                 list[i] = malloc(sizeof(struct typed_item));
27                 list[i]->t = i;
28         } 
29          
30         /* do tests on list */
31          
32         printf("%s\n", success_string);
33 
34         /* cleanup */
35 }
36 
37 int main(void) 
38 {   
39         test1();
40 }
Here's the console output:

sdann32# gcc stack.c && ./a.out

sdann32#

I expected "Success" but instead get a non-printable character and a newline.  Can you spot the bug?

-        struct typed_item *list[TMAX];
+        struct typed_item *list[TMAX+1];

This is a classic off-by-one error causing stack corruption.  The for loop writes one entry past the end of the array and modifies the next variable on the stack.  This was straightforward to track down in gdb.  What was tricky, is that I couldn't repro such a simple bug consistently.  When I ran this test on a different machine, everything worked fine.

My company is in the process of converting from a 32-bit userspace, to a 64-bit userspace.  Because of the way our feature branches are managed in a source tree, often I'm switching between development on a 32-bit build and a 64-bit build.  This bug only reproed on a 32-bit platform as seen above.  The 64-bit system succeeded:

sdann64# gcc stack.c && ./a.out
Success
sdann64#

Since my initial belief was that this issue was caused by stack corruption, I dropped into gdb on the 64-bit machine to examine the test1() stack variables.  To see changes in the stack variables easier, I've initialized the array with a known value of 2.

sdann64# gcc stack.c -g3 -O0
sdann64# gdb ./a.out       
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
This GDB was configured as "amd64-marcel-freebsd"...
(gdb) b test1
Breakpoint 1 at 0x4005d9: file stack.c, line 21.
(gdb) run
Breakpoint 1, test1 () at stack.c:21
21        struct typed_item *list[TMAX] = { (void*)2, (void*)2, (void*)2, (void*)2, (void*)2 };
(gdb) n
22        char *success_string = "Success";
(gdb) n
23        int i = 0;
(gdb) n
25        for (i = T0; i <= TMAX; i++) {
(gdb) p $rbp
$1 = (void *) 0x7fffffffe870
(gdb) p $rsp
$2 = (void *) 0x7fffffffe820
(gdb) x /10xg $rsp
0x7fffffffe820:    0x0000000800902040    0x0000000000000002
0x7fffffffe830:    0x0000000000000002    0x0000000000000002
0x7fffffffe840:    0x0000000000000002    0x0000000000000000
0x7fffffffe850:    0x000000000040065d    0x00000000006fb08c
0x7fffffffe860:    0x0000000000000001    0x0000000000000001

Once past the local variable initialization instructions I check the frame base pointer ($rbp) and the frame stack pointer ($rsp) and see that there's a 0x50 difference between them, or 80 bytes.  Listing this portion of the stack gives the local variables for the function.

(gdb) x /10xg $rsp
0x7fffffffe820:    0x0000000000000002    0x0000000000000002
0x7fffffffe830:    0x0000000000000002    0x0000000000000002
0x7fffffffe840:    0x0000000000000002    0x0000000000000000
0x7fffffffe850:    0x000000000040065d    0x00000000006fb08c
0x7fffffffe860:    0x0000000000000001    0x0000000000000001

Annotated with color above, the red section are saved registers and temporary variables.  The three local variables are highlighted as such:

struct typed_item *list[TMAX];

char *success_string = "Success";

int i = 0;


Just as I suspected, when asking the compiler for 80 bytes of stack storage space, the allocation is padded to 96 bytes.  It is this extra 16 bytes which allows my accidental off-by-one overwrite to have no functional affect on the program.  I've never run into this issue on 32-bit platforms, so I'm now mindful that 64-bit stack padding can mask bugs that appear on other architectures.

(All testing done with Intel x64 processor, FreeBSD 7.3, and gcc 4.2.)