Checking for buffer overflows in unit tests

Functions that take a buffer and a corresponding buffer size as a parameter are ubiquitous. As an example, consider this function declaration:

  Routine Description:
    ...queries some value...

    BufferSize - size of buffer in WCHARs.
    Buffer     - buffer to store information in.

  Return Value:
    ERROR_SUCCESS on success.
    ERROR_INSUFFICIENT_BUFFER if the buffer is to small.
DWORD QueryInformation(
  __in UINT BufferSize,
  __out_ecount(BufferSize) PWSTR Buffer

If we are to write a testcase for this function, it is important to check that the function properly handles buffers of insufficient size. Let us say that we expect this function to return a file path, so its is safe to assume that a buffer size of 2 is guaranteed to be too small and a buffer size of MAX_PATH should be large enough. A straightforward testcase could thus look like this:

  WCHAR SmallBuf[ 2 ];
  WCHAR ReasonableBuf[ MAX_PATH ];

  // Use buffer that is guranteed to be too small.
    _countof( SmallBuf ), 
    SmallBuf ) );

  // Use a reasonable buffer size.
  TEST( ERROR_SUCCESS == QueryInformation( 
    _countof( ReasonableBuf ), 
    ReasonableBuf ) );

  ... test contents of ReasonableBuf ...

If the buffer size handling of QueryInformation is fundamentally flawed, this test case will catch the error. However, QueryInformation may contain a more subtle flaw such as an off-by-one error, that only manifests itself when the buffer is exactly one element too small. In this case odds are that this test case will fail to notice the mistake.

A handy idiom to overcome this is to use brute-force and just test all buffer sizes from 0 to a reasonable maximum size (MAX_PATH in this case). In order to catch buffer overflows, we use the same trick as employed by most debug heaps — we pad the end of the buffer with a guard value and make sure that the value is still intact after the function has used the buffer. An off-by-one error is now highly likely to be caught. The test case may look like this:

  WCHAR Buffer[ MAX_PATH ];
  UINT BufferSize;

  // Try all buffer sizes.
  BOOL WorkedAtLeastOnce = FALSE;
  for ( BufferSize = _countof( Buffer ); 
                                 BufferSize > 0; BufferSize-- )
    DWORD Result;

    // Write guard value to end of buffer..
    Buffer[ BufferSize - 1 ] = 0xDEAD;

    // Pass the buffer to the function under test.
    Result = QueryInformation( 
      BufferSize - 1,    // account for guard value.
      Buffer );

    // Check that guard value is intact - regardless of whether
    // function failed or not.
    TEST( Buffer[ BufferSize - 1 ] == 0xDEAD );

    // Expect either ERROR_INSUFFICIENT_BUFFER or 
        ( Result == ERROR_SUCCESS && 
                     ... check contents of Buffer ... ) );

    WorkedAtLeastOnce |= ( Result == ERROR_SUCCESS );
  TEST( WorkedAtLeastOnce );

Of course, it is now important to make sure that the function under test succeeded at least once, hence the additional check.



About me

Johannes Passing, M.Sc., living in Berlin, Germany.

Besides his consulting work, Johannes mainly focusses on Win32, COM, and NT kernel mode development, along with Java and .Net. He also is the author of cfix, a C/C++ unit testing framework for Win32 and NT kernel mode, Visual Assert, a Visual Studio Unit Testing-AddIn, and NTrace, a dynamic function boundary tracing toolkit for Windows NT/x86 kernel/user mode code.

Contact Johannes: jpassing (at) acm org

Johannes' GPG fingerprint is BBB1 1769 B82D CD07 D90A 57E8 9FE1 D441 F7A0 1BB1.

LinkedIn Profile
Xing Profile
Github Profile

%d bloggers like this: