Archive for the 'Tools' Category

cfix studio Beta 2 to add support for EXE-based unit tests

N.B. cfix studio was the code name of what has become Visual Assert

The biggest shortcoming of the current cfix studio version certainly is that it requires all tests be implemented in a DLL. Conceptually, keeping test cases separated from the remaining code certainly is a good idea — and implementing tests in a DLL is a way to accomplish this. However, there are many projects in which such separation is either not feasible or just too much effort.

The good news is that with Beta 2, this will finally change: EXEs become first class-citizens in cfix studio and it will not matter any more whether your tests are part of a DLL or EXE project — you can just put them where you think is appropriate.

Take a classic MFC/GUI application project as an example: It is pretty common for these kinds of projects that most, if not all, application logic is part of a single Visual Studio project that compiles into a single EXE. There may be some additional DLLs or LIBs, but by and large, the EXE itself is where most of the interesting things happen.

The upcoming Beta 2 release now allows you to implement all your unit tests as part of the same EXE project. This means that your tests have access to all classes, functions and resources that are part of the project — all of which you would not easily have access to if you implemented the tests in a separate DLL.

Of course, embedding unit tests into an executable raises two questions:

  1. How to strip the tests from the final release?
  2. How on earth will you be able to run these tests without having main() create windows, load files, play sounds, etc each time?

Thankfully, C/C++ has a preprocessor and Visual C++ has the “exclude from build” feature which allows you to exclude certain files whenever the project is built using a specific configuration. Using any of these two features, the first question is easily answered.

The second problem is more tricky — but thankfully, it has already been solved for you: When cfix studio runs unit tests, it is well aware of that running main() might have, let’s say interesting effects — so what it does is simple: It just makes sure that main() is never run! Not only does this ensure that the tests run silently, i.e. without windows popping up etc, it also has the benefit that all unit tests “see” the application in a pristine state: Rather than having to worry about which state main() has brought the application into, you can initialize and clean up any state you need in your Setup and Teardown functions1.

To make a long story short: You can write unit tests in EXE projects in exactly the same manner as you would in a DLL project. No special considerations needed, no project settings that need to be changed, no additional boilerplate code to write. And when you run the EXE outside cfix studio, i.e. hit F5 in Visual Studio or launch the EXE directly, you will not even notice that the EXE houses some unit tests — everything works as normal.

Sounds good? Then wait a few more days and see it in action!

1: Needless to say, all global variables are initialized, constructors are run, etc. All CRT initialization happens as normal; only main()/WinMain() is not run. And yes, it also works for apps that link statically to MFC and therefore do not have a “regular” WinMain().


More Context Menu Handlers for Everyday Use

Although Windows Explorer may actually not be the brightest spot of Windows, it is still, for most users, among the most often used pograms. Customizing it to speed up certain tasks is thus a natural desire.

A while ago, I wrote about how to extend the context menu by new commands that allow MSI packages to be installed/uninstalled with logfiles being created. The registry entries I used were:

Windows Registry Editor Version 5.00

@="&Logged Install"

@="msiexec.exe /l* \"%1-install.log\" /i \"%1\" %*"

@="L&ogged Uninstall"

@="msiexec.exe /l* \"%1-uninstall.log\" /x \"%1\" %*"

@="msiexec.exe /l* \"%1.log\" /i \"%1\" %*"

While I do not use Windows Installer every day, I am a heavy user of cmd.exe command prompts. Another set of custom verbs I use on my machines therefore deal with opening command line windows. Getting Windows Explorer to open a “normal” command prompt using the context menu is not hard and it has been demonstrated on various places. The idea becomes truly powerful, though, when the commands are specialized to open special kinds of command windows:

  • A plain command prompt
  • An elevated command prompt (using elevate.exe)
  • A WDK command prompt (WLH-chk)
  • A WDK command prompt (WLHA64-chk)
  • A Visual Studio 2005 command prompt
  • etc …

To distinguish the different types of consoles, I like to use different colors — The Visual Studio command prompt is white/green, the elevated prompt is green/blue, and so on. The following script puts it all together (mind the static paths):

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Directory\shell\Open DDK Console here (WLH-chk)]
@="Open DD&K Console here (WLH-chk)"

[HKEY_CLASSES_ROOT\Directory\shell\Open DDK Console here (WLH-chk)\command]
@="C:\\Windows\\System32\\cmd.exe /k C:\\WinDDK\\6000\\bin\\setenv.bat C:\\WinDDK\\6000\\ chk WLH && cd /D %1 && color 1f"

[HKEY_CLASSES_ROOT\Directory\shell\Open DDK Console here (WLHA64-chk)]
@="Open DD&K Console here (WLHA64-chk)"

[HKEY_CLASSES_ROOT\Directory\shell\Open DDK Console here (WLHA64-chk)\command]
@="C:\\Windows\\System32\\cmd.exe /k C:\\WinDDK\\6000\\bin\\setenv.bat C:\\WinDDK\\6000\\ chk AMD64 && cd /D %1 && color 1f"

[HKEY_CLASSES_ROOT\Directory\shell\Open Default Console here]
@="Open Default Conso&le here"

[HKEY_CLASSES_ROOT\Directory\shell\Open Default Console here\command]
@="cmd.exe /K \"title %1 && cd /D %1\""

[HKEY_CLASSES_ROOT\Directory\shell\Open Elevated Console here]
@="Open Ele&vated Console here"

[HKEY_CLASSES_ROOT\Directory\shell\Open Elevated Console here\command]
@="d:\\bin\\elevate.exe /K \"title %1 && color 1a && cd /D %1\""

[HKEY_CLASSES_ROOT\Directory\shell\Open VS.Net 2005 Console here]
@="Open VS.Net 200&5 Console here"

[HKEY_CLASSES_ROOT\Directory\shell\Open VS.Net 2005 Console here\command]
@="cmd.exe /K \"cd /D %1 && \"C:\\Program Files (x86)\\Microsoft Visual Studio 8\\VC\\vcvarsall.bat\" && color 2f\""

Finally, if you perform backups to the cloud from time to time and do want to upload unencrypted data or for other reasons encrypt specific files occasionaly, it may also be practical to have two GPG commands in your context menu — one to (symetrically) encrypt, and one to decrypt:

Windows Registry Editor Version 5.00





@="\"c:\\Program Files (x86)\\GNU\\GnuPG\\gpg.exe\" -d -i -o %1.plain %1"

@="Encrypt with GPG (symmetric)"

@="\"c:\\Program Files (x86)\\GNU\\GnuPG\\gpg.exe\" -c %1"

Introducing cfix studio, the Visual Studio AddIn for C/C++ Unit Testing

N.B. cfix studio was the code name of what has become Visual Assert

There is little doubt that native code, and C and C++ in particular, is here to stay. And still, it is pretty obvious that when it comes to tools and IDEs, it is the managed world that has gotten most attention from tool vendors over the past years.

While there are lots and lots of useful tools for native development, many of them probably even better than their managed counterparts, there are some areas where the managed language fraction is far ahead: One of these areas certainly is IDE support for unit testing.

JUnit for Eclipse, TestDriven.Net for Visual Studio and MS Test make test-driven development so much more convenient and efficient that it is almost ridiculous that using command line tools is still state of the art for C/C++ development.

That said, there is great news: With cfix studio, there finally is a solution filling in this gap! cfix studio, based on the cfix unit testing framework, is a Visual Studio-AddIn that allows you to easily write, manage, run, and debug your unit tests from within Visual Studio. No fiddling with command line tools, complex configuration, or boilerplate code required!

Among lots of other features, cfix studio also has first-class support for multi-architecture development – you can easily switch back and forth between 32-bit and 64-bit and can even mix tests of different architectures in a single test run. Needless to say, cfix studio, like cfix, is also fully compatible to WinUnit.

If that has caught your interest, you are invited to check out the first beta version of cfix studio:

Download cfix studio Beta 1

It is free, quick to install and comes with a set of example projects. Give it a try — and please let me know about all your crticism, suggestions and other feedback!

Here are some screenshots of cfix studio in action:

Test Explorer
The Test Explorer allows you to start a single or set of tests, the Run Window shows the results

Run Window
Run Window: Viewing test progress

Failed Assertion
Run Window: Viewing test results and details of a failed assertion

Debgging a failed assertion
When running in the debugger, a failed assertion will hit a breakpoint and the Run Window will show additional details

cfix 1.4 released

Today, a new version of cfix, the open source unit testing framework for user and kernel mode C and C++, has been released. cfix 1.4, in addition to the existing feature of allowing test runs to be restricted to specific fixtures, now also allows single testcases to be run in isolation, which can be a great aid in debugging. Besides several minor fixes, the cfix API has been slightly enhanced and cfix now degrades more gracefully in case of dbghelp-issues.

Updated cfix binaries and source code are now available for download

cfix 1.3.0 Released, Introducing WinUnit Compatibility

cfix 1.3, the latest version of the unit testing framework for C/C++ on Windows, has just been released. As announced in the last blog post, the major new feature of this release is WinUnit compatibility, i.e. the ability to recompile existing WinUnit test suites into cfix test suites without having to change a single line of code.

To demonstrate that this compatibility indeed works, consider the following simple example:

#include "WinUnit.h"

  WIN_ASSERT_STRING_EQUAL( "foo", "bar" "Descriptive message");

Compile it:

cl /I %CFIX_HOME%\include /LD /EHa /Zi winunittest.cpp /link /LIBPATH:%CFIX_HOME%\lib\i386

Or, in case %INCLUDE% and %LIB% already happen to be set properly:

cl /LD /EHa winunittest.cpp

Note that the only difference to compiling the test for WinUnit ist that a different include path is used — rather than the original winunit.h, cfix’ own winunit.h is used, which in turn implements the WinUnit API on top of the existing API.

The resulting DLL is a valid cfix DLL and its tests can be run in the usual manner. As the example contains a failing test, cfix will print the stack trace and error description to the console:

D:\sample>cfix32 -ts -z winunittest.dll
cfix version (fre)
(C) 2008-2009 - Johannes Passing -
[Failure]      winunittest.DummyTest.DummyTest
      winunittest.cpp(5): DummyTest

      Expression: Descriptive message: [foo] == [bar] (Expression: "foo" == "bar")
      Last Error: 0 (The operation completed successfully. )

      cfix!CfixpCaptureStackTrace +0x40
      cfix!CfixPeReportFailedAssertion +0xd2
      winunittest!cfixcc::Assertion::Relate ...
      winunittest!DummyTest +0x9c
      cfix!CfixsRunTestRoutine +0x33
      cfix!CfixsRunTestCaseMethod +0x27
      cfix!CfixsRunTestCase +0x25

Of course, cfix also supports WinUnit fixtures, as the following example, taken from the original WinUnit article on MSDN demonstrates:

#include "WinUnit.h"
#include <windows.h>

// Fixture must be declared.

  TCHAR s_tempFileName[MAX_PATH] = _T("");
  bool IsFileValid(TCHAR* fileName);

// Both SETUP and TEARDOWN must be present. 
  // This is the maximum size of the directory passed to GetTempFileName.
  const unsigned int maxTempPath = MAX_PATH - 14; 
  TCHAR tempPath[maxTempPath + 1] = _T("");
  DWORD charsWritten = GetTempPath(maxTempPath + 1, tempPath);
  // (charsWritten does not include null character)
  WIN_ASSERT_TRUE(charsWritten  0, 
    _T("GetTempPath failed."));

  // Create a temporary file
  UINT tempFileNumber = GetTempFileName(tempPath, _T("WUT"), 
    0, // This means the file will get created and closed.

  // Make sure that the file actually exists
    _T("File %s is invalid or does not exist."), s_tempFileName);

// TEARDOWN does the inverse of SETUP, as well as undoing 
// any side effects the tests could have caused.
  // Delete the temp file if it still exists.
  if (IsFileValid(s_tempFileName))
    // Ensure file is not read-only
    DWORD fileAttributes = GetFileAttributes(s_tempFileName);
    if (fileAttributes & FILE_ATTRIBUTE_READONLY)
          fileAttributes ^ FILE_ATTRIBUTE_READONLY),
          _T("Unable to undo read-only attribute of file %s."),

    // Since I'm testing DeleteFile, I use the alternative CRT file
    // deletion function in my cleanup.
      _T("Unable to delete file %s."), s_tempFileName);

  // Clear the temp file name.
    ARRAYSIZE(s_tempFileName) * sizeof(s_tempFileName[0]));

BEGIN_TESTF(DeleteFileShouldDeleteFileIfNotReadOnly, DeleteFileFixture)
    _T("DeleteFile did not delete %s correctly."),

BEGIN_TESTF(DeleteFileShouldFailIfFileIsReadOnly, DeleteFileFixture)
  // Set file to read-only
  DWORD fileAttributes = GetFileAttributes(s_tempFileName);
    fileAttributes | FILE_ATTRIBUTE_READONLY));

  // Verify that DeleteFile fails with ERROR_ACCESS_DENIED
  // (according to spec)

  bool IsFileValid(TCHAR* fileName)
    return (GetFileAttributes(fileName) != INVALID_FILE_ATTRIBUTES);

Compiling and running this test yields the expected output:

d:\sample>cl /I %CFIX_HOME%\include /LD /EHa /Zi fixture.cpp /link /LIBPATH:%CFIX_HOME%\lib\i386
d:\sample>cfix32 -ts -z fixture.dll
cfix version (fre)
(C) 2008-2009 - Johannes Passing -
[Success]      fixture.DeleteFileFixture.DeleteFileShouldDeleteFileIfNotReadOnly
[Success]      fixture.DeleteFileFixture.DeleteFileShouldFailIfFileIsReadOnly

       1 Fixtures
       2 Test cases
           2 succeeded
           0 failed
           0 inconclusive


All compatibility has its limitations — although cfix supports all major WinUnit constructs and assertions, there are a small number of known limitations, which are listed in the documentation. And although I am confident that most WinUnit code should compile and run just fine, it is, of course, possible, that further limitations pop up. In such cases, I would welcome an appropriate bug report and will try to fix cfix accordingly.


In order to have cfix be a fully adequate replacement for cfix, the cfix documentation has additionally been augmented to include a documentation of the entire WinUnit API.

Technical background

Technically, implementing the compatibility layer went rather smoothly. On the one hand, WinUnit and cfix have similar architectures, which makes many things easier. On the other hand, WinUnit has a clean, single public header file (contrast that to CppUnit!), which also simplified things. And as WinUnit is limited to C++, I was able to use C++ templates (in combination with some preprocessor macros) to implement the entire WinUnit compatibility layer without having to change a single line of cfix itself. Rather, the WinUnit macros/classes are all mapped onto the existing cfix C++ API, which already includes most of what was neccessary to implement the WinUnit functionality.


In case have been using WinUnit the past and have a set of existing WinUnit-based test suites, give cfix a try — Not only should it be a full-featured replacement for WinUnit, you can also expect to see, and benefit from new features in upcoming releases!

Last but not least, the release contains a number of minor bugfixes. So upgrading is recommended even if you do not intend to use the new WinUnit compatibility feature.

cfix can be downloaded here.

Effective Leak Detection with the Debug CRT and Application Verifier

Programming memory leaks in C or C++ is easy. Even careful programming often cannot avoid the little mistakes that finally end up in your program having a memory leak. Thankfully, however, there are plenty of helpful tools that assist in finding leaks as early as possible.

One especially helpful tool for leak detection is the debug CRT. Although the leak detection facilities provided by the debug CRT are not as far-reaching as those of, say, UMDH, using the debug CRT is probably the most friction-less way of identifying leaks.

Of course, the debug CRT will only track allocations of the CRT heap. That is, allocations performed using malloc or, in case of C++, the default operator new.

So how to enable allocation tracking? As it turns out, it is already enabled by default for the debug heap — so changing the CRT flags using _CrtSetDbgFlag usually is not even neccessary. All there is to do is to call _CrtDumpMemoryLeaks() at the end of the program.

When exactly is “the end of the program”? That depends on which CRT you use. Each CRT uses a separate heap and thus, must have its resources be tracked separately. If your application EXE and your DLLs all link against the DLL version of the CRT, the right moment to call _CrtDumpMemoryLeaks() is at the end of main(). If you use the static CRT, the right moment is when the respective module is about to unload — for an EXE, this is the end of main() again (atexit is another option). For a DLL, however, this is DllMain (in the DLL_PROCESS_DETACH case).

To illustrate how to make use of this CRT feature, consider the following leaky code:

#include <crtdbg.h>

class Widget
  int a;

  Widget() : a( 0 )

void UseWidget( Widget* w )

int __cdecl wmain()
  Widget* w = new Widget();
  UseWidget( w );

  return 0;

Running the debug build (i.e. a build using the debug CRT) of this program will yield the following output in the debugger:

Detected memory leaks!
Dumping objects ->
{124} normal block at 0x008C2880, 4 bytes long.
 Data:  00 00 00 00 
Object dump complete.

So we have a memory leak — allocation #124 is not freed. The default procedure to locate the leak now is to include a call to _CrtSetBreakAlloc( 124 ) in the program and run it in the debugger — it will break when allocation #124 is performed. While this practice is ok for smaller programs, it will fail as soon as your program is not fully deterministic any more — most likely because it uses multiple threads. So for many programs, this technique is pretty much worthless.

But before continueing on the topic of how to find the culprit, there is another catch to discuss. Let’s include this snippet of code into our program:

class WidgetHolder
  Widget* w;

  WidgetHolder() : w( new Widget() )

    delete w;

WidgetHolder widgetHolder;

No leak here — we are properly cleaning up. But let’s see what the debugger window tells:

Detected memory leaks!
Dumping objects ->
{125} normal block at 0x000328C0, 4 bytes long.
 Data:  00 00 00 00 
{124} normal block at 0x00032880, 4 bytes long.
 Data:  00 00 00 00 
Object dump complete.

Urgh. But the reason should be obvious — when main() is about to return, ~WidgetHolder has not run yet. As a consequence, WidgetHolder’s allocation has not been freed yet and _CrtDumpMemoryLeaks will treat this as a leak. Unfortunately, there is no good way to avoid such false positives. Of course, this only holds for C++. For C, this problem does not exist.

Ok, back to the problem of locating the leak. We know that allocation #124 is the problem, but assuming our program does more than the simplistic example, breaking on #124 during the next runs is likely to lead us to ever changing locations. So this information is worthless. That leaves the address of the leaked memory — 0x008C2880.

At this point, we can leverage the fact that the CRT heap is not really a heap but just a wrapper around the RTL heap. Therefore, we can use the incredibly powerful debugging facilities of the RTL heap to help us out.

In order to fix a leak, it is usually extremely helpful to locate the code having conducted the allocation. Once you have this information, it is often trivial to spot the missing free operation. As it turns out, the the RTL heap’s page heap feature offers this capability.

Open Application Verifier and enable Heap Checks for our application. By default, this enables the full page heap, but the normal page heap is enough for our case.

Note that for the following discussion, I assume you are using the Visual Studio debugger.

Set a breakpoint on the statement immediately following the _CrtDumpMemoryLeaks() statement and run the application until it breaks there. This time, the locations 0x02CDFFA0 and 0x02CDFF40 are reported as being leaked. Do not continue execution yet.

Rather, open WinDBG and attach noninvasively to the debugged process. VisualStudio is already attached, so we cannot perform a real attach, but a noninvasive attach does the trick.

In WinDBG, we now use the !heap extension to query page heap information:

0:000> !heap -p -a 0x02CDFF40
    address 02cdff40 found in
    _HEAP @ 2cd0000
      HEAP_ENTRY Size Prev Flags    UserPtr UserSize - state
        02cdfef8 000c 0000  [00]   02cdff20    00028 - (busy)
        Trace: 02dc
        776380d8 ntdll!RtlDebugAllocateHeap+0x00000030
        6a2fab29 MSVCR80D!malloc+0x00000019
        6a34908f MSVCR80D!operator new+0x0000000f
        4115c9 Leak!WidgetHolder::WidgetHolder+0x00000049
        415808 Leak!`dynamic initializer for 'widgetHolder''+0x00000028
        6a2e246a MSVCR80D!_initterm+0x0000001a
        411d33 Leak!__tmainCRTStartup+0x00000103
        411c1d Leak!wmainCRTStartup+0x0000000d
        767b19f1 kernel32!BaseThreadInitThunk+0x0000000e
        7764d109 ntdll!_RtlUserThreadStart+0x00000023

0:000> !heap -p -a 0x02CDFFA0
    address 02cdffa0 found in
    _HEAP @ 2cd0000
      HEAP_ENTRY Size Prev Flags    UserPtr UserSize - state
        02cdff58 000c 0000  [00]   02cdff80    00028 - (busy)
        Trace: 02e0
        776380d8 ntdll!RtlDebugAllocateHeap+0x00000030
        6a2fab29 MSVCR80D!malloc+0x00000019
        6a34908f MSVCR80D!operator new+0x0000000f
        411464 Leak!wmain+0x00000044
        411dd6 Leak!__tmainCRTStartup+0x000001a6
        411c1d Leak!wmainCRTStartup+0x0000000d
        767b19f1 kernel32!BaseThreadInitThunk+0x0000000e
        7764d109 ntdll!_RtlUserThreadStart+0x00000023

Aha, stack traces! The remaining analysis is almost trivial: 0x02CDFF40 has been allocated on behalf of WidgetHolder::WidgetHolder. WidgetHolder::WidgetHolder, however, is not indirectly invoked by wmain, but rather by MSVCR80D!_initterm! That is a strong indication for this being a global object that can be ignored in this analysis.

0x02CDFFA0, in turn, is allocated by wmain, so this is a real leak. But which allocation is it, exactly? lsa will tell us:

0:000> lsa Leak!wmain+0x00000044
    33: }
    35: int __cdecl wmain()
    36: {
>   37: 	Widget* w = new Widget();
    38: 	UseWidget( w );
    40: 	_CrtDumpMemoryLeaks();
    41: 	return 0;
    42: }

There you go, we have found the culprit.

Although simple, I have found this technique to be very effective in practice, as it enables you to find leaks as you develop your code. As Application Verifier should be enabled anyway for any application you are developing on, the technique also turns out to be a lot less laborious than it may seem. It almost certainly is a lot more convenient than routinely doing UMDH runs. To be fair, however, UMDH is able to catch more leaks (non CRT-leaks), so additionally using UMDH remains being a good idea.

Application Verifier: Thread cannot own a critical section

Whereas most Verifier error messages are rather descriptive, this one is not really:

VERIFIER STOP 00000200 : pid 0x2B4: 
Thread cannot own a critical section. 

0000104C : Thread ID.
05274FDC : Critical section address.
09D88FE0 : Critical section debug information address.
010135D4 : Critical section initialization stack trace.

Why should not this thread be allowed to own a critical section? What is Application Verifier trying to tell us? Luckily, the stack trace gives a hint:


So in fact, Application Verifier has detected that the thread currently being terminated is still owning a critical section. What a stupid error message for a simple problem…


About me

Johannes Passing, M.Sc., living in Berlin, Germany.

Besides his consulting work, Johannes mainly focusses on Win32, COM, and NT kernel mode development, along with Java and .Net. He also is the author of cfix, a C/C++ unit testing framework for Win32 and NT kernel mode, Visual Assert, a Visual Studio Unit Testing-AddIn, and NTrace, a dynamic function boundary tracing toolkit for Windows NT/x86 kernel/user mode code.

Contact Johannes: jpassing (at) acm org

Johannes' GPG fingerprint is BBB1 1769 B82D CD07 D90A 57E8 9FE1 D441 F7A0 1BB1.

LinkedIn Profile
Xing Profile
Github Profile