Posts Tagged 'xunit'

cfix 1.3.0 Released, Introducing WinUnit Compatibility

cfix 1.3, the latest version of the unit testing framework for C/C++ on Windows, has just been released. As announced in the last blog post, the major new feature of this release is WinUnit compatibility, i.e. the ability to recompile existing WinUnit test suites into cfix test suites without having to change a single line of code.

To demonstrate that this compatibility indeed works, consider the following simple example:

#include "WinUnit.h"

BEGIN_TEST(DummyTest)
{
  WIN_ASSERT_STRING_EQUAL( "foo", "bar" "Descriptive message");
}
END_TEST

Compile it:

cl /I %CFIX_HOME%\include /LD /EHa /Zi winunittest.cpp /link /LIBPATH:%CFIX_HOME%\lib\i386

Or, in case %INCLUDE% and %LIB% already happen to be set properly:

cl /LD /EHa winunittest.cpp

Note that the only difference to compiling the test for WinUnit ist that a different include path is used — rather than the original winunit.h, cfix’ own winunit.h is used, which in turn implements the WinUnit API on top of the existing API.

The resulting DLL is a valid cfix DLL and its tests can be run in the usual manner. As the example contains a failing test, cfix will print the stack trace and error description to the console:

D:\sample>cfix32 -ts -z winunittest.dll
cfix version 1.3.0.3340 (fre)
(C) 2008-2009 - Johannes Passing - http://www.cfix-testing.org/
[Failure]      winunittest.DummyTest.DummyTest
      winunittest.cpp(5): DummyTest

      Expression: Descriptive message: [foo] == [bar] (Expression: "foo" == "bar")
      Last Error: 0 (The operation completed successfully. )

      cfix!CfixpCaptureStackTrace +0x40
      cfix!CfixPeReportFailedAssertion +0xd2
      winunittest!cfixcc::Assertion::Fail<std::...
      winunittest!cfixcc::Assertion::Relate<std...
      winunittest!cfixcc::Assertion::Relate ...
      winunittest!cfixcc::Assertion::RelateStri...
      winunittest!DummyTest +0x9c
      cfix!CfixsRunTestRoutine +0x33
      cfix!CfixsRunTestCaseMethod +0x27
      cfix!CfixsRunTestCase +0x25
      ...

Of course, cfix also supports WinUnit fixtures, as the following example, taken from the original WinUnit article on MSDN demonstrates:

#include "WinUnit.h"
#include <windows.h>

// Fixture must be declared.
FIXTURE(DeleteFileFixture);

namespace
{
  TCHAR s_tempFileName[MAX_PATH] = _T("");
  bool IsFileValid(TCHAR* fileName);
}

// Both SETUP and TEARDOWN must be present. 
SETUP(DeleteFileFixture)
{
  // This is the maximum size of the directory passed to GetTempFileName.
  const unsigned int maxTempPath = MAX_PATH - 14; 
  TCHAR tempPath[maxTempPath + 1] = _T("");
  DWORD charsWritten = GetTempPath(maxTempPath + 1, tempPath);
  // (charsWritten does not include null character)
  WIN_ASSERT_TRUE(charsWritten  0, 
    _T("GetTempPath failed."));

  // Create a temporary file
  UINT tempFileNumber = GetTempFileName(tempPath, _T("WUT"), 
    0, // This means the file will get created and closed.
    s_tempFileName);

  // Make sure that the file actually exists
  WIN_ASSERT_WINAPI_SUCCESS(IsFileValid(s_tempFileName), 
    _T("File %s is invalid or does not exist."), s_tempFileName);
}

// TEARDOWN does the inverse of SETUP, as well as undoing 
// any side effects the tests could have caused.
TEARDOWN(DeleteFileFixture)
{
  // Delete the temp file if it still exists.
  if (IsFileValid(s_tempFileName))
  {
    // Ensure file is not read-only
    DWORD fileAttributes = GetFileAttributes(s_tempFileName);
    if (fileAttributes & FILE_ATTRIBUTE_READONLY)
    {
      WIN_ASSERT_WINAPI_SUCCESS(
        SetFileAttributes(s_tempFileName, 
          fileAttributes ^ FILE_ATTRIBUTE_READONLY),
          _T("Unable to undo read-only attribute of file %s."),
          s_tempFileName);
    }

    // Since I'm testing DeleteFile, I use the alternative CRT file
    // deletion function in my cleanup.
    WIN_ASSERT_ZERO(_tremove(s_tempFileName), 
      _T("Unable to delete file %s."), s_tempFileName);
  }

  // Clear the temp file name.
  ZeroMemory(s_tempFileName, 
    ARRAYSIZE(s_tempFileName) * sizeof(s_tempFileName[0]));
}

BEGIN_TESTF(DeleteFileShouldDeleteFileIfNotReadOnly, DeleteFileFixture)
{
  WIN_ASSERT_WINAPI_SUCCESS(DeleteFile(s_tempFileName));
  WIN_ASSERT_FALSE(IsFileValid(s_tempFileName), 
    _T("DeleteFile did not delete %s correctly."),
    s_tempFileName);
}
END_TESTF

BEGIN_TESTF(DeleteFileShouldFailIfFileIsReadOnly, DeleteFileFixture)
{
  // Set file to read-only
  DWORD fileAttributes = GetFileAttributes(s_tempFileName);
  WIN_ASSERT_WINAPI_SUCCESS(
    SetFileAttributes(s_tempFileName, 
    fileAttributes | FILE_ATTRIBUTE_READONLY));

  // Verify that DeleteFile fails with ERROR_ACCESS_DENIED
  // (according to spec)
  WIN_ASSERT_FALSE(DeleteFile(s_tempFileName));
  WIN_ASSERT_EQUAL(ERROR_ACCESS_DENIED, GetLastError());
}
END_TESTF

namespace
{
  bool IsFileValid(TCHAR* fileName)
  {
    return (GetFileAttributes(fileName) != INVALID_FILE_ATTRIBUTES);
  }
}

Compiling and running this test yields the expected output:

d:\sample>cl /I %CFIX_HOME%\include /LD /EHa /Zi fixture.cpp /link /LIBPATH:%CFIX_HOME%\lib\i386
d:\sample>cfix32 -ts -z fixture.dll
cfix version 1.3.0.3340 (fre)
(C) 2008-2009 - Johannes Passing - http://www.cfix-testing.org/
[Success]      fixture.DeleteFileFixture.DeleteFileShouldDeleteFileIfNotReadOnly
[Success]      fixture.DeleteFileFixture.DeleteFileShouldFailIfFileIsReadOnly


       1 Fixtures
       2 Test cases
           2 succeeded
           0 failed
           0 inconclusive

Limitations

All compatibility has its limitations — although cfix supports all major WinUnit constructs and assertions, there are a small number of known limitations, which are listed in the documentation. And although I am confident that most WinUnit code should compile and run just fine, it is, of course, possible, that further limitations pop up. In such cases, I would welcome an appropriate bug report and will try to fix cfix accordingly.

Documentation

In order to have cfix be a fully adequate replacement for cfix, the cfix documentation has additionally been augmented to include a documentation of the entire WinUnit API.

Technical background

Technically, implementing the compatibility layer went rather smoothly. On the one hand, WinUnit and cfix have similar architectures, which makes many things easier. On the other hand, WinUnit has a clean, single public header file (contrast that to CppUnit!), which also simplified things. And as WinUnit is limited to C++, I was able to use C++ templates (in combination with some preprocessor macros) to implement the entire WinUnit compatibility layer without having to change a single line of cfix itself. Rather, the WinUnit macros/classes are all mapped onto the existing cfix C++ API, which already includes most of what was neccessary to implement the WinUnit functionality.

Conclusion

In case have been using WinUnit the past and have a set of existing WinUnit-based test suites, give cfix a try — Not only should it be a full-featured replacement for WinUnit, you can also expect to see, and benefit from new features in upcoming releases!

Last but not least, the release contains a number of minor bugfixes. So upgrading is recommended even if you do not intend to use the new WinUnit compatibility feature.

cfix can be downloaded here.

Advertisements

Embracing WinUnit

Around two years ago, in early 2007, after having read about, having tried, and finally having dismissed numerous existing unit testing frameworks for C, I resigned and started thinking about creating a new unit testing framework. Having been accustomed to NUnit and JUnit, I found most frameworks clumsy to use — some “frameworks” like MinUnit are a joke, some frameworks like CUnit require lots of boilerplate code to be written, some frameworks only support C++ but not C, and some manage to combine the worst properties of them all (CppUnit).

However, it was not before end of 2007 until I finally found the time to actually start working on what would later become cfix. About half way through the initial coding phase, in the February 2007 issue, MSDN magazine featured the article Simplified Unit Testing for Native C++ Applications, introducing WinUnit, a unit testing framework for unmanaged C++.

Enter WinUnit

On the one hand, it was nice to see someone thinking the same about current unit testing frameworks and coming up with a new solution. But given the effort that had already gone into cfix, I was not exactly amused about this article — after all, WinUnit implements one of the core ideas of cfix, namely, to separate the test runner (winunit.exe/cfix32.exe) from the actual tests (DLLs) and using PE file introspection to identify fixtures and test cases. So although WinUnit already generated significant positive feedback, I continued development of cfix — not only would cfix at least add the benefit of supporting C in addition to C++, after investigating WinUnit a bit, I still saw lots of room for improvement.

Now, one year later, the situation has changed. Contrary to what one might have expected, WinUnit has not evolved into a serious project — it has not gotten past the MSDN article and the accomanying download link: No new features, no fixes, no blog, no community — by now, WinUnit seems pretty much dead to me. Sure, nothing prevents you from keep using WinUnit, but using tools for which no further development seems to take place is somewhat dissatisfying to me.

Good News

Given this situation and the architectural simlarity of both testing frameworks, it therefore just makes sense to take the next logical step and have cfix embrace WinUnit!

That is, the upcoming cfix 1.3 release will be compatible to WinUnit by allowing developers to take existing test cases written against the WinUnit API and recompile them into cfix test cases without requiring any code to be changed.

With such compatibility in place, transitioning from WinUnit to cfix will thus become a snap. Better yet, because no code has to be changed, the option to switch back and forth between cfix and WinUnit is retained, giving existing WinUnit users maximum flexibility at minimal risk.

The 1.3 release of cfix is due in a couple of days. Once released, I will get a bit more into detail about WinUnit compatibility.

By the way…

today is the first anniversary of cfix :)

Introducing cfix, a unit testing framework for C/C++ on Win32

I am happy to announce that the unit testing framework cfix I have developed over the past weeks and months is now available on Sourceforge as a first release candidate. It is licensed under the GPL, both binaries and source code are available.

Background

cfix is a framework for authoring and running xUnit-like testcases written in C or C++. The aim of the tool is to provide a development experience very similar to frameworks such as JUnit or NUnit. Due to the nature of C and C++, current unit testing frameworks for C and C++ hardly reach the ease of use of JUnit or NUnit. In particular, it is noticable that significantly more code has to be written to implement a test suite.

Languages like Java and the various .Net languages, as well as scripting languages, all provide reflection facilities. Unit testing frameworks for these languages can therefore rely on reflective features in order to minimize the amount of code required to define a test suite. Provided a library, the framework can find and identify test cases and is able to selectively run them.

Lacking similar reflective facilities, the route most unit testing frameworks for C and C++ have chosen is to oblige the developer to explicitly define test cases and fixtures. Taking CUnit as an example, the developer has to make explictit function calls to define a test suite, add test cases to the suite and to finally run this suite. CppUnit simplifies this a bit, but still requires the developer to implement quite some amount of initialization code. Another important drawback of this approach is the fact that no real separation between test code and test runner is done. Often, even the choice whether to use a graphical or a console frontend for running test is tied to this initialization code.

The aim of cfix is to overcome these limitations and to provide an easy to use, yet powerful unit testing framework for C and C++. Rather than having to write tedious initialization code, the developer merely has to define a fixture using the macros shown in the example below.

Like JUnit and NUnit (but unlike all frameworks for C/C++, with WinUnit being the notable exception), cfix implements a separation in between test code and the test runner. Unit tests are compiled into a DLL rather than an EXE file. This DLL thus only contains the code to be tested. The entire logic and user interface required to actually run unit tests is contained in the cfix-provided testrunner, cfix32.exe/cfix64.exe.

Moreover, cfix has been designed to make debugging of unit tests as easy as possible — the frameworks notices when a debugger is present and allows you to break in as soon as some assertion fails.

Example

So let’s have a look at a minimalistic cfix unit test:

#include 

void Test1()
{
  int a = 1;
  int b = 1;
  CFIX_ASSERT( a + b == 2 );
}

CFIX_BEGIN_FIXTURE( MyMinimalisticFixture )
  CFIX_FIXTURE_ENTRY( Test1 )
CFIX_END_FIXTURE()

After compiling and linking the code into a DLL using

cl /Zi /LD cfix-sample.c

the unit test can be run with the testrunner cfix32 (or cfix64):

[Success]      cfix-sample.dll.MyMinimalisticFixture.Test1

Needless to say, cfix is not limited to such simple tests — have a look at the Tutorial to learn more about the features, installation and usage of cfix.

Enough background information — go ahead and…

Of course, I’d love to read your feedback — you can reach me via passing at users.sourceforge.net!


Categories




About me

Johannes Passing, M.Sc., living in Berlin, Germany.

Besides his consulting work, Johannes mainly focusses on Win32, COM, and NT kernel mode development, along with Java and .Net. He also is the author of cfix, a C/C++ unit testing framework for Win32 and NT kernel mode, Visual Assert, a Visual Studio Unit Testing-AddIn, and NTrace, a dynamic function boundary tracing toolkit for Windows NT/x86 kernel/user mode code.

Contact Johannes: jpassing (at) acm org

Johannes' GPG fingerprint is BBB1 1769 B82D CD07 D90A 57E8 9FE1 D441 F7A0 1BB1.

LinkedIn Profile
Xing Profile
Github Profile
Advertisements