Posts Tagged 'tdd'

Visual Assert Beta 3 released

A third beta release of Visual Assert is now available for download on www.visualassert.com.

Visual Assert, in case you have not tried it yet, is an Add-In for Visual Studio that adds unit testing capabilities to the Visual C++ IDE: Based on the cfix unit testing framework, Visual Assert allows unit tests to be written, run, and debugged from within the IDE. Pretty much like Junit/Eclipse, TestDriven.Net or MSTest, but for real, native code — code written in C or C++.

Bugs, bugs, bugs, bugs

Alas, there were still a few of them in the previous two beta releases. Luckily though, almost all I received from users or found by internal testing were relatively minor in nature. Still, I want Visual Assert to be as high quality as possible and decided to add this third beta release into the schedule and take the time to focus on — you guessed it — bugfixing, bugfixing, bugfixing, and bugfixing.

Speaking of bug reports, I have to thank all users of Visual Assert and cfix who reported bugs, suggested new features or provided general feedback. Your input has been, and still is highly appreciated. Although I had to postpone any feature suggestions to a later release, I tried hard to resolve all bugs and have them fixed in this new release.

Download, Try it, Share Your Opinion

Of course, using the new Beta version is free. So whether you have already used the previous beta or not, whether you are a unit testing newbie or write unit tests on a daily basis, be sure to give the new version a try. And of course, do not forget to let me know about your feedback, suggestions, found bugs, etc.!

Advertisements

Visual Assert Beta 2 Released

The Beta 2 release of Visual Assert (formerly named cfix studio) is now available for download. The release marks a major step in the development of Visual Assert for that it not only comprises a number of bugfixes but also introduces major new features. The two most important certainly are support for EXE targets and Wizard assistance.

Support for EXE Targets

As announced in a previous post and also discussed in the post about the cfix 1.5 release, Visual Assert now fully supports unit tets emedded in EXE modules.

Previous releases required all unit tests to be compiled and linked into DLLs. In fact, the usage of DLLs has many advantages and therefore remains the recommended practice. However, for certain projects, such a requirement can turn out to be a true obstacle: Whenever the code to be tested is not exported from a DLL or part of a static library (LIB), accessing this code from within such a test DLL could become quite a challenge.

The fact that Visual Assert Beta 2 now fully supports EXE modules means the following: You can now place your unit tests wherever you think they fit best. Whether they are part of a DLL or an EXE, whether you create separate “unit test” projects or intermingle your test code with other code — it is now all up to you. Wherever you placed your tests, Visual Assert will find them and will provide a consistent user experience.

And the best part of the support for EXE modules is that it is totally non-intrusive: You do not have to change your main/WinMain function, much less any other code or build settings. And when run “outside” Visual Assert, i.e. launched directly or in the Visual Studio Debugger, the application will behave as normal.

Needless to say, the cfix 1.5 command line test runners, cfix32.exe and cfix64.exe also have been updated to properly deal with EXE modules.

The Wizard

Although neither the WinUnit API nor the cfix C and C++ API require much boilerplate code to be written, there still is some amount of code that more or less all unit tests share. Thanks to the new Wizard, you can now have Visual Assert generate this code for you. This really helps creating new fixtures more quickly!

Download, Try it, Share Your Opinion

Of course, using the new Beta version is free. So whether you are a full time tester or a unit testing sceptic, download the new release and try it by yourself. And of course, your feedback, both positive and negative, is always welcome and can be posted here.

Download Visual Assert Beta 2

Embracing WinUnit

Around two years ago, in early 2007, after having read about, having tried, and finally having dismissed numerous existing unit testing frameworks for C, I resigned and started thinking about creating a new unit testing framework. Having been accustomed to NUnit and JUnit, I found most frameworks clumsy to use — some “frameworks” like MinUnit are a joke, some frameworks like CUnit require lots of boilerplate code to be written, some frameworks only support C++ but not C, and some manage to combine the worst properties of them all (CppUnit).

However, it was not before end of 2007 until I finally found the time to actually start working on what would later become cfix. About half way through the initial coding phase, in the February 2007 issue, MSDN magazine featured the article Simplified Unit Testing for Native C++ Applications, introducing WinUnit, a unit testing framework for unmanaged C++.

Enter WinUnit

On the one hand, it was nice to see someone thinking the same about current unit testing frameworks and coming up with a new solution. But given the effort that had already gone into cfix, I was not exactly amused about this article — after all, WinUnit implements one of the core ideas of cfix, namely, to separate the test runner (winunit.exe/cfix32.exe) from the actual tests (DLLs) and using PE file introspection to identify fixtures and test cases. So although WinUnit already generated significant positive feedback, I continued development of cfix — not only would cfix at least add the benefit of supporting C in addition to C++, after investigating WinUnit a bit, I still saw lots of room for improvement.

Now, one year later, the situation has changed. Contrary to what one might have expected, WinUnit has not evolved into a serious project — it has not gotten past the MSDN article and the accomanying download link: No new features, no fixes, no blog, no community — by now, WinUnit seems pretty much dead to me. Sure, nothing prevents you from keep using WinUnit, but using tools for which no further development seems to take place is somewhat dissatisfying to me.

Good News

Given this situation and the architectural simlarity of both testing frameworks, it therefore just makes sense to take the next logical step and have cfix embrace WinUnit!

That is, the upcoming cfix 1.3 release will be compatible to WinUnit by allowing developers to take existing test cases written against the WinUnit API and recompile them into cfix test cases without requiring any code to be changed.

With such compatibility in place, transitioning from WinUnit to cfix will thus become a snap. Better yet, because no code has to be changed, the option to switch back and forth between cfix and WinUnit is retained, giving existing WinUnit users maximum flexibility at minimal risk.

The 1.3 release of cfix is due in a couple of days. Once released, I will get a bit more into detail about WinUnit compatibility.

By the way…

today is the first anniversary of cfix :)

cfix 1.2 introduces improved C++ support

cfix 1.2, which has been released today, introduces a number of new features, the most prominent being improved support for C++ and additional execution options.

New C++ API

To date, cfix has primarily focussed on C as the programming language to write unit tests in. Although C++ has always been supported, cfix has not made use of the additional capabilities C++ provides. With version 1.2, cfix makes C++ a first class citizen and introduces an additional API that leverages the benefits of C++ and allows writing test cases in a more convenient manner.

Being implemented on top of the existing C API, the C++ API is not a replacement, but rather an addition to the existing API set.

As the following example suggests, fixtures can now be written as classes, with test cases being implemented as methods:

#include <cfixcc.h>

class ExampleTest : public cfixcc::TestFixture
{
public:
  void TestOne() 
  {}
  
  void TestTwo() 
  {}
};

CFIXCC_BEGIN_CLASS( ExampleTest )
  CFIXCC_METHOD( TestOne )
  CFIXCC_METHOD( TestTwo )
CFIXCC_END_CLASS()

To learn more about the definition of fixtures, have a look at the respective TestFixture chapter in the cfix documentation.

Regarding the implementation of test cases, cfix adds a new set of type-safe, template-driven assertions that, for instance, allow convenient equality checks:

void TestOne() 
{
  const wchar_t* testString = L"test";
  
  //
  // Use typesafe assertions...
  //
  CFIXCC_ASSERT_EQUALS( 1, 1 );
  CFIXCC_ASSERT_EQUALS( L"test", testString );
  CFIXCC_ASSERT_EQUALS( wcslen( testString ), ( size_t ) 4 );
  
  //
  // ...log messages...
  //
  CFIX_LOG( L"Test string is %s", testString );
  
  //
  // ...or use the existing "C" assertions.
  //
  CFIX_ASSERT( wcslen( testString ) == 4 );
  CFIX_ASSERT_MESSAGE( testString[ 0 ] == 't', 
    L"Test string should start with a 't'" );
}

Again, have a look at the updated API reference for an overview of the new API additions.

Customizing Test Runs

Another important new feature is the addition of the new switches -fsf (Shortcut Fixture), -fsr (Shortcut Run), and -fss (Shortcut Run On Failing Setup). Using these switches allows you to specify how a test run should resume when a test case fails.

When a test case fails, the default behavior of cfix is to report the failure, and resume at the next test case. By specifying -fsf, however, the remaining test cases of the same fixture will be skipped and execution resumes at the next fixture. With -fsr, cfix can be requirested to abort the entire run as soon as a single test case fails.

What else is new in 1.2?

Download

As always, cfix 1.2 is source and binary compatible to previous versions. The new MSI package and source code can now be downloaded on Sourceforge.

cfix is open source and licensed under the GNU Lesser General Public License.

cfix 1.1 introduces NT kernel mode unit tests

cfix 1.1 introduces a number of new features. The most important among these is the additional ability to write kernel mode unit tests, i.e. unit tests that are run in kernel mode. Needless to say, cfix 1.1 still supports user mode unit tests.

All contemporary unit testing frameworks focus on unit testing in user mode. Certainly, the vast majority of testing code can be assumed to be targeting user mode, so this does not come at a surprise. Tools for driver testing, of which there are quite a few, focus on integration testing — they usually test whether the driver works in its entirety.

While these tools are very useful indeed, they do not support true unit testing — i.e. offering the ability to test individual routines or subsystems of a driver. To perform such tests, it would be neccessary to write a separate test driver or revert to other techniques such as this one.

cfix 1.1 fills in this gap and offers the ability to write kernel mode tests. That way, individual parts of what may eventually become a driver can thoroughly be tested in isolation, without neccessitating much boilerplate code.

Example

Writing a kernel mode unit test is as easy as writing a user mode unit test — the API is the same for user and kernel mode tests. Even the tools, cfix32 and cfix64 are the same for both modi. The only true difference is that kernel mode tests require slightly different build settings.

The following listing shows an example for a kernel mode unit test — but the same code could just as well be compiled into a user mode unit test.

#include <cfix.h>

static void FixtureSetup()
{
  CFIX_ASSERT( 0 != 1 );
}

static void FixtureTeardown()
{
  CFIX_LOG( L"Tearing down..." );
}

/*++
  Test routine -- do the actual testing.
--*/
static void Test1()
{
  ULONG a = 1;
  ULONG b = 1;
  CFIX_ASSERT_EQUALS_ULONG( a, b );
  CFIX_ASSERT( a + b == 2 );
  
  // You are free to use all WDM APIs here!
  
  CFIX_LOG( L"a=%d, b=%d", a, b );
}


/*++
  Define a test fixture. 
--*/
CFIX_BEGIN_FIXTURE( MyFixture )
  CFIX_FIXTURE_ENTRY( Test1 )

  CFIX_FIXTURE_SETUP( FixtureSetup )
  CFIX_FIXTURE_TEARDOWN( FixtureTeardown )
CFIX_END_FIXTURE()

Once built, the test can be run from the command line:

C:\cfix\bin\i386>cfix32 -nologo -kern ktest.sys
Module: ktest (ktest.sys)
  Fixture: MyFixture
    Test1

For a more detailed discussion and more example code, please refer to the tutorial.

Architecture

For user mode code, the cfix architecture roughly looks like this:

The tests are compiled into a DLL. Using the testrunner application cfix32 or cfix64, one or more fixtures defined in the DLL can be run and the results are reported to the console or to a log file.

For kernel mode code, the acrhitecture looks a little different. The tests are compiled into a driver rather than into a DLL. The driver is verly lightweight and, besides the tests, contains only very little cfix-provided code (basically, just a DriverEntry implementation).

When cfix32 or cfix64 is requested to run a kernel mode tests, it will load the Reflector, a driver that contains the kernel mode fraction of the testing framework. Relaying control operation and output through the reflector, the kernel mode unit tests can be run.

All these additional steps are performed without additional user intervention — the drivers are installed, loaded and stopped automatically. From a user perspective, running a kernel mode tests feels just like running a user mode test.

More…

cfix 1.1 introduces additional new features. I will discuss some of them over the next weeks. In any case, whether you have not used cfix yet or are a cfix 1.0 user, you should go straight to the download page now.

cfix 1.0.1 adds support for Windows 2000

Despite the fact that mainstream support for Windows 2000 has ended in 2005 and the system is well on its way to retirement, Windows 2000 is still in wide use today. As such, it remains being an important target platform for many software packages.

The fact that cfix has not provided support for Windows 2000 was thus unfortunate — after all, if Windows 2000 is among the target platforms of your software, you should be able to run your tests on this platform.

cfix 1.0.1 remedies this shortcoming and adds Windows 2000 i386 SP4 as an supported platform.

The updated packages are available for download on Sourceforge.


Categories




About me

Johannes Passing, M.Sc., living in Berlin, Germany.

Besides his consulting work, Johannes mainly focusses on Win32, COM, and NT kernel mode development, along with Java and .Net. He also is the author of cfix, a C/C++ unit testing framework for Win32 and NT kernel mode, Visual Assert, a Visual Studio Unit Testing-AddIn, and NTrace, a dynamic function boundary tracing toolkit for Windows NT/x86 kernel/user mode code.

Contact Johannes: jpassing (at) acm org

Johannes' GPG fingerprint is BBB1 1769 B82D CD07 D90A 57E8 9FE1 D441 F7A0 1BB1.

LinkedIn Profile
Xing Profile
Github Profile
Advertisements