Archive for March, 2009

RCW Reference Counting Rules != COM Reference Counting Rules

Avoiding COM object leaks in managed applications that make use of COM Interop can be a daunting task. While diligent tracking of COM object references and appropriate usage of Marshal.ReleaseComObject usually works fine, COM Interop is always good for surprises.

Recently having been tracking down a COM object leak in a COM/.Net-Interop-centric application, I noticed that the CLR did not quite manage the reference count on my COM object as I expected it to do — more precisely, it incremented the referece count of a COM object when it was passed (from COM) as a method parameter to a callback implemented in .Net — which, of course, contradicts the rules of COM. So while RCWs indeed mostly follow the rules of COM reference counting, they obviously do not do follow the rules in their entirety. Once I spotted this difference, it was easy to find an explanation of this very topic by Ian Griffiths, which is worth quoting [reformatted by me]:

[…]
And by the way, the reference counting is kind of similarish to COM, in that, as you point out, things get addrefed when they are passed to you. But they’re actually not the same. Consider this C# class that
implements a COM interface:

public class Foo : ISomeComInterface
{
  public void Spong(ISomeOtherComInterface bar)
  {
    bar.Quux();
  }
}

Suppose that Spong is the only member of ISomeComInterface. (Other than the basic IUnknown members, obviously.) This Spong method is passed another COM interface as a parameter. And let’s suppose that some non-.NET client is going to call this Spong method on our .NET object via COM interop.

The reference counting rules for COM are not the same as those for the RCW in this case.

For COM, the rule here is that the interface is AddRefed for you before it gets passed in, and is Released for you after you return. In other words, you are not required to do any AddRefing or Releasing on a COM object passed to you in this way *unless* you want to keep hold of a reference to it after the call returns. In that case you would AddRef it.

Compare this with the RCW reference count. As with COM, the RCW’s reference count will be incremented for you when the parameter is passed in. But unlike in COM, it won’t be decremented for you automatically when you return.

You could sum up the difference like this:

  • COM assumes you won’t be holding onto the object reference when the method returns
  • The RCW assumes you *will* be holding onto the object reference when the method returns.

So if you don’t plan to keep hold of the object reference, then the method should really look like this:

public void Spong(ISomeOtherComInterface bar)
{
  bar.Quux();
  Marshal.ReleaseComObject(bar);
}

According to the COM rules of reference counting, this would be a programming error. But with RCWs, it’s how you tell the system you’re not holding onto the object after the method returns.

Pretty counter-intuitive… Plus, I am not aware of any official documentation on this topic.

cfix 1.3.0 Released, Introducing WinUnit Compatibility

cfix 1.3, the latest version of the unit testing framework for C/C++ on Windows, has just been released. As announced in the last blog post, the major new feature of this release is WinUnit compatibility, i.e. the ability to recompile existing WinUnit test suites into cfix test suites without having to change a single line of code.

To demonstrate that this compatibility indeed works, consider the following simple example:

#include "WinUnit.h"

BEGIN_TEST(DummyTest)
{
  WIN_ASSERT_STRING_EQUAL( "foo", "bar" "Descriptive message");
}
END_TEST

Compile it:

cl /I %CFIX_HOME%\include /LD /EHa /Zi winunittest.cpp /link /LIBPATH:%CFIX_HOME%\lib\i386

Or, in case %INCLUDE% and %LIB% already happen to be set properly:

cl /LD /EHa winunittest.cpp

Note that the only difference to compiling the test for WinUnit ist that a different include path is used — rather than the original winunit.h, cfix’ own winunit.h is used, which in turn implements the WinUnit API on top of the existing API.

The resulting DLL is a valid cfix DLL and its tests can be run in the usual manner. As the example contains a failing test, cfix will print the stack trace and error description to the console:

D:\sample>cfix32 -ts -z winunittest.dll
cfix version 1.3.0.3340 (fre)
(C) 2008-2009 - Johannes Passing - http://www.cfix-testing.org/
[Failure]      winunittest.DummyTest.DummyTest
      winunittest.cpp(5): DummyTest

      Expression: Descriptive message: [foo] == [bar] (Expression: "foo" == "bar")
      Last Error: 0 (The operation completed successfully. )

      cfix!CfixpCaptureStackTrace +0x40
      cfix!CfixPeReportFailedAssertion +0xd2
      winunittest!cfixcc::Assertion::Fail<std::...
      winunittest!cfixcc::Assertion::Relate<std...
      winunittest!cfixcc::Assertion::Relate ...
      winunittest!cfixcc::Assertion::RelateStri...
      winunittest!DummyTest +0x9c
      cfix!CfixsRunTestRoutine +0x33
      cfix!CfixsRunTestCaseMethod +0x27
      cfix!CfixsRunTestCase +0x25
      ...

Of course, cfix also supports WinUnit fixtures, as the following example, taken from the original WinUnit article on MSDN demonstrates:

#include "WinUnit.h"
#include <windows.h>

// Fixture must be declared.
FIXTURE(DeleteFileFixture);

namespace
{
  TCHAR s_tempFileName[MAX_PATH] = _T("");
  bool IsFileValid(TCHAR* fileName);
}

// Both SETUP and TEARDOWN must be present. 
SETUP(DeleteFileFixture)
{
  // This is the maximum size of the directory passed to GetTempFileName.
  const unsigned int maxTempPath = MAX_PATH - 14; 
  TCHAR tempPath[maxTempPath + 1] = _T("");
  DWORD charsWritten = GetTempPath(maxTempPath + 1, tempPath);
  // (charsWritten does not include null character)
  WIN_ASSERT_TRUE(charsWritten  0, 
    _T("GetTempPath failed."));

  // Create a temporary file
  UINT tempFileNumber = GetTempFileName(tempPath, _T("WUT"), 
    0, // This means the file will get created and closed.
    s_tempFileName);

  // Make sure that the file actually exists
  WIN_ASSERT_WINAPI_SUCCESS(IsFileValid(s_tempFileName), 
    _T("File %s is invalid or does not exist."), s_tempFileName);
}

// TEARDOWN does the inverse of SETUP, as well as undoing 
// any side effects the tests could have caused.
TEARDOWN(DeleteFileFixture)
{
  // Delete the temp file if it still exists.
  if (IsFileValid(s_tempFileName))
  {
    // Ensure file is not read-only
    DWORD fileAttributes = GetFileAttributes(s_tempFileName);
    if (fileAttributes & FILE_ATTRIBUTE_READONLY)
    {
      WIN_ASSERT_WINAPI_SUCCESS(
        SetFileAttributes(s_tempFileName, 
          fileAttributes ^ FILE_ATTRIBUTE_READONLY),
          _T("Unable to undo read-only attribute of file %s."),
          s_tempFileName);
    }

    // Since I'm testing DeleteFile, I use the alternative CRT file
    // deletion function in my cleanup.
    WIN_ASSERT_ZERO(_tremove(s_tempFileName), 
      _T("Unable to delete file %s."), s_tempFileName);
  }

  // Clear the temp file name.
  ZeroMemory(s_tempFileName, 
    ARRAYSIZE(s_tempFileName) * sizeof(s_tempFileName[0]));
}

BEGIN_TESTF(DeleteFileShouldDeleteFileIfNotReadOnly, DeleteFileFixture)
{
  WIN_ASSERT_WINAPI_SUCCESS(DeleteFile(s_tempFileName));
  WIN_ASSERT_FALSE(IsFileValid(s_tempFileName), 
    _T("DeleteFile did not delete %s correctly."),
    s_tempFileName);
}
END_TESTF

BEGIN_TESTF(DeleteFileShouldFailIfFileIsReadOnly, DeleteFileFixture)
{
  // Set file to read-only
  DWORD fileAttributes = GetFileAttributes(s_tempFileName);
  WIN_ASSERT_WINAPI_SUCCESS(
    SetFileAttributes(s_tempFileName, 
    fileAttributes | FILE_ATTRIBUTE_READONLY));

  // Verify that DeleteFile fails with ERROR_ACCESS_DENIED
  // (according to spec)
  WIN_ASSERT_FALSE(DeleteFile(s_tempFileName));
  WIN_ASSERT_EQUAL(ERROR_ACCESS_DENIED, GetLastError());
}
END_TESTF

namespace
{
  bool IsFileValid(TCHAR* fileName)
  {
    return (GetFileAttributes(fileName) != INVALID_FILE_ATTRIBUTES);
  }
}

Compiling and running this test yields the expected output:

d:\sample>cl /I %CFIX_HOME%\include /LD /EHa /Zi fixture.cpp /link /LIBPATH:%CFIX_HOME%\lib\i386
d:\sample>cfix32 -ts -z fixture.dll
cfix version 1.3.0.3340 (fre)
(C) 2008-2009 - Johannes Passing - http://www.cfix-testing.org/
[Success]      fixture.DeleteFileFixture.DeleteFileShouldDeleteFileIfNotReadOnly
[Success]      fixture.DeleteFileFixture.DeleteFileShouldFailIfFileIsReadOnly


       1 Fixtures
       2 Test cases
           2 succeeded
           0 failed
           0 inconclusive

Limitations

All compatibility has its limitations — although cfix supports all major WinUnit constructs and assertions, there are a small number of known limitations, which are listed in the documentation. And although I am confident that most WinUnit code should compile and run just fine, it is, of course, possible, that further limitations pop up. In such cases, I would welcome an appropriate bug report and will try to fix cfix accordingly.

Documentation

In order to have cfix be a fully adequate replacement for cfix, the cfix documentation has additionally been augmented to include a documentation of the entire WinUnit API.

Technical background

Technically, implementing the compatibility layer went rather smoothly. On the one hand, WinUnit and cfix have similar architectures, which makes many things easier. On the other hand, WinUnit has a clean, single public header file (contrast that to CppUnit!), which also simplified things. And as WinUnit is limited to C++, I was able to use C++ templates (in combination with some preprocessor macros) to implement the entire WinUnit compatibility layer without having to change a single line of cfix itself. Rather, the WinUnit macros/classes are all mapped onto the existing cfix C++ API, which already includes most of what was neccessary to implement the WinUnit functionality.

Conclusion

In case have been using WinUnit the past and have a set of existing WinUnit-based test suites, give cfix a try — Not only should it be a full-featured replacement for WinUnit, you can also expect to see, and benefit from new features in upcoming releases!

Last but not least, the release contains a number of minor bugfixes. So upgrading is recommended even if you do not intend to use the new WinUnit compatibility feature.

cfix can be downloaded here.


Categories




About me

Johannes Passing, M.Sc., living in Berlin, Germany.

Besides his consulting work, Johannes mainly focusses on Win32, COM, and NT kernel mode development, along with Java and .Net. He also is the author of cfix, a C/C++ unit testing framework for Win32 and NT kernel mode, Visual Assert, a Visual Studio Unit Testing-AddIn, and NTrace, a dynamic function boundary tracing toolkit for Windows NT/x86 kernel/user mode code.

Contact Johannes: jpassing (at) acm org

Johannes' GPG fingerprint is BBB1 1769 B82D CD07 D90A 57E8 9FE1 D441 F7A0 1BB1.

LinkedIn Profile
Xing Profile
Github Profile