Posts Tagged 'COM'

Mixing 32 and 64-bit components in a single MSI

Definetely one my pet peeves about Windows Installer is how it deals with instruction set architectures (ISAs). Looking at Windows NT history, supported ISAs have come (amd64, IA-64) and gone (Alpha, PowerPC, MIPS) — yet most of the time, there was more than one ISA being officially supported. Having to ship binaries for multiple ISAs therefore always has been on the agenda for many ISVs.

Needless to say, supporting multiple ISAs requires special consideration when developing setup packages and providing separate packages — one for each ISA — has become common practice to approach this. This approach makes perfect sense: Given the incompatibility of most ISAs, nobody needs Alpha binaries on a MIPS system or amd64 binaries on a IA-64 machine, so there seems little reason to mix ISAs within a single package.

Unsurprisingly, Windows Installer, which was created somewhere around 2000, also goes this route and encourages developers to provide separate packages for each ISA.

However, with the advent of amd64/x64/IA-32e/Intel 64/whateveryoucallit, the situation has changed: Because i386 and amd64 are so closely related and compatible, there are now plenty of situations where combining binaries of differing ISAs (i.e. amd64 and i386) in a single installer package makes perfect sense. Examples for this include:

  • A package comprises a shell extension as well as a standalone App. For certain reasons (maybe the use of VB6), there only is a 32 bit version of the App. The shell extension, in contrast, is available for both, i386 and amd64. Whether you put everything into one package or provide separate packages for each ISA, one of them will comprise a mixture of ISAs.
  • SDKs for unmanaged code usually include .lib and .dll files for multiple architectures. Shipping separate packages for i386 and amd64 (containing different binaries but the same headers, docs, etc.) may please the Windows Installer gods, but seems redundant, a waste of disk space, and user-unfriendly.

Thanks to the msidbComponentAttributes64bit flag, mixing architectures in a single MSI package is technically possible: You mark the package as being 32 bit and set said flag for all 64-bit components. Rather than splitting your setup into multiple packages, you can conveniently combine everything into one.

When reading the documentation (and ICE requirements, more on this later) carefully though, it turns out that this is not quite what the Windows Installer team invented this flag for. Anyway, it works fine, problem solved.

Hmmm…

If only there was not ICE80.

ICE80, alas, is critical if you intend to conform to the Requirements for the Windows Vista Logo Program for Software:

Applications must use the Windows Installer (MSI) or ClickOnce for installation. Windows Installation packages must not receive any errors from the Internal Consistency Evaluators (ICEs) listed here:

1-24, 27-31, 33-36, 38, 40-57, 59, 61-63, 65, 67-72, 74-84, 86-87, 89-94, 96-99

ICE80 mainly states that (1) you should not install 64 bit components to 32 bit directories (e.g. Program Files vs. Program Files (x86)) and (2) you should not use 64 bit components in a 32 bit package.

(1) is fair enough, although it raises the question where you should install your software to without splitting it in two or violating other ICE rules. Worse yet, (2) effectively means that said way to create multi-ISA packages, creating 32 bit packages with some components marked with msidbComponentAttributes64bit, is illegal alltogether.

So to be logo’ed, there seems to be no other way than providing separate packages, maybe along with (urgh!) a meta-package that installs the other two.

If there are more important things on your schedule than getting a Vista logo, ICE80 seems like something that can safely be ignored. Indeed, this is what I have done several times, including in case of the cfix installer.

Anyway, let’s ignore ICE80 once more and hold on to the plan of building a 32-bit package containing both, 32-bit and 64-bit components.

Urgh…

For an SDK that is installed on 64-bit Windows, it will usually make sense to install both, 32 and 64 bit .lib and .dll files etc. On 32-bit Windows, installing 64-bit components may seem odd, but due to the existence of amd64 compilers for i386, it still makes sense to install them or at least offer them as optional feature.

So far, so good. Things get interesting, though, when COM registration comes into play. Naturally, a 32 bit installer package sees the system like any other 32 bit application does. Most importantly, this means that Registry Reflection and File System Redirection applies.

Now consider a package that contains both a 32-bit and a 64-bit version of some COM server, each installed to a separate directory. COM Registration either be performed through the Class or the Registry table. Provided that the msidbComponentAttributes64bit flag has been used properly, such a package will work great on 64 bit systems thanks to Registry Reflection: The regsitry entries will be written to the proper (reflected) locations and both COM servers will work properly.

Now think what happens on 32-bit Windows: (1) There is no Registry Reflection and (2) Windows Installer silently ignores msidbComponentAttributes64bit flags. Result: The installation will run just as smooth as on the 64-bit system. However, while installing the files continues to works flawlessly, the registry will be left in a less-than-optimal state: Due to the nonexistence of Registry Reflection, the registration entries of both COM servers will have been written to the same location!

Needless to say, the server whose registration entries were written first will now be unusable.

In a way, Windows Installer has taken its revenge for breaking the rules.

Bottom line: Mixing 32 and 64-bit components in a single MSI works fine in many cases, but is against the MSI rules and can lead to further problems. And while I am still convinced that providing separate, ISA-specific packages is wrong or at least inconvenient in certain situations, it is definitely the safer and “right” way to go.

(Note: Windows Installer 4.5 introduced multi-package transactions, which allow reliable and transactional multi-package setups to be built so that splitting a setup into multiple packages can be implemented without much pain. However, very few users already have Windows Installer 4.5 installed and Windows 2000 is not even supported by this release. For many of us, relying on this feature therefore is not really an option.)

Advertisements

RCW Reference Counting Rules != COM Reference Counting Rules

Avoiding COM object leaks in managed applications that make use of COM Interop can be a daunting task. While diligent tracking of COM object references and appropriate usage of Marshal.ReleaseComObject usually works fine, COM Interop is always good for surprises.

Recently having been tracking down a COM object leak in a COM/.Net-Interop-centric application, I noticed that the CLR did not quite manage the reference count on my COM object as I expected it to do — more precisely, it incremented the referece count of a COM object when it was passed (from COM) as a method parameter to a callback implemented in .Net — which, of course, contradicts the rules of COM. So while RCWs indeed mostly follow the rules of COM reference counting, they obviously do not do follow the rules in their entirety. Once I spotted this difference, it was easy to find an explanation of this very topic by Ian Griffiths, which is worth quoting [reformatted by me]:

[…]
And by the way, the reference counting is kind of similarish to COM, in that, as you point out, things get addrefed when they are passed to you. But they’re actually not the same. Consider this C# class that
implements a COM interface:

public class Foo : ISomeComInterface
{
  public void Spong(ISomeOtherComInterface bar)
  {
    bar.Quux();
  }
}

Suppose that Spong is the only member of ISomeComInterface. (Other than the basic IUnknown members, obviously.) This Spong method is passed another COM interface as a parameter. And let’s suppose that some non-.NET client is going to call this Spong method on our .NET object via COM interop.

The reference counting rules for COM are not the same as those for the RCW in this case.

For COM, the rule here is that the interface is AddRefed for you before it gets passed in, and is Released for you after you return. In other words, you are not required to do any AddRefing or Releasing on a COM object passed to you in this way *unless* you want to keep hold of a reference to it after the call returns. In that case you would AddRef it.

Compare this with the RCW reference count. As with COM, the RCW’s reference count will be incremented for you when the parameter is passed in. But unlike in COM, it won’t be decremented for you automatically when you return.

You could sum up the difference like this:

  • COM assumes you won’t be holding onto the object reference when the method returns
  • The RCW assumes you *will* be holding onto the object reference when the method returns.

So if you don’t plan to keep hold of the object reference, then the method should really look like this:

public void Spong(ISomeOtherComInterface bar)
{
  bar.Quux();
  Marshal.ReleaseComObject(bar);
}

According to the COM rules of reference counting, this would be a programming error. But with RCWs, it’s how you tell the system you’re not holding onto the object after the method returns.

Pretty counter-intuitive… Plus, I am not aware of any official documentation on this topic.

Working Around TlbImp’s Cleverness

TlbImp, the .Net tool to create Interop assemblies from COM type libraries, contains an optimization that presumably aims at making the consumption of the Interop assembly easier, but ultimately is a nuisance. Consider the following IDL code:

import "oaidl.idl";
import "ocidl.idl";

[
  uuid( a657ef35-fea1-40ad-86d8-bb7b6085a0a3 ),
  version( 1.0 )
]
library Test
{
  
  [
    object,
    uuid( 84b2f017-b8fe-4c2c-87b8-0587b4bf5507 ),
    version( 1.0 ),
    oleautomation
  ]
  interface IFoo : IUnknown 
  {
    HRESULT Foo();
  }

  [
    object,
    uuid( 13d950d6-beb3-4dd3-957b-88b0e5eb5e3f ),
    version( 1.0 ),
    oleautomation
  ]
  interface IBar : IUnknown 
  {
    HRESULT CreateFoo( 
      [out, retval] IFoo **Foo
      );
  }

  [
    uuid( e01ea769-410c-4915-a48c-3522a8087a52 ),
    noncreatable 
  ]
  coclass Foo
  {
    interface IFoo;
  }

  [
    uuid( dca66832-fe3b-4658-a975-442b5678a9ec )
  ]
  coclass Bar
  {
    interface IBar;
  }
}

Two things are worth noting about this IDL code: First, IBar::CreateFoo is declared to “return” an IFoo*, and second, there is only one coclass implementing IFoo, namely Foo. As a consequence, TlbImp attempts to be clever and assumes that if IBar::CreateFoo “returns” an IFoo*, it relly must be an Foo* that is returned. So in the resulting Interop assembly, the IBar interface will look as follows:

[
  ComImport, InterfaceType((short) 1), 
  Guid("13D950D6-BEB3-4DD3-957B-88B0E5EB5E3F"), 
  TypeLibType((short) 0x100)
]
public interface IBar
{
  [return: MarshalAs(UnmanagedType.Interface)]
  [MethodImpl(MethodImplOptions.InternalCall, 
   MethodCodeType=MethodCodeType.Runtime)]
  Foo CreateFoo();
}

Contrary to what the IDL defines, IBar::CreateFoo returns Foo, i.e. a concrete class.

First of all, the assumption underlying this optimization certainly is somewhat flaky as it is not quite in accord with the rules of COM — after all, the implementation of Bar is free to return whatever coclass implementing IFoo seems appropriate and is not limited to returning Foo coclass instances.

While this might not be a real problem in practice, the optimization has another an unpleasant effect on the testability of the library consuming the interface: To properly test this library, it might be a good idea to implement mock or stub implementations of IFoo and IBar. Unfortunately, due to the “optimized” return type of IBar::CreateFoo(), this turns out to be not quite easy, as the following code suggests:

class FooStub : IFoo
{
  public void Foo() 
  {}
}

class BarStub : IBar
{
  public Foo CreateFoo()
  {
    // XXX: FooStub implements IFoo, but Foo is required!
    return new FooStub()
  }
}

Workarounds

As pointless as TlbImp’s behavior in this regard might be, it is quite easy to work around this issue.

The first workaround is to define a dummy coclass in the IDL and declare it to implement IFoo as well:

[
  uuid( ed93b3e6-104b-43d6-be34-972d7519bc62 )
]
coclass Dummy
{
  interface IFoo;
}

Now that there are two candidate coclasses, TlbImp will not be able to infer the coclass from the interface and will not be able to apply its optimization. The drawback of this approach is, of course, that the additional Dummy coclass consitutes additional baggage in the IDL and the Interop assembly.

The second option is to forego declaring the interfaces of Foo in the coclass. These declarations mainly serve informative purposes and are not mandatory, so we can omit them. To satisfy MIDL’s requirement of naming at least one interface, we can just put in IUnknown:

[
  uuid( e01ea769-410c-4915-a48c-3522a8087a52 ),
  noncreatable 
]
coclass Foo
{
  interface IUnknown;
}

Again, TlbImp will not be able to apply its optimization and the resulting Interop assembly will contain a proper, mockable, interface IBar.

Needless to say, this approach has its own drawbacks. As a consequence of the missing interface implementation declarations, class Foo (in the Interop assembly) will not contain any methods and is basically useless — you are obliged to use the interface.

More importantly, however, the .Net class Foo will not implement IFoo — so although QueryInterface’ing IFoo from Foo would work, a statement like IBar bar = new BarClass() will now lead to a compiler error:

Cannot implicitly convert type ‘Test.BarClass’ to ‘Test.IBar’. An explicit conversion exists (are you missing a cast?)

Although I consider the second option to be the cleaner approach, it is therefore best used for noncreatable coclasses only.


Categories




About me

Johannes Passing lives in Berlin, Germany and works as a Solutions Architect at Google Cloud.

While mostly focusing on Cloud-related stuff these days, Johannes still enjoys the occasional dose of Win32, COM, and NT kernel mode development.

He also is the author of cfix, a C/C++ unit testing framework for Win32 and NT kernel mode, Visual Assert, a Visual Studio Unit Testing-AddIn, and NTrace, a dynamic function boundary tracing toolkit for Windows NT/x86 kernel/user mode code.

Contact Johannes: jpassing (at) hotmail com

LinkedIn Profile
Xing Profile
Github Profile
Advertisements