Initial COM Design Requirements

Systems built from binary components offer several advantages over monolithic systems. You can revise and replace individual components in the field without affecting other parts of the system. This development style allows component suppliers to improve and revise their code far more effectively. It also allows reuse of generic components in many applications.

From the beginning, the Windows operating system has been based on the principle of binary code reuse. The Windows API is a call-level interface made up of thousands of functions written in C and assembly language. The Win32 API is available to all applications through a series of operating system DLLs. A client can link to a DLL at run time and call exported functions. Languages such as C, C++, and Visual Basic can link to DLLs as long as they have the definitions of functions and structures from a C header file at compile time. The main limitation of this type of DLL is that it's simply a collection of global functions and data structures that don't contain classes or objects and therefore don't offer an object-oriented solution.

Binary reuse alone wasn't hard to achieve on the Windows platform. However, moving binary reuse into an object-oriented paradigm proved more challenging. C++ was one of the first and most popular object-oriented programming (OOP) languages for Windows developers. It provides support for powerful OOP features that are missing in C, such as encapsulation, polymorphism, and inheritance. Unfortunately, C++ was designed to create monolithic applications and is therefore tricky to use for component-based development.

It's possible to export a C++ class from a DLL, but with many limitations. Most of the problems arise because the ANSI C++ standard defines the requirements only for the compilation process. The original designers of C++ assumed that all of the source code for an application would be compiled and linked at the same time. Therefore, the C++ standard doesn't specify how objects should be laid out in memory. This leads to severe problems in component-based development.

The lack of a binary standard causes compatibility problems among C++ compilers. Compiler vendors such as Microsoft, Symantec, and Borland have implemented C++ features such as exceptions, function overloading, and run-time type information (RTTI) in proprietary ways. This means that using these features creates vendor-specific dependencies among binary components. For instance, you can't throw an exception from a class in a DLL built with the Borland compiler and catch it in a client application built with the Microsoft compiler. The only way to use all of the advanced C++ features in component-based development is to standardize all development on a compiler from a single vendor. This is clearly not an acceptable solution in open software design.

The intuitive solution to this problem is to get rid of all the C++ features that create vendor-specific dependencies. It seems reasonable that if all C++ development is restricted to using a set of C++ features that are compatible across all major compiler vendors, the language can be used for building binary components. This solves some of the problems, but not all of them. There is one remaining problem with C++ that turns out to be the biggest one of all.

The problem is that C++ doesn't support encapsulation at the binary level because the language was designed for building monolithic applications. When a C++ class author marks a data member as private, it is off-limits to client code, as you would expect. However, this encapsulation is enforced only in the syntax of the language. Any client that calls New on a class must have knowledge of the object's data layout. The client is responsible for allocating the memory in which the object will run. This means that the client must have knowledge of each data member for a class regardless of whether it is marked public, protected, or private.

These layout dependencies aren't a problem in a monolithic application because the client always recompiles against the latest version of the class. But in component-based development, this is a huge problem. The layout of a class within a DLL can't change without breaking all the clients that use it. The following C++ code shows a client and two versions of a class in a DLL.

 // In SERVER.DLL - Version 1 // Each object will require 8 bytes of memory. class CDog{ private:     double Weight; } // In CLIENT.EXE // Compiled against Version 1 of DLL // Client allocates 8 bytes for object. CDog* pDog = new CDog; // In SERVER.DLL - Version 2 // Each object will require 16 bytes of memory. // Replacing older DLL will break client. class CDog{ private:     double Weight;     double Age; } 

When the first version of the DLL is replaced in the field by the second version, a big problem arises. The client application continues to create objects that are 8 bytes in size, but each object thinks that it's 16 bytes. This is a recipe for disaster. The newer version of the object will try to access memory that it doesn't own, and the application will likely fail in strange and mysterious ways. The only way to deal with this is to rebuild all client applications whenever the object's data layout changes. This eliminates one of the biggest benefits of component-based development: the ability to modify one binary file without touching any of the others.

How can you replace the DLL without breaking any of the client applications that use it? The object's data layout must be hidden from the client. The client must also be relieved of the responsibility of calling the C++ New operator across a binary firewall. Some other agent outside the client application must take on the responsibility of creating the object. Over the past decade, many C++ programmers have wrestled with the problems of using C++ in component-based development, and the C++ community has devised a few techniques for solving this problem.

One popular technique for solving the binary encapsulation problem is to use a handle-based approach. You can see a great example of this in Windows SDK applications and the Win32 API. The client deals with handles to objects such as windows, files, and device contexts. The client calls a global function to create an object. The client receives an integer identifier, which acts as a logical pointer to an entity such as a window object. The client communicates with the window by calling other global functions in the Win32 API and passing the handle to indicate which window to act on. The operating system responds by performing the desired operation on the window. The downside to this approach is that the client code doesn't have an object-oriented feel. Therefore, it doesn't meet one of the main requirements of COM.

Abstract Base Classes as Interfaces

Another technique for solving the binary encapsulation problem with C++ is the use of abstract base classes. In C++, an abstract base class is used to define method signatures that don't include any implementation. (This should sound familiar.) Here's an example of what an abstract base class looks like in C++:

 class IDog{     virtual void Bark() = 0;     virtual void RollOver(int rolls) = 0; }; 

A C++ member function (such as a method) that's marked with the =0 syntax is a pure virtual function. This means that it doesn't include an implementation. It also means that the class that defines it isn't a creatable type. Therefore, the only way to use an abstract base class is through inheritance, as shown below.

 class CBeagle: public IDog{   virtual void Bark()     {/* Implementation */}   virtual void RollOver(int Rolls);     {/* Implementation */} }; 

C++ programmers who use abstract base classes and a little extra discipline can achieve the reusability, maintainability, and extensibility benefits that we covered in Chapter 2. The extra discipline required is to avoid the definition of both data storage and method implementations inside an abstract base class. As a result, an abstract base class can formalize the separation of interface from implementation. A C++ client application can communicate with a CBeagle object through an IDog reference, which allows the client application to avoid building any dependencies on the CBeagle class.

You should see that a C++ abstract base class can be used as a logical interface. Even though the language has no real support for interface-based programming, advanced techniques in C++ allow you to reap the most significant benefits. In fact, C++ programmers were the ones who pioneered the concepts of interface-based programming by using abstract base classes in large component-based applications. This technique has been used in numerous projects and is described in depth in Large-Scale C++ Software Design, by John S. Lakos. The principles of interface-based programming as implemented in C++ have had a profound effect on the development of COM.

Creating a Binary Standard

The creators of COM concluded that if they removed vendor-specific features of the C++ language and used abstract base classes, they could achieve object-oriented binary reuse. DLLs could hold class definitions. Client applications could use objects created from these DLLs as long as they communicated through abstract base classes. DLLs could be revised and replaced in the field without adversely affecting the client applications that used them. Arriving at this series of conclusions was a big milestone for the Microsoft engineers. They had devised a way to achieve binary reuse in an object-oriented paradigm using C++.

A language that offers polymorphism must provide a way to dynamically bind clients to different versions of the same method signature. A client application can contain code programmed against a generic data type, and any type-compatible object can be plugged in at run time. Different languages and compilers approach this requirement of dynamic binding in different ways. Because of its low-level nature, C++ happens to have one of the fastest binding techniques available. Dynamic binding in C++ is based on a highly efficient dispatching architecture that relies on the use of virtual functions.

The C++ compiler and linker make dynamic binding possible by generating an array of function pointers called a vTable. (The v stands for virtual.) A vTable represents a set of entry points into an object. Each method defined in an abstract base class has one entry point. If the client knows the calling syntax and acquires the function pointer of a particular method, it can access the object without any knowledge of the concrete class from which it was created.

Figure 3-1 shows what a vTable looks like in memory. Note that an object can have more than one vTable. This simply means that an object can expose more than one interface. vTables are automatically created and populated by the C++ compiler on the object side. The C++ compiler can also generate the client-side binding code that invokes methods through the vTable function pointers. It's fortunate that all popular C++ compilers treat vTables in the same way at the physical level. Whether vTable compatibility between compilers was fate or a stroke of good luck is irrelevant. The Microsoft engineers decided to take this physical layout and make it the standard for all COM objects.

Sometime after the completion of the OLE2 project, a team of engineers drafted the COM Specification, a document that defines the rules for COM programming. The rules state that COM objects must adhere to a specific memory layout. Each COM object must implement one or more COM interfaces. COM clients can communicate with objects only through these interfaces. COM obtains information about an object and a list of the interfaces the object supports from a coclass. A coclass is a visible concrete COM implementation that can be used by external COM clients to create objects. Coclasses are packaged for distribution in binary files called COM servers. A server must also support the COM infrastructure for creating and activating objects. As you will see, a server does this by exposing well-known entry points to the system.

Figure 3-1. In both C++ and COM, clients are dynamically bound to objects through the use of vTables. Each physical vTable represents a logical interface implemented by the object.

COM and Language Independence

In COM, clients and objects must communicate through vTables. This means that COM objects as well as COM clients must be savvy in their use of function pointers. Luckily, the compiler helps C++ programmers by doing most of the work behind the scenes. C programmers aren't so lucky. To create a COM object in C, you must simulate a C++-style vTable by manually creating and populating an array of function pointers. On the client side, you must manually acquire the function pointer and explicitly invoke the method through it. This makes straight C unattractive for writing COM code by hand.

But what about Visual Basic and Java programmers? Many developer tools and languages such as these have no built-in support for dealing with function pointers. To create or use COM objects, a development tool or language must follow the rules in the COM Specification, which state that vTable binding must be used to conduct all client-object communications. Many higher-level tools and languages need assistance to be able to participate in COM programming.

Visual Basic provides this assistance by adding support to its compiler and adding a Visual Basic-to-COM mapping layer in the run-time engine. After all, the COM Specification defines the rules clearly. The Visual Basic team knew exactly what it would take to make objects vTable-compliant. On the client side, the Visual Basic compiler creates the vTable binding code that is required to access a COM object. Fortunately, when a language or a tool uses a mapping layer, it can hide many of the underlying details from its programmers. This is why Visual Basic is so much easier to use than C++ for programming COM.

Introducing Interface Definition Language (IDL)

In COM, clients bind to objects at run time. However, to properly communicate with an object, a client must know a few things at compile time. In particular, it must have the following pieces of information:

• The type of object it wants to create and use (the coclass)

• The interface or interfaces it will use to communicate with the object

• The calling syntax for each method in the interface or interfaces

To achieve language independence, COM must provide a universal way for servers to publish information about the interfaces and the coclasses they contain. COM has standardized on a language called interface definition language (IDL). IDL provides a way to define a set of interfaces and coclasses in a manner that is language-neutral. This means that any COM-capable language can be used to implement or use the definitions from an IDL source file. The IDL source file must be fed to the Microsoft IDL (MIDL) compiler. The MIDL compiler generates a few source files used by C and C++ programmers and a special binary database called a type library.

A type library is a catalog that describes interfaces, coclasses, and other resources in a server. Each interface is defined with a set of methods; each coclass is defined with one or more interfaces. Type library files have many possible extensions, including .tlb, .dll, .exe, .olb, and .ocx. When you create a server with Visual Basic, the type library is generated without the use of IDL and is automatically bundled into the server's binary image.

A development tool such as Visual Basic requires the use of a type library for building a client application against a COM server. The type library provides the information that Visual Basic needs at compile time to create the client-side vTable binding code. You can import a type library into a Visual Basic project by opening the References dialog box from the Project menu. This dialog box presents a list of all the type libraries registered on your developer workstation.

Using IDL

C++ and Java developers create type libraries using IDL. In COM, IDL is the one and only official language for describing what's inside a server. When the COM team began formalizing the COM Specification, it became obvious that C++ and C couldn't be used to define COM interfaces. C and C++ weren't designed to define functions that extend process boundaries, and they therefore allow parameter definitions that are extremely vague. For instance, if a C++ method defines a parameter as a pointer, what does the pointer actually point to? What data must actually move between the client process and the object process? A mere pointer can't define what needs to be moved between the two processes. IDL solves this problem by using syntax that describes method parameters without ambiguity.

IDL looks a lot like C, but it adds a few object-oriented extensions. It also allows the specification of attributes for entities such as type libraries, coclasses, interfaces, methods, and parameters. Here is a watered-down example of what IDL looks like:

 library DogServerLib {     interface IDog {         HRESULT Bark();         HRESULT RollOver([in] int Rolls);     };     interface IWonderDog{         HRESULT FetchSlippers();     };       coclass CBeagle {         interface IDog;         interface IWonderDog;     }; } 

This example shows how type libraries, the interface, and coclasses are defined in an IDL source file. Note that each method definition has a return value of type HRESULT. COM requires this standard return value to allow detection of dead objects or network failures in a distributed environment. Chapter 5 describes in more detail how HRESULTs work.

When C++ or Java programmers want to create a COM object, they must first define the appropriate IDL and feed it to the Microsoft IDL (MIDL) compiler. Visual Basic programmers, on the other hand, don't go through this process because the Visual Basic IDE creates type libraries directly from Visual Basic code. Visual Basic programmers never have to work with IDL.

You can live a productive life as a Visual Basic/COM programmer without ever seeing or understanding IDL. However, if you learn the basics of IDL, you will be a better COM programmer. This is especially true if you are concerned with interoperability among components written in different languages, such as Visual Basic and C++. With an understanding of IDL, you can see exactly what Visual Basic is doing behind the scenes. A COM utility named Oleview.exe can help you reverse-engineer a type library into a readable text-based version of IDL. Figure 3-2 shows how you can use this utility to examine the IDL of coclasses and interfaces built into your ActiveX DLLs and ActiveX EXEs. Be sure you set Oleview.exe to expert mode when attempting to read a type library. If you don't do this, you won't see the type libraries in the left-side tree view control of Oleview.exe.

Figure 3-2. OleView.exe lets you examine and modify many aspects of your COM servers. This example shows how Oleview.exe's type library viewer lets you reverse-engineer IDL from a COM server created with Visual Basic.

How Does Visual Basic Map to COM?

COM requires that every object implement at least one interface. Visual Basic makes things easy for you by creating a default interface in each creatable class. All of the public methods and properties from a class are placed in a default interface. For instance, assume that you have a class CCollie with the following public interface:

 Public Name As String Public Sub Bark()     ' implementation End Sub 

Visual Basic creates a hidden interface named _CCollie from the public properties and methods of the class module. The fact that this interface is marked hidden in the type library means that other Visual Basic programmers can't see it in the Object Browser or through IntelliSense. (Visual Basic also hides any type name that begins with an underscore "_".) Visual Basic then creates a coclass named CCollie that implements _CCollie as the default interface. The basic IDL looks like this:

 [hidden] interface _CCollie{     [propget] HRESULT Name([out, retval] BSTR* Name);     [propput] HRESULT Name([in] BSTR Name);     HRESULT Bark(); }; coclass CCollie {     [default] interface _CCollie; }; 

This transparent one-to-one mapping allows Visual Basic classes to be COM-compliant without any assistance from the programmer. Naive Visual Basic programmers have no idea what's really going on. Any Visual Basic client can write the following code to use this class:

 Dim Dog As CCollie Set Dog = New CCollie Dog.Name = "Lassie" Dog.Bark 

In the code above, the variable declared with the type CCollie is transparently cast to the _CCollie reference. This makes sense because a client must communicate with an object through an interface reference. This also makes COM programming easy in Visual Basic. As long as there is a logical one-to-one mapping between a class and the interface that you want to export, you don't have to create user-defined interfaces. However, if you don't employ user-defined interfaces, you can't really tap into the power of interface-based designs.

Chapter 2 showed you how to implement user-defined interfaces in a Visual Basic application. When you define an interface in Visual Basic 5 with a PublicNotCreatable class and implement it in a class, the resultant IDL code looks something like this:

 interface IDog {     HRESULT Name([out, retval] BSTR* Name);     HRESULT Name([in] BSTR Name);     HRESULT Bark(); }; interface _CBeagle { }; coclass CBeagle {     [default] interface _CBeagle;     interface IDog; }; 

Even when your class contains no public members, Visual Basic automatically creates a default interface of the same name preceded by an underscore. You can't change this to make another interface the default. In the next chapter, you will see that on occasion you must put functionality in the default interface. This means that you must put public members in your class. This is always the case when you are creating Visual Basic objects for automation clients.

Visual Basic 6 works differently than Visual Basic 5. When you mark a class module as PublicNotCreatable, Visual Basic 6 still creates a coclass and default interfaces. For instance, when you create the interface IDog with a property and a method in a class module marked as PublicNotCreatable, the resulting IDL looks like this:

 [hidden] interface _IDog : IDispatch {     HRESULT Name([out, retval] BSTR* Name);     HRESULT Name([in] BSTR Name);     HRESULT Bark(); }; [noncreatable] coclass IDog {     [default] interface _IDog; }; 

This means that in Visual Basic 6 you can no longer simply create a COM interface as you could in Visual Basic 5. PublicNotCreatable class modules always produce an interface and a noncreatable coclass. The noncreatable attribute means that the class can't be instantiated from a COM client. However, any code that lives inside the same server can create objects from a PublicNotCreatable class. In Visual Basic 6, you should think of these as Public-Not-Externally-Creatable classes.

It turns out that the differences between Visual Basic 5 and Visual Basic 6 are hidden inside the type library. The code you write for Visual Basic 6 is the same way as it is in Visual Basic 5. In the example above, IDog is a coclass and _IDog is a hidden interface. Whenever you use the type IDog with the Implements keyword or use it to create object references, Visual Basic silently casts it to _IDog.

Distributing Interface Definitions

Your classes can also implement interfaces that are defined in external type libraries. To do this, you must import the type library using the References dialog box (accessed from the Project menu). These type libraries can be built with either Visual Basic or the MIDL compiler. Once your project can see the interface definition, you can use the Implements keyword in a class module. In large projects whose designs depend on user-defined interfaces, it often makes sense to distribute the interface definitions in a type library that's independent of any servers that implement them.

You can create a stand-alone type library in Visual Basic. You create an ActiveX DLL or ActiveX EXE project and select the Remote Server Files option on the Components tab of the Project Properties dialog box. (This option is available only in the Enterprise Edition of Visual Basic.) When you build the server, Visual Basic also creates a type library (*.tlb) file that holds only the interface definitions. This file can be distributed to any programmer who needs to compile code against the interface definitions. The type library can be used by developers of both server and client applications.

Writing Visual Basic-Friendly IDL

Some developers prefer to define their interfaces in IDL instead of Visual Basic. This is especially true for projects that also use C++ or Java. Visual Basic type libraries have a reputation for being very messy when used in any environment other than Visual Basic. And it's important to understand that many things expressed in IDL don't work in Visual Basic. Enterprise-level designers should be aware of what works in Visual Basic and what doesn't. If you intend to use Visual Basic in a project, make sure that all the interfaces you create are Visual Basic-friendly.

One way to write Visual Basic-friendly IDL source code is to leverage the Visual Basic IDE. Start by defining your interfaces in a Visual Basic server, and build the DLL or the EXE. Then use Oleview.exe to reverse-engineer the IDL text, and use that as a starting point for your IDL source file. If you follow this approach, you'll know that your interfaces can be implemented and used in Visual Basic projects. Chapter 6 describes how to create type libraries with IDL in greater depth, but for now here's a short list of rules to keep in mind when you're creating Visual Basic-compatible IDL source files:

• Method names can't start with an underscore (_).

• Don't use COM [out] parameters unless you also use [retval].

• All methods must return an HRESULT.

• Don't use unsigned long integers or unsigned short integers.

• Don't use parameters that contain pointers.

• Don't derive one custom COM interface from another custom interface. Visual Basic-compatible interfaces must derive from IUnknown and IDispatch.