The Magic of Metadata To solve all these problems .NET must provide an underlying set of services that is available to all languages at all times. It also has to understand enough about an application to be able to provide these services. Serialization provides a simple example. Every programmer at some time or another has to write code to save data. Why should every programmer have to reinvent the wheel of how to persist nested objects and complicated data structures? Why should every programmer have to figure out how to do this for a variety of data stores? .NET can do this for the programmer. Programmers can also decide to do it themselves if required. To see how this is done, look at the Serialize sample associated with this chapter. For the moment, ignore the programming details of VB.NET, which will be covered in the next four chapters, and focus on the concepts.  ' Serialize.vb Imports System Imports System.Collections Imports System.IO Imports System.Runtime.Serialization.Formatters.Soap <Serializable()> Class Customer Public name As String Public id As Long End Class Module Test Sub Main() Dim list As New ArrayList() Dim cust As New Customer() cust.name = "Charles Darwin" cust.id = 10 list.Add(cust) cust = New Customer() cust.name = "Isaac Newton" cust.id = 20 list.Add(cust) Dim x As Customer For Each x In list Console.WriteLine(x.name & ": " & x.id) Next Console.WriteLine("Saving Customer List") Dim s As New FileStream("cust.txt", _ FileMode.Create) Dim f As New SoapFormatter() f.Serialize(s, list) s.Close() Console.WriteLine("Restoring to New List") s = New FileStream("cust.txt", FileMode.Open) f = New SoapFormatter() Dim list2 As ArrayList list2 = CType(f.Deserialize(s), ArrayList) s.Close() For Each x In list2 Console.WriteLine(x.name & ": " & x.id) Next Console.WriteLine("Press enter to continue...") Console.ReadLine() End Sub End Module We have defined a Customer class with two fields: a name and an id . The program first creates an instance of a collection class that will be used to hold instances of the Customer class. We add two Customer objects to the collection and then print out the contents of the collection. The collection is then saved to disk. It is restored to a new collection instance and printed out. The results printed out will be identical to those printed out before the collection was saved. [1] [1] The sample installation should have provided an executable file Serialize.exe that you can run. If not, double-click on the Visual Studio .NET solution file that has the .sln suffix. When Visual Studio comes up, hit Control-F5 to build and run the sample. We wrote no code to indicate how the fields of the customer object are saved or restored. We did have to specify the format (SOAP) and create the medium to which the data was saved. The .NET Framework classes are partitioned so that where you load/save, the format you use to load/save, and how you load/save can be chosen independently. This kind of partitioning exists throughout the .NET Framework. The Customer class was annotated with the Serializable attribute in the same way the Public attribute annotates the name field. If you do not want your objects to be serializable, do not apply the attribute to your class. If an attempt is then made to save your object, an exception will be thrown and the program will fail. [2] [2] Comment out the Serializable attribute in the program (you can use the Visual Basic comment ' syntax) and see what happens. Attribute-based programming is used extensively throughout .NET to describe how the Framework should treat code and data. With attributes, you do not have to write any code; the Framework takes the appropriate action based on the attribute. Security can be set through attributes. You can use attributes to have the Framework handle multithreading synchronization. Remoting of objects becomes straightforward through the use of attributes. The compiler adds this Serializable attribute to the metadata of the Customer class to indicate that the Framework should save and restore the object. Metadata is additional information about the code and data within a .NET application. Metadata, a feature of the Common Language Runtime, provides such information about the code as -
Version and locale information. -
All the types. -
Details about each type, including name, visibility, and so on. -
Details about the members of each type, such as methods, the signatures of methods , and the like. -
Attributes. Since metadata is stored in a programming-language-independent fashion with the code, not in a central store such as the Windows Registry, it makes .NET applications self-describing . The metadata can be queried at run-time to get information about the code (such as the presence or absence of the Serializable attribute). You can extend the metadata by providing your own custom attributes. In our example, the Framework can query the metadata to discover the structure of the Customer object in order to be able to save and restore it. Types Types are at the heart of the programming model for the CLR. A type is analogous to a class in most object-oriented programming languages, providing an abstraction of data and behavior, grouped together. A type in the CLR contains -
Fields (data members) -
Methods -
Properties -
Events There are also built-in primitive types, such as integer and floating-point numeric types, string, and so on. We will discuss types under the guise of classes and value types when we cover VB.NET. | .NET Framework Class Library The Formatter and FileStream classes are just two of more than 2,500 classes in the .NET Framework that provide plumbing and system services for .NET applications. Some of the functionality provided by the .NET Framework includes -
Base class library (basic functionality such as strings, arrays, and formatting) -
Networking -
Security -
Remoting -
Diagnostics -
I/O -
Database -
XML -
Web services that allow us to expose component interfaces over the Internet -
Web programming -
Windows user interface Interface-Based Programming Suppose you want to encrypt your data and therefore do not want to rely on the Framework's serialization. Your class can inherit from the ISerializable interface and provide the appropriate implementation. (We will discuss how to do this in a later chapter.) The Framework will then use your methods to save and restore the data. How does the Framework know that you implemented the ISerializable interface? It can query the metadata related to the class to see if it implements the interface! The Framework can then use either its own algorithm or the class's code to serialize or deserialize the object. Interface-based programming is used in .NET to allow your objects to provide implementations to standard functionality that can be used by the Framework. Interfaces also allow you to program using methods on the interface rather than methods on the objects. You can program without having to know the exact type of the object. For example, the formatters (such as the SOAP formatter used here) implement the IFormatter interface. Programs can be written using the IFormatter interface and thus are independent of any particular current (binary, SOAP) or future formatter and still work properly. Everything Is an Object So if a type has metadata, the runtime can do all kinds of wonderful things. But does everything in .NET have metadata? Yes! Every type, whether it is user-defined (such as Customer ) or part of the Framework (such as FileStream ), is a .NET object. All .NET objects have the same base class, the system's Object class. Hence, everything that runs in .NET has a type and therefore has metadata. In our example, the serialization code can walk through the ArrayList of customer objects and save each one as well as the array it belongs to, because the metadata allows it to understand the object's type and its logical structure. Common Type System The .NET Framework has to make some assumptions about the nature of the types that will be passed to it. These assumptions are the Common Type System (CTS). The CTS defines the rules for the types and operations that the Common Language Runtime will support. It is the CTS that limits .NET classes to single implementation inheritance. Since the CTS is defined for a wide range of languages, not all languages need to support all features of the CTS. The CTS makes it possible to guarantee type safety, which is critical for writing reliable and secure code. As we noted in the previous section, every object has a type, and therefore every reference to an object points to a defined memory layout. If arbitrary pointer operations are not allowed, the only way to access an object is through its public methods and fields. Hence, it is possible to verify an object's safety by analyzing only the object. There is no need to know or analyze all the users of a class. How are the rules of the CTS enforced? The Microsoft Intermediate Language (MSIL or IL) defines an instruction set that is used by all .NET compilers. This intermediate language is platform-independent. The MSIL code can later be converted to a platform's native code. Verification for type safety can be done once based on the MSIL; it need not be done for every platform. Since everything is defined in terms of MSIL, we can be sure that the .NET Framework classes will work with all .NET languages. Design no longer dictates language choice; language choice no longer constrains design. MSIL and the CTS make it possible for multiple languages to use the .NET Framework, since their compilers produce MSIL. This is one of the most visible differences between .NET and Java, which in fact share a great deal in philosophy. ILDASM The Microsoft Intermediate Language Disassembler (ILDASM) can display the metadata and MSIL instructions associated with .NET code. It is a very useful tool both for debugging and for increasing your understanding of the .NET infrastructure. You can use ILDASM to examine the .NET Framework code itself. [3] Figure 2-1 shows a fragment of the MSIL code from the Serialize example, where we create two new customer objects and add them to the list. [4] [3] ILDASM is installed on the Tools menu in Visual Studio .NET. It is also found in the Microsoft.NET\FrameworkSDK\Bin subdirectory. You can invoke it by double-clicking on its Explorer entry or from the command line. If you invoke it from the command line (or from VS.NET), you can use the /ADV switch to get some advanced options. [4] Open Serialize.exe and click on the plus (+) sign next to Test. Double-click on Main to bring up the MSIL for the Main routine. Figure 2-1. Code fragment from Serialize example.  The newobj instruction creates a new object reference using the constructor parameter. Stloc stores the value in a local variable. Ldloc loads a local variable. [5] [5] You can read about the MSIL instruction set and the CLR in the book Compiling for the .NET Common Language Runtime (CLR) by John Gough, Prentice Hall, 2002. Language Interoperability Having all language compilers use a common intermediate language and common base class make it possible for languages to interoperate . But since all languages need not implement all parts of the CTS, it is certainly possible for one language to have a feature that another does not. The Common Language Specification (CLS) defines a subset of the CTS representing the basic functionality that all .NET languages should implement if they are to interoperate with each other. This specification enables a class written in Visual Basic .NET to inherit from a class written in C++ with Managed Extensions or C#, or to make interlanguage debugging possible. An example of a CLS rule is that method calls need not support a variable number of arguments, even though such a construct can be expressed in MSIL. CLS compliance applies only to publicly visible features. A class, for example, can have a private member that is non-CLS compliant and still be a base class for a class in another .NET language. For example, VB.NET code should not define public and protected class names that differ only by case sensitivity, since languages such as VB.NET are not case sensitive. Private fields could have case-sensitive names. Microsoft itself is providing several CLS-compliant languages: C#, Visual Basic .NET, and C++ with Managed Extensions. Third parties are providing additional languages (there are over a dozen so far). ActiveState is implementing Perl and Python. Fujitsu is implementing COBOL. Managed Code In the serialization example, a second instance of the Customer object was assigned to the same variable ( cust ) as the first instance without freeing it. None of the allocated storage in the example was ever deallocated. .NET uses automatic garbage collection to reclaim memory. When memory allocated on the heap becomes orphaned, or passes out of scope, it is placed on a list of memory locations to be freed. Periodically, the system runs a garbage collection thread that returns the memory to the heap. By having automatic memory management, the system has eliminated memory leakage, which is one of the most common programming errors. In most cases, memory allocation is much faster with garbage collection than with classic heap allocation schemes. Note that variables such as cust and list are object references, not the objects themselves. This makes the garbage collection possible. Garbage collection is one of several services provided by the Common Language Runtime (CLR) to .NET programs. Data that is under the control of the CLR garbage collection process is called managed data. Managed code is code that can use the services of the CLR. .NET compilers that produce MSIL can produce managed code. Code is typically verified for type safety before compilation. This step is optional and can be skipped for trusted code. One of the most significant differences between verified and unverified code is that verified code cannot use pointers. Code that uses pointers could subvert the CTS and access any memory location. You cannot program with pointers in VB.NETthat is one of the differences between VB.NET and C#. Type-safe code cannot be subverted. A buffer overwrite is not able to corrupt other data structures or programs. Methods can only start and end at well-defined entry and exit points. Security policy can be applied to type-safe code. For example, access to certain files or user interface features can be allowed or denied . You can prevent the execution of code from unknown sources. You can prevent access to unmanaged code to prevent subversion of .NET security. Type safety also allows paths of execution of .NET code to be isolated from one another. Assemblies Another function of the CLR is to load and run .NET programs. .NET programs are deployed as assemblies. An assembly is one or more EXEs or DLLs with associated metadata information. The metadata about the entire assembly is stored in the assembly's manifest. The manifest contains, for example, a list of the assemblies upon which this assembly is dependent. In our Serialize example there is only file in the assembly, Serialize.exe . That file contains the metadata as well as the code. Since the manifest is stored in the assembly and not in a separate file (like a type library or registry), the manifest cannot get out of sync with the assembly. Figure 2-2 shows the metadata in the manifest for this example. [6] Note the assembly extern statements that indicate the dependencies on the Framework assemblies mscorlib and System.Runtime.Formatters.SOAP . These statements also indicate the version of those assemblies that serialize.exe depends on. [6] Open Serialize.exe in ILDASM and double-click on the MANIFES item. Figure 2-2. Manifest for the Serialize assembly.  Assemblies can be versioned, and the version is part of the name for the assembly. To version an assembly it needs a unique name. Public/private encryption keys are used to generate a unique (or strong) name. Assemblies can be deployed either privately or publicly. For private deployment, all the assemblies that an application needs are copied to the same directory as the application. If an assembly is to be publicly shared, an entry is made in the Global Assembly Cache (GAC) so that other assemblies can locate it. For assemblies put in the GAC, a strong name is required. Since the version is part of the assembly name, multiple versions can be deployed side by side on the same machine without interfering with each other. Whether you use public or private deployment, there is no more DLL Hell. Assembly deployment with language interoperability makes component development almost effortless. We discuss issues of deployment in Chapter 9. JIT Compilation Before executing on the target machine, MSIL has to be translated into the machine's native code. This can be done either before the application is called or at runtime. At runtime, the translation is done by a just-in-time (JIT) compiler. The Native Image Generator (Ngen.exe) translates MSIL into native code ("pre-translation") so that it is already translated when the program is started. The advantage of pre-translation is that optimizations can be performed. Optimizations are generally impractical with JIT because the time it takes to do the optimization can be longer than it takes to compile the code. Startup time is also faster with pre-translation because no translation has to be done when the application starts. The advantage of JIT is that it knows what the execution environment is when the program is run and can make better assumptions, such as register assignments, when it generates the code. Only the code that is actually executed is translated; code that never gets executed is never translated. In the first release of .NET, the Native Image Generator and the JIT compiler use the same compiler. No optimizations are done for Ngen; its only current advantage is faster startup. For this reason, we do not discuss Ngen in this book. Performance You may like the safety and ease-of-use features of managed code, but you might be concerned about performance. Early assembly language programmers had similar concerns when high-level languages came out. The CLR is designed with high performance in mind. With JIT compilation, the first time a method is encountered , the CLR performs verifications and then compiles the method into native code (which will contain safety features, such as array bounds checking). The next time the method is encountered, the native code executes directly. Memory management is designed for high performance. Allocation is almost instantaneous, just taking the next available storage from the managed heap. Deallocation is done by the garbage collector, which has an efficient multiple-generation algorithm. You do pay a penalty when security checks have to be made that require a stack walk, as we will explain in Chapter 16. Web pages use compiled code, not interpreted code. As a result, ASP.NET is much faster than ASP. For 98 percent of the code that programmers write, any small loss in performance is far outweighed by the gains in reliability and ease of development. High-performance server applications might have to use technologies such as ATL Server and C++.  | 
