Alternative Implementation Strategies

Python for .NET

An alternative implementation strategy for Python for .NET would be to leverage CPython and attempt to implement .NET support as a regular Python extension module. This was not considered at the outset of the project, as a primary motivation for Microsoft's involvement was to prove that the .NET runtime system and Intermediate Language were capable of supporting the language. The existing implementation demonstrates that this is indeed true, so now that the focus must switch to providing the best compatibility and/or interoperability with CPython, this idea warrants consideration.

A full analysis of this option is beyond the scope of this appendix. However, there are a number of obvious limitations with this approach:

• Python code would not be verifiable . Thus, it would be impossible to write trusted content in Python.

• Other .NET languages would be unable to inherit from (or otherwise treat as a class) Python objects. It is quite likely that the reverse is possible: CPython programs would be able to inherit from .NET classes. This may be a trade-off worth making. Additionally, enhancements to .NET could assist in this goal.

• Any other .NET tools that require .NET support, such as possibly ASP.NET or the Windows Forms designer tool, would not support Python.

To all intents and purposes, this option is supported today using existing Python and .NET facilities. The .NET COM Interoperability features mean that many .NET features are available today. Indeed, the compiler itself depends on this to be able to use Reflection::Emit .

The Compiler

A number of alternative strategies were considered for the compiler.

Use a Different Intermediate Language

This solution would involve compiling the Python source code and generating some other .NET-compatible language rather than .NET Intermediate Language. The most likely target language for this scenario would be C#.

It was decided that the Python compiler should target .NET IL if at all possible. It was felt that this would provide the tightest integration with the .NET Framework, particularly in areas such as debugging and diagnostics. As there appeared to be no significant impediment to using IL directly, this option was rejected.

Use the Unmanaged Emit API

The "unmanaged emit API" is a traditional, C-style DLL in the standard Windows tradition. The Reflection::Emit library is a set of .NET classes. Both APIs are designed to take .NET IL and create .NET assemblies.

This solution would involve using the "unmanaged" .NET API instead of the .NET classes. However, there were two reasons why this option was rejected:

• The advice of the Microsoft staff was that the Reflection::Emit APIs are the new official way of creating assemblies. The unmanaged API existed only to provide a solution to the boot-strapping problem; you can't create a managed .NET API for compilers until you have compilers to write the API in!

• Accessing these unmanaged APIs from Python code was more difficult than accessing a COM API.

Using the Reflection::Emit classes appeared to offer the shortest implementation time, so this alternative was chosen over the unmanaged option.

The Runtime

There were no identified alternatives to the basic scheme of having a .NET runtime implement many of the Python semantics. Although improvements to the compiler would mean less reliance on the runtime system, it is believed that the compiler will always require the runtime for some operations.

There are, however, a number of different implementation strategies possible. The first version of the compiler always works with .NET object references, rather than the PyObject structure as defined earlier in this appendix. However, this meant that any given Object had its IPyType interface looked up many times throughout its life, which had a large performance impact. Performance testing showed that using a value type (instead of a .NET class) provided the best performance.

The Microsoft support staff used their internal testing tool to determine that the IPyType interface lookup for a given .NET object is the biggest hot spot in the runtime. However, as these performance tools are not available to the Python for .NET project, meaningful tuning work is not possible. Hopefully, accessible performance analysis tools will appear soon, allowing further analysis of the runtime performance.

The Library

The Python for .NET compiler and runtime have been designed to allow arbitrary .NET assemblies to be used as Python modules. Thus, it should be possible to write the standard library in any language. The modules written in Python and compiled by the .NET compiler could have been written in C#. However, it was felt that writing the library in Python wherever possible would provide significant benefits, as witnessed in CPython.

Another strategy that has not been investigated would be to port the existing C modules to managed C++ or C#. This has not been explored, partly because the managed C++ compiler was previously not stable enough for serious use, but primarily because it seemed a huge and improbable task. It should be investigated, as there is a huge body of existing C code that is otherwise useless in the Python for .NET environment.

The only other identified strategy would be to attempt to have the compiler inline all the standard library calls. Although this is done for some common built-in functions, it was not considered a viable option for the entire standard library and would place too great a burden on compiler maintenance.