Why Use an Existing Scripting Language

After you determine whether you need a scripting language, you must decide whether to use an existing one or roll your own. This decision has to be made thoughtfully because it can have serious implications either way.

Creating and maintaining a scripting language from scratch is a significant undertaking. Having good scripting support for a game means that both the language and the interpreter must be designed, implemented, and maintained well. Many games use in-house scripting languages that are very poor, many times because the task was underestimated. Other reasons include lack of careful design or the assumption that the language would be used only to solve the specific problem that was originally in mind.

Even though scripting language is adopted in games to solve single specific tasks, such as menus, by the time a game is done scripts tend to be used for far more things than originally intended. Even if a scripting language stays true to its specific mission, by the time the next game comes around, it is just too inconvenient to give up a system in which the team has invested so much time and is so familiar. Of course, a new project can require tasks that are entirely different from those of the original game for which the scripting language was designed.

For example, BGScript, which is the scripting language created for Baldur’s Gate, was intended only for scripting the combats. Yet, it was eventually used in multiple games and used for tasks that were beyond the sight and intention of its original designers. By the time the language and the interpreter were changed to meet the radical new requirements, it was nearly unusable. It become so convoluted and hard to use that after the substantial amount of time and money spent for its development and maintenance, it had to be put aside. In fact, two of the developers at BioWare wrote an article titled “How Not to Implement a Basic Scripting Language,” which discusses all the problems they had.

As another example, Ensemble Studios had used a custom scripting language for Age of Empires II. Even though the language was well designed and thought out, it suffered from inherit limitations that were not a problem or concern for AOE II. The limitations were enough to render the system inherently inappropriate for Age of Mythology. The language allowed a developer to define rules by mapping a set of conditions to a set of actions. The engine behind the script would trigger a rule when its conditions were met. When a rule was triggered, its corresponding actions were executed. This rule-based system was simply not powerful enough to represent and manipulate data that was more sophisticated. A rule-based system is in a sense similar to a function that has a list of if-then checks. What if there is a need to loop through a list of objects? What if there is a need to define and use custom data structures to represent alternate views of the world to give the NPCs a more humanlike view? Such tasks could not be implemented directly in the scripts.

You should be cautious about designing a new language, even if it is a simple one. If a language is very simple, it is likely that it will need a lot of growth before it can handle more sophisticated tasks. Over the years, a substantial amount of effort has been put into designing the languages we use today. Even if you were to create a custom scripting language, you should use the syntax and rules of an existing one. When possible, you should make every effort to avoid creating a new language. By adopting an existing scripting language, you can save a lot of time and money by using a language that has matured and been used by many developers.

Keep in mind that if you have a large number of scripts written and you need to make a change that breaks the backward compatibility of the language, you either must make double standards to hack the change or modify all the old scripts. As you can guess, neither solution is pleasant or safe. As a mater of fact, you can use a subset of an existing language. This is what Quake III does. They use a subset of C as their scripting language. Unreal uses a Java-like language and WarCraft III uses a language similar to Basic.

If you don’t go with an existing scripting language, after you design a language or go with the syntax of an existing language, you must write a parser and a compiler. Writing a parser is not necessarily a difficult task. Actually, it sounds easy enough to make anyone think that it can be written without any problem. Writing a parser, however, is very time consuming, and the code tends to be very long. A full-blown scripting language has so many dependencies that it will be rather easy to create bugs and even harder to maintain. Even if you write your own scripting language, do not write the parser. Writing your own parser is the easiest way to waste a lot of time. Instead, use Lex and Yacc or their equivalent tools to alleviate some of the burden.

Making your own language means that you also have to design and implement your own interpreter. An interpreter is composed of components that can verify and translate the source to the language of the virtual machine, and execute the compiled representation, and a memory manager, which can allocate, free, and compact the memory used by the scripting engine. How much time does it take to design, implement, and test these components? They are definitely not trivial tasks. The interpreter is actually more important than the specific scripting language. One way to measure the amount of work involved is to look at an existing interpreter and try to see how much time went into it to make it work well. These tasks are so involved that they can take a single person a year of solid research, design, and development before the interpreter can be used. Many companies that use an in-house scripting language have a dedicated developer to design, implement, and maintain the system. If this is not the case, it is because they cannot afford to do so, they do not realize how big of a task it is, or someone else is maintaining the language for them, perhaps an independent group or a third-party company.

Plenty of open-source systems are available that can be used to avoid implementing such a system from scratch. Of course, if a studio can afford to tackle the problem by having dedicated developers, it will work out just fine. For example, even though Quake III uses C files as its scripting language, the C files are compiled to bytecodes that are interpreted by the Quake Virtual Machine (QVM).

It is also important to know that compilers exist that can convert existing languages to the bytecodes of an existing virtual machine. This is exactly what Jython (or JPython) does. Jython is a compiler that compiles Python code to Java bytecode. Many different compilers are available that can convert other languages to Java bytecodes. Similarly, Microsoft .NET VM is just a C# VM. Languages such as Visual Basic , Managed C++, and J# are simply compilers that convert different languages to C# bytecodes

It is also important to note that if you want to use an existing language to run on a platform for which a VM does not exist, you must compile the interpreter for the specific platform. Interpreters written in ANSI C can easily be compiled for various platforms. For example, Lua can be compiled to run on PCs as well as on PS2.