Chapter 3: Main Paradigms of the Executable Code Analysis

Overview

In Section 2.4, I provided several simple examples of analysis and correction of executable code. The main goal of this chapter is to provide some theoretical basis. Using the material provided here, you'll be able to investigate more difficult cases.

When studying code analysis, it is necessary to understand that this is not the same thing as decompilation (conversion of binary executable code into a program written in some high-level programming language). Although I will provide analysis of the algorithmic structure and main constructs of high-level programming languages in this chapter, its main goal isn't reconstruction of the program's source code (which in general is impossible). Rather, the main aim of this material is to provide an understanding of the program operating logic. Examples in Section 2.4 demonstrate code analysis techniques aimed at solving specific tasks (code analysis in the specified context). These problems were solved without even trying to understand, which constructs of specific programming language were used. However, you won't be able to solve more difficult problems without knowing these constructs, understanding how they are converted during the compilation, and discovering the form, in which they are present in the binary code after the compilation is completed.

Even for a single programming language there might be lots of different compilers — consider the C++ programming language. In addition, every compiler usually has several compilation modes, which, as a rule, are related to the methods of optimizing the resulting code and adding various check procedures into it (such as checks for going beyond the buffer limits). All of these concepts are illustrated in Fig. 3.1. In general, it is impossible to study this entire hierarchy. This is not a serious problem, however, because studying this is unnecessary. The only thing that you need to do is understand the patterns, according to which the executable code is formed.

Figure 3.1: The language-executable code hierarchy

I hope that the material provided in this chapter will help you master these patterns. The executable code in this chapter is all analyzed on the basis of IDA Pro, which is the best contemporary disassembler. Reference information about this disassembler will be provided in Chapter 5.