Chapter 12: Self-Modification Basics

image from book  Download CD Content

Self-modifying code is encountered in many viruses, protection mechanisms, network worms, cracksims, and other programs of this sort . Although the technique of creating isn't a secret, high-quality implementations become fewer in number with every year. An entire generation of hackers has grown up believing that self-modification under Windows is either impossible or too difficult. In reality, however, this is not so.

Getting Acquainted with Self-Modifying Code

Covered with a veil of mystery and accompanied with an unimaginable number of myths, legends, and puzzles, self-modifying code has unavoidably become a thing of the past. The golden age of the self-modification is gone. By the time noninter-active debuggers such as debug.com and disassemblers such as Sourcer appeared, self-modification seriously complicated code analysis. However, with the release of IDA Pro and Turbo Debugger the situation has changed.

Self-modification doesn't prevent tracing; therefore, it is "transparent" for the debugger. The situation is slightly more difficult with static analysis. A disassembler displays the program in the form it had at the moment of creating a dump or loading from the source file. It implicitly relies on the assumption that no machine command has changed in the course of its execution. Otherwise, reconstruction of the algorithm would be incorrect and the hacker's boat would have an enormous leak. However, if self-modification were detected , then there would be no difficulties correcting the disassembled listing. Consider the example in Listing 12.1.

Listing 12.1: An example of inefficient use of self-modifying code
image from book
 FE 05 ... INC byte ptr DS:[foo] ; Replace JZ (opcode 74 xx)                                 ; with JNZ (75 xx).  33 C0     XOR EAX, EAX          ; Set the zero flag.   foo:   74 xx     JZ bar                ; Jump, if the zero flag is set.  E8 58 ... CALL protect_proc     ; Call to the protected function. 
image from book
 

Analyze the lines in bold. First, the program resets the EAX register to zero, sets the zero flag, and then, if this flag is set (and it is), goes to the foo label. However, the sequence of actions is the reverse of the one described. It is missing one important detail. The inc byte prt ds:[foo] construct inverts the conditional jump command and the protect_proc procedure gains control instead of the bar label. An excellent protection technique, isn't it? I don't want to disappoint you, but any clever hacker will immediately notice the inc byte prt ds:[foo] construct, because it inevitably catches the eye, and will disclose the dirty trick.

What if this construct is placed in a different branch of the program, far from the code being modified? This trick might be successful with any other disassembler, but not with IDA Pro. Consider the cross-reference that IDA Pro has automatically created (Listing 12.2), which points directly to the inc byte ptr loc_40100f line.

Listing 12.2: IDA Pro automatically recognizes the self-modification of the code
image from book
 text:00400000   INC  byte ptr loc_40100F ; Replacing JZ with JNZ text:00400000 ; text:00400000 ; Multiple lines of code here text:00400000 ; text:0040100D   XOR  EAX,  EAX text:0040100F text:0040100F loc_40100F:  ; DATA XREF: .text:00401006     w  text:0040100F   JZ   short  loc_401016  ; Reference to the  text:0040100F  ; self-modifying code  text:00401011   CALL xxxx 
image from book
 

That's it! Self-modification in its pure form doesn't solve any problem, and its fate is predefined if no additional protection measures are taken. The best means of overcoming cross-references is a textbook in mathematics for elementary school. This is not a joke! Elementary arithmetic operations with pointers blind the automatic analyzer built into IDA Pro and make cross-references miss their targets.

An improved variant of self-modifying code might appear, for example, as shown in Listing 12.3.

Listing 12.3: Improved version of self-modifying code that deceives IDA Pro
image from book
  MOV  EAX,  offset foo + 669h ; Direct EAX to the false target.   SUB  EAX,  669h              ; Correct the target.  INC  byte  ptr DS:[ERX]      ; Replace JZ with JNZ.       ;       ; Multiple lines of code here       ;       XOR EAX, EAX                 ; Set the zero flag. foo:       JZ   bar                     ; Jump if the zero flag is set.       CALL protect_proc            ; Call the protected function. 
image from book
 

What happens in this case? In first place, the offset of the command to be modified increased by some value (conventionally called delta) is loaded into the EAX register. It is important to understand that these computations are carried out by the translator at the compile stage and only the final result is included in the machine code. Then the delta value correcting the target is subtracted from the EAX register. This aims EAX directly at the code that needs to be modified. Provided that the disassembler doesn't contain a CPU emulator and doesn't trace pointers (and IDA Pro doesn't carry out these actions), these commands create a single cross-reference aimed at a fictitious target far from the "battleground" and irrelevant to the self-modifying code. At the same time, if the fictitious target is located in the region beyond the limits of [image Base; image Base + image Size ] , cross-reference won't be created.

The disassembled code obtained using IDA PRO (Listing 12.4) confirms this statement.

Listing 12.4: Disassembled listing of self-modifying code without cross-references produced by IDA Pro
image from book
 .text:00400000 MOV  EAX,  offset  _printf+3 ; Fictitious target  .text:00400005 SUB  EAX,  669h             ; Stealth correction                                            ; of the target .text:0040000A INC  byte  ptr [eax]        ; Replace  JZ  with JNZ. .text:00401013 XOR  EAX,  EAX  .text:00401015 JZ   short  loc_40101C      ; No cross-reference!  .text:00401017 CALL protect_proc 
image from book
 

The generated conditional jump points into the middle of the _printf library function, which happened to be at that location. The self-modifying code doesn't attract attention in the background behind other machine commands. Thus, the cracker cannot be sure that it is jz and not jnz . In this case, such a trick doesn't seriously complicate the analysis because the protected procedure ( protect_proc ) is nearby. In fact, it is under the very nose of the hacker, so the attention of any true hacker will be immediately drawn to it. However, if you implement a selfmodifying algorithm that checks serial numbers by replacing ror with rol , the hacker will swear for a long time, wondering why the keygen doesn't work. After the hacker starts the debugger, the swearing will become louder because the hacker will find the deceiving trick with stealthy replacement of one command with another one. By the way, most hackers proceed in exactly this way, using a debugger and disassembler in combination.

More advanced protection technologies are based on dynamic code encryption. Note that encryption is a kind of self-modification. Until the moment the binary code is fully decrypted, it is unsuitable for disassembling . If the decryptor is stuffed with an antidebugging technique, direct debugging also becomes impossible.

Static encryption (typical for most anticrack protectors) is considered hopeless and having no future. This is because the hacker can wait until the moment of decryption completion, make a dump, and then investigate it using standard tools. Protection mechanisms try to thwart such an attempt. They corrupt the import table, overwrite the PE header, set the page attributes to NO_ACCESS , and so on. However, experienced hackers cannot be deceived by these tricks and cracking cannot be retarded for long. Any such protector, even the most sophisticated one, can be easily removed manually, and for some standard protectors there are even automatic crackers.

At no instance must the program code be decrypted entirely . Adopt one important rule ” decrypt the code by fragments , one at a time. At the same time, the de-cryptor must be designed to ensure that the hacker cannot use it for decrypting the program. The typical vulnerability of most protection mechanisms is as follows : The hacker finds the entry point of the decryptor, restores its prototype, and then uses it to decrypt all of the encrypted block, thus obtaining a usable dump at the output. Furthermore, if the decryptor is a trivial XOR , it will be enough to find where the keys are stored; then the hacker will be able to decrypt the program on his or her own.

To avoid these situations, protection mechanisms must use polymorphous technologies and code generators. It is practically impossible to automate the decryption of a program composed of hundreds of fragments encrypted using dynamically-generated encryptors. However, such protection is hard to implement. Before taking on ambitious goals, it is better to concentrate on the basics.



Shellcoder's Programming Uncovered
Shellcoders Programming Uncovered (Uncovered series)
ISBN: 193176946X
EAN: 2147483647
Year: 2003
Pages: 164

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net