15.5 NTLM Version 2

NTLMv2, as it's called, has some additional safeguards thrown into the recipe that make it more complex and hopefully more secure than its predecessors. There are, however, two small problems with NTLMv2:

• Good documentation on the inner workings of NTLMv2 is rare.

• Although it is widely available, NTLMv2 does not seem to be widely used.

Regarding the first point, Appendix B of Luke K. C. Leighton's book DCE/RPC over SMB: Samba and Windows NT Domain Internals provides a recipe for NTLMv2 authentication. We'll do our best to expand on Luke's description. The other option, of course, is to look at available Open Source code.

The second point is really a conjecture, based in part on the fact that it took a very long time to get NTLMv2 implemented in Samba and few seemed to care. Indeed, NTLMv2 support had already been added to Samba-TNG by Luke and crew, and needed only to be copied over. It seems that the delay in adding it to Samba was not a question of know-how, but of priorities.

Another factor is that NTLMv2 is not required by default on most Windows systems. When challenge/response is negotiated, even newer Windows versions will default to using the LM/NTLM combination unless they are specifically configured not to.

15.5.1 The NTLMv2 Toolbox

We have already fussed with the DES algorithm and toyed with the MD4 algorithm. Now we get to use the HMAC-MD5 Message Authentication Code hash. This one's a power tool with razor -sharp keys and swivel-action hashing. The kind of thing your Dad would never let you play with when you were a kid. Like all good tools, though, it's neither complex nor dangerous once you learn how it works.

HMAC-MD5 is actually a combination of two different algorithms: HMAC and MD5. HMAC is a M essage A uthentication C ode (MAC) algorithm that takes a hashing function (such as MD5) and adds a secret key to the works so that the resulting hash can be used to verify the authenticity of the data. The MD5 algorithm is basically an industrial-strength version of MD4. Put them together and you get HMAC-MD5.

HMAC-MD5 is quite well documented, ^[13] and there are a lot of implementations available. It's also much less complicated than it appears in Figure 15.3, so we won't need to go into any of the details. For our purposes, what you need to know is that the HMAC_MD5() function takes a key and some source data as inputs, and returns a 16-byte (128-bit) output.

^[13] MD4 is explained in RFC 1320 and MD5 is in RFC 1321; HMAC in general, and HMAC-MD5 in particular, is written up in RFC 2104 an embarrassment of riches! As usual with this sort of thing, a deeper understanding can be gained by reading about it in Bruce Schneier's Applied Cryptography, Second Edition (see the References section).

Figure 15.3. HMAC-MD5

The HMAC-MD5 is a popular tool for use in message authentication. It is lightweight, powerful, efficient, and ergonomic.

Hmmm... Well, it's not actually quite that simple. See, MD4, MD5, and HMAC-MD5 all work with variable-length input, so they also need to know how big their input parameters are. The function call winds up looking something like this:

 hash16 = HMAC_MD5(Key, KeySize, Data, DataSize); 

There is, as it turns out, more than one way to skin an HMAC-MD5. Some implementations use a whole set of functions to compute the result:

• the first function accepts the key and creates an initial context ,

• the second function may be called repeatedly, each time passing the context and the next block of data,

• and the final function is used to close the context and return the resulting hash.

Conceptually, though, the multi-function approach is the same as the simpler example shown above. That is: Key and Data in, 16-byte hash out.

Not Quite Entirely Unlike Standard Alert

The HMAC-MD5 function can handle very large Key inputs. Internally, though, there is a maximum keysize of 64 bytes. If the key is too long, the function uses the MD5 hash of the key instead. In other words, inside the HMAC_MD5() function there is some code that does this:

 if(KeySize > 64)   {   Key = MD5(Key, KeySize);   KeySize = 16;   } 

In his book, Luke explains that the function used by Windows systems is actually a variation on HMAC-MD5 known as HMACT64, which can be quickly defined as follows :

 #define HMACT64(K, Ks, D, Ds) \         HMAC_MD5(K, ((Ks > 64)?64:Ks), D, Ds) 

In other words, the HMACT64() function is the same as HMAC_MD5() except that it truncates the input Key to 64 bytes rather than hashing it down to 16 bytes using the MD5() function as prescribed in the specification.

As you read on, you will probably notice that the keys used by the NTLMv2 challenge/response algorithm are never more that 16 bytes, so the distinction is moot for our purposes. We bother to explain it only because HMACT64() may be used elsewhere in CIFS (in some dark corner that we have not visited) and it might be a useful tidbit of information for you to have.

Another important tool is the older NTLM hash algorithm. It was described earlier but it is simple enough that we can present it again, this time in pseudo-code:

 uchar *NTLMhash(uchar *password)   {   UniPasswd = UCS2LE(password);   KeySize   = 2 * strlen(password);   return(MD4(UniPasswd, KeySize));   } 

The ASCII password is converted to Unicode UCS-2LE format, which requires two bytes per character. The KeySize is simply the length of that (Unicode) password string, which we calculate here by doubling the ASCII string length (which is probably cheating). Finally, we generate the MD4 hash (that's MD 4 , not MD 5 ) of the password, and that's all there is to it.

Note that the string terminator is not counted in the KeySize . That is common behavior for NTLM and NTLMv2 challenge/response when working with Unicode strings.

The NTLM Hash is of interest because the SMB/CIFS designers at Microsoft (if indeed such people truly exist any more, except in legend) used it to cleverly avoid upgrade problems. With LM and NTLM, the hash is created from the password. Under NTLMv2, however, the older NTLM (v1) Hash is used instead of the password to generate the new hash. A server or Domain Controller being upgraded to use NTLMv2 may already have the older NTLM hash values in its authentication database. The stored values can be used to generate the new hashes no password required. That avoids the nasty chicken-and-egg problem of trying to upgrade to NTLMv2 Hashes on a system that only allows NTLMv2 authentication.

15.5.2 The NTLMv2 Password Hash

The NTLMv2 Hash is created from:

• the NTLM Hash (which, of course, is derived from the password),

• the user 's username, and

• the name of the logon destination.

The process works as shown in the following pseudo-code example:

 v1hash  = NTLMhash(password); UniUser = UCS2LE(upcase(user)); UniDest = UCS2LE(upcase(destination)); data    = uni_strcat(UniUser, UniDest); datalen = 2 * (strlen(user) + strlen(destination)); v2hash  = HMAC_MD5(v1hash, 16, data, datalen); 

Let's clarify that, shall we?

v1hash

The NTLM Hash, calculated as described previously.

UniUser

The username, converted to uppercase UCS-2LE Unicode.

UniDest

The NetBIOS name of either the SMB server or NT Domain against which the user is trying to authenticate.

data

The two Unicode strings are concatenated and passed as the Data parameter to the HMAC_MD5() function.

datalen

The length of the concatenated Unicode strings, excluding the nul termination. Once again, doubling the ASCII string lengths is probably cheating.

v2hash

The NTLM Version 2 Hash.

A bit more explanation is required regarding the destination value (which gets converted to UniDest ).

In theory, the client can use NTLMv2 challenge/response to log into a standalone server or to log into an NT Domain. In the former case, the server will have an authentication database of its very own, but an NT Domain logon requires authentication against the central database maintained by the Domain Controllers.

So, in theory, the destination name could be either the NetBIOS name of the standalone server or the NetBIOS name of the NT Domain (no NetBIOS suffix byte in either case). In practice, however, the server logon doesn't seem to work reliably. The Windows systems used in testing were unable to use NTLMv2 authentication with one another when they were in standalone mode, but once they joined the NT Domain NTLMv2 logons worked just fine. ^[14]

^[14] The lab in the basement is somewhat limited which, in turn , limits my ability to do rigorous testing of esoteric CIFS nuances . You should probably verify these results yourself. Andrew Bartlett (him again!) turned up an interesting quirk regarding the NTLMv2 Response calculation when authenticating against a standalone server. It seems that the NT Domain name is left blank in the v2hash calculation. That is: destination = "";

15.5.3 The NTLMv2 Response

The NTLMv2 Response is calculated using the NTLMv2 Hash as the Key . The Data parameter is composed of the challenge plus a blob of data which we will refer to as "the blob." The blob will be explained shortly. For now, just think of it as a mostly-random bunch of garblement. The formula is shown in this pseudo-code example:

 blob = RandomBytes(blobsize); data = concat(ServerChallenge, 8, blob, blobsize); hmac = HMAC_MD5(v2hash, 16, data, (8 + blobsize)); v2resp = concat(hmac, 16, blob, blobsize); 

Okay, let's take a closer look at that and see if we can force it to make some sense.

1. The first step is blob generation. The blob is normally around 64 bytes in size, give or take a few bytes. The pseudo-code above suggests that the bytes are entirely random, but in practice there is a formula (explained below) for creating the blob.

2. The next step is to append the blob to the end of the challenge. This, of course, is the same challenge sent by the server and used by all of the other challenge/response mechanisms.

   

3. The challenge and blob are HMAC'd using the NTLMv2 Hash as the key.

4. The NTLMv2 Response is created by appending the blob to the tail of the HMAC_MD5() result. That's 16 bytes of HMAC followed by blobsize bytes of blob.

   

If the client sends the NTLMv2 Response, it will take the place of the NTLM Response in the SESSION_SETUP_ANDX.Case Sensitive -Password field. Note that, unlike the older NTLM Response, the NTLMv2 Response algorithm uses 128-bit encryption all the way through.

15.5.4 Creating the Blob

If you have ever taken a college-level Invertebrate Zoology course, you may find the dissection of the blob to be nauseatingly familiar. The rest of you... try not to be squeamish. One more warning before we cut into this: The blob's structure may not matter at all. We'll explain why a little later on.

Okay, now that the disclaimers are out of the way, we can get back to work. The blob does have a structure, which is more or less as follows:

4 bytes

The value seen in testing is consistently 0x01010000 . (Note that those are nibbles , not bits.) The field is broken out as follows:

1 byte

Response type identification number. The only known value is 0x01 .

1 byte

The identification number of the maximum response type that the client understands. Again, the only known value is 0x01 .

2 bytes

Reserved. Must be zero ( 0x0000 ).

4 bytes

The value seen in testing is always 0x00000000 . This field may, however, be reserved for some purpose.

8 bytes

A timestamp, in the same 64-bit format as described back in Section 13.3.1 on page 227.

8 bytes

The "blip": An eight-byte random value, sometimes referred to as the "Client Challenge." More on this later, when we talk about LMv2 challenge/response.

4 bytes

Unknown.

Comments in the Samba-TNG code and other sources suggest that this is meant to be either a 4-byte field or a pair of 2-byte fields. These fields should contain offsets to other data. That interpretation is probably based on empirical observation, but in the testing done for this book there was no pattern to the data in these fields. It may be that some implementations provide offsets and others just fill this space with left-over buffer garbage. Variety is the spice of life.

variable length

A list of structures containing NetBIOS names in Unicode.

4 bytes

Unknown. (Appears to be more buffer garbage.)

The list of names near the end of the blob may contain the NT Domain and/or the server name. As with the names used to generate the NTLMv2 Hash, these are NetBIOS names in uppercase UCS-2LE Unicode with no string termination and no suffix byte. The name list also has a structure:

2 bytes

Name type.

0x0000

Indicates the end of the list.

0x0001

The name is a NetBIOS machine name (e.g. a server name).

0x0002

The name is an NT Domain NetBIOS name.

0x0003

The name is the server's DNS hostname.

0x0004

The name is a W2K Domain name (a DNS name).

2 bytes

The length, in bytes, of the name. If the name type is 0x0000 , then this field will also be 0x0000 .

variable length

The name, in uppercase UCS-2LE Unicode format.

The blob structure is probably related to (the same as?) data formats used in the more advanced security systems available under Extended Security. ^[15]

^[15] Luke Kenneth Casson Leighton's book DCE/RPC over SMB: Samba and Windows NT Domain Internals gives an outline of the structure of the data blob used in NTLMv2 Response creation. Using Luke's book as a starting point, the details presented above were worked out during a late-night IRC session. My thanks to Andrew Bartlett, Richard Sharpe, and Vance Lankhaar for their patience, commitment, and sudden flashes of insight. Thanks also to Luke Howard for later clarifying some of the finer points.

15.5.5 Improved Security Through Confusion

Now that we have the formula worked out, let's take a closer look at the NTLMv2 challenge/response algorithm and see how much better it is than NTLM.

With the exception of the password itself, all of the inputs to NTLMv2 are known or knowable from a packet capture. Even the blob can be read off the wire, since it is sent as part of the response. That means that the problem is still a not-so-simple case of solving for a single variable: the password.

The NTLMv2 Hash is derived directly from the NTLM (v1) Hash. Since there is no change to the initial input (the password), the keyspace is exactly the same. The only change is that the increased complexity of the algorithm means that there are more encryption hoops through which to jump compared to the simpler NTLM process. It takes more computer time to generate a v2 response, which doesn't impact a normal login but will slow down dictionary and brute force attacks against NTLMv2 (though Moore's Law may compensate). Weak passwords (those that are near the beginning of the password dictionary) are still vulnerable.

Another thing to consider is the blob. If the blob were zero length (empty), the NTLMv2 Response formula would reduce to:

 v2resp = HMAC_MD5(v2hash, ServerChallenge); 

which would still be pretty darn secure. So the question is this: Does the inclusion of the blob improve the NTLMv2 algorithm and, if so, how?

Well, see, it's like this... Instead of being produced by the key and challenge alone, the NTLMv2 Response involves the hash of a chunk of semi-random data. As a result, the same challenge will not always generate the same response. That's good, because it prevents replay attacks... in theory.

In practice, the randomness of the challenge should be enough to prevent replay attacks. Even if that were not the case, the only way that the blob could help would be if it, too, were non-repeating and if the server could somehow verify that the blob was not a repeat. That, quite possibly, is why the timestamp is included.

The timestamp could be used to let the server know that the blob is "fresh" that is, that it was created a reasonably short amount of time before it was received. Fresh packets can't easily be forged because the response is HMAC-signed using the v2hash as the key (and that's based on the password which is the very thing the cracker doesn't know). Of course, the timestamp test won't work unless the client and server clocks are synchronized, which is not always the case.

In all likelihood the contents of the blob are never tested at all. There is code and commentary in the Samba-TNG source showing that they have done some testing, and that their results indicate that a completely random blob of bytes works just fine. If that's true, then the blob does little to improve the security of the algorithm except perhaps by adding a few more CPU cycles to the processing time.

Bottom line: NTLMv2 challenge/response provides only a minimal improvement over its predecessor.

This isn't the first time that we have put a lot of effort into figuring out some complex piece of the protocol only to discover that it's almost pointless, and it probably won't be the last time either.

Email

 From: Ronald Tschalr      To: Chris Hertel Subject: The point of client nonces In section 15.5.5 you talk about the "client challenge" a bit, but miss the point of it: the client nonce (as it should really more correctly be called) is there to prevent precomputed dictionary attacks by the server, and has nothing to do with replay attacks against the server (which, as you correctly state, is what the server-challenge is for). If there's no client nonce, then a rogue server can pick a fixed server-nonce (server-challenge), take dictionary, and precompute all the responses. Then any time a client connects to it it sends the fixed challenge, and upon receipt of the client's response it can do a simple database lookup to find the password (assuming the password was in the dictionary). However, if the client adds its own bit of random stuff to the response computation, then this attack (by the server) is not possible. Hence the client-nonce. Even with client nonces a rogue server can still try to use a dictionary to figure out your password, but the server has to run the complete dictionary on each response, instead of being able to precompute and use the results for all responses. 

15.5.6 Insult to Injury: LMv2

There is yet one more small problem with the NTLMv2 Response, and that problem is known as pass-through authentication. Simply put, a server can pass the authentication process through to an NT Domain Controller. The trouble is that some servers that use pass-through assume that the response string is only 24 bytes long.

You may recall that both the LM and NTLM responses are, in fact, 24 bytes long. Because of the blob, however, the NTLMv2 response is much longer. If a server truncates the response to 24 bytes before forwarding it to the NT Domain Controller almost all of the blob will be lost. Without the blob, the Domain Controller will have no way to verify the response so authentication will fail.

To compensate, a simpler response known as the LMv2 response is also calculated and returned alongside the NTLMv2 response. The formula is identical to that of NTLMv2, except that the blob is really small.

 blip = RandomBytes(8); data = concat(ServerChallenge, 8, blip, 8); hmac = HMAC_MD5(v2hash, 16, data, 16); LMv2resp = concat(hmac, 16, blip, 8); 

The "blip," as we've chosen to call it, is sometimes referred to as the "Client Challenge." If you go back and look, you'll find that the blip value is also included in the blob, just after the timestamp. It is fairly easy to spot in packet captures. The blip is 8 bytes long so that the resulting LMv2 Response will be 24 bytes, exactly the size needed for pass-through authentication.

If it is true that the contents of the blob are not checked, then the LMv2 Response isn't really any less secure than the NTLMv2 Response even though the latter is bigger.

The LMv2 Response takes the place of the LM Response in the SESSION_SETUP_ANDX.CaseInsensitivePassword field.

15.5.7 Choosing NTLMv2

The use of NTLMv2 is not negotiated between the client and the server. There is nothing in the protocol to determine which challenge/response algorithms should be used.

So, um... how does the client know what to send, and how does the server know what to expect?

The default behavior for Windows clients is to send the LM and NTLM responses, and the default for Windows servers is to accept them. Changing these defaults requires fiddling in the Windows registry. Fortunately, the fiddles are well known and documented so we can go through them quickly and get them out of the way. ^[16]

^[16] A quick web search for "LMCompatibility" will turn up a lot of references, Microsoft Knowledge Base Article #147706 among them.

The registry path to look at is:

 HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\LSA 

On Windows 9x the variable is called LMCompatibility , but on Windows NT and 2000 it is LMCompatibilityLevel . That variable may not be present in the registry, so you might have to add it. In general, it's best to follow Microsoft's instructions when editing the registry. ^[17]

^[17] ...so that if something goes wrong you can blame them, and not me.

The settings for LMCompatibilityLevel are as follows:

Level	Description	Client Implications	Server Implications
	The Default	LM and NTLM responses are sent by the client.	The server or Domain Controller will compare the client's responses against the LM, NTLM, LMv2, and NTLMv2 responses. Any valid response is acceptable.
1	NTLMv2 Session Security	This level does nothing to change the algorithm used to generate the response. Instead, at this level and higher a feature called NTLMv2 Session Security is supported. Session Security is only used with Extended Security, and must be negotiated between the client and the server. Session Security is an advanced topic, and won't be covered here.
2	NTLM Authentication	The LM Response is not sent by the client. Instead, the NTLM Response is sent in both password fields. Replacing the LM Response with the NTLM Response facilitates pass-through authentication. Servers need only hand the 24-byte contents of the SESSION_SETUP_ANDX.Case-InsensitivePassword field along to the Domain Controller.	The server or Domain Controller will accept a valid LM, NTLM, LMv2, or NTLMv2 response.
3	NTLMv2 Authentication	The client sends the LMv2 and NTLMv2 responses in place of the older LM and NTLM values.	The server or Domain Controller will accept a valid LM, NTLM, LMv2, or NTLMv2 response.
4	NTLM Required	The client sends the LMv2 and NTLMv2 responses.	At this level, the server or Domain Controller will not check LM Responses. It will compare responses using the NTLM, LMv2, and/or NTLMv2 algorithms.
5	NTLMv2 Required	The client sends the LMv2 and NTLMv2 responses.	The server or Domain Controller will compare the client's responses using the LMv2 and NTLMv2 algorithms only.

That's just a quick overview of the settings and their meanings. The important points are these:

• The password hash type is not negotiated on the wire, but determined by client and/or server configuration. If the client and server configurations are incompatible, authentication will fail.

• The SMB server or Domain Controller may try several comparisons in order to determine whether or not a given response is valid.