Validate I18N

Validate I18N

Strings, including Unicode, can be invalid in several ways. For example, a string might contain binary values that do not map to any character or the string might contain characters with semantics outside the domain of the application, such as control characters within a URL. Such invalid strings can pose security threats if your code does not handle them properly.

Starting with Microsoft Windows .NET Server 2003, a new function, IsNLSDefinedString, helps verify that a string contains only valid Unicode characters. If IsNLSDefinedString returns true, you know that it contains no code points that CompareString will ignore (such as undefined characters or ill-matched surrogate pairs). Your code will still need to check for application-specific exceptions.

Visual Validation

Even with normalization, many characters in Unicode will appear identical to the user. For example, is actually two Unicode characters ( plus ), not five ASCII range characters. There is no way the user can reliably determine this from the visual display. Therefore, do not rely on the user to recognize that a string contains invalid characters. Either eliminate visual normalization or assist the user (for example, by allowing the user to view the binary values).

Do Not Validate Strings with LCMapString

You can use LCMapString to generate the sorting weights for a string. An application can store these weights (a series of integers) to improve performance when comparing the string with other strings. However, using the LCMapString-generated weights is not a reliable way to validate a string. Even though LCMapString returns identical weights for two strings, either string might contain invalid characters. In particular, LCMapString completely ignores undefined characters. Either use the new function, IsNLSDefinedString, or perform your own conservative validation.

Use CreateFile to Validate Filenames

Just because CompareString says two strings are equal (or unequal) does not mean that every part of the system will agree. In particular, CompareString might determine that two strings NTFS considers distinct are equal and vice versa. Always validate the string with the relevant component. For example, to verify that a string matches an existing filename, use CreateFile and check the error status.