NLS Format Analysis: Wine locale.c vs. ReactOS Infrastructure

Background

`dll/win32/kernelbase/wine/locale.c` is the Wine-derived implementation of the kernelbase locale API. It targets the Windows Vista+ (NT 6.0+) NLS architecture, which uses a substantially different set of NLS section types, file formats, and kernel APIs compared to what ReactOS currently implements (Windows XP/2003, NT 5.x).

The tool `sdk/tools/txt2nls/main.cpp` (note: file is `.cpp`, not `.c`) converts `.txt` codepage definitions to binary NLS files. It is only one part of the
full NLS infrastructure.

NtGetNlsSectionPtr Section Type Mapping

The central dispatch mechanism is NtGetNlsSectionPtr(type, id, unk, &ptr, &size).
The type numbers differ completely between the two eras.

Windows Vista+ / Wine (correct mapping, confirmed by wine test)

ReactOS Current (`subsystems/win/basesrv/nls.c`)

Key observations:

• Types 1–8 are the old XP/2003 CSR-based mechanism; they do not exist in Vista+.
• Type 9 is stubbed out; `locale.c` calls it unconditionally on startup.
• Type 12 in ReactOS maps to `geo.nls`; in Vista+ it maps to normalization files. This is a direct conflict that will cause `IsNormalizedString`/`NormalizeString` to receive geo data instead of normalization tables.

Function Stubs in ntdll

From `dll/ntdll/def/ntdll.spec`:

RtlGetLocaleFileMappingAddress is the most critical missing export.
load_locale_nls() calls it during init_locale() and if it returns failure the entire locale subsystem fails to initialise.

NLS File Format Differences

1. locale.nls — Major format change

Old format (ReactOS, XP/2003):

• Binary table, no magic header, ~209 KB
• Per-LCID records in an undocumented 2003-era layout
• Read by old `kernel32` via NtGetNlsSectionPtr(2, ...)

New format (Vista+, required by Wine `locale.c`):

• Accessed via RtlGetLocaleFileMappingAddress()
• Starts with NLS_LOCALE_HEADER at offset 0 (magic `'NSDS'` at offset 0x0C)
• Followed by: NLS_LOCALE_DATA[] array, sorted LCID index, sorted LCNAME index, string pool, calendar array, {}and embedded geo data{} (new struct geo_id[] + struct geo_index[] with its own sub-header)

The NLS_LOCALE_HEADER and NLS_LOCALE_DATA types are defined only in `sdk/include/wine/winternl.h` (because only Wine code uses them today).
ReactOS NDK headers (`sdk/include/ndk/`) do not have these types. If `RtlGetLocaleFileMappingAddress` is to be implemented inside ntdll/RTL,
these types will need to be added to a suitable ReactOS header.

Impact: load_locale_nls() reads locale data, geo IDs, geo index, and character maps entirely from the pointer returned by RtlGetLocaleFileMappingAddress. None of this works today on ReactOS.

Tooling gap: No tool exists in ReactOS to produce a `locale.nls` in the new format. Wine's `nls/locale.nls` (generated by its `tools/make_unicode` script) is a compatible source; alternatively, a new tool would need to consume the MS locale data.

Note also that `sdk/lib/rtl/locale.c` currently resolves LCID↔name via a hardcoded RtlpLocaleTable[] array. This approach is orthogonal to
RtlGetLocaleFileMappingAddress; the two need to be reconciled.

2. `sortdefault.nls` — New file, missing in ReactOS

Old format (ReactOS, XP/2003):

• Two separate files served via old CSR types:
    - `sortkey.nls` (type 4, ~262 KB): 4-byte sort weights per Unicode code point
    - `sorttbls.nls` (type 5, ~21 KB): sort table metadata and filenames for locale-specific sort files (e.g. `big5.nls`, `prcp.nls`)

New format (Vista+, required by Wine `locale.c`):

• Single file `sortdefault.nls` served via NtGetNlsSectionPtr(9, 0, ...)
• Header layout (from load_sortdefault_nls()):

struct {

       UINT sortkeys;   // offset to sort key table (UINT per Unicode code point)

       UINT casemaps;   // offset to casemap table (l_intl.nls format, USHORT pairs)

       UINT ctypes;     // offset to CT_CTYPE1/2/3 table

       UINT sortids;    // offset to sort ID block

};  

* Sort ID block: version + guid_count + {{struct sortguid[] }}(each: 16-byte GUID + flags + compression/exception/casing offsets)

• After sort IDs: expansion count + struct sort_expansion[] (2×WCHAR per entry)
• After expansions: compression count + struct sort_compression[] + compression data
• After compression data: multiple-weights block + struct jamo_sort[]

Tooling gap: No tool exists in ReactOS to build `sortdefault.nls`.
Wine generates it from its `tools/make_unicode` script using Unicode data.

3. Normalization NLS files — Missing entirely

Old format (ReactOS, XP/2003): Not present.

New format (Vista+, required by Wine `locale.c`):

• Four files keyed by normalization form (NormalizationC=1, D=2, KC=5, KD=6), served via `NtGetNlsSectionPtr(12, form, ...)`
• Parsed via `struct norm_table` header (defined in `locale.c`):
  • File name (13 WCHARs), checksum, Unicode version, normalization form
  • Offsets to: combining class table, property tables (level 1 + 2), decomposition hash + map + sequence tables, composition hash + sequence tables
• Used by RtlNormalizeString() (also stubbed)

Conflict: ReactOS currently maps type 12 to `geo.nls`. The type 12 slot must be reassigned to normalization. Geo data in the new architecture is embedded in `locale.nls` itself (see section 1 above).

Tooling gap: No tool exists in ReactOS to build normalization NLS files. Wine's `nls/Normalize{C,D,KC,KD}.nls` files can serve as a source.

4. Casemap table (type 10)

Old format (ReactOS): `l_intl.nls` served via old CSR type, also referenced directly from ExpNlsSectionPointer in the kernel.

New format (Vista+): Served via NtGetNlsSectionPtr(10, 0, ...). The format is the same `l_intl.nls` USHORT-pair layout — `locale.c` explicitly notes /* casemap table, in l_intl.nls format */ for `sort.casemap`. The content of ReactOS's existing `l_intl.nls` (4870 bytes) should be compatible once type 10 is implemented.

5. Codepage NLS files (`c_*.nls`) — Minor header difference, mostly resolved

Both old and new code use the same 26-byte NLS_FILE_HEADER (13 WORDs).
The NDK header `sdk/include/ddk/ntnls.h` already defines the correct layout:

typedef struct _NLS_FILE_HEADER {

    USHORT HeaderSize;           // = 13 (WORDs)

    USHORT CodePage;

    USHORT MaximumCharacterSize; // 1 = SBCS, 2 = DBCS

    USHORT DefaultChar;

    USHORT UniDefaultChar;

    USHORT TransDefaultChar;     // Unicode → CP fallback: Unicode of DefaultChar

    USHORT TransUniDefaultChar;  // CP → Unicode fallback: CP of UniDefaultChar

    UCHAR  LeadByte[12];

} NLS_FILE_HEADER; 

Old tool (`sdk/tools/create_nls/create_nls.c`):

• Uses a different in-memory layout (BYTE DefaultChar[2] + unknown1 }}/ {{{}unknown2)
• Always writes `'?'` (0x003F) for TransDefaultChar and TransUniDefaultChar
• Reads data from the host OS via GetCPInfoExA — Windows-only

New tool (`sdk/tools/txt2nls/main.cpp`):

• Uses the correct layout matching `ntnls.h`
• Properly computes TransDefaultChar and TransUniDefaultChar
• Reads from portable `.txt` source files; runs cross-platform

The `txt2nls` tool already generates correct NLS files. All codepage `.nls` files in `media/nls/` are now built by `txt2nls` from the `.txt` sources in
`media/nls/src/`. The `create_nls.c` tool is obsolete for this purpose.

Files not yet converted: `c_856.nls` and `c_878.nls` are listed in `media/nls/CMakeLists.txt` as static (manually generated) rather than built
from `.txt` sources. They may still be in the old format with `'?'` for the translated default chars, and should have `.txt` sources added so they can be
rebuilt by `txt2nls`.

Summary of Required Changes

Critical (locale.c cannot initialize without these)

1. Implement RtlGetLocaleFileMappingAddress in `sdk/lib/rtl/` or `dll/ntdll/`. Must map and return a pointer to a Vista+-format `locale.nls` image.
2. Implement NtGetNlsSectionPtr type 9 (sortdefault) in ntdll/ntoskrnl. Must serve `sortdefault.nls` in the new unified format.
3. Implement NtGetNlsSectionPtr type 10 (casemap) in ntdll/ntoskrnl. Can reuse `l_intl.nls` data (format is already compatible).
4. Fix NtGetNlsSectionPtr type 12 conflict in `subsystems/win/basesrv/nls.c`: change from `geo.nls` to the normalization NLS files. Geo data must instead be embedded in the new `locale.nls`.
5. Create new `locale.nls` in Vista+ format (`NLS_LOCALE_HEADER` + `NLS_LOCALE_DATA[]`). Wine's `nls/locale.nls` can serve as a base.
6. Create `sortdefault.nls` in the new unified format. Wine's `nls/sortdefault.nls` can serve as a base.
7. Add NLS_LOCALE_HEADER and NLS_LOCALE_DATA type definitions to a ReactOS NDK header (e.g. `sdk/include/ndk/rtltypes.h` or a new `sdk/include/ndk/nlstypes.h`),  so that RtlGetLocaleFileMappingAddress can be implemented without depending on Wine's `winternl.h`.

Important (affects correctness)

8. Implement RtlNormalizeString in `sdk/lib/rtl/`. Requires normalization NLS files served via NtGetNlsSectionPtr(12, ...).

9. Create normalization NLS files (`Normalize{C,D,KC,KD}.nls`). Wine's `nls/Normalize*.nls` can serve as a base.

10. Implement `NtInitializeNlsFiles` in ntdll. Used by some callers to prime the locale file mapping before RtlGetLocaleFileMappingAddress is called.

11. Reconcile `sdk/lib/rtl/locale.c` hardcoded RtlpLocaleTable[] with the file-based data from `locale.nls`. Currently both represent locale↔LCID
    mappings independently. Long term the RTL should derive this data from `locale.nls` rather than a compile-time table.

Lower priority

12. Convert `c_856.nls` and `c_878.nls` from static files to `.txt`-sourced builds via `txt2nls`, ensuring TransDefaultChar / TransUniDefaultChar are correct.

13. Deprecate `sdk/tools/create_nls/create_nls.c`. It is superseded by `txt2nls`. The files it would produce have incorrect TransDefaultChar values.

14. Document the old CSR NLS types 1–8 in `basesrv/nls.c` as XP/2003-era compatibility only. They should be preserved for any remaining old code paths but are invisible to Vista+ callers.

File Reference