On OEM IDs (v1.3, 2006-04-17)
Introduction
In 1999, a message was posted to the OpenDOS mailing list at delorie.com. In brief, it describes a problem with using MS-DOS (or Windows 95/98/ME) on partitions created with DR-DOS FDISK, which can result in serious data corruption. The problem seems to arise because of two factors; if either is changed, it goes away. The factors are:
- DR-DOS FDISK chose to use 4k clusters, on a partition where other FDISKs (or FDISK + FORMAT combinations [1]) would have used 2k clusters.
- DR-DOS FDISK formatted the drive with an OEM ID of "DRDOS 7" where other FDISKs/formatters would have used "MSDOS5.0" or "IBM 3.3".
So, why should this combination of circumstances cause trouble?
Background
In this document, I am concerned only with the behaviour of hard drives formatted with the FAT12 or FAT16 filesystems. I don't cover floppies, FAT32, or other filesystems such as HPFS and NTFS.
For practical purposes, all[2] FAT12/FAT16 partitions start with a standard boot sector. This is laid out as follows:
| Offset from start | Meaning |
|---|---|
| 0 | 3 byte jump to boot code. |
| 3 | OEM ID (or OEM label). 8 bytes ASCII. |
| 11 | BIOS Parameter block (BPB), describing number of tracks, number of sectors, cluster size, root directory size, FAT size and so on. BPBs written by earlier versions of MS-DOS are 25 bytes long; later ones are 51 bytes. |
| 36 or 62 | Boot code |
The OEM ID is pretty poorly documented. Microsoft's knowledge base simply describes it as a string of characters. The Windows 2000 resource kit says that Windows 2000 sets it to be "MSDOS5.0" 'to preserve compatibility with MS-DOS'. Microsoft's paper FAT: General Overview of On Disk Format says "Microsoft operating systems don't pay any attention to this field". This is incorrect, as is quite easy to prove.
Reproducing the problem
You will need:
- A PC emulator. PC emulators (VMware, Virtual PC, Bochs etc.) are preferred because it's much quicker and you can set up an isolated test environment; but if necessary, you can use a real PC with a completely blank hard drive.
- A boot floppy for DR-DOS 7.03.
- A boot floppy for the DOS version you want to test.
- A sector editor such as Norton Diskedit.
Proceed as follows:
- Create a new virtual machine.
- Boot from the DR-DOS 7.03 floppy, and type FDISK. Create a primary DOS partition that's 126 megabytes in size.
- Reboot when prompted. At the A> prompt, type CHKDSK C: and verify that the cluster size is 4096 bytes.
- Create a subdirectory on the newly formatted C: drive and copy some text files into it.
- Now reboot from the MS-DOS disc, and try to read the text files. If the problem is manifesting itself (which it should be), you will find that either the contents of the subdirectory cannot be seen, or (if it can) that the text files are scrambled.
- Using your sector editor, go to the boot sector of drive C:. The OEM ID should read "DRDOS 7"; change it to "IBM 3.3".
- Reboot from the MS-DOS disc and try to read the text files again. This time it ought to work.
More tests
Having set up the test system, I tried various different OEM IDs to see which ones work and which ones don't. After establishing that behaviour was the same whether the first 5 letters read "DRDOS" or "MSDOS", I tended to leave them reading "MSDOS".
| OEM ID | Result |
|---|---|
| DRDOS 7 | Fails |
| DRDOS7.0 | Works |
| IBM 3 | Fails |
| IBM 3.3 | Works |
| IBM 3.3 | Works |
| IBM3.3 | Fails |
| MSDOS 5 | Fails |
| MSDOS . | Fails |
| MSDOS... | Fails |
| MSDOS1.0 | Fails |
| MSDOS2.1 | Fails |
| MSDOS3.0 | Fails |
| MSDOS3.1 | Works |
| MSDOS3/0 | Works |
| MSDOS300 | Works |
| MSDOS5 0 | Fails |
| MSDOS5-0 | Fails |
| MSDOS5.0 | Works |
| MSDOS5/0 | Works |
| MSDOSA.B | Works |
| OPENDOS7 | Works |
What's going on? According to this USENET posting, MS-DOS is trying to work around bugs in early versions of MS-DOS or OS/2. One of these workarounds involves calculating the cluster size from first principles and using that in preference to the cluster size in the BPB. DR-DOS FDISK chose to use 4k clusters but the calculation comes up with 2k. The result is that for each file on the drive, MS-DOS and DR-DOS have different ideas about which sectors it occupies.
The USENET article also suggests that MS-DOS requires the OEM ID to be formed "xxxxxY.Y" (where x is any character and Y must be a digit) but this doesn't quite match the results above, where "xxxxxY/Y" and "OPENDOS7" worked.
So, having exhausted other avenues, I cast caution to the winds and broke out the Bochs debugger.
What MS-DOS Actually Checks
MS-DOS does various tests on the OEM ID, but the one that seems to causes all the trouble is this:
C:\>debug io.sys -u2672 0CC5:2672 2E CS: 0CC5:2673 817F08332E CMP WORD PTR [BX+08],2E33 0CC5:2678 72DD JB 2657 0CC5:267A 7507 JNZ 2683 0CC5:267C 2E CS: 0CC5:267D 807F0A31 CMP BYTE PTR [BX+0A],31 0CC5:2681 72D4 JB 2657
What it is trying to do is check if the version number in the last 3 bytes of the OEM ID is less than "3.1" (which explains why "MSDOS3.1" works and "MSDOS3.0" doesn't). The check is in two parts:
- Check if the first two bytes of the number are less than "3.". If so, do the workarounds.
- If the first two bytes were "3.", check if the last byte is less than "1". If so, do the workarounds.
Unfortunately it tries to do the first step using a word comparison. Since it's on a little-endian computer, what actually happens is is this:
- Check if the middle byte of the number is less than ".". If so, do the workarounds.
- If the middle byte of the number is greater than ".", don't do the workarounds.
- If the middle byte is exactly equal to ".":
- Compare the first byte to "3". If it's less, do the workarounds.
- If the first byte is equal to "3", check if the last byte is less than "1". If so, do the workarounds.
The end result is that if the middle byte of the number (the last-but-one byte of the OEM ID) is not ".", the comparison doesn't do what it's supposed to. If this byte is less than ".", the result can be very damaging to your data.
How to change the OEM ID in DRDOS utilities
Since MS-DOS objects to the "DRDOS 7" OEM ID, one solution is to change the DR-DOS utilities (FDISK, FORMAT and SYS) so that they don't write this ID. The utilities in question are compressed with PKLITE (which can be downloaded here) so proceed as follows:
- Decompress the utility: PKLITE -x filename.COM
- Open the decompressed utility in a binary editor (again, Norton's Diskedit is suitable for this purpose).
- Search for the "DRDOS 7" string. Replace it with something that MS-DOS doesn't object to, such as "IBM 3.3".
- If you want, compress the utility again: PKLITE filename.COM
How long has it been like this?
The code fragment above appears to be present in all versions of MS-DOS from 4.01 up to 7.10 (Windows 98). I couldn't find it in the Windows ME version of IO.SYS, but if you boot it from a floppy the same problem occurs; possibly the WinME IO.SYS is compressed so my simple binary search didn't work.
In MS-DOS 3.3 the check is simpler:
C:\>debug io3.sys -u3dfd 0CC5:3DFD 26 ES: 0CC5:3DFE 813E0800332E CMP WORD PTR [0008],2E33 0CC5:3E04 75F4 JNZ 3DFA 0CC5:3E06 26 ES: 0CC5:3E07 803E0A0031 CMP BYTE PTR [000A],31 0CC5:3E0C 72EC JB 3DFA
This version of the check does the workarounds if the first two bytes of the number are not "3.", or the last byte is less than "1". So it would do them for OEM IDs "MSDOS4.0", "MSDOS5.0" and so on. Since MS-DOS 3.3 doesn't support partitions bigger than 32Mb I was not able to test this behaviour on my 126Mb test partition.
Conclusion
If the last-but-one byte of the OEM ID is less than ".", the OEM ID check in MS-DOS will not believe the BPB, but instead use calculated default values for some fields. This can cause filesystem corruption if other operating systems that access the drive do believe the BPB.
It's probably safest to stick to "xxxxxY.Y" (or "xxxxY0.Y") as the format of your OEM ID. Additionally, some versions of IBM PC-DOS (not MS-DOS) apparently require the label to start "IBM " so it looks like the best label to use is "IBM m.n" where m.n is either 3.3 or 5.0.
Other Enlightening Web Links
- I found this USENET posting quite a long time after writing the article. It gives a detailed description of the tests that different versions of MS-DOS and PC-DOS (up to 5.0) make.
- NOVOLTRK is a Registry script which stops Windows 9x overwriting the OEM ID on floppy discs. It includes a comprehensive list of OEM IDs used by different DOS versions.
- Why the MS-DOS 6+ OEM ID is "MSDOS5.0" and not "MSDOS6.0".
- MS-DOS 4.0 and hard drive corruption.
- The Groklaw discussion which set me off discovering all this.
Revision History
- 1.3 (2006-04-17): Added a link to a Google posting which gives precise details of what tests DOS versions up to MS-DOS 5 use.
- 1.2.1 (2004-12-06): Changed Google links from English (hl=en) to Latin (hl=la). For now, at least, this means they use the old Groups interface, and therefore link precisely to the right article rather than the start of a thread.
- 1.2 (2004-07-09): Clarified what is done by FDISK and what by FORMAT.
- 1.1 (2004-06-20): Added details of how to change the OEM ID string in the DRDOS utilities.
- 1.0 (2004-06-18): Initial release.
Footnotes
1. DRDOS FDISK formats the drive in the process of creating the partition. MSDOS FDISK does not, and so FORMAT.COM has to be used to finish the job.
2. A very few FAT12 implementations don't use the standard boot sector. The custom version of MS-DOS for early Apricot PCs, and DOS Plus for the BBC Master 512, are the only ones I know of.
John Elliott 20 June 2004