LDBS disk image format v0.2

John Elliott, 3 March 2017

Introduction

LDBS is a disk image format originally designed for internal use in LibDsk, with the intended use case being for archiving FM / MFM floppy disks. This specification is provided for comment and suggestions, in case the format proves useful in other situations.

The suggested file extension is .ldbs.

The reference implementation for this version of LDBS is ldbs.c / ldbs.h, supplied with LibDsk v1.5.3.

The file structure is defined at two levels: a generic block store, and the specific block types used to specify a disk image.

Rationale: Why a new disk image format? As mentioned above, it was designed for use as an intermediate format in LibDsk. The aim is to have feature parity with CPCEMU extended .DSK, but in a more extensible format with greater support for in-place rewrites. It is not, for example, possible to reformat a track from 8 to 9 512-byte sectors in an EDSK file, without rewriting all subsequent tracks.

Change history

0.2 (16 March 2017)

• Added a warning against storing file offsets in custom blocks.
• Implemented a few more helper methods. The disk image format itself remains unchanged.

0.1 (16 March 2016)

Original version.

Data storage: Block Store

All 16-bit and 32-bit values are stored in Intel format (low byte first).

File offsets and block lengths are 32 bit little-endian doublewords. To avoid signed / unsigned issues, they should not exceed 2^31.

Header

All LDBS files begin with a 20-byte header:

If the offsets at 0x0008 / 0x000C / 0x0010 are non-zero, they must be file offsets of valid data blocks. It is an error for them to point to any other location in the file.

Within the block store layer the track directory is optional, but a valid LDBS disk image must have a track directory.

Although the block store layer is currently only defined as containing a disk image, it would be usable as a container for other file formats; this would be indicated by a different value at offset 0x0004. The use of the block store for other file formats is not covered further by this document.

Data blocks

The header is followed by one or more data blocks (a file with no data blocks is structurally valid, but would not contain a track directory). There is no constraint on the order of blocks in a file; a track header does not have to be stored together with its associated sectors, for example.

For an LDBS disc image, is recommended that a given data block should not exceed 64k in size.

There are no alignment requirements; blocks can begin at any file offset.

Each data block has its own 20-byte header:

The header is followed immediately by the block data.

If a block is recorded in the track directory (see below) its block type is used to identify it, and so must be unique within the file. The block type is not required to be unique for blocks not recorded in the directory.

If the offset at 0x0010 is non-zero, it must point to another valid data block.

If the length of the block on disk is greater than the length of the block contents, an implementation is encouraged to fill the remainder with zero bytes, but is not obliged to. Similarly a free block should be filled with zeroes, but is not required to be.

Data blocks can be enumerated either by stepping through the file in linear order, or by following the used / free linked lists.

It is an error for there to be a 'free' block on the 'used' linked list, or a 'used' block on the 'free' linked list.

Operations

Typical operations performed at block level are:

• Get the length of a block. Given an offset, load the block header at that offset and return its length.
• Load a block. Given an offset, load the block header and data at that offset.
• Add a block. Search the free list for an entry greater than or equal to the required size. If one is found, remove the entry from the free list, add it to the used list, and populate with the data. If not found, append a new block of the required size to the end of the file, and add it to the used list.
• Delete a block. Remove the block from the used list, add it to the free list, and zero its block type.
• Update a block. If the new size is less than or equal to the block's current size on disk, write the data into the block at its current location. Otherwise, treat as delete followed by add.

Note that there is no requirement to coalesce adjacent free blocks, or to split a block if the size requested is less than its current size on disk.

Data storage: Disc image

Track directory

A disk image must contain a track directory block. The intention is that an implementation will use the track directory to locate data rather than performing linear searches of the file or walking its linked lists.

The format of the track directory is:

The format of a directory entry is one of Txxx, INFO, CREA, DPB or GEOM, or a custom block type (beginning with a lowercase 'a'-'z'). In each case, this is the block type of the corresponding block in the store.

Reference to a track header. There is one of these for each track in the disk image.

Reference to a comment block. There is at most one of these in the disk image file.

Reference to a creator block. There is at most one of these in the disk image file.

Reference to a geometry block. There is at most one of these in the disk image file.

Reference to a CP/M Disk Parameter Block block. There is at most one of these in the disk image file. It will only be present on disks containing a CP/M filesystem.

Custom block types

Other block types than the ones listed here may be defined in later versions of this spec; implementations should ignore unknown block types. It may, for example, be useful at some point to create a 'BOOT' block type containing the keystrokes necessary to boot a disk, or a 'PICT' block containing a picture of the disk.

All future block types defined by this spec will start with a capital letter, 'A'-'Z'. Block types beginning with a lowercase letter 'a'-'z' can be used for private purposes — there is, of course, no guarantee that someone else won't pick the same block type for their own use.

You should avoid storing file offsets in your custom blocks. There is nothing stopping an LDBS implementation from rearranging blocks in a file so they end up at different offsets (for example, to remove unused space). However, such an implementation won't be aware of offsets held in custom blocks, and so won't update them. This isn't just a theoretical danger: ldbs_clone() reverses the order of blocks when duplicating an LDBS file.

The LibDsk TeleDisk, CopyQM and QRST drivers use custom blocks to hold details from the original file headers which aren't used by other disk image formats. These are, respectively, 'utd0', 'ucqm', and 'uqrs'.

Track header

0 => unknown
1 => Single density (125 kbit/s or 150 kbit/s FM) or double density (250 kbit/s or 300 kbit/s MFM)
2 => High density (500 kbit/s MFM, or [unlikely] 250 kbit/s FM)
3 => Extended density (1000 kbit/s MFM, or [unlikely] 500 kbit/s FM)

0 => unknown
1 => FM
2 => MFM

Each sector descriptor is formed:

Bit 7: End of cylinder
Bit 5: Data error in ID or data field
Bit 2: No data
Bit 0: Missing address mark in ID or data field

Bit 6: Control mark (sector marked as deleted data)
Bit 5: Data error in data field
Bit 0: Missing address mark in data field

Sector data

Since the track directory does not record the locations of individual sectors, the block type field is not required to be unique. If an LDBS file contained more than 256 cylinders, for example, sectors on cylinder 256 would have the same block type as those on cylinder 0.

Comment block

The comment block is optional, and contains a human-readable comment describing the disk image.

On character sets: If possible, stick to ASCII. Failing that, UTF-8.

Creator block

The creator block is optional, and contains human-readable text naming the utility that created the disk image.

DPB block

The DPB block is optional, and contains a CP/M Plus Disk Parameter Block describing the filesystem.

Rationale: CP/M filesystems are not self-describing. If the disk image was generated by an emulator or utility that is aware of the CP/M filesystem, it may be helpful to record the filesystem parameters used. This is also necessary for round-trip compatibility with the YAZE YDSK format.

Geometry block

The geometry block is optional. It contains the drive geometry used by LibDsk when it last formatted a track in the disk image. On a 'straightforward' disk image where all tracks have the same format, this will give the geometry of the entire file.

0 => Single sided, or alternating sides. Track n is cylinder 
                             (n / heads) head (n % heads).

1 => Out and back. Tracks go (head 0) 0,1,2,3,...,37,38,39 then
                             (head 1) 39,38,37,...2,1,0 

2 => Out and out.  Tracks go (head 0) 0,1,2,3,...,37,38,39 then
                             (head 1) 0,1,2,3,...,37,38,39

3 => Extended surface.  As SIDES_ALT, but sectors on head 1 identify
                           as head 0, with numbers in sequence 
                           eg: Head 0 has sectors 1-9, head 1 has 10-18 

Note that the representation of the data rate and recording mode isn't the same as in a track header.

Rationale: If the imaged disk contains no superblock or other means to specify its own format, this can be used to suggest a geometry for accessing it.

Compression

The only form of compression is omitting sectors that are blank or unreadable (by setting the 'copies' field in the sector header to 0). Should it prove necessary to compress LDBS disk images further, this should be done using an external utility such as gzip (LibDsk can transparently decompress gzipped disk images).

Reference Implementation

ldbs.h / ldbs.c in LibDsk 1.5.3 contain an X11-licensed reference implementation of LDBS. A few sample utilities are also supplied:

• dsk2ldbs: Converts DSK / EDSK disk images to LDBS.
• ldbs2dsk: Converts LDBS disk images to DSK / EDSK.
• ldbstest: Sanity test of block layer functions.
• ldbsdump: Displays the contents of an LDBS file.
• ldbscopy: Copies one LDBS file to another using the ldbs_clone() function. As currently written this happens to reverse the order of blocks in the file, so all block offsets will change.

The relevant files can also be downloaded separately.

Some example LDBS files can be downloaded here.