Starting with how the Retail IPL determines that the system and disk are valid.
1. Check that the expansion PAK RAM is present.
2. Try to Read the Real Time Clock on the Disk Drive Controller. Test that the Controller is functioning, no diagnostic failures and no communication failures.
3. Read the Disk ID in Sector Mode (Disk LBA 14, Alternate 15)
4. Try to Read LBA 12 in Sector Mode with no retries. This will need to result in an unrecoverable read error for a valid retail disk.
4. Read the Disk System Data in Sector Mode with no retries (LBA 0, Alternates 1,8,9) For System Data to be valid the first four bytes must be 0xE848D316. The Disk Type (byte 5, bits 7-4) must be 1, 2 or 3.
5. Load the Disk Program starting at LBA 24 in Block/Track mode. The Block Count is given by System Data bytes 6/7. The Destination is given by System Data word 0x1C.
6. Check the Controller ID Register. It must be equal to 3.
7. Check the IPL ROM byte at offset 0x9FF00. It must equal 0xC3.
8. If the Disk ID sector byte 7, bit 7 is set the disk program is encrypted. The loader will decrypt this in two stages.
9. Reset the system state and jump to the Disk Program.
To read Development Disks the IPL ROM would need in step 4 to have no error reading LBA 12, in step 5 the first four bytes of system data must be 0x00000000 and be read from LBA 2 (Alternates 3,10,11) and in step 6 the controller ID register would return the value 4.
Neat! Please submit as soon as you have something that won't bomb everyone's compiles, because it would suck if work like this got lost. (I know first hand, having lost some pretty significant stuff at various times).
Next, issuing commands to the controller
Each command finishes by issuing a MECHA controller interrupt which signals on interrupt line 1 (CARTRIDGE Interrupt) to the main deck CPU. The interrupt is cleared by setting the MECHA interrupt reset bit in the control/status register.
First set the track with a Read/Seek Command to the desired track. Check that the Index Lock signal in the track register is set when the track is found. Stop the Sequencer (Raw Sector Format). Set the Sector Size and Sectors per Block in the appropriate registers. Restart the Sequencer. Reset the BM(Buffer Manager?) Set the Start Sector index in the control/status register and set the bit to start the BM. Reading sectors only looks at the C1 (Error detection) bytes. It will not use the C2 (Error Correction bytes. Each sector read will set the DREQ (Data Request) flag and signal a BM interrupt when the sector buffer is ready. When 17 sectors have been read without and C1 Errors, the BM is reset to end the transfer.
Starts like reading a sector. If reading a track the XFER_BLKS flag is set in the control/status register before starting the BM. When the data sectors (set to 85) in the block have been transferred, the BM starts reading the C2 sectors to the C2 buffer. The first three only signal a BM interrupt, with out setting the DREQ flag. The fourth (last) C2 sector sets the C2_XFER flag when it sends an interrupt. After the C2 sectors, there is a gap sector which will not be read to a buffer, but will send a BM interrupt with no flags set. If in block mode, the transfer stops. In in track mode, the transfer continues to the second (last) block of the track.
I have decided to post this information for now, as the current disk dumps contain the 'payload' of the tracks, but lack the rest of the structure of the disk. This includes the C2 sectors, Gap sector and alternate/spare tracks for each zone. The lack of alternate/spare tracks presents a slight problem. The list of tracks is kept in the System Data area and the host driver will translate LBA requests that skip over these spare tracks and the Read/Seek commands will be issued to the correct physical track. I would prefer that the emulation code not have to read this list to know how to find the correct offsets, but that the spare tracks were present in the dump. I doubt that any of the games abuse the flexibility of the disk drive, but it may be possible that some of them look for these spare tracks or possibly use a different number of sectors per block or bytes per block. I still need to work out the best way to represent the disk structure. Currently I am using quickload to avoid any commitment to a format.
I should possibly also follow up with Two or Three Things I Don't Know about the N64DD.
Unfortunately, I'm just way too bad of a developer to be really able to develop a dumper that really dumps everything of a 64DD disk. I don't even know the basics of a disk format other than sectors and C1/C2.
The dumper I made is based of the SDK code, slightly hacked to dump the System Area that the SDK likes to hide. I also tried to force it to dump development disks (it works, but the data is in the wrong order of Disk Types different than 0).
I originally tried to do stuff manually with libdragon, but this library is just too shitty for 64DD uses as it does not support CART interrupts (can be added, but I have no idea how to properly do that, and from what other devs say, libdragon isn't really reliable).
If the least I can do is at least get more developers' interest, that would be great ^^
I've found a little more on reading and writing:
For the BM the only signal to decide between read or write is by sending a Read/Seek Command or a Write/Seek Command. The Data field for these is formatted as bit 12 selects Head 0 or 1 and bits 11-0 select the track.
Also the SDK translates LBA's for head 1 in reverse track order per zone. E.g. for zone 0 on head 0 the first block is on track 0 of 146 tracks for the zone. For the corresponding zone on head 1 the first block starts at track 146. (This would reduce head movement for transfers spanning zones)
Also, also: the disk type field (byte 5 of System Data) is divided into a high nibble and a low nibble. The low nibble is the Disk Type 0-7 of the SDK, but the high nibble signifies something else. The IPL will only accept high nibble values of 1,2 or 3 and the game code will only accept a value of 1. This could possibly designate single density / double density disks. The whole byte is sent as data for the controller's Set Disk Type Command.
Update on reads and writes :
Writes set BM Mode = 0 and Reads set BM Mode = 1. I believe this controls the interrupt and data request sequence, but in itself does not signal the transfer as a read or write.
Currently have all known dumps running.