I dumped the 6805 visually, then disassembled the code and found one obvious bit error in the self-test code. After I fixed that I ran the self-test code and the ROM checksum didn't match. It just does an XOR of every byte and only one bit was wrong, so I checked that column in the ROM array and found a bit I had missed. Now the checksum is correct, but since it's just an XOR, it's possible that I still have other errors that cancel out.
Putting 9V on the timer pin for self-test mode moves the vectors up 8 bytes in memory, so you get new reset and interrupt addresses. The self-test code checks the ports then does a RAM test- it fills RAM with the pattern 00 80 40 20 10 08 04 02 01 then verifies it, then shifts the pattern to 80 40 20 10 08 04 02 01 00 and verifies it, etc.
Then it does the ROM test, XORing all the bytes from $080 to $7FF (including unused bytes, which must all read the same and therefore don't change the checksum). The ROM test is done with self-modifying code, using some of the residue from the RAM test! I guess they were trying to cram as many tests as possible into the smallest amount of ROM as possible.
I noticed that the old sim6805 program coded the CLRA opcode incorrectly by also clearing the carry bit (which messed up the RAM test code), so I checked the MAME/MESS code. CLRA and CLR direct are OK, but CLRX and the 2 CLR indexed opcodes clear the carry bit. The data sheets I have say that carry is unaffected.
I don't see a way to dump ROM using the test mode (normally or "creatively"), so I guess we'll have to figure out non-user mode. Unfortunately, my 6805 die got pretty cooked and I don't have a top-metal shot, so it's probably not too useful.
