Previous Thread
Next Thread
Print Thread
Page 1 of 9 1 2 3 4 5 6 7 8 9
#30761 06/25/07 10:08 AM
Joined: Jan 2006
Posts: 138
M
Marty Offline OP
Senior Member
OP Offline
Senior Member
M
Joined: Jan 2006
Posts: 138
I've been thinking more about the use of XML for image info the last few days and I've even finished writing a custom XML parser. I'd like to present some ideas for how the data could be structured. While it's a very subjective field, my hope is that many people could give their input and contribute to this thread so that we can work towards making a nice, clean and flexible format. Databases, romsets and other stuff comes later. Once we have the format, we can take it from there. Right now I want more action and less blah blah blah.

The following layout is not neccesarily the result of deep thinking. It's just something I threw together to encourage a technical discussion.

random example:

Code:
<game name="Weird Science" system="nes.ntsc" crc="C8BAA563"> 
  <hardware board="nes-weirdrom" mapper="1">
     <prg type="rom" size="128k" crc="09234FB0"> (*) </prg>
     <prg type="ram" size="8k" />
     <prg type="nvram" size="8k"> (*) </chr>
     <prg type="eeprom" size="512"> (*) </chr>
     <chr type="rom" size="256k" crc="FE037AB1"> (*) </chr>
     <chr type="ram" size="8k" />
     <nmt mirroring="0101" />
     <mmc>mmc1b2</mmc>
  </hardware>
  <peripherals>
    ..discuss
  </peripherals>
  <dipswitches>
    ..discuss
  </dipswitches>
  <something else>
    ..discuss
  </something else>
</game>

(*) - optional reference to file

real world example using Shadowgate US version:

Code:
<game name="Shadowgate" system="nes.ntsc" crc="6A1F628A"> 
  <hardware board="nes-tkrom" mapper="4">
     <prg type="rom" size="128k" crc="591364C9" />
     <prg type="nvram" size="8k" />
     <chr type="rom" size="128k" crc="05414DD9" />
     <nmt mirroring="ctrl" />
     <mmc>mmc3b</mmc>
  </hardware>
</game>

Some attribute descriptions:

<game>

crc: crc-32 of combined roms in prg+chr order
system: nes.ntsc, nes.pal, famicom, vs.unisystem, vs.dualsystem, playchoice-10, <insert clone?>

<hardware>

board: pcb/unif name (if available, make something up otherwise?)
mapper: %0..255 (controversial, discuss!)

<prg/chr/nmt>

size: % (in bytes), %k (in kibibytes)
type: rom, ram, nvram (battery-backed), eeprom

mirroring:
0000 - single-screen from PPU $2000
1111 - single-screen from PPU $2400
0101 - vertical
0011 - horizontal
0123 - four-screen (uses extra 2k RAM)
ctrl - controlled by mmc
<xxxx> - any other combination

<xxxx> instead of the commonly refered 'horizontal/vertical/four-screen' good or bad? Discuss!

Some tags and attributes could be optional and have documented default values if unspecified. Ideas for which and what settings? Discuss!

Joined: Feb 2004
Posts: 2,383
Likes: 98
Very Senior Member
Online Content
Very Senior Member
Joined: Feb 2004
Posts: 2,383
Likes: 98
Not generic enough. I still think a netlist is better.

Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
Are you prepared to create one for all known NES cart boards? Because I don't think that's a real common skill smile

I think what Marty has is definitely a good start, but my technical knowledge of NES mappers isn't sufficient to offer a good critique.

Joined: May 2007
Posts: 21
S
Member
Offline
Member
S
Joined: May 2007
Posts: 21
How will you handle games that have multiple PRG roms?

Joined: Oct 2005
Posts: 351
R
Senior Member
Offline
Senior Member
R
Joined: Oct 2005
Posts: 351
crc32 was hacked years ago
why not add some sha1

Joined: Jan 2006
Posts: 138
M
Marty Offline OP
Senior Member
OP Offline
Senior Member
M
Joined: Jan 2006
Posts: 138
Originally Posted By Vas Crabb
Not generic enough. I still think a netlist is better.

Maybe so, but I think what you're suggesting will be too difficult to accomplish and work with. Besides, the board name alone will tell everything there is to know about the cartridge minus some specifics like chip sizes and revisions and other things depending on context. That's speaking from an emulation standpoint however, but I think that's just the right balance. The cart has been identified and if someone wants to learn more about the NES-TKROM board used for Shadowgate he/she can look it up.

Originally Posted By Sotho Tal Ker
How will you handle games that have multiple PRG roms?

That's easy. Using the unlicensed Action 52 cart as an example:

<hardware ... >
<prg type="rom" size="512k" crc="64E40C10" />
<prg type="rom" size="512k" crc="69FDC534" />
<prg type="rom" size="512k" crc="7E43E9E1" />
<chr type="rom" size="512k" crc="596442EC" />
</hardware>

Originally Posted By MESSfan
crc32 was hacked years ago
why not add some sha1

While I don't think it's worth worrying too much about CRC vandals, a more safe hash function could indeed be desired.

<game ... crc="baaaaaad" sha1="91a009e610b5e2b9776ac540dcdcada6f8cfc507">

Any other hash function to be considered, or will the two do for now?

Joined: May 2007
Posts: 21
S
Member
Offline
Member
S
Joined: May 2007
Posts: 21
Maybe MD5 as a third option. Those 3 (CRC32,MD5,SHA1) are currently used by ClrMame aswell.

And as I see you determine the order of the PRG roms just by their order in the XML.

Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
Hacking simultaneous CRC+SHA1 is all but impossible, there's no need for a third smile

Joined: May 2007
Posts: 95
M
Member
Offline
Member
M
Joined: May 2007
Posts: 95
Under peripherals, you might describe the game's expected input methods. Weather the game wants just one controller, two controllers, a light gun, etc. IMO, it should also cover 1-player games that have hidden features using the 2nd-player controller (eg, The Legend of Zelda and its trick that allows you to save the game without having to die).

Also, IMO, it would be ideal if there was some sort of iNES -> New Format converter. A database would probably have to be provided to fill in details that iNES might not describe, but such a script/program would make it much easier than downloading new ROMs (think of the dial up users!).

Last edited by Mike S.; 06/27/07 07:35 AM.
Joined: Jun 2007
Posts: 1
J
JDG Offline
Member
Offline
Member
J
Joined: Jun 2007
Posts: 1
I like the general idea, Marty. Here are a couple of suggestions.

*You should be able to define multiple systems. For instance, some games were released with no ROM changes whatsoever in Japan, the US, and Europe, so in those cases you could have something like this: system="famicom,nes.ntsc,nes.pal". Also, I would recommend adding "ffe" and "dr.pcjr" to the list of system types.
*Allow more than one board type to be specified. The emulator would simply use the first such valid type. For instance, Teenage Mutant Ninja Turtles (US, Ultra Games) has been found on all the following boards: Nintendo's NES-SLROM-04, NES-SLROM-05, and NES-SLROM-06, and Konami's 351908. All of these boards are functionally identical and the ROM is exactly the same for every one. But it would be nice to be able to accurately document the range of actual hardware that they were played on. (By the way, board information is from BootGod's excellent database site.) Listing board manufacturer would be good, too.
*In line with the above suggestion, allow two classes for board types: "confirmed" and "virtual". Confirmed means that this is the exact board name on the actual hardware PCB. Virtual means that it is a compatible board that was never used, but would work, and that emulators could more easily recognize. For instance, "4 Nin Uchi Mahjong" (Japan, Nintendo) is on a black blob board that is called HVC-FJ. That is the real board name. But, it is 100% compatiable with standard NES/HVC-NROM-128. There are tons of bootleg NROM games that use boards with weird or no names. By using an added "virtual" name in addition to the real one, we can ensure compatibility with emulators, while still preserving the actual board name.
For instance, "Ski or Die" (US, Konami) was published only on a 351908 Konami board. But this is 100% equivalent to SLROM. So, the board descriptions would be set up something like this:
<board class="confirmed" name="351908" manufacturer="Konami" />
<board class="virtual" name="NES-SLROM" />

*Rather than a mirroring tag, allow tags for the actual solder pads on the board. This better describes what is actually going on in the hardware, and allows for flexibility with stuff like the MMC5 ExROM boards that have more solder pads than just H and V. For instance, something like this (the below example is standard vertical mirroring):
<pad h="bridged" v="open" />
*I would recommend standardizing on MD5 as a hash algorithm. It's relatively simple to implement. I'm not worried about deliberate collisions with CRC32, but accidental ones; the latter cannot entirely be ruled out even if the probability is low. MD5, on the other hand, makes an accidental collision nearly impossible.
*How about appending the XML to a standard *.NES image? The prg and chr fields, instead of specifying file names, could give the size and the offset (from start of file). This would offer the best of both worlds: a cleanly documented and implemented format for new emulators, while maintaining backwards compatibility with old ones. You'd only have 16 bytes of overhead. (You would probably also want to add a special tag right before the XML, so the parser could easily find it and not confuse it for ROM data if the header was corrupt.)
If desired, I will try to write up a full draft spec and submit it to the board (and Marty) for approval.

Joined: Jan 2006
Posts: 18
B
Member
Offline
Member
B
Joined: Jan 2006
Posts: 18
You have my support Marty, if we can get something finalized, I'll be happy to add conversion support for it. JDG, I like your ideas, it's as if you've been reading my mind smile

A netlist would be better, but, it would be such a nightmare to write all of those up. The major advantage to using a netlist (IMO) is that it would simplify the emulator in the respect that it would only need to emulate the various components and leave the connection details to the container format. I can't even count in my head how many boards use a 74/161 but connected in different ways. Anyways, I just can't see the amount of work required being done.

Regarding the "confirmed" and "virtual" board names, that's pretty much how I've been trying to handle the names in the db, with the UNIF version being the virtual name. I could use some help coming up with some standardized names though :| For example, Jaleco's PCBs are usually named using the games catalog ID, but many of them use the same board but still have the unique name. So I try to use something for the UNIF name that can be shared between them, but the results are not always pretty. For example I just did one last night, JF-19, which currently is assigned JALECO-74*08/32/161/174/174, bleh :|

I don't really like the idea of appending the XML to the end of an iNES file though. I'd rather see it in a zip along with one file per ROM. Having stuff in a zip keeps it open to being able to put other things in as well such as those ADPCM speech synth ROMs, should they ever get dumped or just the recordings of them like we have now.

Joined: Jan 2006
Posts: 138
M
Marty Offline OP
Senior Member
OP Offline
Senior Member
M
Joined: Jan 2006
Posts: 138
Originally Posted By Sotho Tal Ker
And as I see you determine the order of the PRG roms just by their order in the XML.

Yeah, I figured it's simplest that way. Another way would be to do something like this:

<prg ... id="0" />
<prg ... id="1" />
<prg ... id="2" />

I'm not sure there's anything to gain with that approach though. Anyone of a different opinion?

Originally Posted By Mike S.
Under peripherals, you might describe the game's expected input methods. Weather the game wants just one controller, two controllers, a light gun, etc. IMO, it should also cover 1-player games that have hidden features using the 2nd-player controller (eg, The Legend of Zelda and its trick that allows you to save the game without having to die).

Yeah, I was planning on that. I find it to be a bit tricky to come up with something elegant, mostly because of the difference between the NES and the Famicom units and also with the availability of four-player games.

Perhaps something like this could work:

Code:
<peripherals>
  <controller port="0">controller name</controller>
  <controller port="1">controller name</controller>
  <controller port="2">controller name</controller>
  <controller port="3">controller name</controller>
  <controller port="exp">controller name</controller>
</peripherals>

port 0..1: NES standard ports. Two standard pads connected by default (hardwired on the Famicom).
port 2..3: if specified, will tell that a four-player adapter is connected for which the controllers will go through instead.
exp: Expansion port. All Famicom controllers go through this port.

The design may seem a bit sketchy and the meaning may change depending on how things are specified. Does anyone have any better idea on how to tackle this?

As for the controller names. What should be prefered? Short and descriptive names or their actual names? For comparision:

Light gun <-> Zapper, Hyper Shot (Space Shadow)
Floor mat <-> Family Fun Fitness, Family Trainer, Power Pad
Glove <-> Power Glove (Super Glove Ball)
Robot <-> Family Computer Robot, R.O.B
Trackball <-> Hori Track (Operation Wolf)

It may turn difficult to come up with meaningful names for some of the special Famicom controllers due to little information. I'll list some of them:

- Arkanoid controller
- Mouse (Educational Computer 2000)
- Top Rider bike
- Mahjong controller (Ide Yousuke Meijin no Jissen Mahjong)
- Bop bag (Exciting Boxing)
- Tap tap mat (Pokkun Moguraa)
- Pachinko controller (Pachinko Daisakusen)
- Tablet (Oeka Kids)

Quote:
Also, IMO, it would be ideal if there was some sort of iNES -> New Format converter. A database would probably have to be provided to fill in details that iNES might not describe, but such a script/program would make it much easier than downloading new ROMs (think of the dial up users!).

Sure, I could put something together later on.

Originally Posted By JDG
*You should be able to define multiple systems. For instance, some games were released with no ROM changes whatsoever in Japan, the US, and Europe, so in those cases you could have something like this: system="famicom,nes.ntsc,nes.pal".

Right, of course. I completely forgot about that when I wrote things down. Multiple names seperated by comma works for me.

Quote:
Also, I would recommend adding "ffe" and "dr.pcjr" to the list of system types.

Some people may argue against allowing ffe (Front Far East copier images), but I'm personally fine with both of them. In Addition, I think "noac" (NES-on-a-ship) should be considered as well. There are actually quite a few games (Chinese mostly) made for it that will not work properly on a standard Famicom/NES console (timing issues etc).

Quote:
*Allow more than one board type to be specified. The emulator would simply use the first such valid type. For instance, Teenage Mutant Ninja Turtles (US, Ultra Games) has been found on all the following boards: Nintendo's NES-SLROM-04, NES-SLROM-05, and NES-SLROM-06, and Konami's 351908. All of these boards are functionally identical and the ROM is exactly the same for every one. But it would be nice to be able to accurately document the range of actual hardware that they were played on. (By the way, board information is from BootGod's excellent database site.) Listing board manufacturer would be good, too.
*In line with the above suggestion, allow two classes for board types: "confirmed" and "virtual". Confirmed means that this is the exact board name on the actual hardware PCB. Virtual means that it is a compatible board that was never used, but would work, and that emulators could more easily recognize. For instance, "4 Nin Uchi Mahjong" (Japan, Nintendo) is on a black blob board that is called HVC-FJ. That is the real board name. But, it is 100% compatiable with standard NES/HVC-NROM-128. There are tons of bootleg NROM games that use boards with weird or no names. By using an added "virtual" name in addition to the real one, we can ensure compatibility with emulators, while still preserving the actual board name.
For instance, "Ski or Die" (US, Konami) was published only on a 351908 Konami board. But this is 100% equivalent to SLROM. So, the board descriptions would be set up something like this:
<board class="confirmed" name="351908" manufacturer="Konami" />
<board class="virtual" name="NES-SLROM" />

*Rather than a mirroring tag, allow tags for the actual solder pads on the board. This better describes what is actually going on in the hardware, and allows for flexibility with stuff like the MMC5 ExROM boards that have more solder pads than just H and V. For instance, something like this (the below example is standard vertical mirroring):
<pad h="bridged" v="open" />

Excellent ideas! I was hoping that someone more experienced could help improve this section.

Quote:
*I would recommend standardizing on MD5 as a hash algorithm. It's relatively simple to implement. I'm not worried about deliberate collisions with CRC32, but accidental ones; the latter cannot entirely be ruled out even if the probability is low. MD5, on the other hand, makes an accidental collision nearly impossible.

Yeah, SHA-1 was brought up earlier but if I had to choose I'd personally go with MD5 as I have more experience with it. However, there's nothing preventing all three (crc,sha1,md5) from being allowed as far as the XML format goes. The only issue I can see is that it would add more pressure on software that's going to use it. Perhaps it would be best to restrict it to two hashing functions - CRC-32 (the simple and fast), and MD5 or SHA-1 (the more safe and secure). In this case, I'd cast my vote on CRC-32 and MD5.

Quote:
*How about appending the XML to a standard *.NES image? The prg and chr fields, instead of specifying file names, could give the size and the offset (from start of file). This would offer the best of both worlds: a cleanly documented and implemented format for new emulators, while maintaining backwards compatibility with old ones. You'd only have 16 bytes of overhead. (You would probably also want to add a special tag right before the XML, so the parser could easily find it and not confuse it for ROM data if the header was corrupt.)

I think it's probably best to leave the NES files alone. The XML format is very flexible and can serve multiple purposes without breaking compatibility. An emulator could for example search for a matching XML file (or an entry from an XML database) while loading a NES file and just use that instead of the file-header.

Quote:
If desired, I will try to write up a full draft spec and submit it to the board (and Marty) for approval.

Thanks, that would be great and it would help offloading some work for me. smile

Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
My understanding is that there's a workable crack/spoof for MD5 out there. This is why MAME went with SHA1 after initially flirting with MD5.

That said, it's pretty unlikely you could come up with a file that spoofs *both* CRC32 and MD5.

Last edited by R. Belmont; 06/29/07 02:46 AM.
Joined: Feb 2004
Posts: 2,383
Likes: 98
Very Senior Member
Online Content
Very Senior Member
Joined: Feb 2004
Posts: 2,383
Likes: 98
The "crack" for MD5 isn't really a crack. The algorithm, given a file and its MD5, will tell you which bits you can change in the file without affecting the MD5. There is not known algorithm that will let you create or alter an arbitrary file to match a given MD5.

The SHA1 "crack" isn't really a crack, either. It's just a weakness in the algorithm that allows you to reduce the space that you do a brute force attack in by a factor of 131072.

Joined: May 2007
Posts: 10
F
Member
Offline
Member
F
Joined: May 2007
Posts: 10
Quote:
I don't really like the idea of appending the XML to the end of an iNES file though. I'd rather see it in a zip along with one file per ROM. Having stuff in a zip keeps it open to being able to put other things in as well such as those ADPCM speech synth ROMs, should they ever get dumped or just the recordings of them like we have now.


I want to agree with this. The old iNES format should have no influence on this new strategy. The rom data should be left alone so that it doesn't take elaborate tools and detection schemes to document and verify roms. Most dumpers that I know have moved completely in favor of DAT files. They're easily modifiable and viewable for dumpers and users alike, but they can't ignore subjective, appended data like headers. And as this new format will prove, they really shouldn't have to.

Joined: May 2007
Posts: 95
M
Member
Offline
Member
M
Joined: May 2007
Posts: 95
A MAME-like approach to storing the ROMs would be nice, sans the stupid MAME things (eight-char filename limits, all game-specific info is documented in the emulator instead of the game set, etc). Multiple ROMs don't belong concatenated together, even if they're on the same cartridge.

This post might be redundant, but I'm getting a feeling that you're planning on keeping iNES files.


Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
Filenames in individual ROMs in MAME are not limited to 8.3, only the outer zip names are.

Joined: Mar 2002
Posts: 1,257
Likes: 44
H
hap Offline
Very Senior Member
Offline
Very Senior Member
H
Joined: Mar 2002
Posts: 1,257
Likes: 44
IMO the zip container containing raw ROM images and an XML board description is the best solution suggested so far. Forget iNES ever existed, it doesn't have any value for future preservation.

(Of course iNES files will keep existing, the purpose of this project is not to kick the iNES format off the throne. Instead of for gaming (iNES), this is for accurate documentation/preservation, right?)

Joined: Jan 2006
Posts: 3,690
Very Senior Member
Offline
Very Senior Member
Joined: Jan 2006
Posts: 3,690
right!

I have a question anyway: how to obtain these raw data? is copynes reliable? if yes, bootgod database is a wonderful reference for this. if not, who would start to dump once more the carts?

EDIT: and notice that I really support this zip with prg+chr+xml idea, I also started to split back a few images following bootgod data... so that we have data available when the xml files are ready (and there will be probably a TOSEC datfiles for these, sooner or later, as I did also for the samples...)

Last edited by etabeta78; 06/30/07 06:26 PM.
Joined: Oct 2005
Posts: 351
R
Senior Member
Offline
Senior Member
R
Joined: Oct 2005
Posts: 351
Originally Posted By Marty
Originally Posted By MESSfan
crc32 was hacked years ago
why not add some sha1

While I don't think it's worth worrying too much about CRC vandals, a more safe hash function could indeed be desired.

<game ... crc="baaaaaad" sha1="91a009e610b5e2b9776ac540dcdcada6f8cfc507">

Any other hash function to be considered, or will the two do for now?


CRC vandals have gone as far as hacking hidden intros into roms and reverse the CRC back to the proper dump. So I think it's worth worrying.

As R.Belmont stated, the crc32+sha1 combo is secure enough, adding more will just slow down the checks.

*edit

On a not related note, will the roms be splitted or still joined?
That's unclear. I favor splitted sets, it looks much better...

Last edited by MESSfan; 06/30/07 07:32 PM.
Joined: Jan 2006
Posts: 18
B
Member
Offline
Member
B
Joined: Jan 2006
Posts: 18
Originally Posted By MESSfan

CRC vandals have gone as far as hacking hidden intros into roms and reverse the CRC back to the proper dump. So I think it's worth worrying.


Are you serious!? Here I always thought to myself there was no real reason to use anything beyond CRC32 because the chances of an actual (as in not intentional) collision are practically 0 and I didn't think anyone would be enough of a jack ass to do it on purpose (silly me :|).

I never used to collect SHA-1 for the DB either, but a few people requested it some time ago so I added it anyways. FYI, you can view the SHA-1 by hovering over the the CRC32 on the website. Not terribly useful like that, but it's there smile

Originally Posted By etabeta78
is copynes reliable?


As long as the mapper bank-switching info is known and correct, yes it is. If not, this mis-information gets built into the dumping program and in turn the banks will come out in the wrong order. If emulators also use this bad info for implementation, then you really have a problem because the game will appear to run fine because the errors cancel each other out. I've been keeping a close eye on this though and I don't believe anything in the DB is "dirty" like this.

Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
There were some instances of vandalized MAME ROMs with the CRC32 hacked to appear original as well. (Worse, people started burning the modified ROM into an EPROM and sold the resulting region-hacked PCBs on eBay, meaning we had to give up on trying to find legit alternate regions for that game). That's why there's 2 hashes now smile

Last edited by R. Belmont; 06/30/07 10:46 PM.
Joined: May 2007
Posts: 10
F
Member
Offline
Member
F
Joined: May 2007
Posts: 10
Originally Posted By BootGod

Are you serious!? Here I always thought to myself there was no real reason to use anything beyond CRC32 because the chances of an actual (as in not intentional) collision are practically 0 and I didn't think anyone would be enough of a jack ass to do it on purpose (silly me :|).


I can elaborate on this a little bit. There are "dumping groups" that take an inordinate amount of pride in what they do, so they feel it is necessary to credit themselves in the game with an intro or something. Then to spite archival projects like no-intro, they can hack the modified CRC32 to match the unmodified version's. CRC32 + SHA1 would make this nearimpossible because you simply cannot fake both checksums at the same time.

So yeah, for a while people were just like you, considering only deceit and vandalism out of boredom or twisted self-gratification. But then this less apparent motive became clear, and it wasn't worth risking the verification status of everything when it was so easy to just use two and be done with it.

Last edited by FitzRoy; 07/02/07 11:52 AM.
Joined: Feb 2006
Posts: 81
D
Member
Offline
Member
D
Joined: Feb 2006
Posts: 81
Do the GoodTools still use CRC32?


"Last version was better," says Floyd. "More bugs. Bugs make game fun."
Joined: Feb 2006
Posts: 4
Member
Offline
Member
Joined: Feb 2006
Posts: 4
Quote:
Do the GoodTools still use CRC32?


The newer tools use SHA1, but there still is an option to scan w/ CRC32 should you manually select it.

Joined: May 2007
Posts: 95
M
Member
Offline
Member
M
Joined: May 2007
Posts: 95
Not that people should still being using GoodTools anyway, they're no longer useful as they once were. Better database projects are around now.

Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
Yeah, but from my (admittedly limited) views into the "underground" most ROM distributions are still done in terms of GoodSets.

Joined: Feb 2006
Posts: 81
D
Member
Offline
Member
D
Joined: Feb 2006
Posts: 81
Originally Posted By R. Belmont
Yeah, but from my (admittedly limited) views into the "underground" most ROM distributions are still done in terms of GoodSets.
Which is why I asked the question. I am not a fan of the GoodTools at all (the name is a misnomer as far as I'm concerned), but it remains the "industry standard," so we have to deal with it. The GoodTools have the biggest databases out there as far as I know, but that's only because of all the bad dumps and hacks included with it (which most other DB's don't count). The larger number serves as a marketing advantage, which is why I don't think a competitor will ever overthrow the GoodTools unless it has a similar "unabridged" database.


"Last version was better," says Floyd. "More bugs. Bugs make game fun."
Joined: Jan 2006
Posts: 44
X
Member
Offline
Member
X
Joined: Jan 2006
Posts: 44
Originally Posted By dvdmth
The GoodTools have the biggest databases out there as far as I know, but that's only because of all the bad dumps and hacks included with it
That's actually what I like about the GoodTools. I believe that bad dumps and overdumps can still serve a purpose. For example, bad dumps can be tested on real hardware just as well as good dumps, and the results compared to emulators. In addition, if you dump a ROM and a GoodTool identifies it as bad, then at least you know what it is. (And just for the record, I'm not one of those people who must have bad dumps in their collection.)

No-Intro, on the other hand, only identifies the "best" dumps, which seems a little subjective to me. Take pirate NES games for example. In the NES dat only "pirate originals" are included, but pirates hacked from other games are not. I'm pretty sure that many of these pirate hacks have unique mappers that aren't found in other games. Is it really worth excluding these games that have unique mappers?

I can understand why some people don't like the GoodTools, though. If Cowering disagrees with something that a lot of other people support, it won't get supported in the GoodTools. With that said, I doubt Cowering will support iNES 2.0 and the XML idea.

Edit: Sorry, I shouldn't have turned this into a big discussion thread. The point of this thread is to work on the XML format, not give personal opinions about GoodTools...

Last edited by xamenus; 07/03/07 06:52 AM.
Joined: May 2007
Posts: 21
S
Member
Offline
Member
S
Joined: May 2007
Posts: 21
Unless the good tools consist of wrong information themselves. Which is usually the case. Since his goodtools als closed source and he is the single maintainer of everything the day will come when there are no updates anymore. wink
And since nobody has his sources what is confirmed by who, when, how, you would have to restart collecting info, redumping every game to see if it matches and and and...

I will partially quote bootgods post from the no-intro boards to demonstrate how inaccurate the goodtools (goodnes in this case) are:
Originally Posted By bootgod
Ok on to the good stuff. Note that I was using the latest goodnes to come up with the following things, in the future I will switch over to using BigFred's dats to make things easier.

The following changes need to be made regarding revisions:
4 Nin Uchi Mahjong (J).nes is RevA
Dragon Quest III (J).nes was marked RevA on back, but has v0 ROM (so leave alone I guess)
Dragon Quest IV (J).nes is RevA
Family Mahjong II - Shanghai heno Michi (J).nes appears to be RevA
Mach Rider (U) [h1].nes is RevA
Super Mario Bros. 3 (J) (PRG0).nes is RevA
Toukyou Pachi Slot Adventure (J) [!].nes is RevA
Tetris (J) [a1].nes is RevA

These have various incorrect goodnes tags that need to be fixed:
Digital Devil Monogatari - Megami Tensei II (J) [b1].nes is not bad [!]
Door Door (J) (FDS Hack).nes is not hack [!]
Dragon Ball 3 - Gokuu Den (J) [a1].nes should be moved to [!] for now
Famicom Jump - Eiyuu Retsuden (J) [f1].nes is not fixed [!]
Lode Runner (J) [b2].nes is not bad [!]
Ninja Jajamaru Kun (J) [t1].nes is not trained [!]
Raid on Bungeling Bay (J) [a1].nes should be moved to [!] for now

I'm not going to bother listing all the games that need to be upgraded to [!]

The following games are incorrectly "fixed" by the goodnes fixnes option (see the DB for correct values):
iNES Header Fixes - [--F---] -> Barcode World (J).nes.new.nes
iNES Header Fixes - [--F---] -> Digital Devil Monogatari - Megami Tensei II (J) [b1].nes.new.nes
iNES Header Fixes - [--F---] -> Door Door (J) (FDS Hack).nes.new.nes
iNES Header Fixes - [--F---] -> Family Boxing (J).nes.new.nes
iNES Header Fixes - [--F---] -> Famista '90 (J).nes.new.nes
iNES Header Fixes - [--F---] -> Famista '91 (J).nes.new.nes
iNES Header Fixes - [--F---] -> Famista '93 (J).nes.new.nes
iNES Header Fixes - [--F---] -> Final Lap (J).nes.new.nes
iNES Header Fixes - [--F---] -> Jajamaru Ninpou Chou (J) [!].nes.new.nes
iNES Header Fixes - [--F---] -> Kaijuu Monogatari (J).nes.new.nes
iNES Header Fixes - [--F---] -> Lasa-r Ishii no Childs Quest (J).nes.new.nes
iNES Header Fixes - [--F---] -> Metro-Cross (J).nes.new.nes
iNES Header Fixes - [--F---] -> Ninjara Hoi! (J).nes.new.nes
iNES Header Fixes - [--F---] -> Pro Yakyuu - Family Stadium '87 (J).nes.new.nes
iNES Header Fixes - [--F---] -> Pro Yakyuu - Family Stadium (J).nes.new.nes
iNES Header Fixes - [--F---] -> SD Gundam - Gachapon Senshi 2 - Capsule Senki (J).nes.new.nes
iNES Header Fixes - [--F---] -> SD Gundam - Gachapon Senshi 3 - Eiyuu Senki (J).nes.new.nes
iNES Header Fixes - [--F---] -> SD Gundam - Gachapon Senshi 4 - NewType Story (J).nes.new.nes
iNES Header Fixes - [--F---] -> Shin 4 Nin Uchi Mahjong - Yakuman Tengoku (J).nes.new.nes
iNES Header Fixes - [--F---] -> Sky Kid (J).nes.new.nes
iNES Header Fixes - [--F---] -> Tanigawa Kouji no Shougi Shinan 2 (J).nes.new.nes
iNES Header Fixes - [--F---] -> Tetris (J) [a1].nes.new.nes


PS: Some Trivia why those tools are named "goodxxx": Cowerings first tool was for the NES and he pondered about naming it "GoodNES" or "NESCafe". He chose the first and named all his other tools subsequently with "good" ;-)

Joined: Oct 2005
Posts: 351
R
Senior Member
Offline
Senior Member
R
Joined: Oct 2005
Posts: 351
a tales of goodtools fun

a while ago, I renamed a bad dump of JRA PAT with goodsnes
the renamed file: NTT JRA PAT - Wide Baken Taiyou (J) [!].sfc
then I wondered about that [!], what could it mean? bad dump?
I searched the docs and found the shocking truth: [!] Verified Good Dump

At that second I rofl'd under the table for a couple of minutes... why? because the rom has a bad checksum. A damned BAD CHECKSUM. A Verified Good Dump with a bad checksum? Such blatant amateurism is to be commonly expected from TOSEC, but from the glorious cowering? I lol'd.

I then realized the seriousness of goodstuff verifications and deleted the goodcrap for good.

Today JRA PAT is redumped and we have a real good dump with good checksum. Happy ending. grin

Joined: Jan 2006
Posts: 3,690
Very Senior Member
Offline
Very Senior Member
Joined: Jan 2006
Posts: 3,690
Quote:
Such blatant amateurism is to be commonly expected from TOSEC


thank god there exists the smart nointro crew tired tired

Last edited by etabeta78; 07/04/07 03:58 PM.
Joined: May 2007
Posts: 21
S
Member
Offline
Member
S
Joined: May 2007
Posts: 21
Does not mean that other dats are without any errors - but in the case of No-Intro the stuff is all open and the errors can be quickly fixed. smile

Well, back on topic:
When will Nestopia with support for this be released? smile
At least: How does the current specification look like? Including the suggestions that were made.

Also you could include as many checksums as you like, when you make an option to turn individual checksums off in the emulator.

Joined: May 2007
Posts: 95
M
Member
Offline
Member
M
Joined: May 2007
Posts: 95
Considering that Marty is both the author of the format and the emulator, I can't imagine it taking long for Nestopia to support it once it's finalized.

My thoughts on checksums: The format should be flexible so newer fields can be added when needed. You can have CRC32, MD5, SHA-1, SHA-256, etc. Programs should ignore unknown fields (maybe give a warning message about the unknown field (since it might indicate a spelling error), but not remove it). This gives increased flexibility at the user's needs. The emulator probably should check at least CRC32 or MD5 (user configurable is optimal) on load so the user can be warned about corrupt files, though I can't imagine it preventing bad ROMs from being distributed since all the malicious users need to do is update the checksum fields.

Joined: May 2007
Posts: 95
M
Member
Offline
Member
M
Joined: May 2007
Posts: 95
I'm sorry to bump, but I'm wondering on weather there's been any news on this format?

I think I'm just a bit anxious to see a new and improved format laugh

Joined: Jan 2006
Posts: 138
M
Marty Offline OP
Senior Member
OP Offline
Senior Member
M
Joined: Jan 2006
Posts: 138
It's coming along nicely even though I'm a bit disappointed with the lack of interest and feedback so far. Luckily, Bootgod is helping me out with the project and we're currently exploring ways to handle the extreme-case-carts. I'm also in the process of adding support for ROM sets and external databases using this format.

Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
If Bootgod is helping you out, I'm not sure you need other feedback smile

Joined: Oct 2005
Posts: 351
R
Senior Member
Offline
Senior Member
R
Joined: Oct 2005
Posts: 351
Way to go Marty & BootGod! You are the men!
I hope the external db can be a CM dat.

Joined: Feb 2006
Posts: 81
D
Member
Offline
Member
D
Joined: Feb 2006
Posts: 81
Originally Posted By MESSfan
I hope the external db can be a CM dat.

Can a CM dat contain INES / NES 2.0 / UNIF info for managing NES ROMs? Because of the issues specific to NES, you almost need a separate ROM tool for this console (like NSRT is for Super NES). I'd love a cross-platform NES-specific ROM tool that can scan ROMs, fix common bad dumps (like the infamous SMB pirate dump), convert between INES, NES 2.0, and UNIF, rename and sort ROMs, etc. Anyone here willing to take on such a project?


"Last version was better," says Floyd. "More bugs. Bugs make game fun."
Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
NSRT is probably not the best example - it's intended primarily to make everyone turn their ROMs into a proprietary format so the author can then charge people to use them smile

Joined: Oct 2005
Posts: 351
R
Senior Member
Offline
Senior Member
R
Joined: Oct 2005
Posts: 351
CM dats can do anything db related, have support for filnames, size, hashes up to sha1, and any extra custom field.

Joined: Feb 2006
Posts: 81
D
Member
Offline
Member
D
Joined: Feb 2006
Posts: 81
Originally Posted By R. Belmont
NSRT is probably not the best example - it's intended primarily to make everyone turn their ROMs into a proprietary format so the author can then charge people to use them smile

Umm... I fail to see your point here. All NSRT does (and only if you tell it to) is use empty space in the 512-byte header at the start of .smc files to provide additional info, such as controllers used by the game. Although NSRT is closed source, the NSRT header is used by Snes9x and ZSNES, both of which are open source. Besides, the NSRT header is designed with backwards compatibility in mind, so any emulator or tool can still read ROMs with an NSRT header, whether the software knows anything about NSRT or not.

I don't think a piece of software has to be open source to be good. It can help, but it certainly isn't necessary.


"Last version was better," says Floyd. "More bugs. Bugs make game fun."
Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
This is *not* an open vs. closed source thing. This is a "people attempting to make money off hobbyist emulation" thing (something which there's a depressing amount of lately). NSRT's original intent (I read this in a forum post by Nach) was to convert ROMs to a compressed proprietary format which they would then charge everyone money to use once it caught on. If that's no longer happening, wonderful.

That said, extending headers is still the wrong way. Just look at the title of this thread.

Joined: Oct 2005
Posts: 351
R
Senior Member
Offline
Senior Member
R
Joined: Oct 2005
Posts: 351
Indeed the whole point is to get rid of headers, not the opposite. Sir dvdmth, headers fail.

Last edited by MESSfan; 08/03/07 12:11 AM.
Joined: Nov 2003
Posts: 459
K
Senior Member
Offline
Senior Member
K
Joined: Nov 2003
Posts: 459
Originally Posted By R. Belmont
NSRT's original intent (I read this in a forum post by Nach) was to convert ROMs to a compressed proprietary format which they would then charge everyone money to use once it caught on. If that's no longer happening, wonderful.


You sure this wasn't a joke post? Can't imagine Nach doing something like that...

Joined: Feb 2006
Posts: 81
D
Member
Offline
Member
D
Joined: Feb 2006
Posts: 81
Originally Posted By R. Belmont
NSRT's original intent (I read this in a forum post by Nach) was to convert ROMs to a compressed proprietary format which they would then charge everyone money to use once it caught on. If that's no longer happening, wonderful.

Citation? There certainly isn't anything in the NSRT documentation that suggests anything like that. If Nach were trying to push a proprietary compression format, then why does NSRT support several different compression types? And if that were the original intent of NSRT, it certainly failed miserably because I never saw an SNES ROM that didn't use a standard compression format such as ZIP (and NSRT has been around for years).
Originally Posted By R. Belmont
That said, extending headers is still the wrong way. Just look at the title of this thread.

Frankly, I don't think this XML idea will succeed at replacing INES (as much as I'd love it to, I doubt it'll happen). What I like about NES 2.0 is its backwards compatibility, which is the primary reason why UNIF failed. A new format will not catch on unless it works with existing emulators and tools.

I like the idea being discussed here, and I sincerely hope it succeeds. At the same time, I don't think we shouldn't also try to help people fix their ROMs so that they will work in emulators that haven't been updated to use the XML data.


"Last version was better," says Floyd. "More bugs. Bugs make game fun."
Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
Backwards compatibility sucks. And NEStopia is much more powerful and influential than it used to be :-)

Joined: Oct 2005
Posts: 351
R
Senior Member
Offline
Senior Member
R
Joined: Oct 2005
Posts: 351
LMAO dvdmth! If it is supported by Nestopia and MESS, hell I would care about other emulators!

Joined: May 2007
Posts: 95
M
Member
Offline
Member
M
Joined: May 2007
Posts: 95
Originally Posted By dvdmth
What I like about NES 2.0 is its backwards compatibility, which is the primary reason why UNIF failed. A new format will not catch on unless it works with existing emulators and tools.

iNES, any version, only cares about giving the people games to play. The goal of the XML-based format is about preserving how the hardware actually works.

Joined: May 2007
Posts: 21
S
Member
Offline
Member
S
Joined: May 2007
Posts: 21
So... what happened to this idea? smile

Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
BootGod seems to have worked it out. It's unclear when there'll be a new NEStopia to take advantage though.

Joined: Jan 2006
Posts: 138
M
Marty Offline OP
Senior Member
OP Offline
Senior Member
M
Joined: Jan 2006
Posts: 138
It certainly will. We're adding the final touches to it for v1.0. cool
I may post an XML schema file here later for viewing and feedback.

Joined: Jan 2006
Posts: 3,690
Very Senior Member
Offline
Very Senior Member
Joined: Jan 2006
Posts: 3,690
it would be really nice smile

Joined: Jan 2006
Posts: 138
M
Marty Offline OP
Senior Member
OP Offline
Senior Member
M
Joined: Jan 2006
Posts: 138
Here's a WIP XML schema file of the database format:

Download Link

Romset schema is not completed yet but should look almost the same. Not being terribly experienced in XML schema creation, it may not be entirely correct and some of the rules may sometimes be too strict or too loose, but it's a start.

For reference, here's how a game element may look:

Code:
<game name="Bird Week" class="Licensed" catalog="TFS-BK" publisher="Toemiland" developer="Lenar" region="Japan" players="1" date="1986-6-3">
  <cartridge system="Famicom" dumper="bootgod" datedumped="2006-12-24" crc="A8A9B982" sha1="22EAF03CC4A148EB7F96CCBDB878170B6D117940">
    <board type="HVC-CNROM" pcb="HVC-CNROM-256K-01" mapper="185">
      <prg name="TFS-BK-0 PRG" size="16k" crc="2A629F7D" sha1="BD0B7D6974CE1E3BC0A8AFB55FEBD0C03332361C"/>
      <chr name="TFS-BK-0 CHR" size="8k" crc="970A934E" sha1="2307A6DAF73984CA4242A93858CAABF9F86469AD">
        <pin number="26" function="CE"/>
        <pin number="27" function="CE"/>
      </chr>
      <chip type="74xx161"/>
      <pad h="1" v="0"/>
    </board>
    <properties>
      <property name="Back Label ID" value="860410"/>
      <property name="Label ID Code" value="TFS-BK"/>
      <property name="Manufacturer ID" value="09"/>
      <property name="Cart Type" value="Standard"/>
      <property name="Cart Producer" value="Famicom"/>
      <property name="Cart Color" value="Aqua"/>
      <property name="Secondary ID" value="FS-2003G"/>
    </properties>
  </cartridge>
</game>

And a minimalistic approach:

Code:
<game>
  <cartridge system="Famicom" crc="A8A9B982" sha1="22EAF03CC4A148EB7F96CCBDB878170B6D117940">
    <board type="HVC-CNROM" mapper="185">
      <prg size="16k" />
      <chr size="8k">
        <pin number="26" function="CE" />
        <pin number="27" function="CE" />
      </chr>
      <pad h="1" v="0" />
    </board>
  </cartridge>
</game>


Joined: Oct 2005
Posts: 351
R
Senior Member
Offline
Senior Member
R
Joined: Oct 2005
Posts: 351
Great Marty!
I can not wait for next Nestopia update!
BTW, will nestopia use split images as the PCB roms, or still joined images?

Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
Very cool smile

Joined: Feb 2006
Posts: 81
D
Member
Offline
Member
D
Joined: Feb 2006
Posts: 81
Whoa, that's really neat! I can tell you spent a lot of time on this project. No wonder Nestopia hasn't been updated lately...

You might want to post this on the nesdev forum as well.


"Last version was better," says Floyd. "More bugs. Bugs make game fun."
Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
nesdev alternately denies there's a problem with iNES and pushes the weak-sauce iNES 2.0. I'm not sure they'll like medicine this strong, but with Marty and bootgod driving it's sure to attract attention :-)

Joined: Dec 2007
Posts: 23
T
Member
Offline
Member
T
Joined: Dec 2007
Posts: 23
That looks great so far, has anyone given any though yet to adding things to the zip file like artwork, cheat.dat, info.dat and so forth?

Or maybe some of these could be in the xml itself?

Also would a basic rom file be named after the PCB? so if the prg and chr weren't split the file would be HVC-CNROM-256K-01.bin ?

It's not fully clear to me what the filenames would be for the file(s) in the zip

Last edited by Tetsuo55; 02/26/08 05:25 PM.
Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
Pretty much the entire point of the zip wrapper is exactly so that manual/label/pcb/cart scans can be included. And there's no need for fixed filenames - the XML points to the ROMs, and presumably there'd be only one XML in the zip so that wouldn't need to be any special name either.

Last edited by R. Belmont; 02/26/08 05:35 PM.
Joined: Oct 2005
Posts: 351
R
Senior Member
Offline
Senior Member
R
Joined: Oct 2005
Posts: 351
Originally Posted By Tetsuo55
Also would a basic rom file be named after the PCB? so if the prg and chr weren't split the file would be HVC-CNROM-256K-01.bin


This is not how ROM are named on the PCB.

ex: http://bootgod.dyndns.org:7777/profile.php?id=2529

PAL-MW-0 PRG
NES-MW-0 CHR

They could be named:

PAL-MW-0.PRG
NES-MW-0.CHR

But for now it is unclear whether Nestopia will use split or joined images.

Joined: Dec 2007
Posts: 23
T
Member
Offline
Member
T
Joined: Dec 2007
Posts: 23
One still has to wonder how frontends would handle the included documentation and artwork.

At first i was thinking of something standard like:

mario kart.zip
(mario kart.bin)
(manual.pdf)
(screenshot.png)
(box.png)\
and so forth

However i then started thinking, how would a frontend or emulator cache 100's of screenshots with the exact same name..


@messfan
Yeah i was talking about non-split roms, when applying xml to current games(regardless of system) we will be stuck at first with merged single files, i assume the best name for those would be that of the pcb, at least untill they get propered with split roms(and is that really proper or just byteswapped)

Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
Not to be mean, but that's the frontend's problem, not ours :-)

Joined: Dec 2007
Posts: 23
T
Member
Offline
Member
T
Joined: Dec 2007
Posts: 23
Doesn't messui support screenshots and such??

Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
I don't know, I've never seen or used messui so that would tend to not be my problem. Also, THIS IS THE NESTOPIA BOARD. MESS can fend for itself when this format becomes the standard.

Joined: Dec 2007
Posts: 23
T
Member
Offline
Member
T
Joined: Dec 2007
Posts: 23
i see your point, but why reinvent the wheel twice i say

Joined: Jan 2006
Posts: 3,690
Very Senior Member
Offline
Very Senior Member
Joined: Jan 2006
Posts: 3,690
I would be more interested in understanding how nestopia will handle the xml inside the zip.

I mean: I assume at start it will support .nes images (to be crc'ed as a whole file minus the header) and use the board & prg & chr info (and the pins at a deeper level) to correctly emulate the game.

but, looking in perspective, wouldn't be better to add hashes to prg & chr in the minimalistic approach as well... on the long term, it would be nice to add support for splitted sets without force the user to edit each xml files...


my 2 cents

Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
Actually, given Bootgod's involvement, there will almost certainly be brand-new certified dumps in the new format (although I'll understand if nobody wants to say as much in public). I strongly doubt the XML-points-to-NES garbage would ever be supported because it would simply encourage bad habits. It's not like the ability to open iNES files is going to go away as soon as this is introduced.

Last edited by R. Belmont; 02/26/08 09:16 PM.
Joined: Jan 2006
Posts: 3,690
Very Senior Member
Offline
Very Senior Member
Joined: Jan 2006
Posts: 3,690
I hope so.

anyway, notice that most of the [!] in GoodNES and the images in NoIntro are exactly the bootgod prg+chr (that's the way I was creating a TOSEC dat bootgod-compliant last summer, before an unfortunate HD failure...)

so at start it could be well possible to simply release the xmls (which wouldn't violate any copyright) and let clrmame put them in the zip...

later, images could simply be splitted in header, prg & chr according to the xml and you would obtain (finally) a correct representation of the content of NES carts...

anyway, it's a bit too early to bother about this. I'll simply wait for Marty and next NEStopia, and see what will happen

Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
Yup. This is the *definition* of "relax and cool things will happen" smile

Joined: Jan 2006
Posts: 138
M
Marty Offline OP
Senior Member
OP Offline
Senior Member
M
Joined: Jan 2006
Posts: 138
Originally Posted By MESSfan
But for now it is unclear whether Nestopia will use split or joined images.

For romsets, it will use whatever the XML tells it to. Consider the following:

1) <prg size="32k" file="combined.rom" ... />

2) <prg id="0" size="16k" file="first.rom" ... />
<prg id="1" size="16k" file="second.rom" ... />

Same end result for an emulator but (2) is more informational (provided they're in fact two distinct chips) and sometimes even necessary if they differ in some ways, like in the case of two WRAM chips where a battery is connected to one but not the other. PRG and CHR will always be treated as two separate entities, meaning each element will point to a different file of choice.

For databases, the source image file can simply be of any type. Standard CRC32+SHA1 lookup of combined PRG+CHR data will be used to find an entry.

As for ROM file naming conventions, I honestly don't know what I like best. xxx.prg/chr? xxx.bin? xxx.rom? xxx? Since it's outside the scope of the format anyway I think I'll just stay out of that one.

Joined: Jan 2006
Posts: 138
M
Marty Offline OP
Senior Member
OP Offline
Senior Member
M
Joined: Jan 2006
Posts: 138
Originally Posted By etabeta78
I would be more interested in understanding how nestopia will handle the xml inside the zip.

It's pretty straightforward. It searches for an XML file inside the ZIP (any name will do), reads it and then, guided by the XML, loads and assembles the raw binary files (also inside the ZIP) to be used for emulation. In other words, NES and UNIF files inside the container won't work. The only way to utilize them with the XML format would be through a database.

Joined: Jan 2006
Posts: 3,690
Very Senior Member
Offline
Very Senior Member
Joined: Jan 2006
Posts: 3,690
thanks for the explaination!

Joined: Feb 2006
Posts: 81
D
Member
Offline
Member
D
Joined: Feb 2006
Posts: 81
Will IPS files currently in existence still be usable with this new format?


"Last version was better," says Floyd. "More bugs. Bugs make game fun."
Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
As long as you let NEStopia do the patching, many (probably not all) IPS files should work.

Joined: Jan 2006
Posts: 18
B
Member
Offline
Member
B
Joined: Jan 2006
Posts: 18
As far as the naming of files inside the container goes, as Marty said, they can be named whatever you want. I've tentatively decided to name them like so:

If they have an actual ID printed on them, that is what will be used. Just like MESSfan's example:

PAL-MW-0.PRG
NES-MW-0.CHR

If they don't have an ID (like epoxy blobs or eproms) I was planning on either using an implied name (e.g. another ROM with the same hash, that does have an ID) or just use the CRC32 for the filename:

43E95611.PRG
B4C1BFB4.CHR

I'm not sure if I'm going to have support to build images in this format in time for the next release of Nestopia, as there is a bit of behind the scenes stuff I need to get done first. I hope to have it ready ASAP though.

Once nice thing about the database aspect is that Nestopia will still have its huge internal DB with data for carts not yet in my system and then you'll be able to overlay one downloaded from my system on top of his. So your not stuck with one or the other and you can get an up-to-date one at any time without needing a new version of Nestopia to be released.

Joined: Jan 2006
Posts: 138
M
Marty Offline OP
Senior Member
OP Offline
Senior Member
M
Joined: Jan 2006
Posts: 138
Originally Posted By R. Belmont
nesdev alternately denies there's a problem with iNES and pushes the weak-sauce iNES 2.0. I'm not sure they'll like medicine this strong, but with Marty and bootgod driving it's sure to attract attention :-)

And it sure did. Well, even if the OP there was unessecarily provocative, at least I got credited for being the creator of "the sunlight, the earth, the community...". Anyway, I'm tired right now so I'll just post some brief comments.

- IPS should work just fine as long as they're soft-patched. PRG+CHR minus 16 and voila.

- There are several excellent free XML parsers available, such as TinyXML. I wrote my own because I'm crazy.

- I'm not interested in leading a crusade to get rid of INES. Every format has its strengths and weaknesses. Apples and oranges. I like oranges.

- Nice work on Schpune Disch. smile

Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
Wow. That thread confirmed everything I ever thought was bad about nesdev - none of them give a flying frak about the actual hardware or software, they just like the free games. I expected that from most people, but it was disappointing to see it from the likes of Blargg.

Ahh well, when MAME and MESS go to xml-in-a-zip-wrapper ROMs at least they can't say we didn't warn them it was the Next Big Thing ;-)

Last edited by R. Belmont; 02/29/08 10:24 PM.
Joined: Feb 2006
Posts: 81
D
Member
Offline
Member
D
Joined: Feb 2006
Posts: 81
Originally Posted By Marty
- IPS should work just fine as long as they're soft-patched. PRG+CHR minus 16 and voila.

Well, you'd better convince Richard Bannister to add soft IPS support in the Mac version, or you can count me out as a customer for the new format (at least the zipfile aspect).
Quote:
- I'm not interested in leading a crusade to get rid of INES. Every format has its strengths and weaknesses. Apples and oranges. I like oranges.

Nor should you. As I said, INES will never go away. You're just adding another alternative for people to use, or not use. What ticked people off at nesdev is the fact that the poster proclaimed the death of INES. That's flat out wrong. Besides, INES files can still benefit from the XML database approach, so it isn't like you gain nothing by sticking with INES.


"Last version was better," says Floyd. "More bugs. Bugs make game fun."
Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
I dunno, once No-Intro or TOSEC or whoever puts up a torrent of new-format ROMs with included manuals, box, cartridge scans, Game Genie codes, and IPS patches (all in each game's zip container) then iNES is gonna start looking anemic.

Soft-patching is a core feature of NST as far as I know, so if Richard doesn't hook it up it's not Marty's fault.

Joined: Jun 2007
Posts: 44
D
Member
Offline
Member
D
Joined: Jun 2007
Posts: 44
Why get Bannister to do it wink I've been hard at work on a full rewrite of OpenNestopia, and with any luck will be able to release right alongside of the official version.

Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
Nifty, looking forward to that then smile

Joined: Jan 2006
Posts: 138
M
Marty Offline OP
Senior Member
OP Offline
Senior Member
M
Joined: Jan 2006
Posts: 138
Originally Posted By R. Belmont
Soft-patching is a core feature of NST as far as I know, so if Richard doesn't hook it up it's not Marty's fault.

The patching actually takes place in the Windows shell code right now. I could change that though and move it to the core instead and add an optional std::istream parameter to the loader functions. Would that work?

Joined: Jun 2007
Posts: 44
D
Member
Offline
Member
D
Joined: Jun 2007
Posts: 44
Originally Posted By Marty
Originally Posted By R. Belmont
Soft-patching is a core feature of NST as far as I know, so if Richard doesn't hook it up it's not Marty's fault.

The patching actually takes place in the Windows shell code right now. I could change that though and move it to the core instead and add an optional std::istream parameter to the loader functions. Would that work?


Would sure make my life easier

Joined: Mar 2001
Posts: 16,923
Likes: 57
R
Very Senior Member
Online Content
Very Senior Member
R
Joined: Mar 2001
Posts: 16,923
Likes: 57
Ahh, that would be why I didn't implement it yet. And yes, please do smile

Joined: May 2007
Posts: 95
M
Member
Offline
Member
M
Joined: May 2007
Posts: 95
Originally Posted By R. Belmont
Wow. That thread confirmed everything I ever thought was bad about nesdev - none of them give a flying frak about the actual hardware or software, they just like the free games. I expected that from most people, but it was disappointing to see it from the likes of Blargg.

Ahh well, when MAME and MESS go to xml-in-a-zip-wrapper ROMs at least they can't say we didn't warn them it was the Next Big Thing ;-)

I just read the thread you're talking about and... wow indeed. It's rare that I see such high levels of ignorance.

Joined: Dec 1969
Posts: 917
Likes: 3
R
Senior Member
Offline
Senior Member
R
Joined: Dec 1969
Posts: 917
Likes: 3
Originally Posted By R. Belmont
Ahh, that would be why I didn't implement it yet. And yes, please do smile


Yup, I'll second that.

Joined: Dec 2007
Posts: 23
T
Member
Offline
Member
T
Joined: Dec 2007
Posts: 23
Bootgod, i hope your reading this..

I saw over at the aformentioned thread over at nesdev http://forums.no.intro.free.fr/redirector.php?url=http%3A%2F%2Fnesdev.parodius.com%2Fbbs%2Fviewtopic.php%3Fp%3D31392%2331392

That you're implementiation of the XML format allows for merging of clones into a single zip, over at no-intro we are hard at work for identifing parents/clones for this kind of merging, maybe using this XML based system will allow for a better more coordinated way to define parents and clones than the current text file list we use, see here for our current system:
http://forums.no.intro.free.fr/showthread.php?t=1999

Joined: Mar 2008
Posts: 1
J
Member
Offline
Member
J
Joined: Mar 2008
Posts: 1
Originally Posted By MESSfan
a tales of goodtools fun


At that second I rofl'd under the table for a couple of minutes... why? because the rom has a bad checksum. A damned BAD CHECKSUM.

Aren't there good dumps with a known incorrect checksum? Or is that more of the goodtools koolaid recirculating?

And to Marty, why not put the ines files inside the xml file, like here?
Code:
<dump format="ines" system="nes" country="us">
NES**@DiskDude!**'***"**7* ****(***(***7  
...
</dump>


I'm assuming the format would support more than one system type?


Sorry for joining the thread late.

Last edited by Justin Goldberg; 03/18/08 06:06 AM.
Page 1 of 9 1 2 3 4 5 6 7 8 9

Moderated by  Marty, R. Belmont 

Link Copied to Clipboard
Who's Online Now
1 members (Reznor007), 22 guests, and 2 robots.
Key: Admin, Global Mod, Mod
ShoutChat
Comment Guidelines: Do post respectful and insightful comments. Don't flame, hate, spam.
Forum Statistics
Forums9
Topics9,102
Posts119,263
Members5,019
Most Online890
Jan 17th, 2020
Our Sponsor
These forums are sponsored by Superior Solitaire, an ad-free card game collection for macOS and iOS. Download it today!

Superior Solitaire
Forum hosted by www.retrogamesformac.com