Home Page
Posted By: FitzRoy Internal Database and Headerless Rom Support - 05/25/07 03:14 AM
Im finally annoyed with headers to the point that Im bothering to register and suggest that Nestopia, along with all other active NES emulators, implement some kind of internal database if a universal one isnt possible. Headers have long been required for NES emulation and I just dont understand why any user should be burdened by something that could be made completely transparent to them.

The information present in a header could easily be moved to a single database file cataloguing all games that actually had PCBs and are unmodified. Once there, it would be far easier and more effective to revise/update than to require each user to change each file every time a new format is decided upon or a new GoodTools is released just so that they can get the best compatibility. GoodTools, of course, is closed source, infrequently released, and the developer is prone to errors because he catalogues infinitely created material like hacks and homebrews.

Header supporters are quick to point out that headers contain information vital to the emulation of a rom, but headers represent no part of rom data. Header formats are created by third parties, and despite containing information integral to the emulation of the rom, the manner in which this information is arranged is always going to be subjective. Putting it on the rom itself to accomplish some kind of feeling of greater accuracy because of its closeness to the rom data is just a load of hooey. Its not a part of the rom and its still a subjective format prone to endless revisionism and format wars. Simply put, this information is better collected and referenced outside of the rom data in a database file. If the emulator could scan a headerless roms checksums and find its attributes within an internal database, then people would only have to worry about having headers on modified files or homebrews and other infinitely creatable material.

If youre deterred by the time it would take to catalogue such a database, Im sure either I or someone else could help you build it so long as we are provided with an easily modifiable entry format.
NEStopia *does* have an internal database smile (Well, actually it's in a separate file, but you know what I mean).
Really? Is it at the level that I'm referring to? I try to load headerless roms and it gives an "invalid file" error.
it still need a header starting with NES and 0x16 long to load the game, but it can be empty and nestopia should be able to recognize the image
Why does there have to be one at all? How do I go about adding empty headers to all of my games? Do you see what I'm getting at with my original post. First I have to remove them, now I have to add a ghost header? Their purpose is to exist?
No, I agree, ROMs should not have headers at all. (I still think they should be separate CHR/PRG images in a zip container, but even getting rid of iNES headers would at least get us into the 21st century).
I second this idea, but I was pointing out that still nestopia does not do that smile
This header format stuff has been endlessly debated at NesDev. I think it's more important that the iNES 2.0 format gets off the ground first. Although it may not be the perfection you're looking for, it's a good intermediary solution for the time being.
I agree to an extent, but at the same time if you're going to force everyone to change all their ROMs it *should* be to an ideal solution.
Originally Posted By xamenus
This header format stuff has been endlessly debated at NesDev. I think it's more important that the iNES 2.0 format gets off the ground first. Although it may not be the perfection you're looking for, it's a good intermediary solution for the time being.


I'm not convinced of this at all. Header dependency has to end at some point and I think it's a good idea to get started. Another revision to the format isn't going to solve their inherent problems. I realize there's a concerted effort behind the iNES format and its someone's pet, but honestly, that shouldn't be getting in the way of doing what makes more sense.
Originally Posted By FitzRoy
I'm not convinced of this at all. Header dependency has to end at some point and I think it's a good idea to get started. Another revision to the format isn't going to solve their inherent problems. I realize there's a concerted effort behind the iNES format and its someone's pet, but honestly, that shouldn't be getting in the way of doing what makes more sense.


The simple fact of the matter is that people who make proprietary ways of classifying ROMs are doing so for no reason other than to have a personal fap-fest over the fact that they're getting someone to do things their way. It makes them feel important. Case in point: NSRT. I agree that removing headers from ROMs altogether is the best way to go, but good luck getting large numbers of people to buy into that.
Posted By: hap Re: Internal Database and Headerless Rom Support - 05/26/07 07:41 AM
A universal database would have been a good suggestion back in the late 90s. 10 years later the world is used to iNES (and some UNIF) headers, it's nigh on impossible to force a change now, except for introducing iNES 2.0 that's backward compatible.

Quote:
The simple fact of the matter is that people who make proprietary ways of classifying ROMs are doing so for no reason other than to have a personal fap-fest over the fact that they're getting someone to do things their way. It makes them feel important.

The initial iNES header format is Marat's pet, he's semi-responsible for the big mess by fapping not updating the format. UNIF (and now iNES 2.0) was created to clean it up, not for self-importance.
The world isn't used to iNES headers, they're used to having games "just work" in emulators[1]. You can easily get there now with a headerless format. It'll be just as hard to switch everything to iNES 2.0 or UNIF as it would be to switch to headerless, so again, why not make it the final switch instead of yet another waypoint? I don't know and I don't care who the person that's pushing iNES 2.0 is, but it's a dumb idea, and arguably even a step backwards from UNIF.

[1] I support this by noting that SNES, Genesis, N64, and GBA games (among others) have largely always been headerless and nobody cares. Headered ROMs are actually something of an anachronism.
Posted By: hap Re: Internal Database and Headerless Rom Support - 05/26/07 08:38 AM
End-users are not the world. For example introducing headerless files would make every NES emulator and related programs stop working, and every IPS translation/hack incompatible. I'm afraid Cowering (unfortunately he's the main console ROMs distributor) would not support this.
Every living NES emulator could have an appropriate update out before the new ROMs started being distributed (and if end users are not the world, who cares anyway?). And Cowering is the problem, not the solution. This is 2007. The concept of a single source for ROMs is so stupid in light of things like BitTorrent I can't begin to imagine it.
[stupid question]
Wouldn't there be a way to strip the headers out of ROMs?

Of course - you just chop the first 512 or whatever bytes off the file.
So why'd there be a problem with changing the ROMS? MAME Changes them all the time and people adapt, I mean other than PokeROM collector's, and even in their case, couldn't there be a simple batch program to "update" the ROMs?

Heck, I have less than 40 megs of Nintendo ROMs, I'd probably be able to download that in one night on dialup if I had to, but I'm sure a batch program would do just as well, right?

MAME makes it look easy because all the decisions are generally made strictly on technical merits. Most of the console "scene" is still too immature to be there yet. The NSRT swindle Moogly mentions is a prime example: SNES ROMs have no header so they want to add one and then charge emu authors to decode it. It's been fabulously successful thus far, as you might imagine smile
Originally Posted By R. Belmont
Every living NES emulator could have an appropriate update out before the new ROMs started being distributed (and if end users are not the world, who cares anyway?). And Cowering is the problem, not the solution. This is 2007. The concept of a single source for ROMs is so stupid in light of things like BitTorrent I can't begin to imagine it.


Authors are probably still leery of old versions, older emulators, etc. that are permanently dependent on them now that headers have been allowed to exist for so long.

The way that the internal database should work is this: there should be an option at first for "use internal database." Upon being checked, it would use the internal database for headerless roms and headered ones (ignoring the header), essentially adding support for rom-only files. This would give advanced users and DAT makers exactly what they want without disturbing GoodTools/iNES dependent newbs. With any luck, the option itself might wise people up to their unnecessary dependencies, and headers could gradually be phased out.

Anyway, Marty has probably seen the thread already, so it's not looking good.
While this is an important subject and people are strongly opinionated around here, my personal main objective as an emulator author is foremost to know what the hell I'm supposed to emulate. A CNROM cart? A TKSROM cart? Or maybe a bootlegged multi-cart with "happy new year" labeled on the board? Obviously, NES cart information is absolutely necessary and where it should go is what's being debated here.

A database taking care of pure ROM files certainly has its uses. It keeps the implementation details away from the user and places the burden on someone else, i.e the db maintainers and emu authors. If something needs to be changed or updated, you do it once and in one place only. The one I personally maintain has surely been handy at times, especially as I can use it to store additional information about a cartridge which the iNES file doesn't have a clue about. Things like exact board used (SAROM/SJROM/SKROM/SXROM), chip revision (MMC1A/MMC1B2) and other things that helps bring up emulation accuracy.

However, as convenient for the user it may be, it's a hefty amount of work and not the most fun thing an emulator author can do - constantly having to keep track of new dumps and whatnot. The reason the database was put to use in the first place was so I could get around the limited design of the iNES file format and deal with some of the common widespread broken ROMs. I never really intended to have the database work as a full-fledged replacement for headered ROMs.

That being said, if an image shouldn't contain anything but pure ROM data it might as well be split into separate files so that PRG-ROM and CHR-ROM data can be distinguished. The full database proposal brings up a few interesting questions. Who will maintain it? What should go in there? Anything under the sun, including homebrews and hacks, or just games that was commercially released when the console was still being actively sold? Anything not in the database would be useless to an emulator since the emulator wouldn't know how to run it. Is that really ok?
Assuming the database thing went forward, here's how I'd answer:

As far as maintainers, the database itself should be GPL or a similar F/OSS license and maintained as a separate project.

What should go in there? I would suggest having separate databases for e.g. Japan official, US official, Europe official, Chinese hacks, homebrew, and so on. Bad dumps should never be added - that's my major beef with the Good* sets.

Is that OK? I'm guessing that anyone still creating NES homebrew is probably closely associated with NESdev and will be able to communicate their needs appropriately.

That all said, I prefer a more distributed approach.

You'd have separate PRG-ROM and CHR-ROM files in a zip (ideally separate per physical chip, but not strictly necessary) and a config file (ini or xml format) which would give the name of the board, the mapper chip details, and so on. IMO that would combine a lot of the advantages of the database and headered approaches.

- It's pretty easy for emulator authors, especially if the board descriptions are compatible with UNIF's
- Anyone can be the maintainer. Someone could pretty easily throw together a utility that used NEStopia's database file to split up current iNES ROMs into the "new" format
- Homebrew authors could easily write their own config file
If I might, I'd just like to throw my two cents on the whole matter.

As stated earlier in this thread, almost all gamers don't really care about the format used for their games, so long as it runs. For now, iNES "works" and they are probably completely unaware of what iNES even is, much less why it causes some trouble with emulation authors, NES developers, and other such people.

If a new format is to be made for better accuracy and technical merit, it should probably get supported in the popular and actively maintained emulators first. To cause as little confusion as possible, this should be done before split ROMs get widely distributed. Some old tools/emulators might not ever support the new format, and when they're needed, there probably would be a relatively simple way of converting to the old iNES format.

Personally, there shouldn't be any intermediate "phase" change by going headerless first. If the goal is to have an accurate way for preserving NES cartridges, there shouldn't be any need for this step. (Especially when the intermediate step might be worse than the final goal, if the new format has an information file containing various information.)
i would go for the .ini/.xml version too

not to mention that, if at a certain point we find out new properties of a board revision (if ever), we should simply spread out a new .xml file (hardly to be considered a copyright infringiment adn clrmame + a dat file could put it in the right place infew seconds) without touching the .prg / .chr
Originally Posted By R. Belmont
No, I agree, ROMs should not have headers at all. (I still think they should be separate CHR/PRG images in a zip container, but even getting rid of iNES headers would at least get us into the 21st century).
I don't know if that was sarcasm, but I totally support this idea.

I think NES roms sets should be like MAME roms sets.
And the emulator (Nestopia or MESS) should provide a driver.
Then we would no longer need headers.
I am sure the No-Intro Project would support those split sets as soon as Nestopia and/or MESS implement such drivers so we can finally get rid of those headers. We would take care of the dump tracking and database part to ease the emulator authors burden.

http://forums.no.intro.free.fr/showthread.php?t=1418
a database of dumps already exists (look for bootgod)

EDIT: moreover, taking a look at the no-intro thread [1], it seems someone do not understand that you cannot simply remove headers and run images as they are: NES carts have very different layouts, with different connections, and different ways to be read by the console.

the emulator have to know which kind of emulation apply to the image. it's like emulating different arcade PCB, no more no less.

therefore, you have to put these informations somewhere: iNES images put those in the header, we should find another way (either in an external database, or in an additional file inside the set)

in both approaches, it would be anyway kind to let homebrew programs & tests a way in and not be too strictly focused on commercial games: tests often stress edge behaviour of the hardware and help to debug emulation issues. it would be stupid to simply throw them away from the emulation world

[1] I would post there but I had some problems with my password and I still cannot login...
Originally Posted By etabeta78
a database of dumps already exists


I thought they were searching serious mainteners wink
And please take no offence but the No-Intro project may be the best choice, if compared to the poor competitors.
Originally Posted By MESSfan
I thought they were searching serious mainteners wink
And please take no offence but the No-Intro project may be the best choice, if compared to the poor competitors.


don't be offended, but I trust much more that single guy which has verified by itself each cart (dumping various carts just to confirm the validity of the dump), than you guys at nointro.

NES is a more serious problem than GB or FDS: you really have to open the carts and put your hands on the chips, so I would trust someone only after he gave me some technical proof (or if he's guru)

and also TOSEC is able to spread out dats if needed (being myself the maintainer of the NES part wink )
Take a closer look at what we are doing. wink
TOSEC is a huge joke. laugh
I think "huge joke" is a bit unfair, but their standards for CD dumping are unintentionally hilarious for sure.
As long as the database would be still compatible with headered images, I'd be willing to support it.

Although I like some of the ideas here, I personally still want to give iNES 2.0 a chance. iNES 2.0 has the advantage of being able to work on older emulators, while the database idea does not.

Plus, having headers will give the emulator something to fall back on. If info for a particular ROM in the database is missing or incorrect, it would have the header to rely on. But if the ROM was headerless, the ROM wouldn't run.

So basically, I support a compromise between the two.
Older emulators can use headered ROMs. If there's really a demand, they'll be kept available (just as you can still get MAME 0.36final sets). I see no need to retard current ones just so morons can keep using NESticle though.
Originally Posted By MESSfan


I love this world because is large enough for both the approaches.

You can continue with your plan to rule the world of emulation, and I can continue to trust bootgod and the pages he puts up showing how he opened ten different copies of the same cart just to check the layout and the contents of the chips.

You can continue your holy war against homebrew and test roms and I can continue to rename them for TOSEC (even if for test rom is a shame not to catalogue sources and readmes... I will find a solution...)

Of course efforts by other guys are welcome (and bragalad and polarz and others already contributed to bootgod database), but I continue to trust single person more than "projects" (for the same reason I would criticize single dats from TOSEC rather than the project as a whole, single changes to MAME rather than its value as a whole etc.)

anyway, I'm leaving for a week, so keep up the discussion: I'll try to check the thread when I can smile
Originally Posted By xamenus

Plus, having headers will give the emulator something to fall back on. If info for a particular ROM in the database is missing or incorrect, it would have the header to rely on. But if the ROM was headerless, the ROM wouldn't run.


If a rom didn't run because of an entry error or because it was too new, it would probably get noticed and fixed fairly quickly. Such a fix wouldn't require any action on the part of the user other than staying up to date with the emulator. Header errors are less practical for the typical user to fix.

As for what should be included in the database, I think that the emulator db should stick only to licensed material which was commercially available. Third party dbs could extend this by providing a db with betas, test carts, and pirates. Hacks/translations are infinitely creatable and are modifications of already databased material. Is there a patch format so that the patch stays separate of the rom? That would solve some things... like, you want to play a translation, you name the patch the same as the other roms, put it in the patch directory, and turn on an option, and nestopia detects it and soft-patches the original rom(s)? Nobody actually needs the thousand separate mario hacks just because GoodTools collects them. I think the soft-patch functionality would be sufficient for actual users, even if it means manually renaming and moving some things to get it working (as described in patch readme).
Are 16 bytes really that scary? Is the desire to force headerless ROM images on the world just driven by a few individuals' concept of what's "right", regardless of pragmatism?

Also, I think identifying file types by file extension only is...dirty, but maybe that's the same type of idealism the anti-header people express. wink
Posted By: hap Re: Internal Database and Headerless Rom Support - 05/27/07 06:41 PM
Originally Posted By R. Belmont
(...) As far as maintainers, the database itself should be GPL or a similar F/OSS license and maintained as a separate project.
definitely

Quote:
That all said, I prefer a more distributed approach.

You'd have separate PRG-ROM and CHR-ROM files in a zip (ideally separate per physical chip, but not strictly necessary) and a config file (ini or xml format) which would give the name of the board, the mapper chip details, and so on. IMO that would combine a lot of the advantages of the database and headered approaches.
That's basically the same as what we have now: decentralized metadata. I mean, there's no difference between this and the UNIF format with its sections (info, PRG, CHR) in separate files.
Mednafen: depends on your view of the purpose of emulation. There's nothing wrong with the status quo if your only desire for it is to play free games. If you actually want your grandchildren to be able to see and understand NES games, then we need a better archival format. (Similarly, it's important that the GNU Radio guys get NTSC decoding working so there's some way to show the output of real hardware then). The fact that emulators have to second-guess the header info currently to achieve max compatibility is just silly.

hap: I personally like the UNIF format, and I'm OK with it even for archival purposes, but it doesn't seem to have taken off despite fairly wide support in emulators. I'm not sure if it's just inertia, or that nobody knows about NEStopia yet, or that nobody powerful on NESdev is pushing it the way they are iNES 2.0.
Originally Posted By hap
That's basically the same as what we have now: decentralized metadata. I mean, there's no difference between this and the UNIF format with its sections (info, PRG, CHR) in separate files.


There is a difference. In UNIF, you're still dealing with a header-like format that works on a per-ROM basis separate of the emulator. Once again, to effectuate changes is far less practical than a single database file included with the emulator. I think that UNIF could serve a nice secondary purpose in the presence of a database because it could become a requirement for infinitely creatable material not in the database (hacks, homebrews, etc). If Marty and other emulator authors got together and defined a database format that they could all use and that users could edit, we will have simplified emulation and removed a dependency and complexity that really doesn't serve much of a current purpose other than to annoy verifiers and make people dependent on GoodTools.

Originally Posted By mednafen
Are 16 bytes really that scary? Is the desire to force headerless ROM images on the world just driven by a few individuals' concept of what's "right", regardless of pragmatism?


What's most interesting about this statement is that you make it appear as though the world is against removing headers while a few nutjobs are supporting it. In reality, the vast majority of users just want their games to work, a small group of people support (retaining) headers and a seemingly larger group of people support getting rid of them to some extent.

I'm one of those people that believes that emulation is "not" just about the masses who want nothing more than for their games to work. The people who have a passion for the systems and take the time to learn about how they work, write emulators, or dump games and record their checksums, are probably mostly in agreement with a change in the way roms are stored and emulated. In that regard, yes, I think it's scary that complacence could affect the outcome of this.

EDIT: Somehow, I forgot a "not" which really helps with what I was meaning to say...
Posted By: hap Re: Internal Database and Headerless Rom Support - 05/28/07 05:06 PM
Have you considered the fact that some game revisions are equal when it comes to ROM contents, but have a different hardware configuration? It's impossible to include that in a universal database using ROM checksums as private key.

Quote:
There is a difference. (...)
No, Belmont suggested a "per-ROM" solution, but stored in a much cleaner way than say, iNES. I think I have to agree with him that iNES 2.0 is not a good solution for accurate archival purposes (even though it would make every game playable).
prg+chr+xml would probably be the way to go for me - you could put that easily in a container format.
The XML would work as description how to access the game: Storing information that would otherwise to be guessed:
board name, mapper info, wram, prg size, chr size, developer, publisher, release... well any information you could probably want in there. That is up to those who actually have the info smile

Of course this information has to come from a trusted source. If everyone could mash half-way working info together, it is nothing better than iNES - just more "modern".
BootGods webpage is a very good example how things can work - every information is backed up by pcb scans. Its only drawback is that the information is not in an "easy to use" format for emulators.

In the current no-intro set there are like 150 unrecognized games. But without the actual pcb info there is no way to tell which mapper they should have. (In the past it was trial and error - try all mapper numbers until one works and use that)

Btw - what speaks against a MAME-like approach with verifying the roms? Nestopia does already warn you if you have a bad dump...
Verifying ROMs would be the best way to go. Last time I checked Nintendo don't release games for their original Entertainment System, so there is a limit on the amount of unique games ("clones" included), excluding fan-made homebrew games.

That said, would there be clones of NES games like in MAME or a simpler CHR+PRG regardless of identical images? There is a possibility of a game having a different program revision but the same CHR ROM. Not that zipped games are larger than around 300K!
Originally Posted By hap
Have you considered the fact that some game revisions are equal when it comes to ROM contents, but have a different hardware configuration? It's impossible to include that in a universal database using ROM checksums as private key.


If there were, you could probably just have a window pop-up with a selection upon detection of this.

Originally Posted By hap
Have you considered the fact that some game revisions are equal when it comes to ROM contents, but have a different hardware configuration?


Can you name any examples for this?
Posted By: hap Re: Internal Database and Headerless Rom Support - 05/29/07 05:14 PM
IIRC, I read about a Turtles game with 2 different hardware revisions. I can't find the source where I read that.

I've invited the other NesDev guys to join the discussion, they decided to discuss over there instead (thus as a whole this discussion is getting kind of messy): http://nesdev.parodius.com/bbs/viewtopic.php?t=3387&postdays=0&postorder=asc&start=0
Their objections are just weird, although tepples did a nice job rebutting them. And some of them point to exactly why a zip container is the right thing: if you want to include manual and label scans, external sample ROMs, strategy guides, cheat codes, or whatever you can just toss 'em in the zip and it won't hurt anything. You could even include patch files for translations/cheats/etc and have the emulator prompt which ones to apply.

The one thing I agree with is that it would be hard to redump everything, but I think we know enough about a majority of already dumped games to create at least provisional "znes" (I personally like ".nez" for the zip format) versions of most games (additional info could be added as needed). Certainly bootgod's database would be useful here.
Just to weigh in, as an electrical engineer, I think the best format is a zip container with one file per ROM (flat binary, Intel hex or something like that) and programmable logic device (JEDEC would be my format of choice). The cartridge configuration can be described with a textual netlist - this is intelligible by humans (it's just a list of which pins are connected together), and easy to parse in software. On top of this, you can put in PCB scans, user guide PDFs and whatever else.
I'm certainly no expert, but I myself would prefer any format that would archive the cartridges as accurately as possible. Say the new format contained information on how the chips are layed on the PCB and made it possible to emulate -- even if it were too slow for common computers (the NesDev forum claims that, but it might as well be FUD in favor of iNES), it could be optionally loaded in an emulator (they don't seem to have much trouble without such information currently).

I work at a museum on weekends, and I understand how important it is to properly preserve items. There is no way we would replace those old Vinyl 78s with some MP3s, it's just not proper and loses all value in studying the old format. One day, say in a few decades, real working NES/Famicoms might be nearly-non-existent, and if all we had were files in the clunky iNES format, nobody would learn anything. You can't recreate a cartridge based on an iNES container, we need a format that would allow you to do so. I really couldn't care less if it makes all hacks incompatible with the new format--that's inevitable (though NINJA is supposedly format-neutral (eg, a single patch works for both iNES and UNIF formats)). Hell, if a cartridge is stored as accurately as possible, it would be perfectly feasible to create a program that can convert it into the lesser formats if the need arises (like how you can convert FLAC to MP3 without re-ripping a CD).

Back in the day, those old vinyls were thought to be a convenient way of playing back your favorite music. Very few people thought it was necessary to preserve them, lucky for us many of those vinyls have been preserved, and we even have the technology today to use them without damaging them (laser turntables). Be it music or video games, it represents a part of the culture that cannot be ignored, historians and game-enthusiasts alike need to have a way of studying the era. We cannot afford to be careless -- it is our culture we can either throw away or preserve for ages to come.
Especially with older systems like the NES - ten years ago people couldn't care less about the 'normal' Nintendo as there was the Super Nintendo, Mega Drive (Genesis), Saturn, PlayStation, Nintendo 64 etc, all delivering much better graphics. However there was quite a few games which never made it past the 8-bit stage, which people do miss.

My opinion is the ZIP file is the virtual cart or PCB where the chips live as such. I'd leave documentation in a separate file - the game's instruction book was never hidden inside the cartridge, even though there is probably enough space inside it!

MAME doesn't need the extra information like artwork or user's manuals in the ROM container, it's optional (particularly for bezels, marquees etc).

Originally Posted By "Mike S."
One day, say in a few decades, real working NES/Famicoms might be nearly-non-existent


NES consoles don't break, they're not called a pee-ess-two. wink
People are seriously claiming that a zip container would be too much for low-end computers? Zip was invented on 8088s when half of NESdev wasn't even a sperm yet. Gimme a break.

Besides, any half-accurate NES emulator's going to have requirements dwarfing those of an unzip implementation.
Originally Posted By Heihachi_73
Especially with older systems like the NES - ten years ago people couldn't care less about the 'normal' Nintendo as there was the Super Nintendo, Mega Drive (Genesis), Saturn, PlayStation, Nintendo 64 etc, all delivering much better graphics. However there was quite a few games which never made it past the 8-bit stage, which people do miss.

Not only that, but many game series' had their roots on the NES (Final Fantasy, Super Mario Bros., The Legend of Zelda, Metroid, etc). Many people love to look back on how it all started, how far they have come. It's simply unacceptable to say "we have better consoles, destroy all the old ones!"
You know. We also stopped using a.out about 8 years ago too smile. We have ELF now. God forbid file formats upgrade and improve over time.
Originally Posted By Heihachi_73

NES consoles don't break, they're not called a pee-ess-two. wink


I had 2 NES systems break, the cartridge reader part broke.

Hmmm, my PS2 is 2 years old, I'll bet it lasts longer than many XBOX360s!
Regarding the central database idea, I have two (minor) objections. It would make an emulator more an "accessory to infringement", especially if the database is just commercial games. I'm also unclear on the motivation for dividing information for a given cartridge into two pieces that are stored separately. The main benefit I can imagine is that it allows easy repeated updating of the description of many cartridges at once, where storing the information individually in the cartridge files would require a batch of patches and application to each one every time. But why must this information be updated so often? Is it not possible to take a particular cartridge, dump it and examine the circuit board in full, then input a complete, correct description of it (in whatever format you want)? Or perhaps we want to be able to keep revising the database information, for example start out with just mapper names, then later add full hardware descriptions or something?

Quote:
You can't recreate a cartridge based on an iNES container

Even though an iNES file lacks scans of the circuit board, it still has sufficient information to make a cartridge. For example, the mapper number is a reference to a mapper description that's elsewhere (in an emulator's case, it's in the source code). Any description of a cartridge will have references to external material, no matter how detailed. There's no reason to describe the common aspects separately for each cartridge.
Well, yes, but iNES requires more reference to external material than most (is that mapper #119 varient #62 or #63?). And currently you have to get that material from the Anointed Priests of NESdev. That's really my goal with console emulation: make it so adults are in charge and nobody's lording anything over anyone else. Ville and Moogly did a wonderful job emptying an Uzi into the tires of N64 eliteness (with an unexpected assist from Ziggy), now it's other consoles' turns ;-)
It would be hard and expensive to redump every rom, so for beginning the database would have to be filled with preliminary information taken from iNES headers. (I doubt all the betas and prototypes can be redumped, so this information would be unconfirmed).

A simple csv database would be easy to make, I guess.
Originally Posted By Marty
While this is an important subject and people are strongly opinionated around here, my personal main objective as an emulator author is foremost to know what the hell I'm supposed to emulate. A CNROM cart? A TKSROM cart? Or maybe a bootlegged multi-cart with "happy new year" labeled on the board? Obviously, NES cart information is absolutely necessary and where it should go is what's being debated here.

A database taking care of pure ROM files certainly has its uses. It keeps the implementation details away from the user and places the burden on someone else, i.e the db maintainers and emu authors. If something needs to be changed or updated, you do it once and in one place only. The one I personally maintain has surely been handy at times, especially as I can use it to store additional information about a cartridge which the iNES file doesn't have a clue about. Things like exact board used (SAROM/SJROM/SKROM/SXROM), chip revision (MMC1A/MMC1B2) and other things that helps bring up emulation accuracy.

However, as convenient for the user it may be, it's a hefty amount of work and not the most fun thing an emulator author can do - constantly having to keep track of new dumps and whatnot. The reason the database was put to use in the first place was so I could get around the limited design of the iNES file format and deal with some of the common widespread broken ROMs. I never really intended to have the database work as a full-fledged replacement for headered ROMs.

That being said, if an image shouldn't contain anything but pure ROM data it might as well be split into separate files so that PRG-ROM and CHR-ROM data can be distinguished. The full database proposal brings up a few interesting questions. Who will maintain it? What should go in there? Anything under the sun, including homebrews and hacks, or just games that was commercially released when the console was still being actively sold? Anything not in the database would be useless to an emulator since the emulator wouldn't know how to run it. Is that really ok?


Originally Posted By R. Belmont
Assuming the database thing went forward, here's how I'd answer:

As far as maintainers, the database itself should be GPL or a similar F/OSS license and maintained as a separate project.

What should go in there? I would suggest having separate databases for e.g. Japan official, US official, Europe official, Chinese hacks, homebrew, and so on. Bad dumps should never be added - that's my major beef with the Good* sets.

Is that OK? I'm guessing that anyone still creating NES homebrew is probably closely associated with NESdev and will be able to communicate their needs appropriately.

That all said, I prefer a more distributed approach.

You'd have separate PRG-ROM and CHR-ROM files in a zip (ideally separate per physical chip, but not strictly necessary) and a config file (ini or xml format) which would give the name of the board, the mapper chip details, and so on. IMO that would combine a lot of the advantages of the database and headered approaches.

- It's pretty easy for emulator authors, especially if the board descriptions are compatible with UNIF's
- Anyone can be the maintainer. Someone could pretty easily throw together a utility that used NEStopia's database file to split up current iNES ROMs into the "new" format
- Homebrew authors could easily write their own config file


I find these two comments to be of the most significance in this whole thread. So I'm going to give my opinion and my answer on these.

Some people are showing concern of loss of translation, hack, and home brew support if a new database is devised. This in my opinion is wholly illogical as just because we replace iNES does not mean the emulator must drop legacy support.

Included in each games 'set' I would not only include the split .PRG and .CHR file as well as the .XML file containing specific hardware and other useful information I would include an .NES file. The .NES file would be nothing more then a legacy iNES header and emulators that do not wish to conform to the new database would need only read the archive and combine the .NES .PRG and .CHR files in ram and run the ROM as if it was any common NES ROM found today. I find this to be a very simple and logical solution.

Such a database should only include official commercial games. Translations, Hacks, and home brew should only be supported with the above mentioned legacy iNES support. Should a user require the proper ROM to apply a translation's to a simple 'build iNES' option could be provided. You browse though your ROM list and find the game you need. Right click it or open a drop menu and along with other options such as 'play', 'audit', 'settings' you would find the 'build iNES' option.

the database itself should be treated very much like MAME. The emulator itself could also take a few hints from MAME. Instead of emulating different CPUs as drivers we treat the individual boards or mappers as drivers. I have really always felt the NES has much in common with arcade machine design rather then traditional game console design. Treating it as so seems logical to me.

Anyway these are just my opinions. I am no programmer or anything and my only real past experience with emulation advancement was a lot of work I put into NSRT (witch most people here including myself now hate) before that program fell to the dark side and started with the NSRT header BS and attempts to force the tool on people as well as the inclusion of fake ROMs in the database just to annoy the so called collectors. I used the early development of that program as a means of cleaning up a lot of the mess in SNES ROMs that was common at the time. Fixed a lot of interleave issues and tons of over dumps that have gone undetected for over a decade in some cases. SNES in my opinion is still a mess but it is a bit better. I'm kind of in the mood to see if I can do anything to help with NES now.
Instead of giving the task of converting to the older iNES format to the emulator, why not write a script to convert them? It can't be that hard.

Is the NES really so different from consoles of its era? Once again, I don't know the details, but it'd surprise me if the NES is the only one with various boards and expansions. Even the Super NES had different boards and expansions (did the Genesis? That console had the Sega CD and 32X to add capabilities, but I don't see any reason it can't be built-into the cart). By the 5th generation, Nintendo Power wrote that one reason the Nintendo 64 was better than the PlayStation was that cartridge-based systems could be expanded by extra hardware in the cart itself (though I can't think of any N64 game that actually did this, probably a contributing factor to Nintendo's choice of using DVDs in the next console (gameCube)); while that might also show some lack of foresight in Nintendo's engineering and marketting departments, it does show that such technology was so popular in the 3rd and 4th generations they felt it could last into the 5th. It's hard to believe that the NES had a unique feature. If you compare it to the 6th/7th generation consoles (5th gen too if you think of Saturn and PS1), sure, but you need to be fair and think in the cartridge-era of video games.
Yes, the NES *is* a unique case. No other console[1] had cartridges that 1) contained separate program and graphics ROMs 2) had over 150 unique kinds of cartridges. By contrast, there are maybe a dozen unique kinds of SNES carts, and half a dozen Genesis (Virtua Racing had a DSP to generate the polygons and thus it remains unemulated). By contrast, GBA and NDS carts are, with the exception of the 2 or 3 GBA tilt/rumble carts, differentiated only by save media type (none vs. EEPROM vs. battery SRAM vs. Flash) - the "mapper" is always exactly the same.

[1] The TI99-4/A computer also had "split" cartridges, but they weren't split by code vs. graphics and it wasn't a console.
© Forums