Home Page
Posted By: R. Belmont Fixed software lists - 01/05/10 04:28 AM
Sorry, I goofed up and killed the earlier version of this thread after a spam one appeared. I think I can summarize the earlier one as:

- Someone noted that the .HSI files fit well into how MESS already works and can work with the wiki for compiling the lists
- Haze will only accept fully hardcoded data inside the executable like MAME (currently) works
- Everyone agrees current console dump formats suck, but good luck changing them
Posted By: Guru Re: Fixed software lists - 01/05/10 06:02 AM
urgghh. Do you know how long it took to type up that first post?
Please ban yourself then go sit in the corner facing the wall. Thanks laugh
Posted By: Justin Re: Fixed software lists - 01/05/10 07:26 AM
Browser cache to the rescue
Posted By: judge Re: Fixed software lists - 01/05/10 08:11 AM
Please see mail to mailing list for a possible solution (also usable for cartridge based systems in MAME).
Posted By: Guru Re: Fixed software lists - 01/05/10 08:51 AM
I see. I see smile
OK RB, you can unban yourself now, the software lists are in. well, something is happening anyway. Thanks judge!

Oh and you left one out....

- Guru will only accept fully hardcoded data inside the executable like MAME (currently) works

smile
Posted By: Curt Coder Re: Fixed software lists - 01/05/10 11:48 AM
Yet Another Dump list.

- Good*
- No-Intro
- TOSEC

Isn't 3 competing sources enough already?
Posted By: JoJo Re: Fixed software lists - 01/05/10 12:01 PM
Originally Posted By Curt Coder
Yet Another Dump list.

- Good*
- No-Intro
- TOSEC

Isn't 3 competing sources enough already?


I guess these submits are here to show how the code could work. Nothing (except emu-politics wink ) should prevent from using an already established source once the framework has stabilized.
Posted By: judge Re: Fixed software lists - 01/05/10 12:30 PM
We wouldn't be competing, ideally we would only add known good dumps documenting the software released for a system. Anything not in the list can of course still be loaded through the already existing routines.
Posted By: Curt Coder Re: Fixed software lists - 01/05/10 01:31 PM
Originally Posted By judge
We wouldn't be competing, ideally we would only add known good dumps documenting the software released for a system. Anything not in the list can of course still be loaded through the already existing routines.


Known good according to who? NoIntro or Goodxxx? Or some other party?
Posted By: judge Re: Fixed software lists - 01/05/10 01:56 PM
That may differ per system and per media.

For the epoch game pocket computer we got direct rom dumps from the chips on the cartridges, so we know those are good.

The lists will require time and effort to create and expand them. A lot of information is already out there and it makes sense to re-use that information.
Posted By: Haze Re: Fixed software lists - 01/05/10 01:56 PM
Originally Posted By Curt Coder
Yet Another Dump list.

- Good*
- No-Intro
- TOSEC

Isn't 3 competing sources enough already?


Well, prior to MAME the arcade scene was just as fragmented, slowly but surely everybody start supporting MAME sets, and now nobody looks back.

Due to the fact it's also an emulator MESS could easily become de-facto reference just as MAME is now.

If you know your history you'll see what an enormous help MAME doing things 'the right way(tm)' has been to the arcade emulation scene. It's a long hard road (mainly because it's been left so late) but it would be kinda nice to have all the official Sega part numbers documented etc. rather than just Sonic 2 [UE] [!] or whatever. Having a proper database would provide a mechanism for that. (of course, non-licensed games often don't have proper labels, but it's the same in MAME)

Really, it's about time the console scene matured, and stopped supporting things the way they do because it's 'easier' With that attitude MAME would still be using MGD2 and NeorageX dumps.

I know it's not a popular opinion, but I've never tried to be popular, just ensured things get done correctly.




Posted By: Guru Re: Fixed software lists - 01/05/10 02:17 PM
Originally Posted By judge
We wouldn't be competing, ideally we would only add known good dumps documenting the software released for a system. Anything not in the list can of course still be loaded through the already existing routines.


there's probably a lot of repetition between the lists because no one cooperates. once the lists are in MESS there is no competition, everyone bows down and submits to the MESS team. Well it works for MAME anyway wink
Posted By: ElBarto Re: Fixed software lists - 01/05/10 03:10 PM
Originally Posted By Haze

It's a long hard road (mainly because it's been left so late) but it would be kinda nice to have all the official Sega part numbers documented etc. rather than just Sonic 2 [UE] [!] or whatever. Having a proper database would provide a mechanism for that. (of course, non-licensed games often don't have proper labels, but it's the same in MAME)


I've started this some time ago, I still have ~300 scans to add.
The wiki is script generated so I can do the same for the software list.
Posted By: ht1848 Re: Fixed software lists - 01/05/10 03:16 PM
Hey this is great. Since I have been following MESS this has always been one of those wish list items for me. I know there is some complicated things to work out and I am sure the team will figure it out.

It is all about continuous improvement, set a baseline and keep tweaking. Nothing has to be perfect from the beginning.

I have a few questions but I want to study what was submitted before I foot-mouth myself.

Great work!
Posted By: Haze Re: Fixed software lists - 01/05/10 03:42 PM
Originally Posted By Haze
Originally Posted By Curt Coder
Yet Another Dump list.

- Good*
- No-Intro
- TOSEC

Isn't 3 competing sources enough already?


Well, prior to MAME the arcade scene was just as fragmented, slowly but surely everybody start supporting MAME sets, and now nobody looks back.

Due to the fact it's also an emulator MESS could easily become de-facto reference just as MAME is now.

If you know your history you'll see what an enormous help MAME doing things 'the right way(tm)' has been to the arcade emulation scene. It's a long hard road (mainly because it's been left so late) but it would be kinda nice to have all the official Sega part numbers documented etc. rather than just Sonic 2 [UE] [!] or whatever. Having a proper database would provide a mechanism for that. (of course, non-licensed games often don't have proper labels, but it's the same in MAME)

Really, it's about time the console scene matured, and stopped supporting things the way they do because it's 'easier' With that attitude MAME would still be using MGD2 and NeorageX dumps.

I know it's not a popular opinion, but I've never tried to be popular, just ensured things get done correctly.


The other big advantage of the MAME database is of course that we consider the information in MAME to be free, you can use it how you want, you can create awesome sites like MAWS, you can incorporate it in your emulator (infact, it's far better that way) MAME doesn't try to claim ownership over factual information like some of the other databases do, it just presents it as free-to-use, as-accurate-as-possible data and provides handy means of exporting it. Many of the proprietory rom managers seem to somehow think they own filenames ;-)
Posted By: Guru Re: Fixed software lists - 01/05/10 03:58 PM
Originally Posted By ElBarto

I've started this some time ago, I still have ~300 scans to add.
The wiki is script generated so I can do the same for the software list.


what's that byteswapped column about? if you dump the ROM using an EPROM reader then that's it, no byte swapping is required. The emulator should load it as-is. If you've 'fixed' them to work with some other emulator then you should unfix it. That's one of the reasons the console skene is in such a mess.
that list is farking huge. how many have you redumped? just those listed or is there more to come? that's going to be a very expensive exercise if you don't own them all frown

the list in MESS can only work with known CRCs etc otherwise there's nothing to 'check'. it will be possible to load anything but the file names will be generic instead of MPR-xxxxx etc.
btw, the file names should be MPR-xxxxx.IC1

Posted By: Guru Re: Fixed software lists - 01/05/10 04:01 PM
Originally Posted By Haze
Many of the proprietory rom managers seem to somehow think they own filenames ;-)


not sure which ROM managers you mean but the idea with hard coded lists is to get native .exe support in clrmame and then the fanboys will create torrents for the software and that will be the end of looking for stuff smile
Posted By: incog Re: Fixed software lists - 01/05/10 04:03 PM
Originally Posted By ElBarto

I've started this some time ago, I still have ~300 scans to add.
The wiki is script generated so I can do the same for the software list.


Reminds me a lot of bootgod's NES pages, good going smile
Posted By: ElBarto Re: Fixed software lists - 01/05/10 04:19 PM
Originally Posted By Guru

what's that byteswapped column about? if you dump the ROM using an EPROM reader then that's it, no byte swapping is required. The emulator should load it as-is. If you've 'fixed' them to work with some other emulator then you should unfix it. That's one of the reasons the console skene is in such a mess.


byteswapped column means that the checksums are calculated with a byteswapped rom (like a direct eeprom dump).
When I say byteswapped it's according to current dump.

Originally Posted By Guru

that list is farking huge. how many have you redumped? just those listed or is there more to come? that's going to be a very expensive exercise if you don't own them all frown


More than 300 but I only add them when I scan the pcb/box etc ...
So far I've only added 12 but more than 40 are ready to be added.

Originally Posted By Guru

the list in MESS can only work with known CRCs etc otherwise there's nothing to 'check'. it will be possible to load anything but the file names will be generic instead of MPR-xxxxx etc.
btw, the file names should be MPR-xxxxx.IC1



Yeah I should change the filename to chip.ic#
Posted By: Guru Re: Fixed software lists - 01/05/10 04:21 PM
Originally Posted By incog
Reminds me a lot of bootgod's NES pages, good going smile

it looks like it's still active.
http://bootgod.dyndns.org:7777/
looks pretty damn good. the individual ROM CRCs and ROM labels are listed too. perhaps we should base the NES software list from that site.
can someone contact him and ask if he'll let us use the info. or better maybe, if he has a text list of the game title/rom name/crc etc info that we need for the MESS list.
Posted By: ranger_lennier Re: Fixed software lists - 01/05/10 07:14 PM
More info on Game Pocket Computer releases--

The Mahjong box lists it as game 3 and says copyright 1984. The cart says copyright 1984.

The Reversi box lists it as game 4 and says copyright 1984. The cart says copyright 1984.

Most of my Game Pocket Computer stuff is with Guru, so he could probably provide more info.
Posted By: judge Re: Fixed software lists - 01/05/10 07:21 PM
Thank you.

Maybe it makes sense to add some string field for software/serial/release number...
Posted By: Stiletto Re: Fixed software lists - 01/05/10 07:33 PM
there's a SNES page like these too, somewhere.
Posted By: judge Re: Fixed software lists - 01/05/10 08:08 PM
Here is another example for the BBC Bridge Companion cartridges: http://git.redump.net/cgit.cgi/mess/tree/src/mess/software/bbcbc_cart.c

Posted By: Guru Re: Fixed software lists - 01/06/10 05:12 AM
Originally Posted By judge
Maybe it makes sense to add some string field for software/serial/release number...


MAME usually just adds a note in the src on the GAME line. we don't want to go overboard with tons of info that not everyone cares about. just list that kind of thing in the src.
Posted By: Guru Re: Fixed software lists - 01/06/10 05:15 AM
Originally Posted By ranger_lennier
Most of my Game Pocket Computer stuff is with Guru, so he could probably provide more info.


more info? like what? like the PCB is green and the ROM is brown and it has 28 little shiny silver legs. come on, its a cart with a single rom in it. the info we have is sufficient and there isn't any more info anyway.
well, no more than this anyway....
Block Maze, (C)1984, game 1
Astro Bomber, (C)1984, game 2
that's about it.
that would make Soukoban game 5
Posted By: Justin Re: Fixed software lists - 01/06/10 05:27 AM
Yeah, I'm definitely a believer in keeping it simple and letting some other kind of wiki or database handle all the minutiae and scans etc.
Posted By: judge Re: Fixed software lists - 01/06/10 04:13 PM
Originally Posted By Guru
well, no more than this anyway....
Block Maze, (C)1984, game 1
Astro Bomber, (C)1984, game 2
that's about it.
that would make Soukoban game 5


Thank you, that completes all the release years and game numbers for the software list.
Posted By: Darkstar Re: Fixed software lists - 01/07/10 05:40 AM
Sorry for slightly hijacking this thread, but...

I was recently thinking about all those different "good" dump lists before (TOSEC, No-Intro, Redump, whatever) and wondered wether it would be possible to unify them into one big ultimate "list of all known dumps". I actually spent some time thinking about this. The ideas are:
- one huge database with entries for all ROMs (identified via different/multiple CRCs), CHDs, Tapes, floppy images, whatever
- references to where this particular dump comes from (so that it can be tracked back or cross-checked)
- metadata added to the dumps (i.e. chip type, "byteswapped" flags, you-name-it)
- online "perma-link" style references (via either SHA/CRC checksum or unique ID) which quickly shows you something like "this is a good dump according to TOSEC" or "this was a good dump in MAME 0.77 but it was redumped and replaced by [this dump]" etc.
- cross-links to other information sources for each game (MAWS, LemonAmiga, ...)
- possibility to export the db to various formats (dat, hsi, xml,...)
- import tools to (manually/semi-automatically) import dump lists from various sources
- most importantly: completely open database and tools

The idea is not to abolish the various "good-lists" but to create a superset which can be used as an online reference tool, but across all dumping groups and emulation systems.
Also, the database would not only contain "known good dumps" but also old dumps that have been replaced, known-bad dumps (with information what is broken), and hacked/cracked/trained versions (as long as data on those exist), so that you can identify almost every file through it.

Anyone ever thought about something like this? Feasible? Helpful? Or just helping the pirates?

As I said I already thought a lot about this and I even wrote up some sort of braindump that I could clean up and post somewhere if anyone's interested.

-Darkstar
Posted By: Stiletto Re: Fixed software lists - 01/07/10 06:02 AM
Whoa, I see I'm not the only hyper-ambitious person.

I would say go for it aside from knowing that some of these databases claim you can't repurpose them like this. Can't name any names off the top of my head but I'm sure it would rub some of those people the wrong way.

Obviously MAMEDEV doesn't care, not what I'm talking about. smile
Posted By: Guru Re: Fixed software lists - 01/07/10 07:11 AM
it might be ok as a big software list but the idea is to have smaller console and computer specific lists so heads dont explode with information overload.
just check the mess source for an example
http://git.redump.net/cgit.cgi/mess/tree/src/mess/software

to keep track of what was fixed and replaced etc you need a MAWS for MESS. Once the software lists are in maybe cutebutwrong will add a MESS section :-)
Posted By: Lord Nightmare Re: Fixed software lists - 01/07/10 02:22 PM
Speaking of Software lists, is there any way to combine this with RPK support, i.e. for having a cartridge which has more than one rom inside it in one unified archive?

LN
Posted By: R. Belmont Re: Fixed software lists - 01/07/10 02:35 PM
Sure. The list would contain the hash of the XML file inside the RPK, which establishes the same chain of trust you'd have with single files.
Posted By: etabeta78 Re: Fixed software lists - 01/07/10 02:45 PM
wouldn't be possible to replace the fake rpk format with xml inside a common zip? i.e. to directly add .xml as supported extension to the drivers?

opening a mounted zipped image, the MESS driver should look for the xml extension before anything else and, if the xml is present, then it should read nodes into it

if no xml is present, MESS should check other extensions to see if any supported file is present (say some .bin cart) and load it

as a bonus, replacing rpk with plain zip, we might use the transparent zip handling from MAME core and the whole design whould possibly reduce the duplicated code...

also, all the ti99 improvements would not be lost because files would only require .rpk to be renamed back to .zip, but the xml would remain inside the archive (and those xmls were the part which required more work)
Posted By: R. Belmont Re: Fixed software lists - 01/07/10 03:31 PM
Sure, but that's OT for this thread and doesn't actually gain us anything (except that apparently using the rpk extention really bothers you ;-)
Posted By: judge Re: Fixed software lists - 01/07/10 04:18 PM
rpk can still be used for software with more than 1 rom/file/whatever which mess does not know about or which mess does not want to support (example: archives with bad dumps or hacks).

Posted By: etabeta78 Re: Fixed software lists - 01/07/10 11:02 PM
Originally Posted By R. Belmont
Sure, but that's OT for this thread and doesn't actually gain us anything (except that apparently using the rpk extention really bothers you ;-)


it's the same reason why I always liked to have .bin extensions for cart dumps in place of .sms, .gg, .smc, .gb, .gba and other silly names wink

Originally Posted By judge
rpk can still be used for software with more than 1 rom/file/whatever


well, rpk requires a .xml inside it... hence MESS could look directly inside the xml to decide where to load each file

That said, stop with the OT: I'm really happy judge managed to finally complete this long-awaited submission and I look forward into helping a bit with the list creation (and/or the list addition to listxml) smile
Posted By: ranger_lennier Re: Fixed software lists - 01/08/10 12:08 AM
Originally Posted By etabeta78
I'm really happy judge managed to finally complete this long-awaited submission and I look forward into helping a bit with the list creation (and/or the list addition to listxml) smile


So is it full speed ahead with software list submissions, or are we still working out the details?
Posted By: Guru Re: Fixed software lists - 01/08/10 06:31 AM
it should be full speed ahead with software list as long as they are like the 2 examples there ATM.
unfortunately the other important parts are not done yet so you cant get a list of which games a console/computer supports and there's no xml output of software so it can't be hooked into clrmame yet either.
Posted By: Robbbert Re: Fixed software lists - 01/08/10 07:28 AM
Originally Posted By etabeta78
it's the same reason why I always liked to have .bin extensions for cart dumps in place of .sms, .gg, .smc, .gb, .gba and other silly names wink


It's probably the reason I have a bunch of .bin files with no idea what system they are for...

Hopefully this new listing will sort that out..
Posted By: Darkstar Re: Fixed software lists - 01/08/10 07:39 AM
Originally Posted By robbbert
It's probably the reason I have a bunch of .bin files with no idea what system they are for...


That's actually one of the use-cases I hoped to be able to figure out with a "unified" ROM db: CRC search across all systems smile

-Darkstar
Posted By: judge Re: Fixed software lists - 01/08/10 07:46 AM
The software lists are not final yet; some important things like listxml and somthing similar to -verifyroms are missing. I do have listxml and rom loading working locally, but there is duplicated code and I don't like that.

I do hope we can get this fully supported for the 0.137 release.
Posted By: Guru Re: Fixed software lists - 01/08/10 09:44 AM
Originally Posted By Darkstar
That's actually one of the use-cases I hoped to be able to figure out with a "unified" ROM db: CRC search across all systems smile


eventually (I hope) MESS should work like MAME. So you can check your unknown ROM/bin against MESS.....
MESS -romident unknown.bin

a separate db isn't needed.
Posted By: RColtrane Re: Fixed software lists - 01/08/10 11:04 AM
Originally Posted By robbbert
Originally Posted By etabeta78
it's the same reason why I always liked to have .bin extensions for cart dumps in place of .sms, .gg, .smc, .gb, .gba and other silly names wink


It's probably the reason I have a bunch of .bin files with no idea what system they are for...

Hopefully this new listing will sort that out..


If that matters, I have a different way to deal with my files. As an example, I took the following MSX game from a 'well-known' site and the name of the file explains everything about its contents:

Yie Ar Kung-Fu II - The Emperor Yie-Gah [Konami] [Jp-En] [RC-737] [MSX].rom

In this way, you can have 'generic' file formats for different media types (.rom for cartridge dumps, .cas for cassete dumps...) and for different systems without messing them all with those nasty file formats (.sms, .smc... mentioned above).











Posted By: Heihachi_73 Re: Fixed software lists - 01/08/10 12:03 PM
Hopefully, good riddance to the above example for anyone being forced to type that into the command line to run it. This is the exact same reason is why MAME's ROMset names warofbug, eprom and redufo are so much better.
Posted By: Haze Re: Fixed software lists - 01/08/10 01:19 PM
Originally Posted By Heihachi_73
Hopefully, good riddance to the above example for anyone being forced to type that into the command line to run it. This is the exact same reason is why MAME's ROMset names warofbug, eprom and redufo are so much better.


Indeed, as I said in my example

MESS genesis ghouls

would be heavenly. It might not look as pretty in your filesystem*, but it's a lot easier than either

a) having to rename everything to 8 characters yourself

b) traversing through a gui to launch the game you want each time you need to test something.

*I wouldn't actually oppose MAME / MESS being able to look for roms from 'LONG DESCRIPTION'.zip either, so that the files in the filesystem can look nicer, although the disconnect between the 8 letter tag needed to run it, and the description+filename may then confuse people.
Posted By: netol Re: Fixed software lists - 01/08/10 02:23 PM
Just asking,

Don't you think that this software list is too huge to use 8-character filenames?
Posted By: Haze Re: Fixed software lists - 01/08/10 02:25 PM
btw for Megadrive / SMS / GG the HazeMD database is pretty much up to date. It aggregates several other sources (I guess some ROMs could be bad, but until proven otherwise I haven't deleted them)

I didn't extract the databases from any of the existing tools, the internal database I'm using was created from files renamed using them, but just using a script to convert all the filenames of the roms in a folder to 8 letters (well 6, as I prefix them g_ due to the lack of native system division) It resulted in some ugly filenames, but at the time I didn't feel like fine-tuning them because MESS seemed disinterested in any fixed software support.

The checksums etc. I have are all of the raw binary dumps (not custom SMD format) I haven't byteswapped the roms to be 'proper' dumps, although I've been tempted; I'd rather just add real dumps as they're dumped tho.

The real cart dumps I support are obviously in the correct dumped format.

The 'Radcia' stuff is MD hardware, but is actually a standalone 'TV Game' system. AFAIK HazeMD is the only thing that runs them (although I haven't checked the latest Kega etc.)

The most interesting of the 'Real Dumps' is F22 with a rom loading as follows

ROM_START( g_f22_realdump )
ROM_REGION( 0x400000, "maincpu", 0 ) /* 68000 Code */
ROM_LOAD16_WORD_SWAP( "f-22_intercept_f-202_4.u1", 0x000000, 0x080000, CRC(649771f8) SHA1(6daeca39841f06549373f3a4fd746f3e1c95c328) )
ROM_LOAD16_BYTE( "f-22_intercept_f-202_1.u2", 0x080001, 0x020000, CRC(d3d7cbb8) SHA1(72e56f858bfe88c2529939304ba49eee3fe14353) )
// no even rom (it becomes a 0x1a fill in the cart copier dump)
ROM_END

All the 'combined into one file' dumps have dummy data (usually a fixed fill) for where the even ROM bytes would go, when in reality it should be unmapped.

Last time I checked

GAME( 1996,g_truc96, 0, megadriv, megadri3, megadriv, ROT0, "Miky", "Truco '96 (Argentina)", 0 )
GAME( 19??,g_tc2000, 0, megadriv, megadri3, megadriv, ROT0, "unlicensed", "TC 2000 (Unl) (Argentina)", 0 ) // based on SMGP code

weren't in any other databases.

There is some homebrew stuff in there, but I actually feel that as long as it's tagged as such it should be ok (as long as people don't go overboard) Homebrew often provides very good test cases which ultimately help with the goal of documentation.

I did have to resort to some nasty auto-backup ram detection in the lastest HazeMD due to rebuilding the database, I simply didn't have time to go over all the sets and add the correct backup RAM sizes. I did have to hardcode a few cases where the headers are incorrect on purpose tho. The EEPROM based save stuff has never been supported. Ideally with a fixed databse the hackish auto-detection could be dropped, as it was in the previous release. That database had gone unmaintained for too long tho, which is why I just rebuilt it instead. In a more active and evolving project like MESS this will be less of a problem.

The protection emulation in some of the pirate games might still be imperfect.

The SMS / GG emulation is my own, I only did the rewrite so that the SystemE games in MAME worked properly after Nicola decrypted them. The current MESS code should be better, more accurate and more compatible.

CONSOLE_ROM_LOAD_PIRATE( g_redclf, "the battle of red cliffs - romance of the three kingdoms (unl).mdx", 0x000000 , 0x200005, CRC(44463492) SHA1(244334583fde808a56059c0b0eef77742c18274d) )

is in some weird format, I think it's actually been encrypted for a specific emulator to stop other people emulating it. It should be converted back to a normal ROM. I just didn't get around to it. There were several other similar sets which I never added, and probably got lost due to a general lack of traction with the project.


CONSOLE_ROM_LOAD( g_indy , "indiana jones' greatest adventures (release candidate).md", 0x000000 , 0x200000, CRC(9a01974e) SHA1(d59bb77b83cf912e3cb8a7598cabcf87a273e429) )

shouldn't have been enabled, it isn't publicly available (and I don't have it anymore either, I had it on a USB drive for safe-keeping, and the only time I took that USB drive out with me, as an emergency, it was stolen)

There are one or two sets missing from HazeMD at the moment, but I didn't feel it work making an extra update for it.

Many of the HPZ-dumped prototypes won't work. I have a feeling they were simply dumped from daily builds of carts that didn't work hence the naming. Having worked in the industry I think it's fair to say there are periods where the daily builds really do nothing but crash :-)

CONSOLE_ROM_LOAD( g_gameno, "game no kandume otokuyou (japan).md", 0x000000 , 0x300000, CRC(cdad7e6b) SHA1(31c66bd13abf4ae8271c09ec5286a0ee0289dbbc) )

I don't really know what this is. It's some collection of remakes, but expects all the ROM are to be RAM so that it can copy the games there and run them. The games included also seem to be tagged 'Sega-Net' elsewhere, so maybe they were downloadable, or it's from a CD collection of downloadable games. Was interesting to see what it needed to boot anyway, because prior to that people were having to just hack the roms out of it to run them.

(and on a related note, there is some good 'SegaNet' stuff, the Doki Doki Penguin remake is much nicer than the arcade / SMS versions, I'm surprised nobody hacked it to be a real arcade version ;S)


So yeah, if you want to use anything from HazeMD that's probably all you need to know. Obviously the database needs refining over time (names improving, manufacturer details adding, years adding, backup ram verifying, proper dumps, proper filenames etc.), but if it makes it's way into a project which isn't my sole responsibility then it seems a more worthwhile task as I'll know the changes aren't just going to sit in some obscure emulator I created ;-)

Have fun.






Posted By: Haze Re: Fixed software lists - 01/08/10 02:26 PM
Originally Posted By netol
Just asking,

Don't you think that this software list is too huge to use 8-character filenames?


as long as it's sub-divided by systems, I think it will cope. Clones aren't limited to 8 letters anymore either which eases the pressure significantly.

Not subdividing will make things harder, especially when you consider how many 8-bit titles are used for different games across all the platforms. It's definitely an important consideration to make.

At the end of the day it's meant to benefit the developers the most, by presenting easy to remember, easy to type names. Many users will just use a frontend anyway, and not care.
Posted By: etabeta78 Re: Fixed software lists - 01/08/10 02:52 PM
I think command line will always require a 4th parameter for the involved media, something like

mess genesis -cart ghouls

(which makes even more sense for games released on multiple media, e.g. tapes and floppies on a c64)

AND loading software not on the list should be still available of course (especially for computers)

other than that, as soon as the list format is establish, I think hazemd database would be the most natural starting point for the correspondent MESS database wink
Posted By: judge Re: Fixed software lists - 01/08/10 02:54 PM
Since it's a seperate list we could use different limitations; that is all still to be decided.

Seperate lists seems the best way to go. There were many games released on different consoles that it only makes sense to keep them unique within a list.

Posted By: Haze Re: Fixed software lists - 01/08/10 03:14 PM
Originally Posted By etabeta78
I think command line will always require a 4th parameter for the involved media, something like

mess genesis -cart ghouls

(which makes even more sense for games released on multiple media, e.g. tapes and floppies on a c64)

AND loading software not on the list should be still available of course (especially for computers)

other than that, as soon as the list format is establish, I think hazemd database would be the most natural starting point for the correspondent MESS database wink


There are lots of different media formats in MAME, (games which were on floppy / rom etc.) I don't really think you need an extra switch unless you're specifically trying to load something external to the database. I'd prefer to type

MESS c64 ghouls
or
MESS c64 ghoulsd

rather than

MESS c64 -tape ghouls
or
MESS c64 -disk ghouls

it makes more sense for loading external stuff tho, when the internal database isn't available for it to know what to do.

MESS c64 -tape ghouls_and_ghosts.tap
and
MESS c64 -disk ghouls_and_ghosts_my_hack.fpy
or
MESS c64 -disk ghoulsghosts.chd ;-)

(i don't know what formats C64 uses, so i made them up ;-)

The general rule as far as I'm concerned is that as a developer I want to use the command line, but the LESS i have to type, the better.

Assuming that Ghouls was a 2 disk / Side game, I wouldn't be offended if that launched me into a minimal MAME/MESS gui (not a windows specific thing) where I could choose which of the disks I wanted to mount, or if it was a system which needed a boot disk to boot anything, if that option was presented too. (As i've said before, in the cases of games where you have to install them having a 'parent' pre-installed DOS / Win95 etc. partition that could be associated with any piece of software would help usability 100-fold)

Sliding a bit further off-topic, but I think for computers it's much more important to provide a level of 'hand-holding' than it is with arcade games in MAME. I'd hate to have to learn how to use a japanese command-line window to boot a game on a system I know nothing about if all I wanted to do was try to fix some graphical bugs. I know this moves away from the 'pure' aspect of MAME, but usability and accessibility are important if any real progress is to be made. Likewise, I remember having to make all sorts of MSDOS boot disks back in the day, with exactly the required memory configs but if you asked me how to do that now I'd just give you a blank stare ;-) I guess MAME does kinda do this type of thing with the default EEPROMs tho.

Obviously even with separate lists MESS should be able to rom-ident anything thrown at it (providing it's a valid dump) without having to specify the expected system or so, and loading should be flexible enough that you're not forced to have sub-folders for each system; if I only have genesis games, I'd prefer it to simply look in ROMS, not have to have ROMs/genesis. I don't care if that means I can mix games with the same name from different platforms in the same zip, I just want it to be quick and easy to use, and if it's loading from a databse, it won't matter anyway, it will load what it needs, where it finds it.


Posted By: Micko Re: Fixed software lists - 01/08/10 03:33 PM
Current implementation is using ROM_LOAD and that works fine only for carts and there are lot of other software, think maybe it would be wise to do it a little different.

Since we already have image mounting we could do next
SOFTWARE_START(prog )
IMAGE_CART("cart1", "masc_03a.bin", 0x0000, 0x8000, CRC(cbec2471) SHA1(770ab06b1fcc35da08ce8453b754b8f62520cdd0)
IMAGE_FLOPPY("flop1","floppy.dsk", 0x80000, CRC(cbec2471) SHA1(770ab06b1fcc35da08ce8453b754b8f62520cdd0)
SOFTWARE_END

so it will do something similar as you will manually do when mount that cart and that floppy image to driver.

I am in progress of converting image.c into device to make this transition easier(but currently have lot of work on RL job)

this way we could support multiple images also, so if we have for example two files for flop1 then (similar we already do for nes famicom disk system) we could make a disk switch key (or meny option) so you can put disk 2,3,... when asked in game

Agree that load should be simple as possible so I am for

MESS driver game

This could also make RPK deprecated, since you could define which file is loaded in which device
Posted By: Haze Re: Fixed software lists - 01/08/10 03:42 PM
Originally Posted By Micko

Agree that load should be simple as possible so I am for

MESS driver game

This could also make RPK deprecated, since you could define which file is loaded in which device


As I said elsewhere, it would be nice if the software-lists could be associated with multiple 'drivers' tho.

ie.

MESS megadriv ghouls
MESS megadrivj ghouls
MESS genesis ghouls

would all refernce the same software list, but load up a different machine configuration. (ie, if a system is a clone of another system, it can support the same software list)

Again for computers it becomes more difficult, because you have near unlimited configurations, but that's an issue for another day.
Posted By: Micko Re: Fixed software lists - 01/08/10 03:48 PM
in running_machine you have now compatible_with as global (so visible in MAME too), and that parameter is set in computers and consoles, so those that are clones of and compatible with can share same software lists, think there should be no issues on this.
Posted By: Haze Re: Fixed software lists - 01/08/10 04:15 PM
will this expand to MAME having an implied 'system' of Arcade?

(although it would be nice to have an option to do a full MESS compile, where the explicit system of 'arcade' caused it to run MAME stuff, it would help for development where you have arcade and console games shared hardware and want to test both in the same binary while developing)

Could be messy for NeoGeo I guess, where in reality the software lists would/should be shared, as the AES would run any MVS carts with an adapter* and the rom content is identical anyway (the actual first-run AES releases, tagged with 'h' in MAME are just more recent code revisions which often made their way back onto MVS carts anyway)

* later games did lock out the AES mode, because MVS carts were cheaper, and SNK wanted to crackdown on people using adapters to play them instead of buying the AES carts, so you might need a way of specificaly marking some sets as unavailable for some system configs, in that case not available for 'neoaes'

Afaik the main differences on the AES are a different bios, the lack of multi-slot, and the lack of a watchdog (the games with watchdog protection like Fatal Fury 2 completely skip the watchdog protection check with an AES bios*)


* Fatal Fury 2 is the best protected of the early NeoGeo games with 'watchdog' protection. While most will freeze at a 'for use on original SNK boards only' message if the watchdog fails to kick in and set a value in SRAM during the initial boot Fatal Fury 2 will continue and just break the game in random ways. Combine this with an actual protection device which acts in a similar way and you have the reason why it took so long to properly understand the protection on it and why for a long time emus recommended to just run it in AES mode, which skips part of it.



Posted By: judge Re: Fixed software lists - 01/08/10 04:23 PM
It should be possible to create software lists for things like mvs and stv carts too.

I'd prefer to keep mvs and aes lists separate though; simply because these carts could not be directly plugged into each other's cartridge slots.
Posted By: Haze Re: Fixed software lists - 01/08/10 04:26 PM
Originally Posted By judge
It should be possible to create software lists for things like mvs and stv carts too.

I'd prefer to keep mvs and aes lists separate though; simply because these carts could not be directly plugged into each other's cartridge slots.


I'm pretty sure you could get adapters to do so. It's just a case of the pins being in a different place. You can't plug JPN megadrive carts into a US/EU megadrive without stripping parts of the case away wink
Posted By: judge Re: Fixed software lists - 01/08/10 04:27 PM
Yeah, same with mastergear adapters or adapters for nes carts. Those will be another nice challenge smile
Posted By: etabeta78 Re: Fixed software lists - 01/08/10 04:28 PM
Originally Posted By Haze
As I said elsewhere, it would be nice if the software-lists could be associated with multiple 'drivers' tho.

ie.

MESS megadriv ghouls
MESS megadrivj ghouls
MESS genesis ghouls

would all refernce the same software list, but load up a different machine configuration. (ie, if a system is a clone of another system, it can support the same software list)

Again for computers it becomes more difficult, because you have near unlimited configurations, but that's an issue for another day.


while I haven't checked the current implementation, the original plan was to explicitly list the software lists for systems like they were devices. i.e. you would have a

MDRV_ADD_SOFTWARE_LIST(listname)

inside the driver.

This way you can combine very easily lists with systems, depending on compatibilities... of course, some validation on gamenames might be needed to avoid duplicates...

avoiding duplicates was also the main reason to use a media parameter before the gamename
Posted By: incog Re: Fixed software lists - 01/08/10 04:29 PM
Originally Posted By Haze
You can't plug JPN megadrive carts into a US/EU megadrive without stripping parts of the case away wink


It's not really parts, you just snip 2 plastic corners off the cart slot but it still stands.

However you could also claim that all SMS titles should be merged with the GameGear and Genesis/Megadrive lists as you could also buy converters.
Posted By: Haze Re: Fixed software lists - 01/08/10 04:31 PM
Originally Posted By judge
Yeah, same with mastergear adapters or adapters for nes carts. Those will be another nice challenge smile


Well yeah, my point was just that you can theoretically do it, even if it requires something sitting between the system and the cart. I had plenty of imported Japanese MD games, some of which AFAIK didn't actually get an EU/US release, only some are region locked, or fail due to timing differences. (I believe SMS games are worse for this, as most of the Euro developed releases will run on US systems, but are heavily glitched due to them trying to squeeze out every cycle and they're just not capable of running at 60fps, so break) Quite how you communicate this concept to the user, I don't know, nor have any suggestions for.

Posted By: Haze Re: Fixed software lists - 01/08/10 04:33 PM
Originally Posted By incog
Originally Posted By Haze
You can't plug JPN megadrive carts into a US/EU megadrive without stripping parts of the case away wink


It's not really parts, you just snip 2 plastic corners off the cart slot but it still stands.

However you could also claim that all SMS titles should be merged with the GameGear and Genesis/Megadrive lists as you could also buy converters.


Well.. there probably should be a way of running them with the Genesis VDP model.. (and I seem to remember the Genesis release of Phantasy Star 1 *WAS* the SMS version) but I think as far as merging the entire lists goes, common sense should come first. In the SMS case I'd probably just suggest having an SMS clone which was 'Genesis (in SMS Compatibility Mode)' or so... I already had fun with this on Megatech, where the carts are a mix of Genesis and SMS carts, and you have to be able to switch between them on the fly laugh

The strangest cases will probably always need a 'common sense' handling system. You simply can't implement everything in a generic and expandable framework.

The other tricky cases I can think of are where the carts have extra CPUs in them. Ideally it should be possible to add the extra hardware when the machine is initalized, as it's in the carts, and shouldn't really require different machine structures, or systems to be specified. I imagine this comes with it's own set of problems too however. I guess if the machine configuration is separate from the system name, then it isn't too much of a problem, as different machine configs can be specified with the lists (although it probably isn't so simple, as the different systems will have to represent different configs.. so, it just seems more logical that it should be handled in the cart/driver init instead with CPUs being added with the clocks derived from the already known system, interleave levels increased as requied etc. much as cart specific decryption and installing of cart specific protection handlers is done now)

For now I'd suggest full steam ahead, and dealing with the issues, and refining things at a later date. Not doing anything for fear of future problems doesn't really get you very far. I didn't look at the Type-4 GX games for a long time because I simply thought the mulit-screen would make it a nightmare even if the protection was figured out. In the end I said 'fuck it', did it anyway, and they turned out to be pretty easy :S



Posted By: judge Re: Fixed software lists - 01/08/10 05:09 PM
Heh, -romident is already working locally laugh
Posted By: Stiletto Re: Fixed software lists - 01/08/10 05:29 PM
If you really are serious about this, Darkstar, you should talk to Logiqx (logiqx.com) about the arcade side of things. He has all those DATs for all those other emulators, generally knows ROM histories, etc.
Posted By: ht1848 Re: Fixed software lists - 01/08/10 06:21 PM
Random Idea #1 Is there any consideration for documenting bad dumps? or will we take MAME approach and only document Best-known (which is a fine policy as well).

My thought is that MESS is coming from behind there are a ton of known crap files out there.

Random idea: what if we had a "software/bad_dumps.c" that stored simple CRC and name (and maybe a reason code: overdump, partial, hack...etc) of only known bad files.

This would feed data to support -romident. telling you not only is your file not identified as good, but it is a known bad file, - don't submit it. (in fact delete it!)

clrmame (or similar) could read that data, but only allow deleting of those files, not pokeromcollecting of crap.

[b]Random Idea #2
The "what about homebrews" question always seems to come up (not sure I care totally about homebrews but there are good things about them). What about providing only the framework to be able to add a second file in the software folder so your directory might be...

software/supracan_cart.c (included in MESS)
software/superacan_cart_homebrew.c (3rd party)

The homebrew files could be like a addon and managed separately, if a particular user is interested in homebrews, add that to software folder and compile. Maybe not the best approach but, I like that it maintains flexibility but has core list as official.

Just rambling...Keep up the good work. I am excited no mater what direction this goes.
Posted By: Haze Re: Fixed software lists - 01/08/10 07:14 PM
Originally Posted By ht1848
Random Idea #1 Is there any consideration for documenting bad dumps? or will we take MAME approach and only document Best-known (which is a fine policy as well).


For the love of god, no. One of the worst things about the 'good' tools is that they know every bad dump under the sun. This only serves to keep useless dumps in circulation.

Of course, if there is a bad dump of a rom which is clearly meant to be a different revision, and no good dump of that specific revision then it should be documented and supported so that people know that there is a revision missing for which currently only a bad dump exists.


Originally Posted By ht1848
Random Idea #2 The "what about homebrews" question always seems to come up (not sure I care totally about homebrews but there are good things about them). What about providing only the framework to be able to add a second file in the software folder so your directory might be...

software/supracan_cart.c (included in MESS)
software/superacan_cart_homebrew.c (3rd party)

The homebrew files could be like a addon and managed separately, if a particular user is interested in homebrews, add that to software folder and compile. Maybe not the best approach but, I like that it maintains flexibility but has core list as official.

Just rambling...Keep up the good work. I am excited no mater what direction this goes.


I think the plan is to support homebrew stuff via an external mechanism, although personally I'd actually suggest that useful / interesting ones actually be supported and marked as such. Especially when you're dealing with computer systems there was a huge amount of 'Public Domain' software which could be considered homebrew, but was just as much a part of the software library as anything else. (I probably spent more time playing amiga 'PD' games via the Assassins collections, and standalone discs than actual commercial software because the stuff wasn't much more expensive than blank discs and many of the games were very very good)

Some of the homebrew test cases on the Megadrive are invaluable too, and I think providing they're appropriately marked and documented they should be supported. Some of the translations were also widely used on original hardware due to cart copiers etc. Other 'homebrew' like the 10000 Sonic palette hacks out there, which were never on an original cart and probably made by some 14 year old kid who thought that it would be awesome if sonic was green, pink, orange with a hint of brown are completely worthless however, and those are the type of thing that if somebody REALLY wants to run, they should resort to using the external rom loader.

Again, I'd just advocate 'Common Sense'. If something is useful to document, it should be documented. The most important thing is that a) doing all of this helps document what was on the system, and b) it makes it easier for developers and testers alike.

If somebody wanted to start a misfit-mess project, where all the truly worthless hacks, bad dumps, and 20% complete translations were left then they'd be more than welcome to, as long as they didn't actually start including things which should be in the main version as that just ends up making things more difficult ;-)

Posted By: Robbbert Re: Fixed software lists - 01/08/10 08:47 PM
Originally Posted By incog
Originally Posted By Haze
You can't plug JPN megadrive carts into a US/EU megadrive without stripping parts of the case away wink


It's not really parts, you just snip 2 plastic corners off the cart slot but it still stands.

However you could also claim that all SMS titles should be merged with the GameGear and Genesis/Megadrive lists as you could also buy converters.


Keep the lists separate, BUT remember MESS has a COMPAT field to indicate unrelated (by parent or clone) systems that are able to run each other's software. This is something that should be taken into account in the new listing system.
Posted By: ranger_lennier Re: Fixed software lists - 01/08/10 10:04 PM
I would say we could do without bad dumps, ROMs hacked to run in an emulator, hacks with just a message from the dumper, etc.

Original homebrews, fan translations, and hacks that introduce interesting changes (like the many crazy Super Mario Brothers levels out there) I would want to document accurately. When you get into homebrews, many of the better ones actually are sold, and so could be considered an unlicensed commercial release.
Posted By: Heihachi_73 Re: Fixed software lists - 01/09/10 04:22 AM
Homebrews don't belong in a list of console games, unless they were actually sold dumped from a physical cartridge/disk/tape - as with MAME.
Posted By: Haze Re: Fixed software lists - 01/09/10 11:28 AM
Well, just consider that *Home* consoles were used in the home, reprogrammable carts / copiers weren't all that uncommon, and most of the homebrew has been running on *Home* systems in people's homes.

A lot of it provides very useful test cases, and providing everything that wasn't commercial software is documented as such then keeping such test cases in circulation benefits the software in many ways (Charles' Window Bug Test for example), and I've known plenty of people who run the top quality Wonderboy translation on their original systems, because it's as good as any commercial release even if it's most definitely after-market and would have to be marked as such.

The arcade scene (aka MAME support) is a bit different, people tended to just buy arcade boards, and operate them on location. They didn't make their own, and the majority of 'homebrew' never operated in the intended environment (the arcades)

Anyway, at the end of the day I'd advocate whatever is going to result in the most accurate, bug-free emulation of all the systems with the test cases to verify code changes against, and the provides most accurate documentation of the software running on each system anywhere online. I don't think anybody is going to disgree with that goal, and any steps that help the project reach it.

Just don't be too closed-minded on the issue, the gambling games in MAME are only just starting to recover from such a mess because back in the day somebody decided they weren't arcade games, didn't count, and didn't need to be documented. I really wonder how many have been lost, as a result, or would have otherwise been better documented if they'd been supported earlier.
Posted By: Heihachi_73 Re: Fixed software lists - 01/09/10 12:21 PM
Good point...the main ones I was thinking about was mainly those level hack type games where the majority of the game has been modified, but never been burned to a ROM since it was made, remaining in binary file format from day 1. I know there are lots of useful and really good homebrew programs out there. Maybe they can be added as 'unsupported' - this would be like running MAME with an alternate/hacked ROM set, the GUI will tell you the game is unsupported or is a ROM image.

I can definitely remember then 'casino' games weren't allowed in MAME - down to the point where Lotto Fun was removed just because it sounded like a gambling game! I believe quite a few people back then also thought that all gambling games were 99% mechanical, like the pachinkos, UK fruit machines and traditional Vegas slots (e.g. IGT S-Plus).
Posted By: etabeta78 Re: Fixed software lists - 01/09/10 12:27 PM
MESS, due to emulation of computers where fixed software lists only would stop people from running their own programs, could not support only the list titles.

There would always be the option of running spare software, and this includes both test programs / demos and good hacks. In fact, this would also allow to test new dumps to see if they are good or not.

Not that I'm against documenting/listing homebrew, but I don't see it as a priority.
Posted By: Guru Re: Fixed software lists - 01/09/10 12:33 PM
I could count the number of people who have the capability to re-program OLD carts on one hand. it's not as widespread as you suggest. most homebrew was designed to run on emulators or hacked new flash carts with a USB interface to a common PC. they were not official and most of those companies were sued to hell and back. homebrew console software is not official and they are mostly uninteresting, weak and shallow. its true a lot of released official games were crap too, but they were official in a cart with maskroms etc. not just binary only. we can support them but by only allowing them to run (as general homebrew/not official software), not by hard coding list of them as something we want supported.
obviously you can run whatever you want on a computer so the scope for emulated computers will be almost unlimited there. but there must be a definitive line drawn between official mass-marketed software and some hacker guy writing a demo for his mates, be it console or computer. otherwise we'll have the whole internet listed in these lists going up to PCs running Linux and Aminet for Amiga (which is hundreds of gigabytes of stuff). its going to get out of hand VERY fast. I say dont touch it.
the ideal situation should be we support all official software from official manufacturers and 3rd party software houses and to let the fan boys sort out their precious homebrew all by themselves. we could use the years that the hardware was in production/sold-officially as a guide. any title that was produced outside of the official manufacturers production date is not official. especially Nintendo and Sega stuff because they ran that part of the operation with an iron fist. otherwise let's just support everything in the WayBack Machine and be done with it wink


>back in the day somebody decided they weren't arcade games,

they're not games, they're designed to steal money
Posted By: Drucifer Re: Fixed software lists - 01/09/10 01:07 PM
If you will be using the 8.3 naming convention for the game list, I would suggest creating a common standard for how the games are named. Some suggestions would be:

  • For single word games (Foobartastic) - First 8 characters (foobarta)
  • For games that are common-name sequels (Foobartastic 2) - Drop the last character & replace with the sequel number (foobart2)
  • For multi word games (Uber Foobar Extreme) - First initial of each word except the last word, which completes the 8 characters (ufextrem)


Just a thought...
Posted By: Heihachi_73 Re: Fixed software lists - 01/09/10 01:08 PM
Arcade games are also designed to 'steal' money in a way (isn't being beaten continuously on the final stage and continuing similar to losing coin after coin on a poker machine that won't pay the feature or a decent win?).

Not all gambling games are boring and repetitive as some people might think (at least from an AU perspective where 'pokies' rule the roost), in fact, I can spend much less money at the 'local' than at the arcade if I bet low - generally I can take a lowly $5 in there and play for hours if small wins keep coming up, since I am one of those pathetic types that 'waste' features playing a few credits on 1 line! Of course, if a 'big' win comes up (e.g. it puts me $10 ahead), that's it for the night!

Back to console/computer games though. While a fixed software list would work for ROM images, one thing which won't work is floppy/HD images or tapes, especially for games which save high scores and configuration data straight to the same media. Most PC games for example don't save their data to NVRAM, except for lesser programs which 'forget' everything once you exit the program.
Posted By: Guru Re: Fixed software lists - 01/09/10 01:13 PM
we would just support the official games that are 'out there'
if better dumps come along without high score saves etc then we update it. this is exactly the same as MAME does. we support what we have now until something better comes along.

as for pokies etc, they are banned here by the government. that's probably so only they can make money in their officially sanctioned casinos where they get a huge percentage cut, but the fact that they are banned in public places speaks for itself.

you could argue that arcade games take your money too. but they are skill based, the better players can last a long time. gambling games are rigged, they pay out only a percentage of the money that goes in (usually 30%) and this can be altered using DIPs etc. That means you WILL lose 70% of the time regardless of your skill. that's stealing at the highest level. governments turn a blind eye because they make billions of dollars out of it.

this is getting a little too much OT now. MESS should support the official while-the-console-was-in-production software and let the fan boys sort out the homebrew and home-written stuff. we allow it and turn a blind eye, but not openly endorse it.
Posted By: Guru Re: Fixed software lists - 01/09/10 01:28 PM
Originally Posted By Andy
If you will be using the 8.3 naming convention for the game list, I would suggest creating a common standard for how the games are named. Some suggestions would be:


8.3 isnt necessary. MAME supports more than 8 characters. MESS can too.
What we dont want to support is crap like "Yie Ar Kung-Fu II - The Emperor Yie-Gah [Konami] [Jp-En] [RC-737] [!] [b] [o] [a] [MSX].rom"

etc
Posted By: Haze Re: Fixed software lists - 01/09/10 03:36 PM
Originally Posted By Guru
Originally Posted By Andy
If you will be using the 8.3 naming convention for the game list, I would suggest creating a common standard for how the games are named. Some suggestions would be:


8.3 isnt necessary. MAME supports more than 8 characters. MESS can too.
What we dont want to support is crap like "Yie Ar Kung-Fu II - The Emperor Yie-Gah [Konami] [Jp-En] [RC-737] [!] [b] [o] [a] [MSX].rom"

etc


*for clones* which is fine.

parents are still 8 letters, and while you could argue that there is no point because DOS is dead it does help keep the parent set names sane.

The great thing about the MAME database format is that it doesn't attempt to cram everything into a single filename anyway. You have the set name (8ish letters) a long-description (which is the closest thing to what's in Good* / Tosec), Manufacturer and Year fields, *and* Individual ROM names. As far as documentation is concerned the MAME db format is win-win.

Anyway, I wasn't trying to derail the topic by mentioning gambling games, just giving an example of why preserving information and documenting things while we can is more important than doing it later, when nobody can remember a thing about what was dumped.

Posted By: Haze Re: Fixed software lists - 01/10/10 04:15 PM
What 'cleaning' is needed of the Sega Software lists?

Is it just things like the manufacturer info?

I'm just wondering why it was disabled, it seems more logical to leave it enabled and let it evolve.

In addition to the previous stuff I mentioned, a lot of the HPZ protos are also cut at the last 0xff / 0x00 fill (the data is clearly complete as you can see from comparing sets) I was planning on padding the roms to the correct size at some point. If that's also what you were refering to by cleaning, again, I can take care of it at a later date once they're enabled.

Also I don't know if you used the TinyCDI CHD sets, if not, you're welcome to as long as we're confident that the CHDs are accurately representing the files they've been converted from. They can be created from the images that are around, so I'm not hoarding anything.

Once they get supported I'll start converting the other revisions of the games, and probably even the digital video ones (which I didn't convert beceause we have no way of running yet)
Posted By: judge Re: Fixed software lists - 01/10/10 04:21 PM
The rom names could be better; in some sets i see weird rom lengths; manufacturer info; several years should have been set; mappers/needed features should've been documented or set (they already are in the hsi files).
Posted By: etabeta78 Re: Fixed software lists - 01/10/10 06:08 PM
it's being worked on, but you're more than welcome to contribute

I started from hazemd, and manually changed romsets in needs of a longer name and added some parent/clone relations

manufacturers and years still needs a lot of work, though

@judge: rom lengths come from hazemd, hence I guess David had a reason for those wink
Posted By: etabeta78 Re: Fixed software lists - 01/10/10 06:15 PM
Originally Posted By Haze
What 'cleaning' is needed of the Sega Software lists?

Is it just things like the manufacturer info?

I'm just wondering why it was disabled, it seems more logical to leave it enabled and let it evolve.


I added regions through defined strings which did not cope well with judge's removal of the regions field wink

hence, compile was broken. now they are enabled again.

my whole point in adding these was to have some larger set to play with in addition to the small one (which do not put under real test the softlist code, imho wink )
Posted By: Haze Re: Fixed software lists - 01/10/10 07:52 PM
Originally Posted By judge
The rom names could be better; in some sets i see weird rom lengths; manufacturer info; several years should have been set; mappers/needed features should've been documented or set (they already are in the hsi files).


Yeah, as I mentioned the lengths are weird on some of the protos because the dumping device for them cut 0x00/0xff fills at the end, I've seen others do that before, it's annoying but they can be improved later. Some will also be odd sizes like 3 meg, 2.5 meg etc. because the real carts aren't just 1 rom. Hopefully by supporting and transitioning over to real dumps this will improve with time. I'd actually be tempted to batch process and byteswap all the Genesis roms to native format, because there is a far higher chance of those matching the real dumps for cases which are a single rom, however the roms would then be incompatible with every other emu.

Year / Manufacturer data can be updated over time. Likewise the backup ram ranges for each game / eeprom / whatever they use data (because header guessing is bad)

Posted By: R. Belmont Re: Fixed software lists - 01/10/10 11:12 PM
Fireball attempted to make the reasonable point (while using unreasonable language, hence I killed his post) that CDs with audio tracks probably need a better ripping method before we enshrine them in lists, although that doesn't affect most CD-i titles.

I don't agree that we need to store data-only discs at a very low level though, and reading subchannels for all discs guarantees unreproducable dumps.

For GDROMs since they are by definition all read on real original hardware the offsets are "baked in" and we don't need to care as long as the emulation doesn't add any further offset (it doesn't currently).
Posted By: Heihachi_73 Re: Fixed software lists - 01/11/10 03:33 AM
Best example of a 'wrong' ROM file, Pitfall II on the Atari 2600. 10495 bytes; 8K + 2K + 255 bytes.
Posted By: F1ReB4LL Re: Fixed software lists - 01/11/10 04:38 AM
You could cut a few "unreasonable language" sentences and leave the center part with all the reasons and descriptions.

Originally Posted By R. Belmont
For GDROMs since they are by definition all read on real original hardware the offsets are "baked in" and we don't need to care as long as the emulation doesn't add any further offset (it doesn't currently).


Why bother with decryption on arcade boards, then? Decrypted roms are always perfectly playable (if decrypted properly), emulation of the decryption doesn't add anything into gameplay. "read on real original hardware" means you've dumped a RAW console output instead of original media contents. It's like a table of MCU outputs vs. a dump of the real ROM inside.

Originally Posted By R. Belmont
I don't agree that we need to store data-only discs at a very low level though, and reading subchannels for all discs guarantees unreproducable dumps.


I've told already, that they should be properly cleaned to get the reproducable ones, not just read with CDRDAO, like now.
Posted By: R. Belmont Re: Fixed software lists - 01/11/10 04:53 AM
Quote:
Why bother with decryption on arcade boards, then?


Because the idea is that the MAME media be useful to people who own the boards. Encrypted ROMs must be encrypted to be useful on boards (even though most CPS2 owners now Phoenix theirs to avoid the hassle).

But there's no way for anyone (including Sega) to make GD-ROMs now. The way you run MAME dumps on real boards is to extract the image file from the disc and transmit it to an Ethernet-capable DIMM board with xorloser's Python script. The GDROM images in MAME actually contain more data than is necessary to run them by that method so we're covered on hardware preservation.

Incidentally, in the middle of all this baking in software lists, we need to make sure none of that work breaks the existing mechanisms for loading software.
Posted By: Darkstar Re: Fixed software lists - 01/11/10 08:11 AM
Originally Posted By Guru
Originally Posted By Darkstar
That's actually one of the use-cases I hoped to be able to figure out with a "unified" ROM db: CRC search across all systems smile


eventually (I hope) MESS should work like MAME. So you can check your unknown ROM/bin against MESS.....
MESS -romident unknown.bin

a separate db isn't needed.


True, but this works (obviously) only for ROMs in MESS, not for example for other emulators that MESS doesn't yet support. And I doubt you would be able to identify all those hacked/patched/fantranslated/trained/otherwise modified ROMs via MESS anyway ;-)

-Darkstar

*edit* damn, should have read the whole thread before replying. The last point has been explained enough I think. The first point still remains valid though.
Posted By: Haze Re: Fixed software lists - 01/11/10 01:42 PM
Originally Posted By F1ReB4LL
You could cut a few "unreasonable language" sentences and leave the center part with all the reasons and descriptions.

Originally Posted By R. Belmont
For GDROMs since they are by definition all read on real original hardware the offsets are "baked in" and we don't need to care as long as the emulation doesn't add any further offset (it doesn't currently).


Why bother with decryption on arcade boards, then? Decrypted roms are always perfectly playable (if decrypted properly), emulation of the decryption doesn't add anything into gameplay. "read on real original hardware" means you've dumped a RAW console output instead of original media contents. It's like a table of MCU outputs vs. a dump of the real ROM inside.

Originally Posted By R. Belmont
I don't agree that we need to store data-only discs at a very low level though, and reading subchannels for all discs guarantees unreproducable dumps.


I've told already, that they should be properly cleaned to get the reproducable ones, not just read with CDRDAO, like now.


So what do you propose? In reproducable steps, that anybody can take, and be confident in taking.

Obviously it's a concern, but thus far nobody has come up with anything like a reasonable, reliable solution, only incredibly convoluted ones where you're more likely to introduce errors from simply misunderstanding the instructions.

Most of the initial CDs will be conversions from existing databases to CHD anyway, although obviously there will be mixed sources. It's better to have a documented dump of something, than none at all, even if it's one that might be replaced later if a better dump is made. (For example for CDi 'The Apprentice' isn't in the TOSEC database, and has audio tracks, but not supporting it just because the only dump has unverified offsets would be silly.. (Speaking of which, it's still prone to hanging between menu screens since JD added the CDDA playback)

I'd probably be more confident in our CHD code in general if there was a way to convert back to the common toc/cue/bin etc. formats to verify that the data and metadata contain 100% the same information as before the conversion mind you wink Also since nothing supports burning of CHDs or mounting, it's not especially useful outside of MESS, if for example you want to view files on the disc.

Posted By: Diet Go Go Fan Re: Fixed software lists - 01/11/10 03:45 PM
The software list at git.redump.net/cgit.cgi/mess/ is not loading today. When it worked yesterday, I saw that a few roms were missing. They are: Sega Genesis - "Dyno Blaze" - this prototype rom is listed in the No-Intro dat file "Megadrive - Genesis - other (20070911) but is not in more recent files. The other Genesis roms are all Russian pirates: "Harry Potter", "Pokemon Crazy Drummer", "Pocket Monsters 2", "Deer Hunter", "Death Caliber", "Command & Conquer", and "Commandos". The missing SMS roms are "Super Boy 2" and "Super Boy 4". The two missing GG roms are "Yaiba Adventures" and "Yokaru Yotantori". They appear to be retail Japanese releases.
Posted By: Haze Re: Fixed software lists - 01/11/10 03:52 PM
Originally Posted By Diet Go Go Fan
The software list at git.redump.net/cgit.cgi/mess/ is not loading today. When it worked yesterday, I saw that a few roms were missing. They are: Sega Genesis - "Dyno Blaze" - this prototype rom is listed in the No-Intro dat file "Megadrive - Genesis - other (20070911) but is not in more recent files. The other Genesis roms are all Russian pirates: "Harry Potter", "Pokemon Crazy Drummer", "Pocket Monsters 2", "Deer Hunter", "Death Caliber", "Command & Conquer", and "Commandos". The missing SMS roms are "Super Boy 2" and "Super Boy 4". The two missing GG roms are "Yaiba Adventures" and "Yokaru Yotantori". They appear to be retail Japanese releases.


Yeah, I have Dyno Blaze and some others (it appears to be a rather broken alpha build with a lot of corruption and buggy code tho)

Some of the Russian ones I weren't sure if were legit releases at the time, so never got around to looking at them. Command & Conquer is almost some homebrew demo thing, but I'm told that the author is the same person as was behind many of the other games which were actually released, just this one was unfinished / stolen / abandoned.

Some of the SMS / GG roms were released after the last HazeMD update, so obviously aren't in there. This is the kind of fine-tuning that can be done later, and it's far easier to add new sets to an actively maintained project where a few lines can simply be checked into SVN when they appear than it is for me to release an update of the standalone emus every time I find something new.

I'll be glad when this is all done and officially supported anyway, I can't even find some of the CDi sets I converted myself anymore :-/

Posted By: F1ReB4LL Re: Fixed software lists - 01/11/10 03:55 PM
Originally Posted By R. Belmont
Because the idea is that the MAME media be useful to people who own the boards. Encrypted ROMs must be encrypted to be useful on boards (even though most CPS2 owners now Phoenix theirs to avoid the hassle).


And how can board owners use the MCU roms (especially those, which were read by decapping)? And what's the reason to dump the individual roms inside cartridges, then? Flash carts for consoles use the complete roms, not the per-chip dumps, so they aren't very useful for console owners (maybe for fixing the carts, though, but I haven't heard about such cases yet).

Originally Posted By R. Belmont
But there's no way for anyone (including Sega) to make GD-ROMs now. The way you run MAME dumps on real boards is to extract the image file from the disc and transmit it to an Ethernet-capable DIMM board with xorloser's Python script. The GDROM images in MAME actually contain more data than is necessary to run them by that method so we're covered on hardware preservation.


You've removed 2 of my posts, none of them had mentioned anything about GD-ROMs, only about CD dumps. GD is a special case of CD, though, can be read completely on some PC drives (IIRC, some data can be read even from the area between the low-density and high density areas). But MAME CD dumps are definetely so-so, especially those with audiotracks.

Originally Posted By Haze
So what do you propose? In reproducable steps, that anybody can take, and be confident in taking.

Obviously it's a concern, but thus far nobody has come up with anything like a reasonable, reliable solution, only incredibly convoluted ones where you're more likely to introduce errors from simply misunderstanding the instructions.


Can you give the "reproducable steps, that anybody can take" for dumping the protected ROMs? smile Everything can be done either manually by the professionals or automatically via some smart tool (which, again, should be written by the smart guys), which won't be able to cover all the unusualities from start (only after a number of revisions).

I'll repeat the contents of the deleted post. 1) Data tracks on CDs are stored in scrambled form. Regular dumps are stored in descrambled form, i.e. "decrypted", not representing the actual CD contents. Moreover, there are cases of imperfect mastering, when different firmwares in different drives descramble the data differently (and usually all do it wrong, because trying to fix it), so the original contents can't be recreated anymore. There are drives, which can read all the data as audio, giving the scrambled output for data sectors, so shouldn't be any troubles with it. Also, the offset in this case is perfectly visualised: when the drive decrypts the data sectors, it automatically fixes the offset, but when it reads the audio part - it reads it "as is", so the very end of the data track on mixed-mode CDs is usually present (i.e. doubled) in the very beginning of the following audio track. 2) Subchannels can be dumped in reproducable form (by combining the rereading of each sector a number of times and some 'light' error correction), as a result - drivers can use the actual subchannels data to navigate (RAW TOC dump will also be needed in this case). 3) On pressed discs there are no "empty" sectors, like on CD-Rs, the data always starts at -150 sector and goes upto the edge of CD. -150 to -1 sectors is the 1st track pregap (not the lead-in, like some people think -- lead-in is located before -150 and copied a number of times to increase the durability) and, AFAIK, used by some consoles to sync (Sega CD, for example), so it should be useful to increase the precise of emulation. Most of the drives are able to read from -147, though, not directly from -150. Lead-out sectors are also readable on the drives with overread into lead-out capability, so, in fact, most of the data can be properly dumped.

Originally Posted By Haze
Most of the initial CDs will be conversions from existing databases to CHD anyway, although obviously there will be mixed sources. It's better to have a documented dump of something, than none at all, even if it's one that might be replaced later if a better dump is made. (For example for CDi 'The Apprentice' isn't in the TOSEC database, and has audio tracks, but not supporting it just because the only dump has unverified offsets would be silly.. (Speaking of which, it's still prone to hanging between menu screens since JD added the CDDA playback)


The Apprentice is in the CD-Ready format, the datatrack itself is stored in the 1st track's pregap, so the disc should be dumped in a different way. TOSEC's "method" is always the same, so it can't cover such discs at all.

Originally Posted By Haze
I'd probably be more confident in our CHD code in general if there was a way to convert back to the common toc/cue/bin etc. formats to verify that the data and metadata contain 100% the same information as before the conversion mind you wink Also since nothing supports burning of CHDs or mounting, it's not especially useful outside of MESS, if for example you want to view files on the disc.


While CHD dumps don't have anything special and while there are no unique CHD dumps (only converted ones), I don't think the format will be added into the burning/mounting tools.
Posted By: etabeta78 Re: Fixed software lists - 01/11/10 04:06 PM
Originally Posted By F1ReB4LL
While CHD dumps don't have anything special and while there are no unique CHD dumps (only converted ones), I don't think the format will be added into the burning/mounting tools.


we could however add a chdplayer like MAME offers a ldplayer to e.g. listen an audio cd in CHD form :P
Posted By: R. Belmont Re: Fixed software lists - 01/11/10 04:08 PM
Originally Posted By F1ReB4LL
And how can board owners use the MCU roms (especially those, which were read by decapping)?


You can buy a blank MCU and program it with the MAME image in most common EPROM writers. The thing with the protected MCUs is that they're read-protected - if you buy a new one you can program or read/write it all you want.

Originally Posted By F1ReB4LL
And what's the reason to dump the individual roms inside cartridges, then?


That's more Haze's thing so I'll let him defend it.

Originally Posted By F1ReB4LL
Originally Posted By Haze
So what do you propose? In reproducable steps, that anybody can take, and be confident in taking.

Obviously it's a concern, but thus far nobody has come up with anything like a reasonable, reliable solution, only incredibly convoluted ones where you're more likely to introduce errors from simply misunderstanding the instructions.


Can you give the "reproducable steps, that anybody can take" for dumping the protected ROMs? smile


Regardless, the fact that nobody's coughed up an automated way to read offset-corrected discs that will work on even a majority of drives contributes to the perception that it's all a stunt to make audio ripping elitist again in a world where Steve Jobs is letting the little people stomp their muddy boots all over it. Clearly the author of EAC could create an automated disc ripper with full correction but they choose not to.

Meanwhile the Dumping Union has democratized ROM dumping - there are half a dozen people now with Naomi serial dumping setups and f205v can handle most of the surface-mount crap that used to require Guru.

Originally Posted By F1ReB4LL
I'll repeat the contents of the deleted post. 1) Data tracks on CDs are stored in scrambled form. Regular dumps are stored in descrambled form, i.e. "decrypted", not representing the actual CD contents.


Sure, and I don't know of any common CD format that stores the data in that way. Even DiscJuggler and CloneCD have obviously recognizable "decrypted" data in their files.

Originally Posted By F1ReB4LL
The Apprentice is in the CD-Ready format, the datatrack itself is stored in the 1st track's pregap, so the disc should be dumped in a different way. TOSEC's "method" is always the same, so it can't cover such discs at all.


The dump obviously contains the data track since the game boots in both MESS and CD-i Emulator. I'm not clear what TOSEC has done wrong in that case.

Originally Posted By Haze
I'd probably be more confident in our CHD code in general if there was a way to convert back to the common toc/cue/bin etc. formats to verify that the data and metadata contain 100% the same information as before the conversion mind you wink Also since nothing supports burning of CHDs or mounting, it's not especially useful outside of MESS, if for example you want to view files on the disc.


It's easy enough to add code to output a .cue file (we accept .cue files as input now, after all). And in most cases you can just -extractcd and mount the resulting binary track image with IsoBuster or Daemon Tools or whatever.

I'm glad DVDs and BDs only have data sectors after all this smile
Posted By: F1ReB4LL Re: Fixed software lists - 01/11/10 04:50 PM
Originally Posted By R. Belmont
Regardless, the fact that nobody's coughed up an automated way to read offset-corrected discs that will work on even a majority of drives contributes to the perception that it's all a stunt to make audio ripping elitist again in a world where Steve Jobs is letting the little people stomp their muddy boots all over it. Clearly the author of EAC could create an automated disc ripper with full correction but they choose not to.


Offset correction for scrambled dumps is easy, each data sector has its own synchro at the beginning, so you should read from the first byte of the 0 sector synchro (or from the -150, if capable) upto leadout.

Originally Posted By R. Belmont
Meanwhile the Dumping Union has democratized ROM dumping - there are half a dozen people now with Naomi serial dumping setups and f205v can handle most of the surface-mount crap that used to require Guru.


So? I'm not talking about a few "top-secret" drives, you can easily buy them, they aren't very expensive. You can write a tool to automatize all the dumping routines, if you're going to have many volunteers. The worst thing is that noone in MAME team seems to be familiar with the optical media formats and noone wants to examine it to write a good dumping tool. No magic is needed, just some coding skills and a wish to have the good dumps for preservation.

Originally Posted By R. Belmont
Originally Posted By F1ReB4LL
I'll repeat the contents of the deleted post. 1) Data tracks on CDs are stored in scrambled form. Regular dumps are stored in descrambled form, i.e. "decrypted", not representing the actual CD contents.


Sure, and I don't know of any common CD format that stores the data in that way. Even DiscJuggler and CloneCD have obviously recognizable "decrypted" data in their files.


Because there's no tool focused to dump discs in preservation terms, only to dump the user data somehow and beating some protections. You can use a software descrambler to convert such dumps for burning, nothing special in it. Also, the software descrambler doesn't have those firmware issues I've described above.

Originally Posted By R. Belmont
Originally Posted By F1ReB4LL
The Apprentice is in the CD-Ready format, the datatrack itself is stored in the 1st track's pregap, so the disc should be dumped in a different way. TOSEC's "method" is always the same, so it can't cover such discs at all.


The dump obviously contains the data track since the game boots in both MESS and CD-i Emulator. I'm not clear what TOSEC has done wrong in that case.


TOSEC doesn't have this dump at all and I've described, why.

Originally Posted By R. Belmont
I'm glad DVDs and BDs only have data sectors after all this smile


Each sector on any DVD consists of 2418 bytes: 2048 bytes user data, 16 bytes of header, 302 bytes of error correction data, and 52 bytes of sync data. Header in each sector includes 4 bytes ID, 2 bytes IEC, 6 bytes reserved, and 4 bytes EDC. CSS-encrypted DVDs have a sector key in the reserved 6 bytes of the header. So, the DVD dumps should be _at least_ 2064 bytes/sector (which is also possible to do on many drives).
Posted By: R. Belmont Re: Fixed software lists - 01/11/10 04:53 PM
Please prove that any arcade or console systems use any data on DVDs beyond the 2048 user bytes. And while you're at it I'd like to see software that can actually burn that extra data. If it can't be recreated and isn't used the preservation value of having it is highly questionable (as you yourself tried to argue for ROM dumps).
Posted By: F1ReB4LL Re: Fixed software lists - 01/11/10 05:12 PM
AFAIK, either SecuROM or Safedisc use those sectors (personally I'm interested in CDs, haven't examined both DVDs protections enough to say for sure). StarForce definetely doesn't.
Posted By: Haze Re: Fixed software lists - 01/11/10 05:24 PM
Well your problem is this:

Yes, there are people on the dev team who care about this, myself included.

No, there is nobody on the dev team who understands or is capable of writing a program to deal with this.

So unless you can find somebody, that situation isn't going to change.
Posted By: F1ReB4LL Re: Fixed software lists - 01/11/10 06:01 PM
Originally Posted By Haze
Well your problem is this:


I don't think it's my problem.

At least some kind of 'convention' should be set, CHDMAN should be modified. Some demonstrative dumps can be done manually, before writing anything.
Posted By: R. Belmont Re: Fixed software lists - 01/11/10 06:07 PM
I've said before - if the redump guys (or some other group) could provide an offset-corrected dumping service and release their dumps in some documented format I'd be happy to add the necessary support in MAME and CHDMAN. Until then, we do what we can.
Posted By: Diet Go Go Fan Re: Fixed software lists - 01/11/10 06:23 PM
I'm glad to see that those roms weren't forgotten. - Note - I was responding to Haze's post after my post, but this forum works differently than I thought.
Posted By: Haze Re: Fixed software lists - 01/11/10 06:25 PM
Well for now we can only get on with things, and improve the methods later, hopefully it won't be too late by then.

The Apprentice thing is weird, so yeah, maybe the way the dump has to have a data track before it will boot is wrong, however if you just convert without one, or burn it 'as-is' apparently it doesn't work in a real system anyway? If it is a funky format then it's something that will need further investigation. There is also a dump of 'Lucky Luke' which is similar, although that one apparently requires the digitial vid cart anyway. I'm sure given time the answer will become clearer, and eventually be properly documented and supported. This if anything is another reason why this project is important.

I was atually looking at running some software through my hacked -romident to create some lists, I was going to start with the NGP / NGPC driver, but it seems that the actual driver is broken, both the bios, and all games I've tried just give a white screen?
Posted By: judge Re: Fixed software lists - 01/11/10 06:47 PM
You need to switch the neogeo pocket on.
Posted By: Just Desserts Re: Fixed software lists - 01/11/10 06:47 PM
Originally Posted By Haze
it seems that the actual driver is broken, both the bios, and all games I've tried just give a white screen?


I'm going to ask a dumb question, but did you press the input button corresponding to the Power button?

Edit: Damn you judge
Posted By: Haze Re: Fixed software lists - 01/11/10 06:57 PM
Originally Posted By Just Desserts
Originally Posted By Haze
it seems that the actual driver is broken, both the bios, and all games I've tried just give a white screen?


I'm going to ask a dumb question, but did you press the input button corresponding to the Power button?

Edit: Damn you judge


Ahh.. I wouldn't exactly file that under 'obvious', even if it is pretty much a different take on the classic 'did you remember to plug it in?'

That works anyway.

I'll see if I can throw together a software list for that and the Jag.

*edit* Although I'm pretty surprised the Jaguar is considered working, it seems to run about 5% of games, and in most of those the sound doesn't work or there are significant glitches, and in most case they decide the resolution is something like 640x220 @ 32fps....

This software system is definitely going to need per title flags, because marking an entire driver as working just because one game works is senseless.
Posted By: Just Desserts Re: Fixed software lists - 01/11/10 07:45 PM
Originally Posted By Haze
*edit* Although I'm pretty surprised the Jaguar is considered working


It isn't, whoever marked it as working was being overly enthusiastic, and it should be downgraded back to Not Working.
Posted By: F1ReB4LL Re: Fixed software lists - 01/11/10 08:06 PM
Originally Posted By R. Belmont
I've said before - if the redump guys (or some other group) could provide an offset-corrected dumping service and release their dumps in some documented format I'd be happy to add the necessary support in MAME and CHDMAN. Until then, we do what we can.


Redump actually suffers from the same problems. Subchannels will be added sooner or later, though. Dumps are in descrambled form (scrambled dumps are used in some cases, descrambled with a software descrambler to achieve the needed accuracy), only user data is preserved (no first pregap, no lead-out sectors) -- yes, due to limitations of the available formats. All the needed methods and algorithms are known and described (like detecting the offset or GD-ROM dumping on PC drives), but we're in lack of coders, so I'm afraid no tool is going to be released soon. In the MESS' case, as I've said, a different chd format is needed, so if there's _someone_ able to do the coding part (there should be a specific tool with a special output, compatible with that 'different chd format') - we could discuss it.
Posted By: judge Re: Fixed software lists - 01/11/10 08:14 PM
Originally Posted By Haze

Ahh.. I wouldn't exactly file that under 'obvious', even if it is pretty much a different take on the classic 'did you remember to plug it in?'


Indeed, but it is how the system works. As soon as you plug in all the batteries the microcontroller is running and is waiting for that button to get pushed.

I thought I had that documented on the wiki (can't seem to find that now) ...
Posted By: Stiletto Re: Fixed software lists - 01/11/10 08:22 PM
I am completely willing to see yet another list about offset decoding, but I'm not sure the thread about fixed software lists is the appropriate venue for "things aren't being dumped properly" complaints. Can we fork it off to another thread perhaps?
Posted By: R. Belmont Re: Fixed software lists - 01/11/10 08:38 PM
I don't think a separate thread is necessary because it's easy to list exactly what the state of play is, and additional discussion won't help.

1) There are zero CD rips in existence that meet Fireball's standards.
2) Nobody is able to write a program that could create them.
3) No existing file format could contain them (although it would be trivial to add the necessary data to CHDs without breaking existing rips).
4) We're not CD super-experts nor do we particularly want to be. This is something that ripping groups like Redump/TOSEC/No-Intro should solve.

So, we'll use the rips we have that work for now, and if those other problems are solved we'll switch to those in the future.
Posted By: mizapf Re: Fixed software lists - 01/11/10 09:30 PM
Originally Posted By Micko

Agree that load should be simple as possible so I am for

MESS driver game

This could also make RPK deprecated, since you could define which file is loaded in which device


could you please choose some formulation which sound less terrifying to me? smile Some months ago I spent weeks with creating (just counted) 576 RPKs and reorganising the TI-99 software stock on our FTP server. The RPKs were not just a nice-to-have; we really needed them.

BTW, I think a command line like "MESS <driver> <game>" does not properly match the requirements I have here, where I want to be able to access several plugged-in cartridges. However, maybe I just did not get the idea of your proposal, so probably you could clarify.

Michael

Posted By: etabeta78 Re: Fixed software lists - 01/11/10 10:32 PM
yeah, for computers (and system with more than one slot) the list system needs definitely more thoughts. do not worry: rpk are not going to disappear [1] wink

[1] as I already mentioned, though, we might look into changing a little bit the code (mind, the code, not the format) so that it avoids some duplication currently needed by rpk files... I will write you an email when I have some more spare time, and maybe we can come up with a solution smile
Posted By: franciscohs Re: Fixed software lists - 01/11/10 11:59 PM
Originally Posted By mizapf
[quote=Micko]
BTW, I think a command line like "MESS <driver> <game>" does not properly match the requirements I have here, where I want to be able to access several plugged-in cartridges. However, maybe I just did not get the idea of your proposal, so probably you could clarify.

Michael



I don't know MESS THAT much, so maybe I'm making no sense here, but I think that command line should load whatever is needed for a certain system so that the software being loaded works as expected and additional cartridges, etc. can also be specified on the command line. This syntax has to check whether a certain cart is already in use by the software specified on the command line and warn the user about it.
Posted By: Haze Re: Fixed software lists - 01/12/10 01:27 AM
Originally Posted By franciscohs
Originally Posted By mizapf
[quote=Micko]
BTW, I think a command line like "MESS <driver> <game>" does not properly match the requirements I have here, where I want to be able to access several plugged-in cartridges. However, maybe I just did not get the idea of your proposal, so probably you could clarify.

Michael



I don't know MESS THAT much, so maybe I'm making no sense here, but I think that command line should load whatever is needed for a certain system so that the software being loaded works as expected and additional cartridges, etc. can also be specified on the command line. This syntax has to check whether a certain cart is already in use by the software specified on the command line and warn the user about it.


It's a long-standing problem in MAME too, the Megatech, Megaplay, PC-10, and NeoGeo drivers (at the very least) should all be able to load multiple cartridges.

MAME currently offers no sane way to specify this. Megatech is the worst because you can have a mix of SMS and genesis based games, although there are some disabled multi-game sets in the driver as proof of concept ;-)
Posted By: gigadeath Re: Fixed software lists - 01/12/10 02:58 AM
Here is an updated No-Intro MD dumps file

http://www.speedyshare.com/files/20287666/MD_Dumped_20091227_.zip

if anyone wants to have fun adding chip serials :P

Not every dump comes with its chip serial(s), because not every dumper wants/cares to open its carts (especially Japanese carts with their damn back stickers). Some of my dumps come without serial too because I sold those carts before opening them, many years ago.

But it helps to spot a few carts that contain 2 or more rom chips, and have to be redumped as a priority.
Posted By: Micko Re: Fixed software lists - 01/12/10 06:27 AM
Originally Posted By mizapf

could you please choose some formulation which sound less terrifying to me? smile Some months ago I spent weeks with creating (just counted) 576 RPKs and reorganising the TI-99 software stock on our FTP server. The RPKs were not just a nice-to-have; we really needed them.


Let me change formulation. It became deprecated in form that you do not need to make definition in RPK but you can define the same thing in "software definition list" in code itself. But that doesn't mean that we are going to rule out RPK, in fact same loading routine will probably be used to load from software list or RPK files.

Also would like to ask question to all that are more common with cart games.
Are there any cart games that require change of cart during game play (for any known system) ?
I am just looking to unify mechanism of handling images (carts, floppies, cassettes,...) inside software list so need to know what all we can expect.

Also good idea for thinking would using this sw lists to support "mappers" for nes for example, same as we have for TI carts, since some carts have different hardware inside, and right now it is all shown as part of NES machine and should be in separate device per cart type. Let me just say that I am not common with NES hardware, but it seams to me this is how it can work. But let's leave this to some other release when things settle down.
Posted By: etabeta78 Re: Fixed software lists - 01/12/10 07:45 AM
about NES: we would need to support separate files for PRG/CHR in place of the current .nes images (which glues things together)

I plan to implement these through software lists (I actually also started to work on it, but it will take quite a long time), and for these new images it makes sense to have per cart types as you suggest.

Afterwords, when the MESS driver will be able to handle the new images, we can see if it is worth to extend its use to old iNES files.

However, given the number of different cart types (and the amout of existent images), it is something that has to be introduced little by little wink
Posted By: JoJo Re: Fixed software lists - 01/12/10 08:54 AM
Originally Posted By Micko


Also would like to ask question to all that are more common with cart games.
Are there any cart games that require change of cart during game play (for any known system) ?
I am just looking to unify mechanism of handling images (carts, floppies, cassettes,...) inside software list so need to know what all we can expect.



I'm pretty sure that none of the old consoles/computers supported hotswapping... on the contrary I remember that those old manuals clearly stated that plugging/unplugging carts when the machine was powered on was a big no-no and led to all sort of disasters wink

Nevertheless, some systems supported multicart accessories, and MAYBE some of them have the necessary circuitry to allow hotplugging, therefore the possibility can't be ruled out.

Talking about corner cases and image loading restructuring, there are a couple of points that you might want to take into account: I stumbled upon them while collecting data for a ZX Spectrum fixed list.

1) Some carts/images might require a fixed or minimum quantity of RAM: it would be interesting to provide a directive to specify it in the SOFTWARE_START section. In case of multiple RAM sizes, the existing commandline switch selects the quantity to use.

2) Many carts of the "Action Replay" variety have a pushbutton or dipswitch to configure and operate them: maybe the proper solution to handle them is to define an ad-hoc PCB as in CoCo/TI99 drivers?

3) That's for a farther future: an ARTWORK directive to solve the Vectrex situation that needs a per-cartridge overlay.
Posted By: etabeta78 Re: Fixed software lists - 01/12/10 10:04 AM
about minimum ram: I think there should be some compatibility check at loading and if there is not enough RAM the driver should give a warning message, with the user in charge of selecting the proper ramsize.


other few possible cases to take into account while designing the loading routines:

- take systems like the amstrad+megadrive computers. In theory we should add to them two separate lists: one for the cpc carts/programs and one for the megadrive carts... but what if two entries have the same shortname?

- how to load pcecd games? a tempting solution would be to associate a shortname, say akumajo to both load the cd cart and the chd... but what if the user wants to use a different version of the cd cart? I think we should allow the use to load them separately

mess pce -cart cdcart1 -cdrom akumajo

while requiring some more typing, this kind of solution offers full flexibility...

- systems like megadrive or sms, would require a per-cart init (to avoid filling up the loading routines with strcmp(cart, shortname) ). should we handle these adding a sort of SOFT_INIT similar to DRIVER_INIT, or shall we store some specific setting in a xml file to be included in the ROM_LOAD (quite similarly to default eeprom in MAME > 0.135u4)?

just my 2c


EDIT: also, I am curious about cart dips as well, since some NES pirates has them...
Posted By: Micko Re: Fixed software lists - 01/12/10 10:23 AM
About solution for pcecd and other systems needing few things attached in same time. My idea from start was that software list should be some kind of extension of image mounting mechanism.
So you will in software list define which file should be loaded on what image device.

Example :
SOFTWARE_START( igmks1 )
IMAGE_MOUNT("cart1")
ROM_LOAD( "yrm0442m-184s", 0x00000000, 0x00080000, CRC(32f9c38a) SHA1(1be82afcdf5e2d1a0e873fda4161e663d7a53a85) )
IMAGE_MOUNT("flop1")
IMAGE_LOAD( "test.chd", 0x00000000, 0x00080000, CRC(32f9c38a) SHA1(1be82afcdf5e2d1a0e873fda4161e663d7a53a85) )
SOFTWARE_END

Posted By: judge Re: Fixed software lists - 01/12/10 10:25 AM
Or the pce cd device could be changed to check if there's a "pce cd controller cart" inserted in the system and bail out if that's not the case.
Posted By: etabeta78 Re: Fixed software lists - 01/12/10 10:27 AM
Originally Posted By Micko
About solution for pcecd and other systems needing few things attached in same time. My idea from start was that software list should be some kind of extension of image mounting mechanism.
So you will in software list define which file should be loaded on what image device.

Example :
SOFTWARE_START( igmks1 )
IMAGE_MOUNT("cart1")
ROM_LOAD( "yrm0442m-184s", 0x00000000, 0x00080000, CRC(32f9c38a) SHA1(1be82afcdf5e2d1a0e873fda4161e663d7a53a85) )
IMAGE_MOUNT("flop1")
IMAGE_LOAD( "test.chd", 0x00000000, 0x00080000, CRC(32f9c38a) SHA1(1be82afcdf5e2d1a0e873fda4161e663d7a53a85) )
SOFTWARE_END



but there exist multiple versions of the cart that you can load... why should we decide which one the user is forced to use?!? wink
Posted By: Duke Re: Fixed software lists - 01/12/10 10:28 AM
Yes, I think we should go with "mess pce -cart cdcart1 -cdrom akumajo".
Posted By: Micko Re: Fixed software lists - 01/12/10 10:33 AM
User should not be limited, but software list should be predefined so if you use:

mess pce akumajo

It will load cd rom with default placed cd rom cart (that we think do the best job for that game)

but user can override, so

mess pce akumajo -cart cdcart2

will do the same thing with overriden cart settings

last we will support free image mounting as we have it now

For me idea of software list is making a documented list of available software for some platform, but also a way non-developers can start using MESS without much knowledge of system itself.
Posted By: judge Re: Fixed software lists - 01/12/10 10:42 AM
Let's keep it simple and make it actually work throughout the system before expanding.
Posted By: etabeta78 Re: Fixed software lists - 01/12/10 11:00 AM
Originally Posted By Micko
User should not be limited, but software list should be predefined so if you use:

mess pce akumajo

It will load cd rom with default placed cd rom cart (that we think do the best job for that game)


I thought about this but if you have a system with a same game released on both a cart, a cassette and a floppy, which version should mess choose to load?

option1: you have non-conflicting shortnames among the different lists. while possible, when you have thousands of games (e.g. c64 or zx) it makes the creation of lists quite a nightmare... when you add cross compatibilities among systems (e.g. zx games on samcoupe comes to my mind), the whole scenario is really hard to handle

option2: you require the user to add a small effort to type the media type he wants and mess pick up the correspondent list... (not to mention that half of the users would use a frontend in any case) wink
Posted By: JoJo Re: Fixed software lists - 01/12/10 11:04 AM
Originally Posted By Micko
User should not be limited, but software list should be predefined so if you use:

mess pce akumajo

It will load cd rom with default placed cd rom cart (that we think do the best job for that game)

but user can override, so

mess pce akumajo -cart cdcart2

will do the same thing with overriden cart settings

last we will support free image mounting as we have it now



I find it very intuitive this way - moreover the "compatible with" and "parent of" fields allow for validation of available combinations; if the user really wants to try bogus/illegal pairings, he's free to do it using the present mechanism.
Posted By: Haze Re: Fixed software lists - 01/12/10 11:27 AM
Originally Posted By JoJo
Originally Posted By Micko
User should not be limited, but software list should be predefined so if you use:

mess pce akumajo

It will load cd rom with default placed cd rom cart (that we think do the best job for that game)

but user can override, so

mess pce akumajo -cart cdcart2

will do the same thing with overriden cart settings

last we will support free image mounting as we have it now



I find it very intuitive this way - moreover the "compatible with" and "parent of" fields allow for validation of available combinations; if the user really wants to try bogus/illegal pairings, he's free to do it using the present mechanism.


Yes, this makes the most sense. In it's most basic form launching things should be simple, most of the work should be done for you behind the scenes, and you shouldn't have to remember additional command line switches for basic use.

One of the problems right now with MESS is that it's far too finnicky about what you have to specify on the command line, and where the files need to go. For making the NGPC / Jag lists I actually gave up using MESS (again) because I couldn't find what syntax / file locations it wanted in order to simply load the games. Converting the entire driver over to work in MAME for testing purposes, even with the current updates was easier.

It seems to go against what a lot of software engineers think (especially open source projects) but KEEP IT SIMPLE. Having extended functionality is fine, but use of it shouldn't be required.
Posted By: etabeta78 Re: Fixed software lists - 01/12/10 11:43 AM
let me open a parenthesis here:

Originally Posted By Haze
One of the problems right now with MESS is that it's far too finnicky about what you have to specify on the command line, and where the files need to go. For making the NGPC / Jag lists I actually gave up using MESS (again) because I couldn't find what syntax / file locations it wanted in order to simply load the games. Converting the entire driver over to work in MAME for testing purposes, even with the current updates was easier.


erm... mess is not that hard:

Code:
mess system -device full_path_to_soft


where is the difficulty? (exclude for a moment the ngp requirement of pressing a button to actually start emulation)

since I have written the user manual (in the wiki), I'm quite curious to know what you were uneasy with, so that I can offer tips in the manual for other which could have the same problem

Originally Posted By Haze
It seems to go against what a lot of software engineers think (especially open source projects) but KEEP IT SIMPLE. Having extended functionality is fine, but use of it shouldn't be required.


I'm not so sure it's easier for user to remove the device from command line. say you have launch

Code:
mess c64 renegade


and you have defined renegade to load the floppy version. fine, at start you have to type DLOAD

now say that for another game rainbow there is only the tape version. launching

Code:
mess c64 rainbow


you come up with a screen where you need to type now LOAD!!! how to explain this to average users? and should they learn all the shortnames to be able to know what to do after the emulation started? wouldn't it be better to say them: ok, if you launch

Code:
mess c64 -cass xxxx


you always have to type LOAD, because it is a tape; if you lauch

Code:
mess c64 -flop xxxx


you always have to type DLOAD, because it is a floppy


you know, just because consoles make everything easier we should not forget that MESS emulates more complex systems as well wink
Posted By: Haze Re: Fixed software lists - 01/12/10 11:54 AM
Originally Posted By etabeta78
let me open a parenthesis here:

Originally Posted By Haze
One of the problems right now with MESS is that it's far too finnicky about what you have to specify on the command line, and where the files need to go. For making the NGPC / Jag lists I actually gave up using MESS (again) because I couldn't find what syntax / file locations it wanted in order to simply load the games. Converting the entire driver over to work in MAME for testing purposes, even with the current updates was easier.


erm... mess is not that hard:

Code:
mess system -device full_path_to_soft


where is the difficulty? (exclude for a moment the ngp requirement of pressing a button to actually start emulation)


It's non-obvious, and if you get it wrong, there are no prompts to tell you what you're meant to be doing.

MAME is easy. I just have to remember an 8-letter game name, if I get it wrong, I'm given other suggestions. I can't go wrong.

One of the big advantages of the internal database system is that you can get rid of this pain. Just remember 2 names, the system, and the game short-name. Leave the roms in anywhere in any recognized ROM path, and let MESS take care of the rest.

MESS just seems to have fallen into the typical open-source trap of the programmers knowing how things work, and therefore leaving it completely non-obvious to anybody else.

As for the loading once in the driver, as I've said before, MESS *should* provide some internal hint system for this, an additional screen on startup, and something that can be called up from the tab menu, or even a 'quickstart' option in the tab menu to type it for you. It might not seem pure, but the pure MAME approach really doesn't work as well here. Standalone emus offer such functionality, and it works really well.

Accessibility is vital or the project will just sink in the mud, as it has been doing for years. It's the reason so many open source projects are completely unusable.

I'm pretty sure if firefox required you to manually type each HTTP request it wouldn't have ended up being so popular ;-) Projects like LAME work because if you just give it the name of a wav, it will create an MP3, maybe not with optimal settings, but it will create one. Too many others just leave you thinking 'what the f**k' and no matter how good they are they end up being ignored because it's not obvious how to use them. Open Source games are just as bad, Simutrans should be fun, I like that type of game, but again the accessibility isn't just there, the learning curve is too steep, it makes no sense.




Posted By: Duke Re: Fixed software lists - 01/12/10 12:05 PM
"mess -help"

...
Usage: MESS <system> <device> <software> <options>
...
MESS -listdevices for a full list of supported devices
...

(ok, so this needs to be renamed to -listmedia because of recent changes, but its still useful)
Posted By: JoJo Re: Fixed software lists - 01/12/10 12:08 PM
Originally Posted By Haze


As for the loading once in the driver, as I've said before, MESS *should* provide some internal hint system for this, an additional screen on startup, and something that can be called up from the tab menu, or even a 'quickstart' option in the tab menu to type it for you. It might not seem pure, but the pure MAME approach really doesn't work as well here. Standalone emus offer such functionality, and it works really well.



There already is an "Image Information" option in the TAB menu: it can be a little bit more informative (at the moment is pretty bare...) and displayed at startup.

The next step is reading the excellent system instructions on the wiki: if I'm trying to load a Spectrum tape (mind you, not a cartridge!), it's my duty to know that LOAD"" is typed using the key sequence J-P-P wink
Posted By: Haze Re: Fixed software lists - 01/12/10 12:12 PM
Originally Posted By Duke
"mess -help"

...
Usage: MESS <system> <device> <software> <options>
...
MESS -listdevices for a full list of supported devices
...

(ok, so this needs to be renamed to -listmedia because of recent changes, but its still useful)


It's still not obvious. I don't know what <device> is, eta puts an example of -device ? but that would also imply that <system> is -system .. or does it mean types of devices? is -chd a device? or -hdd or -gdrom ? or ....

I'm not an idiot, but the system doesn't make much sense to me. It never has. Therefore I believe it to be broken, and illogical. For basic use it's a switch too far, and one which with a database system in place doesn't have any reason to exist unless you want to load external files, or override some internal software setting.


Posted By: Haze Re: Fixed software lists - 01/12/10 12:15 PM
Originally Posted By JoJo
Originally Posted By Haze


As for the loading once in the driver, as I've said before, MESS *should* provide some internal hint system for this, an additional screen on startup, and something that can be called up from the tab menu, or even a 'quickstart' option in the tab menu to type it for you. It might not seem pure, but the pure MAME approach really doesn't work as well here. Standalone emus offer such functionality, and it works really well.



There already is an "Image Information" option in the TAB menu: it can be a little bit more informative (at the moment is pretty bare...) and displayed at startup.

The next step is reading the excellent system instructions on the wiki: if I'm trying to load a Spectrum tape (mind you, not a cartridge!), it's my duty to know that LOAD"" is typed using the key sequence J-P-P wink


Well yes, I'll admit, I know the spectrum loading things myself, I owned a spectrum. Having MESS display a reminder of that on startup wouldn't kill anybody, and for games which do require slightly more obscure startup procedures, it would help too (some spectrum games are LOAD"" CODE or whatever). Not much point in documenting things if nobody can use them...

If I didn't know the spectrum I'd be lost without that, and the Wiki isn't always accessible to me when the executable is.
Posted By: Duke Re: Fixed software lists - 01/12/10 12:20 PM
Originally Posted By Haze

It's still not obvious. I don't know what <device> is, eta puts an example of -device ? but that would also imply that <system> is -system .. or does it mean types of devices? is -chd a device? or -hdd or -gdrom ? or ....


"mess -listdevices" tells you (or -listmedia, which would be better).
Posted By: Haze Re: Fixed software lists - 01/12/10 12:37 PM
Originally Posted By Duke
Originally Posted By Haze

It's still not obvious. I don't know what <device> is, eta puts an example of -device ? but that would also imply that <system> is -system .. or does it mean types of devices? is -chd a device? or -hdd or -gdrom ? or ....


"mess -listdevices" tells you (or -listmedia, which would be better).


But the point remains. I shouldn't have to care unless I actually want to do more advanced things, as was given in the example here (using a different cart than the default specified one with the PCE CD games)

Things should 'just work' by default, they don't.

I guess the 'software' field should represent something of a default configuration, in the case of consoles, that means a specific cart mounted, in the case of computers, an installation of the OS and specific disks available to insert, or appropriate system config and tape ready to load. Almost a 'quick start' setting if you will.

Advanced use should still be possible, the user should be able to bring up a list of discs / cds associated with the 'software' they chose, or call up a master list of all the known software that they have which is appropriate for that system. It should be possible to override things in the software configuration from the commandline if they really want to.

Those things should be optional tho, they shouldn't be a requirement. In it's most basic form it should just work, it should be accessible, useable, and easy to test with, without having to have a deep knowledge of how MESS works (if you're used to MAME) or even how the systems work (as prompts should be provided)

Obviously for now getting the lists in is a priority, getting something that works, and can be refined, and helps document the software on the systems is almost a separate issue than the above; once the database is there it can be refactored, exported, and reworked very easily using batch processing tools.

Posted By: etabeta78 Re: Fixed software lists - 01/12/10 12:48 PM
lack of clear errors/warnings pointing out what is wrong is indeed an issue. but this should be fixed separately, hence let me leave it for another discussion...

Originally Posted By Haze
One of the big advantages of the internal database system is that you can get rid of this pain. Just remember 2 names, the system, and the game short-name. Leave the roms in anywhere in any recognized ROM path, and let MESS take care of the rest.

MESS just seems to have fallen into the typical open-source trap of the programmers knowing how things work, and therefore leaving it completely non-obvious to anybody else.


I have written a quite comprehensive user manual exactly to avoid this, but not many users seem to read it (and I'm not referring to you, but to the number of requests about unzipped bioses...)

Originally Posted By Haze
As for the loading once in the driver, as I've said before, MESS *should* provide some internal hint system for this, an additional screen on startup, and something that can be called up from the tab menu, or even a 'quickstart' option in the tab menu to type it for you. It might not seem pure, but the pure MAME approach really doesn't work as well here. Standalone emus offer such functionality, and it works really well.


except that the real system had no such a functionality, so it's a big NO imho.

the user should be pointed to sysinfo.dat (which is displayed in the GUI or readable with text editor) for the instruction about how to load.

this was not my point though! My question is: how should we explain to the user that different shortname would require different commands. i.e. that

mess c64 renegade

requires DLOAD (assuming it is a floppy) while

mess c64 bublbobl

requires LOAD? Asking for "mess c64 -cass bublbobl" already warns the user that he will need to type LOAD and not DLOAD, even if he has never tried that image before! without "cass", the user would have to load a file without knowing what to do next even if he is an expert of the usage of that system, and he would probably lose more time trying out the various commands than the time typing -cass/-flop would have required...

or shall pretend the user to learn the correspondence between the shortname and the correspondent cassette/floppy/cart command?

Originally Posted By Haze
Accessibility is vital or the project will just sink in the mud, as it has been doing for years.


that's because there were only 3-4 devs and, hence, most drivers were completely abandoned, not for the lack of accessibility.

while I agree with some of your points, and I hope we can introduce better error messages to help people launching the exe with the wrong options, I still fear that fopr computer systems with multiple floppy drive & cartslots the usage

mess computer game

is a bit too restrictive and would risk to become even less intuitive than the current approach
Posted By: Haze Re: Fixed software lists - 01/12/10 12:58 PM
Originally Posted By etabeta78

mess c64 renegade

require DLOAD while

mess c64 bublbobl

requires LOAD? How would display the instruction which differs between one item of list and another one? Shall we put in the source each possible case? or shall pretend the user to learn the correspondence between the shortname and the correspondent cassette/floppy/cart command?


Add an 'additional information' field to the SOFTWARE / GAME macros which can point to a useability string. To be quite honest MAME needs this too with the gambling games. Tafoid's instructions are useful, but again are hidden away in a wiki / dat file somewhere, when really they should be more closely tied to the emulator if possible. 'Everything in one place' is a big time saver.

Originally Posted By etabeta78

I have written a quite comprehensive user manual exactly to avoid this, but not many users seem to read it (and I'm not referring to you, but to the number of requests about unzipped bioses...)


'RTFM' isn't exactly user friendly, and often manuals are so bad they're useless. I'm not saying yours is, but it's often the case, so they usually get ignored, or not updated properly. Placing the information where it can be seen is more useful.


Originally Posted By etabeta78

except that the real system had no such a functionality, so it's a big NO imho.


real systems don't have tile-viewers, save states, integrated debuggers etc. and like those features it's a time saver for anybody working on the driver. What's more valuable use of my time? trying to figure out how some obscure system boots, or fixing a graphical bug somebody reported in it?

Originally Posted By etabeta78

that's because there were only 3-4 devs and, hence, most drivers were completely abandoned, not for the lack of accessibility.


poor acessibility might explain the lack of devs too, that and the lack of software lists, which means that picking up somebody else's driver is hard, because you don't really know the status of anything (or what the status should be) or what's actually dumped, or what to fix. Thankfully that is changing.

Originally Posted By etabeta78

mess computer game

is a bit too restrictive and would risk to become even less intuitive that the current approach


with the appropriate hints it provides a good 'quick start' mechanism tho which will work in the majority of cases. For the more advanced ones, additional options can be used.
Posted By: etabeta78 Re: Fixed software lists - 01/12/10 01:00 PM
Originally Posted By Haze

But the point remains. I shouldn't have to care unless I actually want to do more advanced things, as was given in the example here (using a different cart than the default specified one with the PCE CD games)

Things should 'just work' by default, they don't.


again: this could make perfectly sense for consoles, less for other systems.

requiring the user to try -listmedia before running a new system, is not different than requiring a MAME user to know that he cannot launch

mame Street Fighter 2

wink

also, options for the devices are always the same (-cart, -cdrom, -cass, -cass1, -cass2, -flop, -flop1, etc.) and you know what to try after you have used the first 2/3 machines...

Once more, I agree with you about keeping things as simple as possible, but this should not make computer completely too hard to work with


EDIT: Last point, let me state that any quickstart help which makes e.g. cassette not to require loading or something similar would be a HUGE backward step... if you want to save time to load cassette, you simply press Insert and you will speed up the loading. or shall we stop documenting the correct way in which the CPU reads tapes? wink
Posted By: Haze Re: Fixed software lists - 01/12/10 01:03 PM
Originally Posted By etabeta78
Originally Posted By Haze

But the point remains. I shouldn't have to care unless I actually want to do more advanced things, as was given in the example here (using a different cart than the default specified one with the PCE CD games)

Things should 'just work' by default, they don't.


again: this could make perfectly sense for consoles, less for other systems.

requiring the user to try -listmedia before running a new system, is not different than requiring a MAME user to know that he cannot launch

mame Street Fighter 2

wink

also, options for the devices are always the same (-cart, -cdrom, -cass, -cass1, -cass2, -flop, -flop1, etc.) and you know what to try after you have used the first 2/3 machines...

Once more, I agree with you about keeping things as simple as possible, but this should not make computer completely too hard to work with


I'm not proposing anything is made harder to work with, the existing functionality will still be there. The functionality I'm proposing just gives an alternate, easier, option which will work fine in a worthwhile number of cases.
Posted By: etabeta78 Re: Fixed software lists - 01/12/10 01:05 PM
Originally Posted By Haze
I'm not proposing anything is made harder to work with, the existing functionality will still be there. The functionality I'm proposing just gives an alternate, easier, option which will work fine in a worthwhile number of cases.


I know. I'm just pointing out that some systems would not work well with such a simplification and maybe it is better to be aware of it while planning the functionalities, than finding out about it at a later stage wink
Posted By: Haze Re: Fixed software lists - 01/12/10 01:09 PM
Originally Posted By etabeta78
EDIT: Last point, let me state that any quickstart help which makes e.g. cassette not to require loading or something similar would be a HUGE backward step... if you want to save time to load cassette, you simply press Insert and you will speed up the loading. or shall we stop documenting the correct way in which the CPU reads tapes? wink)


That's not what I mean by quickstart anyway, what I mean is that it will type the loading string for you, and start the loading if you select an appropriate option from the tab menu. A bit like an internal 'inp' if you see what I mean? Obviously it can't cover all cases, so just having the 'associated information' screen might be better, but I've seen the feature as mentioned in other emus and it worked pretty well.
Posted By: Dr.Zer0 Re: Fixed software lists - 01/12/10 01:10 PM
What about a "priority" system...
For example:
I have an Game called "dummyGame" relased in 3 format CD/Floppy/Cart
I can define a "media priority" like CD > Floppy > Cart
And then I type:
mess dummySys dummyGame
the emulator will run the CD version

Of course I can even create an override system, where if i type:
mess dummySys dummyGame -cart
mess will run the Cart version, instead of the CD one

My 2c
Sorry, my English is really bad
Posted By: Haze Re: Fixed software lists - 01/12/10 01:14 PM
Originally Posted By Dr.Zer0
What about a "priority" system...
For example:
I have an Game called "dummyGame" relased in 3 format CD/Floppy/Cart
I can define a "media priority" like CD > Floppy > Cart
And then I type:
mess dummySys dummyGame
the emulator will run the CD version

Of course I can even create an override system, where if i type:
mess dummySys dummyGame -cart
mess will run the Cart version, instead of the CD one

My 2c
Sorry, my English is really bad


It's just confusing, and wouldn't mix well if you wanted a system to override things from the command line.

Ideally the goal of all of this should be to have a system which works 'out of the box' for most cases (all cases is quite frankly impossible). A system whereby a site like Mametesters can be setup, and in addition to the 'setname' field (which would be software) there would be a 'system' field. Having different setnames for the Floppy / CD / Pre-installed HDD versions of games would help a lot with that. The same for creating a MAWS-like site.
Posted By: etabeta78 Re: Fixed software lists - 01/12/10 01:32 PM
Now that I better understand you're "autoloading" idea, I have less objections (even if I personally dislike it). Implementation could be really though in some case: systems like c64 or c16 just get ready when you type LOAD. afterwords, they wait for the tape to be manually started.

I don't think it would be easy to implement it. moreover, such a code could possibly be hacked to keep/use credits in bootleg MAME cabinets and this would be a good reason not to add it.


also, to play devil's advocate, let me point out that a Mametesters database and/or a MAWS one would very well be possible also when you have to specify the medium at commandline:

the bug/maws database would look like
Code:
MESS
> system1
      > cass
            > game1, game2, etc
      > flop
            > game1, game2, etc
      > cart
            > game1, game2, etc
> system2
      > cass
            > game1, game2, etc
      > cdrom
            > game1, game2, etc


with a 3rd subfield in addition to the 2 you suggested. wink
Posted By: JoJo Re: Fixed software lists - 01/12/10 01:38 PM
Guys, you're over-engineering it: why did both of you ruled out the possibility to have disk, cassette and cartridge shortname versions of e.g. Rainbow Islands named rbisland_f, rbisland_t and rbisland_c?

A short help text, driver specific is a simple improvement that solves 95% of the issues: what about e.g. for Commodores:

------
Use LOAD"pgm" to load from tape
LOAD"pgm",8 to load from disk
LOAD"$",8 displays disk directory (but erase BASIC programs!)
If the program is in machine code try adding ",1". Use RUN for BASIC programs and SYS <startaddr> for machine code programs.
------

Stick to the end the web address of the Wiki, and be done with it...

If the user still complains, tell him that this is not iMESS wink
Posted By: Haze Re: Fixed software lists - 01/12/10 01:46 PM
Originally Posted By etabeta78
I don't think it would be easy to implement it. moreover, such a code could possibly be hacked to keep/use credits in bootleg MAME cabinets and this would be a good reason not to add it.


Off-topic, but the save state system has already been hacked to do that. I was sent the files from one of the more modern bootlegs, they have save states for every (supported) game, made on the title screen. When a game is selected they copy + edit their default save state file in the front-end before loading, then use the auto-load functionality of MAME to boot straight to the title screen with X credits inserted. You're actually helping them by adding all those save states, but, that's life..

They also use the auto-save function, and upon returning to the frontend they extract the credit byte from the save state made when you quit before deleting the temporary file and returning the remaining credits to the front-end.

It even works 'out the box' with an unmodified copy of MAME as long as you have the appropriate save states, and address where the credits are stored for each game. (of course, they don't use stock MAME, because they get rid of all the warning screens etc. first, but it does work with it) The cheat.xml gives pretty good hints where the credits are stored too as many games have 'infinite credit' cheats.

Why do you think people were _really_ annoyed that the Namco states severely messed up the credits? I have it on good authority that the bootleggers were pretty pissed off about those problems because the Namco games are a key part of any multi-game bootleg. Sad to say, but the bootleggers love your work just as much, if not more than anybody else and I know how crap that feels from working on CPS3.




Posted By: Haze Re: Fixed software lists - 01/12/10 01:50 PM
Originally Posted By JoJo
Guys, you're over-engineering it: why did both of you ruled out the possibility to have disk, cassette and cartridge shortname versions of e.g. Rainbow Islands named rbisland_f, rbisland_t and rbisland_c?


That's effectively what I'm suggesting.

Originally Posted By JoJo

A short help text, driver specific is a simple improvement that solves 95% of the issues: what about e.g. for Commodores:

------
Use LOAD"pgm" to load from tape
LOAD"pgm",8 to load from disk
LOAD"$",8 displays disk directory (but erase BASIC programs!)
If the program is in machine code try adding ",1". Use RUN for BASIC programs and SYS <startaddr> for machine code programs.
------

Stick to the end the web address of the Wiki, and be done with it...

If the user still complains, tell him that this is not iMESS wink



Similar to what I'm suggesting, except I'm saying we could have a couple of predefined statements in the driver, and associate each piece of software with one. That way if a specific game really does require something weird it can be displayed (or if the user needs to be warned that they REALLY need the manual / protection wheel for a set they're trying to boot etc.). This would be also useful in MAME for the gambling games with odd init sequecnes too, if you didn't know better you'd think Super Gran Safari was broken.

Posted By: JoJo Re: Fixed software lists - 01/12/10 02:14 PM
Originally Posted By Haze

Similar to what I'm suggesting, except I'm saying we could have a couple of predefined statements in the driver, and associate each piece of software with one. That way if a specific game really does require something weird it can be displayed (or if the user needs to be warned that they REALLY need the manual / protection wheel for a set they're trying to boot etc.). This would be also useful in MAME for the gambling games with odd init sequecnes too, if you didn't know better you'd think Super Gran Safari was broken.



I see - in this case, we should push the issue to Aaron - there's no point in doing it just for MESS benefit...
Posted By: franciscohs Re: Fixed software lists - 01/12/10 02:52 PM
Originally Posted By JoJo
Guys, you're over-engineering it: why did both of you ruled out the possibility to have disk, cassette and cartridge shortname versions of e.g. Rainbow Islands named rbisland_f, rbisland_t and rbisland_c?


I think this is the way to go, the long descriptions to each of these could be:

Rainbow Islands (Tape version)
Rainbow Islands (Cart Version)
etc.

One of them could be made the parent and the rest are clones. This is self explanatory as to which version you're loading so you would know if you have to use "LOAD" or "DLOAD".

I even think that maybe for systems with software in different type of media, all software should say what's the media they use in the long description, even if there is only one version. In this way you always know what you're loading, is a good way to document on what kind of media certain software was released and also if you wish to use the "advanced" commands "mess system -cass xxxx" you know what rom names you can use when using "-cass".
Posted By: Skito Re: Fixed software lists - 01/12/10 03:48 PM
If mess does move to a

mess system software

command line shortcut, someone(s) will have to map current software to 8.3 name(s) for all systems. Even mapping 8.3 games to potentially multiple softwares loaded on multiple devices, so there will need to be an xml like mapping. Should we start doing that? and how?
Posted By: etabeta78 Re: Fixed software lists - 01/12/10 03:55 PM
1. we have already started, take a look to the software/ directory in the source

2. 8.3 only holds for parents, clones can have longer names

3. if we decide to append _c, _f, _t to the files depending on the media, a possible approach would be to have 8char long names and the suffix to be added by the core: individual list would not contain the _x suffix but the suffix could be added by the core depending on the associated media...

EDIT: thinking back, maybe the approach in 3. would not work well with rebuilding tools and compatible systems... I fear we'll have to clog up the lists with media suffix smirk
Posted By: franciscohs Re: Fixed software lists - 01/12/10 06:53 PM
Originally Posted By etabeta78

I fear we'll have to clog up the lists with media suffix smirk


I understand part of your discomfort, but in the end, it's two different versions of the same game. We have the same on MAME:

vf4evoct Virtua Fighter 4 Evolution (Cartridge)
vf4evoa Virtua Fighter 4 Evolution (Rev A) (GDS-0024A)
vf4evo Virtua Fighter 4 Evolution (Rev B) (GDS-0024B)

Posted By: etabeta78 Re: Fixed software lists - 01/12/10 08:10 PM
I know, but in that case it's a matter of a couple of clones... in this case we would have hundreds of _c / _f at the end...

well, it's probably a bit too early to worry about this: a lot of work has still to be done before system can load from the lists.
Posted By: franciscohs Re: Fixed software lists - 01/12/10 08:22 PM
I know, that's why I said I understand your discomfort... wink

Even so, I think that's the way to go.
Posted By: ranger_lennier Re: Fixed software lists - 01/12/10 09:36 PM
I noticed that the Supervision software list has some overdumps in it (denoted by [o1]). Are these actually any different from the good sets of the same games in there? They should probably be removed or repaired.
Posted By: etabeta78 Re: Fixed software lists - 01/12/10 10:06 PM
That list has still to be cleaned up (description might be inaccurate as well). that's what I meant with "Added preliminary lists for..." in the commit wink
Posted By: Haze Re: Fixed software lists - 01/13/10 12:41 AM
playing with the NGP lists, unlike the Jaguar it's really rather impressive, the only issue(s) I seem to have encountered so far is that the screen 'rolls' in SNK vs. Capcom - Card Fighters 2 - Expand Edition (Japan).ngc

and in Crush Roller something is flashed on the screen after you finish a level, but you don't really have chance to read it, or see what it is?

finally in Pachi-Slot Aruze Oukoku Pocket - Porcano 2 (Japan).ngc there is some flickering on the playfield all the time.

Other than those, there just seem to be some minor (raster?) timing glitches in some games, casusing flickering / jumpy scanlines between screen areas.

I'm using a the driver in MAME tho, which is why I haven't put anything on bugzilla. If somebody wants to confirm those and enter them be my guest.



Posted By: Darkstar Re: Fixed software lists - 01/13/10 05:32 AM
Instead of "fixed" suffices like "_f", "_c" which are essentially part of the ROM name, why not do it this way:
Let's say the game "Bubble Bobble" is called bublbobl. If I launch

mess c64 bublbobl

mess will automatically pick a sane default for me (and optionally print something like "multiple versions of bublbobl exist, using the DISK version" or similar)

If I want a specific version, I could use

mess c64 bublbobl(t)

which would be the same game name (i.e. not a different game name like bublbobl_f would be) but it would tell MESS to prefer the TAPE version. If there is no TAPE version, MESS should probably ignore the "(t)" at the end and launch the disk version anyway, after notifying the user.

Or some kind of selector could be provided, if I run "mess c64 bublbobl" it gives me all available bublbobl ROMs (disk, tape, cart, etc.) and I could select which one I want or simply press "enter" to use the default (after which mess tells me that I can skip this step if I launch the game with "mess c64 bublbobl(f)" next time)

I admit it's not really much better than "mess c64 -cart bublbobl" vs. "mess c64 -flop bublbobl" but it's just another idea.

-Darkstar
Posted By: Donny Re: Fixed software lists - 01/13/10 06:31 AM
would it be taboo to have pseudonyms for each system: c64tape, c64disk, c64cart? each with its own software list?
Posted By: Just Desserts Re: Fixed software lists - 01/13/10 07:06 AM
Originally Posted By Donny
would it be taboo to have pseudonyms for each system: c64tape, c64disk, c64cart? each with its own software list?


And how do you propose to handle combinations of cartridges, tapes, and disks?
Posted By: etabeta78 Re: Fixed software lists - 01/13/10 07:19 AM
Originally Posted By Donny
would it be taboo to have pseudonyms for each system: c64tape, c64disk, c64cart? each with its own software list?


if you have to type

mess c64disk gamename

then you can simply use

mess c64 -disk gamename

wink
Posted By: Guru Re: Fixed software lists - 01/13/10 07:53 AM
Originally Posted By R. Belmont
Meanwhile the Dumping Union has democratized ROM dumping - there are half a dozen people now with Naomi serial dumping setups and f205v can handle most of the surface-mount crap that used to.........


haha. well if 'most' means ~20% then sure. oh, and just make sure you don't need it back fast. or working. or both grin
Posted By: Curt Coder Re: Fixed software lists - 01/13/10 08:32 AM
Originally Posted By etabeta78
Originally Posted By Donny
would it be taboo to have pseudonyms for each system: c64tape, c64disk, c64cart? each with its own software list?


if you have to type

mess c64disk gamename

then you can simply use

mess c64 -disk gamename

wink


Besides what percentage of our "end users" use the command line? I would think most would use the MESSGUI.
Posted By: ElBarto Re: Fixed software lists - 01/13/10 09:32 AM
Originally Posted By Curt Coder

Besides what percentage of our "end users" use the command line? I would think most would use the MESSGUI.


All sdlmess users I think.
Posted By: Curt Coder Re: Fixed software lists - 01/13/10 09:59 AM
Originally Posted By ElBarto
Originally Posted By Curt Coder

Besides what percentage of our "end users" use the command line? I would think most would use the MESSGUI.


All sdlmess users I think.


If you are using linux already, you should be able to figure out even the most cryptic command line options :P
Posted By: Haze Re: Fixed software lists - 01/13/10 10:38 AM
Originally Posted By Darkstar
Instead of "fixed" suffices like "_f", "_c" which are essentially part of the ROM name, why not do it this way:
Let's say the game "Bubble Bobble" is called bublbobl. If I launch

mess c64 bublbobl

mess will automatically pick a sane default for me (and optionally print something like "multiple versions of bublbobl exist, using the DISK version" or similar)

If I want a specific version, I could use

mess c64 bublbobl(t)

which would be the same game name (i.e. not a different game name like bublbobl_f would be) but it would tell MESS to prefer the TAPE version. If there is no TAPE version, MESS should probably ignore the "(t)" at the end and launch the disk version anyway, after notifying the user.

Or some kind of selector could be provided, if I run "mess c64 bublbobl" it gives me all available bublbobl ROMs (disk, tape, cart, etc.) and I could select which one I want or simply press "enter" to use the default (after which mess tells me that I can skip this step if I launch the game with "mess c64 bublbobl(f)" next time)

I admit it's not really much better than "mess c64 -cart bublbobl" vs. "mess c64 -flop bublbobl" but it's just another idea.

-Darkstar


And once again, I feel the need to state 'Keep It Simple'


As for the other message MESSGUI / end users, I think you're forgetting one important fact. The people using the code to develop with are also users, and the comment about linux users understanding over-complex command lines is one of the primary reasons linux will never be popular outside of small circles because this concept of 'Keep It Simple' is too often forgotten.
Posted By: etabeta78 Re: Fixed software lists - 01/13/10 12:05 PM
adding one of

-cart, -cass, -flop

between the system name and the gamename, does not sound over-complex to me. Just something you learn after the first 2 times you use it... as long as we don't require lots of options, I think the keep-it-simple part is satisfied.

EDIT: gamelists for Nintendo systems (gb, gbc, snes, n64 etc.) have been created as well, but due to their size (between 500k and 1mb each) won't be added to the source until lists are used in a meaningful way. I'm just stating this to avoid duplicate efforts.
Posted By: franciscohs Re: Fixed software lists - 01/13/10 01:12 PM
But you would have to know beforehand if a certain software is available in each of those medias to be able to load it.

Additionally, what would be mess output if I type:

mess system bubblebob

I rather have:

bublbobc Bubble Bobble (Cart Version)
bublbobt Bubble Bobble (Tape Version)


Than:

bublbobl Bubble Bobble (Use with -cart switch)
bublbobl Bubble Bobble (Use with -tape switch)


And, additionally, under what filenames would you store each of these versions of the game?, different game names ensures we can have different file names for each game in an unequivocal way.



BTW, I think it was already addressed, but how will versions of certain software for different systems be handled?

Will the main version of Bubble Bobble be named "bublbobl" for each of the systems?, in that case, how do you distinguish the filenames for each system?, different directories?


Also, will I be able to type:

mess bublbobl

and get something like:

"bublbobl" is available for the following systems

nes
sms
snes

type "mess <system> bublbobl" to load the game for the chosen system


Maybe is going too far on easiness, and I definitively wouldn't make any default here, but I think it could be an interesting option given that people are used to type "mame game".
Posted By: etabeta78 Re: Fixed software lists - 01/13/10 01:34 PM
Originally Posted By franciscohs
BTW, I think it was already addressed, but how will versions of certain software for different systems be handled?

Will the main version of Bubble Bobble be named "bublbobl" for each of the systems?, in that case, how do you distinguish the filenames for each system?, different directories?


yeah. you will need different directories for different systems. there is basically NO WAY to create different 8chars names without duplications between c64+zx+nes+snes+gba etc.

is way easier to use bublbobl for each Bubble Bobble game and store them in different directories. Especially because if you are only interested in a single system you can more easily remember the shortname wink


Originally Posted By franciscohs
Also, will I be able to type:

mess bublbobl

and get something like:

"bublbobl" is available for the following systems

nes
sms
snes

type "mess <system> bublbobl" to load the game for the chosen system


i don't think it's really feasable. what if bublbobl is a supported system (see e.g. mk2 chess computer vs. mortal kombat 2)? it is better that

mess gamename

only starts the required system or says that no such a system exists. also, browsing through all the lists at start to implement this option would kill performances
Posted By: Haze Re: Fixed software lists - 01/13/10 02:25 PM
Originally Posted By etabeta78
adding one of

-cart, -cass, -flop

between the system name and the gamename, does not sound over-complex to me. Just something you learn after the first 2 times you use it... as long as we don't require lots of options, I think the keep-it-simple part is satisfied.


but they're redundant.

I don't want to have to type -cart, -flop, -whatever unless I actually want to force a change from the default configuration, or I'm loading an image directly and *need* to tell MESS what type of image it is. Forcing people to learn and type redundant options is pointless, and confusing if they don't know anything about the system.

For basic use of Naomi in MAME you don't have to care that it uses a GDROM, MAME knows this based on the software you're trying to run, and takes care of it. Having to tell the computer something it should already know is another sign of bad design.



Quote:


EDIT: gamelists for Nintendo systems (gb, gbc, snes, n64 etc.) have been created as well, but due to their size (between 500k and 1mb each) won't be added to the source until lists are used in a meaningful way. I'm just stating this to avoid duplicate efforts.


You should submit them, the refinement process can begin now, even if the lists aren't used.
Posted By: Donny Re: Fixed software lists - 01/14/10 12:18 AM
IMHO (random ramblings):

trying to differentiate software by changing just the filename isn't a good idea.... the only way that I can see it working is by using a front-end.

When using c64 for example, we are talking about THOUSANDS of titles. I don't believe you can make enough readable/understandable unique names (especially confined by 8:3).

Using the 8th character as an input device specifier is undesirable as well because that eliminates the ability to use names/clones that happen to use that same character and also limits you even worse at 7:3).

Software lists for consoles is one thing, but for systems that have multiple input devices (cart, tape, disk) is TOTALLY different. I believe you need some kind of device type specified in order to get the proper software. I am a fan of the "-cart, -cass, -flop, -disk" usage syntax.

As for arguments about more complex situations requiring different devices in use at once, you just fallback to the complete mess commandline options.

Maybe a mechanism similar to the MAME way might work for cryptic names. Typing mess <system> <software> could list the approximate matches.

More adventurous could be mess <software>
Posted By: Haze Re: Fixed software lists - 01/14/10 01:50 AM
Originally Posted By Donny
IMHO (random ramblings):

trying to differentiate software by changing just the filename isn't a good idea.... the only way that I can see it working is by using a front-end.

When using c64 for example, we are talking about THOUSANDS of titles. I don't believe you can make enough readable/understandable unique names (especially confined by 8:3).

Using the 8th character as an input device specifier is undesirable as well because that eliminates the ability to use names/clones that happen to use that same character and also limits you even worse at 7:3).


There are no limits on clones. You could, if you want, have
'rainbow' (Rainbow Islands, Cassette Version)
'rainbowdisk' (Rainbow Islands, Disk Version)
'rainbowcart' (Rainbow Islands, Cart Version)

as the set names, and MAME / MESS would be fine with it.

the obvious advantage here is if you typed 'Mess c64 rainbowx' you would get both suggestions.

the '8 letters' rule only applies to parents.

When it comes to the Genesis, where basically everything was on cart, then having to specify '-cart' becomes pointless, and also, technically incorrect for the 'SegaNet' games, which act like carts anyway, but were actually downloadable (and should be supported in some form, because 'emulating' the original download service is just plain impossible because the servers don't exist anymore) Having a fixed list of 'device types' doesn't really aid the documentation at all, and just makes it less usable. As for thousands of sets, MAME already supports thousands of sets, but we always manage to find appropriate 8-letter names.

The whole point in having a database is that MESS will know such things, and the user shouldn't really have to be concerned over the command-line syntax.


Quote:

Software lists for consoles is one thing, but for systems that have multiple input devices (cart, tape, disk) is TOTALLY different. I believe you need some kind of device type specified in order to get the proper software. I am a fan of the "-cart, -cass, -flop, -disk" usage syntax.

As for arguments about more complex situations requiring different devices in use at once, you just fallback to the complete mess commandline options.

Maybe a mechanism similar to the MAME way might work for cryptic names. Typing mess <system> <software> could list the approximate matches.

More adventurous could be mess <software>
Posted By: etabeta78 Re: Fixed software lists - 01/14/10 06:42 AM
"unfortunately", as I said, MESS does not only support consoles, or the whole problem would have not been present and we could simply use your

mess system game

suggestion.

BUT we actually support computers, calculators, chess computers and they cannot be forgotten just because console don't have floppy drives nor printers

moreover, consoles also present situations where you would like to specify devices:

1* atari 2600 supercharger games use a cart + a tape, but I think can work with only the cart as well
2* snes sufami turbo is a cart with two slot: the first subcart is the game, the second optional one unlocks items in the main game (and you can put in the slot any other sufami cart)
3* n64 Mario no Photopie allows to plug SD card in the n64 cart to "upload" and edit your pictures
4* psx vib ribbon allows to swap the game CD with music CD to create new levels, and I think there was another digimon/pokemon style game which unlocked enemies depending on which CD you plugged into the console at runtime.

while 1 can easily be treated with default configuration /romsets which help the users, how to deal with the other ones?

3 could in principle be treated with the option overwriting mentioned by Micko and others, but not really easily. also it would mean that we should be able to mix softlist with general loading (so that you can start mess n64 -cart mariopp -sdmedia my_card)

4 would really represent a challenge (even if it is not strictly related to softlist)

finally, in sufami you would need a ROM_START with bios (the main sufami cart content) and the cart to be plugged into slot1; but then how to describe the options for slot2? it could possibly work with an overwrite at command line, but overwriting the one game with the other would also reload the BIOS? otherwise, we should add fake sets with all the possible combinations of images...

you see, just because these are corner cases, we should at least be aware of them and the whole implementation should not be so rigid that these cannot be supported at a later stage! and while I am definitely enthusiastic about software lists, I am also aware that we cannot just take too many shortcuts for the sake of a few simpler consoles...

if adding a -cart option to a console which only have carts is the only way to add support for home computers[1] as well, then it is a price I'm very happy to pay


[1] even if computers add a whole additional universe of strange corner cases, but those can be taken care little by little
Posted By: Guru Re: Fixed software lists - 01/14/10 09:11 AM
Originally Posted By etabeta78
EDIT: gamelists for Nintendo systems (gb, gbc, snes, n64 etc.) have been created as well, but due to their size (between 500k and 1mb each) won't be added to the source until lists are used in a meaningful way. I'm just stating this to avoid duplicate efforts.


that's like saying we shouldn't add games into MAME until we figure out how the hardware works and emulate it properly. It's kind of counter-productive to hold stuff back. The stuff needs to go in so it can be worked on.

I noticed Sega Game Gear carts are listed too (and many others).
Can we get some quality control enabled on these lists? The columns are more crooked than a dogs back leg ;-)
Nice evenly-spaced columns are much easier to read :-)
Posted By: etabeta78 Re: Fixed software lists - 01/14/10 10:26 AM
Originally Posted By Guru
Originally Posted By etabeta78
EDIT: gamelists for Nintendo systems (gb, gbc, snes, n64 etc.) have been created as well, but due to their size (between 500k and 1mb each) won't be added to the source until lists are used in a meaningful way. I'm just stating this to avoid duplicate efforts.


that's like saying we shouldn't add games into MAME until we figure out how the hardware works and emulate it properly. It's kind of counter-productive to hold stuff back. The stuff needs to go in so it can be worked on.

I noticed Sega Game Gear carts are listed too (and many others).
Can we get some quality control enabled on these lists? The columns are more crooked than a dogs back leg ;-)
Nice evenly-spaced columns are much easier to read :-)


I was referring to the code to validate lists, not the emulation of the systems. your comparison is not fully correct wink
fwiw, there are probably also bogus descriptions and parents with typos and other things to fix.

as soon as the lists get at least parsed/validated by the emu, I will start to fix mistakes. until then, I will keep preparing other lists.

if you want to improve them visually before then, feel free to send your changes and I'll happily apply them (notice that smaller lists have already been formatted appropriately)
Posted By: Haze Re: Fixed software lists - 01/14/10 11:26 AM
Originally Posted By etabeta78
"unfortunately", as I said, MESS does not only support consoles, or the whole problem would have not been present and we could simply use your

mess system game

suggestion.

BUT we actually support computers, calculators, chess computers and they cannot be forgotten just because console don't have floppy drives nor printers

moreover, consoles also present situations where you would like to specify devices:

1* atari 2600 supercharger games use a cart + a tape, but I think can work with only the cart as well
2* snes sufami turbo is a cart with two slot: the first subcart is the game, the second optional one unlocks items in the main game (and you can put in the slot any other sufami cart)
3* n64 Mario no Photopie allows to plug SD card in the n64 cart to "upload" and edit your pictures
4* psx vib ribbon allows to swap the game CD with music CD to create new levels, and I think there was another digimon/pokemon style game which unlocked enemies depending on which CD you plugged into the console at runtime.

while 1 can easily be treated with default configuration /romsets which help the users, how to deal with the other ones?

3 could in principle be treated with the option overwriting mentioned by Micko and others, but not really easily. also it would mean that we should be able to mix softlist with general loading (so that you can start mess n64 -cart mariopp -sdmedia my_card)

4 would really represent a challenge (even if it is not strictly related to softlist)

finally, in sufami you would need a ROM_START with bios (the main sufami cart content) and the cart to be plugged into slot1; but then how to describe the options for slot2? it could possibly work with an overwrite at command line, but overwriting the one game with the other would also reload the BIOS? otherwise, we should add fake sets with all the possible combinations of images...

you see, just because these are corner cases, we should at least be aware of them and the whole implementation should not be so rigid that these cannot be supported at a later stage! and while I am definitely enthusiastic about software lists, I am also aware that we cannot just take too many shortcuts for the sake of a few simpler consoles...

if adding a -cart option to a console which only have carts is the only way to add support for home computers[1] as well, then it is a price I'm very happy to pay


[1] even if computers add a whole additional universe of strange corner cases, but those can be taken care little by little


The other cases you list are what I would consider 'advanced use' cases, where yes, you would want to be able to use the extra switches. There will always be special cases (the 'lock-on mechansim of Sonic & Knuckles on the Megadrive is another, where you plug a cart into a cart.. not even a -cart switch really helps there ;-)

I guess in such cases 'MESS system software' would load the game, however, if you wanted a different cart in the 2nd cart slot you should be able to specify 'MESS system software -cart2 filename' to *directly* specify an additional cart to use.

of course it makes sense to have some configurations build into the software lists, eg. I'd make sure that the Sonic & Knuckles sets with Sonic 3 & Sonic 2 were built in as they were common / intended configurations (basic use), but the user could still use the additional parameter to attach a different cart to it if they wanted (advanced use).

for Vib Ribbon, obviously we can't create lists of every audio CD ever made, so again whatever you do it's going to need an advanced case of somehow letting the user specify an additional CD (you can't simply override the default one anyway because it's needed to boot the game')

in that case, -cd2 or so could be used to specify the 2nd cd directly if you want one, although things could start to get confusing if you have systems with 1/2 drives, at which point your switches need a rethink anyway.

As for the SD card thing, NeoGeo has similar in MAME, for transfering stuff between AES and MVS. Lots of modern games also have card slots for similar, a solution is needed anyway.

The 'MESS system software' syntax isn't imposing any additional limitations on the system, it just provides a base 'software' setup that the user should be able to further tweak with switches. As I've pointed out, there are limitations to your switch idea anyway which will need working around regardless of the method chosen, the basic syntax isn't changing this, the other problems will still need solving.

I'm aware that Computer games often use multiple discs etc. and so the software lists should be able to have multiple discs for each piece of software by default. The discs associated with the software should be displayed in the menu, the user can insert different ones from there (or a blank / save disc if they want?)

In terms of more advanced use, the user should be able to bring up the lists of discs associated with any piece of software at any time from the internal menus. Eg. if they are a advanced user, and want to get hold of all 12 Monkey Island 2 discs on the amiga, they should be allowed to type 'amiga, monkey2' in the internal menus (similar to the quick menu in MAME) and be presented with the appropriate discs, which they can then mount as they want. They should probably also be allowed to create custom lists which get saved out, and reloaded as part of their config for when they run that driver. Again for this to work, there is no need to specify -disk, MESS should know the media types that the user is presented with.

Again, as you can see, the basic idea is just a quick start system, the advanced use of the lists still needs consideration.

If you're going to start coming up with difficult cases just to make the plans sound unworkable, how about an amiga, with a cassette drive connected, running a spectrum emulator, where you want to mount a tape from the spectrum software library ;-) (I guess with my above suggestion the only real issue there is the hardware config one; how do you tell MESS your amiga has a tape drive? (& the upgraded cpu etc. needed for them to work well), otherwise you could bring up the spectrum software list with 'spectrum, manicm' and mount it .. MESS knows it's a tape, MESS knows you have a tape drive, hey.. could be fun ;p) Your switches as suggested offer no real advantages here, the only switches you would want would be to configure the actual hardware.


Posted By: Duke Re: Fixed software lists - 01/14/10 12:27 PM
This is all getting way to complicated. Why can't we just have lists that define "media" (a cartridge, a disk, a cdrom). To load a game then type:

mess sms -cart alexkidd
mess amiga -flop1 monkeyisland_d1 -flop2 monkeyisland_d2
mess pce -cart cdromsys -cdrom game

At least as a start.
Posted By: Haze Re: Fixed software lists - 01/14/10 01:44 PM
Originally Posted By Duke
This is all getting way to complicated. Why can't we just have lists that define "media" (a cartridge, a disk, a cdrom). To load a game then type:

mess sms -cart alexkidd
mess amiga -flop1 monkeyisland_d1 -flop2 monkeyisland_d2
mess pce -cart cdromsys -cdrom game

At least as a start.


Well I'm just looking for any justification at all for these 'cart' and 'flop1' etc. options for basic use. So far nobody has given a convincing argument for them. To me they're illogical, useless and make using MESS significantly harder and should be reserved for more advanced configuration setups, and will have to go away anyway if you do something like the 'spectrum in amiga' case I pointed out.

mess pcecd draculax

what's so hard about that? The 'draculax' software would define that it has one cart, and one cdrom, and by default those would be mounted to the first cart slot, and first (and only) cd-rom drive.

mess pcdcd draculax -cart1 altcart
if i want to override the default cart selection

mess pcecd draculax -cdrom1 othercd
if i want to launch draculax with the default cart but override the cd selection (pointless, but it should be an option)

mess pcecd draculax -cdrom2 mycd
if i want to launch draculax with the default cart + cd, but also have an additional cdrom available as a music CD, that if I really want I could swap in from an internal menu

etc.

for pieces of software which already have multiple discs associated with them there will already be default values for -cdrom1 / -cdrom2 / -cdrom3 / -cdrom4 etc. based on the software selected, available from the internal menus. Do you really think having to manually specify 24 floppies for a 24-floppy game is practical?


why should I have to type mess "pcecd -cart cdromcart1 -cdrom draculax" just to launch something? it makes my head hurt, especially for systems I don't know! (most users aren't going to know, or care that pcecd requires a cd and a cart, let alone know that they have to specify both, which your proposal would force; it would be like being to specify the bios and cart for everything with a bios you want to run in MAME, by default MAME just picks something sensible which you can override if you want, it 'just works' even if there is room for improvement with a more generic implementation)

MESS needs to bring down the learning curve if it's to be widely adopted.

Things like pirate DC games (prior to self-boot) will be the same. The default software name will provide both the 'utopia' boot CD, and the game CD. (Yes, I know piracy helped kill the DC etc. and most of them are horrible downsampled things with music ripped out etc. but MESS, like MAME exists to document things, not pretend they don't exist and you can't document the DC without documenting the piracy that existed)
Posted By: R. Belmont Re: Fixed software lists - 01/14/10 01:47 PM
Can we make the simple cases work before we figure out ways to destroy it? smile

Also, Spectrum emulators for the Amiga loaded tape image files, there wasn't any official C= way to hook up a tape drive ;-)

ETA: MESS already lets you mount different images in the Tab menu. All that's necessary to fulfill most of the cases cited is to let software lists define a set of disks and have those disks appear in the media swapper along with the ability to access arbitrary files on your PC.
Posted By: Haze Re: Fixed software lists - 01/14/10 01:56 PM
Originally Posted By R. Belmont
Can we make the simple cases work before we figure out ways to destroy it? smile

Also, Spectrum emulators for the Amiga loaded tape image files, there wasn't any official C= way to hook up a tape drive ;-)


Hmm, I seem to remember having some sampler box that let you load from the original tapes ;-)
Posted By: Dr.Zer0 Re: Fixed software lists - 01/14/10 05:11 PM
A difficult case...
What about "Software that need to be installed" ? (ex. PC Game ...)
Posted By: R. Belmont Re: Fixed software lists - 01/14/10 05:18 PM
Eh? That's easy. You'd mess at486 duke3d -hard1 my_hdd.chd, install it, and then after that you'd simply play from your HDD image.
Posted By: franciscohs Re: Fixed software lists - 01/14/10 05:19 PM
Originally Posted By Dr.Zer0
A difficult case...
What about "Software that need to be installed" ? (ex. PC Game ...)


I don't think this should be in scope of MESS. I think the software list should have the original install disks and maybe some generic disks, like a DOS boot disk (to continue the PC example) and anything else should be handled by the user.
Posted By: Dr.Zer0 Re: Fixed software lists - 01/14/10 06:11 PM
What do you think about the definition of an default HD (ie. with the operating system in it), and when you launch the game for the first time (ex. to install it) the emulator load the disk and calculates the difference with the default HD (ie create a .diff file). Then when you launch the game a second time, the emulator find the .diff file associated with the game and takes you directly to the system prompt (ie. without loading the discs) ready to start it !
Posted By: R. Belmont Re: Fixed software lists - 01/14/10 07:14 PM
I think that's unnecessary for MESS and overly complex. Harddisks for computer drivers should be outside the scope of the canned software. Also, using .diffs is much slower than just writing to the image as MESS does now. That's OK for MAME where very little of the drive will get written to, but in MESS users are expected to save savegames and other documents.
Posted By: Dr.Zer0 Re: Fixed software lists - 01/14/10 07:20 PM
I understand the points smile Just an idea;)
Posted By: Heihachi_73 Re: Fixed software lists - 01/14/10 07:23 PM
I don't think there should be any fixed software lists for computers (at least PC-compatibles), there's simply way too many programs to even think of; it would be also pointless having a hash/checksum of the 'original' disk image unless it was a known unmodified disk where the program cannot save anything to it. While you could have a hash of the 6 Spear of Destiny install disks and even Windows 3.1 disks, you couldn't have one of Digger, which saved its high scores straight to the disk. Even then, you'd be looking at literally tens of thousands of 'legitimate' games and programs (want a 60MB .hsi file in there?).

I haven't got anything done in the PC driver over here, but what happens if you attempt to install Spear of Destiny using the original six 360K 5.25" floppy images? I gather it would be far easier in MESS than DOSbox. DOSbox fails to install as soon as the installer asks for the second disk (the disk label is hard coded and the installer looks for the correct one; also, AFAIK DOSbox can't unmount/swap disks either unless it's in the DOS prompt).
Posted By: R. Belmont Re: Fixed software lists - 01/14/10 07:29 PM
Oh, I think there's value to having lists for at least the OS install media for starters. If nothing else it's a good way to leverage the ClrMAME Pony Express and make sure there's wide access to the necessary stuff to test all working drivers.
Posted By: Heihachi_73 Re: Fixed software lists - 01/14/10 07:36 PM
That's what I meant, keep it to the critical OS items and install disks, but not user-editable or writeable disks of course.
Posted By: Haze Re: Fixed software lists - 01/14/10 08:05 PM
Originally Posted By R. Belmont
I think that's unnecessary for MESS and overly complex. Harddisks for computer drivers should be outside the scope of the canned software. Also, using .diffs is much slower than just writing to the image as MESS does now. That's OK for MAME where very little of the drive will get written to, but in MESS users are expected to save savegames and other documents.


Isn't the whole XP vitual PC thing in Win7 based on diffs against the original image? or am I misremembering that.
Posted By: Haze Re: Fixed software lists - 01/14/10 08:07 PM
btw I think MESS will at least need some pre-canned 'installed' versions of things like the OS as recognized software images which can be specified / mounted. Having to install a game is one thing, having to mess around and install entire OS each time you want to try something new, especially under an emulator isn't really productive or useful.

I'm hoping the major result of these changes will be
a) better documented software
b) greatly improved usability
c) greatly improved accessibility





Posted By: R. Belmont Re: Fixed software lists - 01/14/10 08:18 PM
But you wouldn't have to install the OS each time, just once per machine you want to mess with. Given that most older machines that even had them came with blank HDDs and you had to install the OS to begin that seems quite reasonable.
Posted By: franciscohs Re: Fixed software lists - 01/14/10 08:26 PM
Originally Posted By Haze
btw I think MESS will at least need some pre-canned 'installed' versions of things like the OS as recognized software images which can be specified / mounted. Having to install a game is one thing, having to mess around and install entire OS each time you want to try something new, especially under an emulator isn't really productive or useful.


I agree, at least some plain installs for significant versions of OSes, but trying to keep it at the minimum. I would say the minimum amount of pre-canned OS images so that all the software lists for a certain system can be run, preferably only one. Also things like a DOS boot disk for PC and things like that.

The problem I see here is, what size CHD do you create for each of the pre-canned OS installs?, do these files grow as the space of the CHD get's filled up or they are a certain size from the start?

I think this has to be handled with a lot of care, as already was said, software lists for certain systems can be almost infinite. In that case, I think only some minimum pre-canned software should be included and that's all.
Posted By: R. Belmont Re: Fixed software lists - 01/14/10 08:35 PM
CHDs are a fixed size and cannot grow. That's why if we supply canned OS images on HDDs at all such "boot drives" must be read-only, with the user supplying a second HDD for installing stuff.
Posted By: Haze Re: Fixed software lists - 01/14/10 08:41 PM
Originally Posted By R. Belmont
CHDs are a fixed size and cannot grow. That's why if we supply canned OS images on HDDs at all such "boot drives" must be read-only, with the user supplying a second HDD for installing stuff.


I honestly don't see the problem with the 'diff' system MAME uses now. Nobody is going to be using the emulated OS as their primary OS, so for simple things like installing software it should be fine..

Posted By: R. Belmont Re: Fixed software lists - 01/14/10 08:49 PM
You'd be surprised how much installing people do in the real world on these things. There's a pretty large need for people to open e.g. weird old word processing files that nothing modern will touch. Anyway, I thought the canned software lists were to coexist with the existing mount methods. If you want to make it only canned software like MAME I'm out, I'll fork MESS and work on the Mac driver my way.
Posted By: Just Desserts Re: Fixed software lists - 01/14/10 08:54 PM
Originally Posted By R. Belmont
If you want to make it only canned software like MAME


Was there ever any doubt?
Posted By: R. Belmont Re: Fixed software lists - 01/14/10 09:31 PM
I do give people the benefit of the doubt, even Haze. Anyway, if he wants to trap himself in .diff land it's his funeral as long as nobody removes or otherwise damages -hard1 =)
Posted By: etabeta78 Re: Fixed software lists - 01/14/10 09:49 PM
for computers, the whole fun is to let people run their own software [1]. having fixed lists only would be, again, a big NO because they would be a loss of functionalities and a step back from current flexibility wink

documenting available software is ok and is an added value. blindly stick only to that just because it works for arcades is stupid


[1] that's why I spent a lot of time debugging c16 tape writing problems, even if tape loading was ok and that was enough obviously to play games
Posted By: R. Belmont Re: Fixed software lists - 01/14/10 09:52 PM
Sure. And as I said, for ease of use it'd even be OK to provide canned boot HDD images. But you can't curtail the ability to use the emulated computers *as* computers.

Real example: I dug up an old technical document that was in MacWrite format. Modern software won't touch it. But I was able to open it in MacWrite on MESS, export to .RTF, mount the HDD image on my G5, copy the .RTF to my LAN, and open it in Word.
Posted By: Dr.Zer0 Re: Fixed software lists - 01/14/10 11:35 PM
The idea that I expressed a few lines above, has the sole purpose of making the mess closer to an user that "just wants to enjoy a game". Of course it did not pretend to replace a much more flexible system. About me, I use PCs for almost 20 years, and I like doing things the "old style". Points of view ...

I take this opportunity to thank all the DEVS for their splendid work on MAME and MESS
Zer0
Posted By: PandMonium Re: Fixed software lists - 01/15/10 02:29 AM
Hi all,

imho the idea of fixed software lists with known good dumps is great not only for helping devs tracking errors/improving things but also to create a bit more order in all the console/computer rom mess [ laugh ].

I'm just curious on how it will work for all systems, in consoles it seems way more obviously than in computers. In cases of some computers with a lot of software for varied purposes (games, OSes, lots of applications, ...), will there be a massive list of this (with some sort of categorization?), or the idea is just to stick with a small list of essential software for those computers?
Other question is if the plan is to dump some stuff again (properly) or for now just stick with available sets that you consider to be 'perfect'.

About the discussion on setnames and command line, imho i would keep it simple at least for now, i'm talking from the outside without knowledge of the size or complexity of these first lists/systems using them, but the default usage should be simple, something like "mess system setname" so anyone could easily launch a given set without having to investigate much about the set or system, if things start getting too big later then they would be reorganized according to the eventual needs.

Thats all, hope i didn't made any mistake here or asked something already answered before but the topic is growing fast and couldn't read it all from the start yet. blush
Posted By: ranger_lennier Re: Fixed software lists - 01/15/10 07:21 AM
Everyone (I think) wants to keep MESS offering the full features of emulated computers. But I see as much value in documenting old computer software as in documenting console games. Precisely because there's so much of it, things can easily fall through the cracks. So, I wouldn't want to rule out software lists for these. Massive software documentation is possible. The Software Preservation Society, for example, has archived over 2500 Amiga programs. And like them, I would suggest using an unaltered disk is possible. If none is available, then use one example of an altered one. And even a cracked image is better than nothing, though it should certainly be considered a bad dump. It's not something to start out with, and it should never be expected to be 100% complete, but we shouldn't just throw up our hands at the outset because it's a really big job.

This does bring up something I hadn't through of, though. If we're already constructing megabyte-size lists, how large are all the lists going to be if someone actually makes an Apple II software list, an Amiga software list, a (this one is admittedly intimidating) PC software list? Should the lists actually be required as part of MESS, or should there be some sort of optional light build so uninterested parties don't have to download a 100MB+ program?
Posted By: judge Re: Fixed software lists - 01/15/10 07:59 AM
The size thing does bring up an interesting point especially when looking at the bigger systems.

Documenting the old software for older (obscure) systems will be valuable; it is usually quite hard to find software for such systems. The 'old and obscure' is already one of the strong points of MESS wink
Posted By: Robbbert Re: Fixed software lists - 01/15/10 09:27 AM
Just a personal view, nobody need take notice...

I always believed that MESS was there to document hardware, not software. However, I guess there's not much harm in it, there is not much else we can do now, with the u1 changes coming along..

I think the software lists (i'm calling them SLs) should be .dat or .xml files compatible with CMP. They should be optional, able to be created by anyone (like most MAME files - cheat, titles, artwork, etc). MESS would use the SL if it is supplied, if not, do the same as now. The SLs would be available as a separate download, for those who want them.

They would only cover software as supplied by companies (carts, cassettes, cds, unaltered virgin disks, and other original media) but not hard disks, since they can (and do) contain anything. Software that contains a unique serial number per copy would need to be commented as such.

If someone makes a SL, and it happens to crash MESS, that is not our problem. After all, products such as browsers and word processors crash every day due to unplanned input.

All we should do is supply guidelines of making SLs, and supply a zip file with our "official" list.

Unlike some, I like to have software run as fast as possible, with as small a footprint as possible (small exe size, small memory usage, small resource needs). This means the smallest download size as well, this is important as sites still have monthly limits, and not everyone has 100mb connections.

This means, therefore that SLs and Artwork not be included in the MESS download, but be downloaded from the relevant pages for those who want it. (we will need to make a SL download page, obviously).

Lastly, from what I've seen both here and on the list, a lot more thought needs to put into the nitty-gritty of the list format, how to represent information, how to deal with software on multiple media or requires multiple media. Therefore, although it is fine to gather and submit SLs, there is no hurry to lock in a format. As you know, if you rush, you end up making mistakes.

We have a good product now, don't kill it with bloat and slowness..

That's all I have to say on this topic, now continue on.. laugh
Posted By: Anna Wu Re: Fixed software lists - 01/15/10 12:18 PM
I agree, the software lists should be optional. Personally, I no need because it blow up the memory usage/space consumption.
Posted By: etabeta78 Re: Fixed software lists - 01/15/10 12:29 PM
Originally Posted By Anna Wu
Personally, I no need because it blow up the memory usage/space consumption.


?!?

given the fact no code has been written so far to actively use lists in MESS, it's definitely too early to state that it will blow up anything smirk

of course, the size of the source might become bigger with lists, but we could offer lists as a separate download (especially because they will be updated less often than the rest of the emulator, after the initial fixes)
Posted By: Dr.Zer0 Re: Fixed software lists - 01/15/10 01:30 PM
Nice idea
Posted By: Stiletto Re: Fixed software lists - 01/15/10 02:45 PM
I was thinking there might be some sort of "list checksumming" utility.

Ideally I would like to see external fixed software lists so that the updating of the lists can be decoupled from the updating of the emulator. However, I was thinking that you could do a "list checksum" in order to prove that the list is valid, and build that into the emulator. If list checksum = 0345592B, then list = genesis.list, go ahead on loading list. However, if an interim update to the list went out, then what?

If you were to go with external lists, you would need some sort of validation process.

*just thinking aloud*
Posted By: Duke Re: Fixed software lists - 01/15/10 03:54 PM
Why would you need a validation process?
Posted By: Haze Re: Fixed software lists - 01/15/10 04:02 PM
Originally Posted By etabeta78
Originally Posted By Anna Wu
Personally, I no need because it blow up the memory usage/space consumption.


?!?

given the fact no code has been written so far to actively use lists in MESS, it's definitely too early to state that it will blow up anything smirk

of course, the size of the source might become bigger with lists, but we could offer lists as a separate download (especially because they will be updated less often than the rest of the emulator, after the initial fixes)


The space / memory the lists take is minimal compared to the memory available on modern systems.

The problem with the dats etc. being *external* is that it means if you want to use functionality like -romident then MESS has to load and parse the files before it will work, this slows things down and is just irritating (and the results will depend on what dats you have, which is also less than desirable)

Nobody is proposing limiting what MESS can do in any way, the proposal is simply that software lists provide a list of media associated with a game in a way which makes accessing it easier (via 8 letter system name + 8 letter software name) and likewise making launching it easier.

All the existing functionality should remain, but again, things will become easier to use due to it being able to call up software lists for known software in addition to the current methods.

MESS does need to become easier to use, whether the 'hardcore' users like it or not. MAME doesn't really have to make such efforts because arcade games are straightforward.

It seems the project is somewhat lacking in any leadership as far as these issues are concerned, so I can only tell you what would make MESS a more attractive project to both use, and work on from my point of view. As somebody who finds the current systems unfriendly, alienating, and hard to work with even to somebody highly familiar with MAME I do hope some of what I'm saying makes sense.

Posted By: Just Desserts Re: Fixed software lists - 01/15/10 04:19 PM
Originally Posted By Haze
The space / memory the lists take is minimal compared to the memory available on modern systems.


To quote Walter Sobchak, "The Chinaman is not the issue here, dude".

Once MESS is on someone's hard drive or resident in memory, how much space it takes up is immaterial. However, it seems for all the world like adding all of these lists has the potential to add well over ten megs, possibly tens of megs, of data to the executable. Are you going to pay the bandwidth overages when MESS's executable size bloats to Jupiterian proportions compared to the already gargantuan MAME?
Posted By: franciscohs Re: Fixed software lists - 01/15/10 04:27 PM
Originally Posted By Just Desserts

Once MESS is on someone's hard drive or resident in memory, how much space it takes up is immaterial. However, it seems for all the world like adding all of these lists has the potential to add well over ten megs, possibly tens of megs, of data to the executable. Are you going to pay the bandwidth overages when MESS's executable size bloats to Jupiterian proportions compared to the already gargantuan MAME?


Is this REALLY a problem?, I mean, I have really no idea how MESS handles this, but it would have to be a REAL problem if I were to change a design decision that IMHO would defeat all the purpose of what's being proposed.

Specially nowadays that there are so many distribution channels available.
Posted By: Haze Re: Fixed software lists - 01/15/10 04:30 PM
Originally Posted By Just Desserts
Originally Posted By Haze
The space / memory the lists take is minimal compared to the memory available on modern systems.


To quote Walter Sobchak, "The Chinaman is not the issue here, dude".

Once MESS is on someone's hard drive or resident in memory, how much space it takes up is immaterial. However, it seems for all the world like adding all of these lists has the potential to add well over ten megs, possibly tens of megs, of data to the executable. Are you going to pay the bandwidth overages when MESS's executable size bloats to Jupiterian proportions compared to the already gargantuan MAME?


well, information uses space, even if you make the lists external, they still take up space and use bandwidth, and if you want to actually document the software and have fully functional -romident etc. then can't be avoided. The information is as big as it is unless you want to revert back to CRC32 only, but that would be dumb.

MESS has a smaller userbase than MAME anyway, I don't think it's really relevant and doubt the impact will be too high.

I think being able to properly identify software is far more important than the size of the exe, or bandwidth used, you'll always find people willing to host mirrors. I believe a lot of hosts are more concerned over CPU usage these days than actual data transfer?

I actually wonder if having the MESS SVN public actually saves bandwidth, or causes more to be used. It would be quite nice IMHO if the MAME SVN was public too.

It's a silly argument, almost as silly as the ROM sites screaming at the MAME project 'Don't support the encrypted NeoGeo sets, they're too big' and 'Your Laserdisc format is fucking stupid becuase the CHDs are 25gig'. The important thing about these projects is that they do what needs doing, at whatever cost, properly.




Posted By: incog Re: Fixed software lists - 01/15/10 04:37 PM
lets get started making these lists then, i just wish hashtab didnt give out all its checksums in upper case
Posted By: Haze Re: Fixed software lists - 01/15/10 04:40 PM
Originally Posted By incog
lets get started making these lists then, i just wish hashtab didnt give out all its checksums in upper case


I've submitted a whole bunch of them for consoles to eta, I kinda wish he would check them in so that the refinement process can start.

really what you want is some custom tool to do a quick pass for you, I used a hacked up -romident from MAME for the first pass, but there are better ways wink (eta has the required hacks too)
Posted By: etabeta78 Re: Fixed software lists - 01/15/10 04:40 PM
in case you're planning to create lists, I have lists for all main consoles already done with CRC32 & SHA1 (including n64, gb, gbc, gba, snes)

nes will follow as soon as I have info on the properly split dumps

jfyi

EDIT: and refinements can start with the present lists. what's holding back progresses is MAME 0.136u1 being still unreleased. we'll have to make MESS properly compile again, before going on with the list work.

about my lists mentioned above: they will add around 3mb to the source, hence I'd like to be sure nobody complains before making the commit. in the weekend, I'll mirror these missing lists for who's interested, though.
Posted By: Haze Re: Fixed software lists - 01/15/10 04:51 PM
Originally Posted By etabeta78
in case you're planning to create lists, I have lists for all main consoles already done with CRC32 & SHA1 (including n64, gb, gbc, gba, snes)

nes will follow as soon as I have info on the properly split dumps

jfyi

EDIT: and refinements can start with the present lists. what's holding back progresses is MAME 0.136u1 being still unreleased. we'll have to make MESS properly compile again, before going on with the list work.

about my lists mentioned above: they will add around 3mb to the source, hence I'd like to be sure nobody complains before making the commit. in the weekend, I'll mirror these missing lists for who's interested, though.


Well FYI the main things that often need doing (aside from improving the 8 letter games and setting up clones) is that due to the filesystem sort order it often ends up putting games tagged with (Beta) before (Europe) etc. so sometimes the first one in the list is the Beta. Obviously it's nothing a bit of shuffling around won't fix, and far less painful than trying to make all the lists by hand :-)

For the megadrive stuff I think it took me about 4 hours of work after the list generation to get things sorted out to a reasonable level as used by the current HazeMD (the previous version was entirely done by hand as I was testing each game in the emulation as I added it, so that made sense) Now that the lists are in MESS I'm more prepared to further refine the names as some of the generated ones are still pretty nasty.

Are there naming preferences in MESS? I think some of the NoIntro names etc. are taken from the boxes, not the games. (Japanese name for Castle of Illusion comes to mind)

Of course to start cleaning things up I'll probably need SVN access, it's an incremental thing and I doubt eta is going to want me to send improvements to 5/6 set names each and every time I have a free 10 minutes to make them. If that's too much of a problem then I can leave it to other people to tidy up the naming tho.
Posted By: etabeta78 Re: Fixed software lists - 01/15/10 05:02 PM
1. about naming, I was using titles from box as well, but I'm not too strict about this, and we could switch as well to ingame titles

2. I tried to update as best as I was able to the names, from 6 to 8 chars, sometimes exploiting MAME names. however, they're far from perfect, hence your refinements are really welcome.

3. I'll leave the megadrive list to you and you can send me patches once every 2-3 days, if you want. no need to send an update every 5/6 lines, because all lists are still preliminary and need more work
Posted By: Stiletto Re: Fixed software lists - 01/15/10 05:24 PM
Originally Posted By Duke
Why would you need a validation process?


If the lists were to be external, I'd like to see MESS able to load only properly formatted lists versus fly-by-night DAT creator lists. Also, it would raise the bar of list creation. Anyone creating a new list would have to have it "blessed" by the validation utility before it could be loaded. This would fall into the plans of Haze and Guru, "official" software lists. But not really - it would simply require that each list be able to be validated by some internal validation process - in other words, it could be a compromise.

But it sounds like you guys are going internal lists, so whatev.
Posted By: R. Belmont Re: Fixed software lists - 01/15/10 05:27 PM
External lists are cool because those of us who aren't going to want to participate in that aspect of the project don't have to deal with their disadvantages.

Loading time is negligible even on fairly slow computers now, and in most cases you only need to load the entries for the current system. Whereas with internal lists you always pay the price linking and loading the executable. MESS with robust lists will easily exponentially expand the linking time and force everyone into tiny builds.
Posted By: Haze Re: Fixed software lists - 01/15/10 05:43 PM
Originally Posted By R. Belmont
External lists are cool because those of us who aren't going to want to participate in that aspect of the project don't have to deal with their disadvantages.

Loading time is negligible even on fairly slow computers now, and in most cases you only need to load the entries for the current system. Whereas with internal lists you always pay the price linking and loading the executable. MESS with robust lists will easily exponentially expand the linking time and force everyone into tiny builds.


I still think internal lists are more idiot proof and robust tho.

Believe me, if the lists are external you'll get people with bad roms, who 'fix' the external list (because it's 'easy' to fix) and then complain that the game doesn't work anyway, as I've said before, the mix-and-match aspect becomes worrying too. People will hang on to old lists, even after we've corrected things because they match their roms. That's a bad thing. Knowing that MAME 0.136 uses MAME 0.136 roms and all bug reports will have been made using a known version and thus known sets is a huge advantage. MESS stands to gain the same advantage from the software lists.

Posted By: R. Belmont Re: Fixed software lists - 01/15/10 05:52 PM
Right, and you can easily make MESS 0.136 accept only the lists that we shipped with it.
Posted By: Haze Re: Fixed software lists - 01/15/10 06:04 PM
Originally Posted By R. Belmont
Right, and you can easily make MESS 0.136 accept only the lists that we shipped with it.


How about we implement it like MAME, and if it really has too much of a negative effect, consider the refactor. I've never really had an issue with link times in MAME at least, a couple of megs of data is nothing for a modern HDD / CPU to link anyway.

Once the data is in there it can be exported refactored and whatever else as much as we need, and I'd rather be able to specify actual init functions in the software lists rather than passing magic 'mapper' numbers between external files and internal functions.

XML is slow and memory intensive to parse, and too verbose to read / edit easily. Even compiled XML isn't too fast to process, so external XML lists would be bad at least, and you'd want something with at least the flexibility and functionality of the existing rom loading. Many current gen games aren't actually loading anything during their 30 second loading sequences, they're actually just parsing XML which only took about half second to actually load, but somebody decided XML was the future so....

Also I worry that such things would end up in MAME, and the more files you split drivers into, the more annoying they become to work on as you end up having to cross-reference more things and have more files open.
Posted By: Heihachi_73 Re: Fixed software lists - 01/15/10 06:45 PM
Human-readable text files beats XML hands down, the XML-based cheat format already turned me off making cheats (well, the cheat engine redesign did, turning it from fully functional and highly usable to being deaf, dumb and blind but with an IQ of 189; still there and very smart but totally helpless to anyone who doesn't know it like the back of their hand).
Posted By: R. Belmont Re: Fixed software lists - 01/15/10 06:57 PM
M1 parses XML entries for about 1700 games in less than 1/10th of a second on a 7 year old Athlon. But until we find out that internal lists won't work it's not all that useful to specify a format.

As far as multiple files, Haze, get a real editor. Your whole outlook on how you develop MAME would change with something like SlickEdit. Visual Studio's is a bad joke and has set back progress in computer science by decades. (If you buy Visual Assist it gets better, but it's still not as powerful as the standalones).
Posted By: Dr.Zer0 Re: Fixed software lists - 01/15/10 07:08 PM
You can always start with the internal lists, and migrate to the external at any time or vice versa wink Create the standard is the "killer job" here..
Posted By: R. Belmont Re: Fixed software lists - 01/15/10 07:12 PM
More precisely, having the data compiled ASAP (these boxes and media aren't getting any newer) is the important part smile
Posted By: gigadeath Re: Fixed software lists - 01/16/10 02:13 PM
Where can I send corrections to the lists (in particular MD list)?
Posted By: etabeta78 Re: Fixed software lists - 01/16/10 02:22 PM
MD fixes to haze, so that you don't overlap your efforts smile

other fixes / suggestions to me (by PM)


it would be nice to slowly move from the current alpha-state to something correct, complete and easily readable smile
Posted By: gigadeath Re: Fixed software lists - 01/16/10 02:35 PM
Since me (and ElBarto) cured the MD dat for No-Intro for years, and the group dumped quite a few MD carts (albeit with the SegaCD cable), I think I can help with MD.

Of course I'm acting as an independent here, so I don't know what the stance of the other No-Intro members is regarding to this.

But since most of the MD work come from me and ElBarto (who I think shares mi views), I don't think there's a problem if I lend a hand here.

I begun correcting things here and there (such as ~ to / to separate western from japanese title for games that switch region depending on the bios, a-la Neo-Geo), uniforming entries from Goodgen, format etc.

In particular I wonder where I could add info on MPR-***** serial numbers. They're BEYOND USEFUL for spotting/comparing MD dumps, since each unique MPR serial is linked to a unique hash. They should be on the list at all costs. But I don't know where to put them.
Posted By: Haze Re: Fixed software lists - 01/16/10 02:42 PM
Originally Posted By gigadeath

In particular I wonder where I could add info on MPR-***** serial numbers. They're BEYOND USEFUL for spotting/comparing MD dumps, since each unique MPR serial is linked to a unique hash. They should be on the list at all costs. But I don't know where to put them.


When the proper MPR-xxxxx roms are added the said rom will also need to be byteswapped into the native format, rather than the 'console emulator' pre-byteswapped format.

They belong in the rom loading part anyway, and if there are multiple roms in the cart, they should be loaded correctly, some cases are quite strange eg. Ghouls and Ghosts

ROM_LOAD16_WORD_SWAP( "mp12605.ic1", 0x000000, 0x020000, CRC(1066C6AB) SHA1(C30E4442732BDB38C96D780542F8550A94D127B0) )
ROM_LOAD16_WORD_SWAP( "mpr12606.ic2", 0x080000, 0x020000, CRC(D0BE7777) SHA1(A44B2A3D427F6973B5C1A3DCD8D1776366ACB9F7) )
ROM_CONTINUE(0x020000,0x60000)

(that's from megatech, but I'm assuming the Genesis cart has the same funky layout) One of the main reasons for doing this is so that the proper roms, with proper sega labels where they exist can replace the existing console dumps.

The existing rom loading is macroized, and for loading proper roms you'll have to either make a new macro, or add them manually using the standard rom loading.

SOFTWARE_START( riserbot )
ROM_REGION( 0x400000, CARTRIDGE_REGION_ROM, 0 ) /* 68000 Code */
ROM_LOAD16_WORD_SWAP( "acclaim,es133-1,rise_of,the_robots.u1", 0x000000, 0x200000, CRC(ed583ef7) SHA1(b9f43d5bf31819a1d76c1495e81cfa1d38bcde1c) )
ROM_LOAD16_WORD_SWAP( "acclaim,es133-2,rise_of,the_robots.u2", 0x200000, 0x100000, CRC(fcf18470) SHA1(09f8ba0b295da42359c354e71b9b7c780a465046) )
SOFTWARE_END

for example, is the rom loading for rise of the robots, with labels from the cart, loading in the proper dumped format (not pre-byteswapped)

For now I'm not making any changes to the list myself, so feel free to just send any fixes / improvements to eta.

It still concerns me that MAME is enforcing lower-case only rom names tho, to me this isn't accurate documentation, but requests to change that have thus far fell on deaf ears.
Posted By: gigadeath Re: Fixed software lists - 01/16/10 02:53 PM
It's true that they belong there, but since we won't have proper dumps for all the carts for many many years (ever?), I think there should be a way to put MPR serials on the list even for the dumps that come from a copier (add that for the most part proper dumps will be identical to copier dumps only byteswapped, contrary to the popular belief multichip MD games are a very small minority).

"Misplaced" MPR info will be removed when a proper dump is available. We have many of those serials already, and they're proven to be really useful, especially to spot bad dumps.
Posted By: Haze Re: Fixed software lists - 01/16/10 03:08 PM
Originally Posted By gigadeath
It's true that they belong there, but since we won't have proper dumps for all the carts for many many years (ever?), I think there should be a way to put MPR serials on the list even for the dumps that come from a copier (add that for the most part proper dumps will be identical to copier dumps only byteswapped, contrary to the popular belief multichip MD games are a very small minority).

"Misplaced" MPR info will be removed when a proper dump is available. We have many of those serials already, and they're proven to be really useful, especially to spot bad dumps.


Well I think you can quite safely byteswap anything you have the mpr number for anyway, afterall, it's 100% more likely to be closer to a proper dump that way wink
Posted By: R. Belmont Re: Fixed software lists - 01/16/10 03:12 PM
FWIW in real life the ROMs probably aren't byteswapped, it's just EPROM dumper software on little-endian hosts swaps reads and writes of 16-bit wide devices (it's an unwritten standard). So it depends on if being able to burn replacements is more important than strict documentation (which in MAME it has been).
Posted By: gigadeath Re: Fixed software lists - 01/16/10 03:29 PM
I won't touch the byteswapping issue of course, since the discussion is heated on MD byteswapping, and has been for years laugh

This is the first time I hear what Arbee is saying, that would put the byteswapping discussion to an end.

Anyway, I was simply making clear that for 90+% of MD roms, a redump isn't *strictly* necessary, them being single-chipped, for those it's simply a matter of discovering if they're really b-swapped or not on actual chip. A redump is *always* nice, but not a priority.

The multi-chipped ones need a redump as a priority of course, some of them are noted in my document.

But whether we consider copier dumps good or not, MPR is fundamental.
Posted By: R. Belmont Re: Fixed software lists - 01/16/10 03:34 PM
Like I said, for MAME it's historically been more of a priority that it be easy for people to burn replacement EPROMs and fix their boards than it is the ROMs be strictly accurate (since on common consumer hardware and software the swapped format *is* correct).

If anyone has a schematic of an MD cartridge it would be easy to verify once and for all - just check if d0-d15 out of the ROM connects in order to d0-d15 on the 68000 bus.

And yes, having MPR numbers is a huge huge thing.
Posted By: gigadeath Re: Fixed software lists - 01/16/10 03:37 PM
Another thing, Mega-Tech carts can be added to the list as "proper dumps",since they have identical hash and MPR-serial to the normal carts. Mega-Tech dumps such as Tetris and GnG, which are multi-chip, will save time and money for a redump. Tetris in particular is impossible to find as a normal cart, so that Mega-Tech dump is especially precious.
Posted By: Haze Re: Fixed software lists - 01/16/10 03:38 PM
Originally Posted By R. Belmont
Like I said, for MAME it's historically been more of a priority that it be easy for people to burn replacement EPROMs and fix their boards than it is the ROMs be strictly accurate (since on common consumer hardware and software the swapped format *is* correct).

If anyone has a schematic of an MD cartridge it would be easy to verify once and for all - just check if d0-d15 out of the ROM connects in order to d0-d15 on the 68000 bus.

And yes, having MPR numbers is a huge huge thing.


Any MD carts dumped with an actual eprom programmer come out swapped vs. the existing console dumps. Several people have dumped them this way. It's just a case of which standard you pick I guess and I'd argue that if you care about the ROM labels you care about using a real programmer, not a cart copier.

As a side note 2 of the Megatech sets in MAME have been pre-swapped to work with console emulators I think, because whoever dumped them assumed that something that would run in the console emulators was the 'right way'.

Anyway, yeah, get the mpr stuff documented, while it only applies to official Sega carts (3rd party had their own numbering) it's good to fill in the holes, and as you say, it's ideal for verification of dumps.

Posted By: etabeta78 Re: Fixed software lists - 01/17/10 12:10 AM
ok, here is a link to snes+n64+gb+gbc+gba lists so that they don't get lost if my HD dies

the current status is the following:

n64, snes, gb : alpha, i.e. setnames almost make sense, even if they can be improved for sure; there are most parent/clones relationships; BUT years and manufacturers still miss and some descriptions are not 100% accurate

gbc, gba : pre-alpha, i.e. setnames have only been automatically generated (there are still things like pokem33, 2games75, etc.) and need to be cleaned; also a bunch of gba dumps are missing.

feel free to step up if you want to clean any of these.

tomorrow, I'll commit gigadeath's fixes to megadrive list. and, fwiw, I fully supporting both addition of part numbers and rom byteswapping (with a cautionary BAD_DUMP flag for the carts which are swapped without confirmation smile )
Posted By: Guru Re: Fixed software lists - 01/17/10 03:09 AM
Originally Posted By R. Belmont
FWIW in real life the ROMs probably aren't byteswapped, it's just EPROM dumper software on little-endian hosts swaps reads and writes of 16-bit wide devices (it's an unwritten standard). So it depends on if being able to burn replacements is more important than strict documentation (which in MAME it has been).


eh? you've been staring at your Mac too long and dreaming its useful (in a virtual world).
EPROM programmers read the ROMs as-is, there is no additional manipulation.
For example most of the Amiga boot ROMs used in WinUAE are no good. They were dumped using the Amiga. If you program them to an EPROM and plug them into the Amiga it won't boot up. The ROMs need a 16-bit byte-swap before they are programmed to the EPROM. I've proved that myself for Amiga boot ROMs when I updated my A600 to 37.35 and it wouldn't boot and I figured out why.
I'm sure any carts that were dumped from the edge connector and were found to be byte-swapped compared to dumping the actual EPROMs are not right and should be byte-swapped. Otherwise if you program the edge-dumped dumps as-is to an EPROM the cart won't work.
Which means of course that being able to burn replacement ROMs to repair real hardware is W-A-Y more important than anything else.
Posted By: R. Belmont Re: Fixed software lists - 01/17/10 03:37 AM
Not quite. For ROMs wider than 8 bits PC based readers/writers store the resulting 16 or 32 bit words in the dump file in Intel little-endian order (which is fine since they have no way to know the endianness of the system you're dumping).

This has the effect that if the ROMs are from a big-endian system the dump file is byte-swapped compared to what the real system (and the emulated processor in MAME) really sees so MAME has to swap them on load. Conversely dumps of 16-bit wide ROMs from little-endian systems (e.g. Namco FL) don't need swapping in MAME.

So technically the copier dumps and BIOS dumps from the Amiga itself are correct as to how the 680x0 sees memory, but you can't convince PC-based EPROM readers to do the right thing with them in many cases so we punt.

(And tempting though it would be to make everyone have to redownload every MAME set ever dumped we're not going to switch to physical byte-level accuracy).
Posted By: gigadeath Re: Fixed software lists - 01/17/10 03:18 PM
As I see it, the problem is not that big. Once the correct "endian-ness" is found (if ever) a swapping tool would take 3 minutes to magically make all the dumps "good". Luckily for MD most games have only one chip. Eventually redumps will come to confirm things once and for all.

On the other hand, there ARE games that absolutely require a redump, such as
-multichip games (not that many)
-some strange EA carts like F22 (already taken care of), Centurion and the first Road Rash
-Quackshot
-etc

Anyway I'm noting all the carts that *absolutely* need a redump, together with a small undumped list of carts that we miss *for sure* (other revisions might exist that we don't even know they exist).
Posted By: etabeta78 Re: Fixed software lists - 01/17/10 03:30 PM
I'll be happy to add your notes whenever they're ready smile
Posted By: ElBarto Re: Fixed software lists - 01/17/10 05:45 PM
Originally Posted By gigadeath
As I see it, the problem is not that big. Once the correct "endian-ness" is found (if ever) a swapping tool would take 3 minutes to magically make all the dumps "good". Luckily for MD most games have only one chip. Eventually redumps will come to confirm things once and for all.

On the other hand, there ARE games that absolutely require a redump, such as
-multichip games (not that many)
-some strange EA carts like F22 (already taken care of), Centurion and the first Road Rash
-Quackshot
-etc


Quackshot "mystery" have already been solved.
Quoting myself from the No-Intro forum :
Quote:



On the Quackshot PCB there is a 512Kb rom.
Normally all the address lines from A1 to A19 (the whole 512Kb range) tied to the chip in the right order.
But here we have the megadrive A18 line tied to the A20 pin of the rom.
This means that we have the first 256Kb of the rom followed by 0xC0000 lenght range of mirrored/zero-ed data then again the last 256Kb of the game.
I think they did that for a crappy (really crappy) copy protection.
For me, even the rom is not good, it's "good" cause it's like this that the megadrive see the cart (that's why you can play it).
None of the existant emulator can emulate quackshot with a real dump (zero-ed data trimmed or dump via eeprom dumper), maybe mess in a near future.
The only thing is to inform people about it so they won't spend hour too see what's wrong with their dump.

On a side note, I 99% sure that their is only one revision of the game worldwide, for now I only dumped three european cart and 2 jap ones. Same.
The v1.0 must be :
1) A hack of the "1.1" trimmed to 512Kb
2) A dump for a chinese pirate cart.
3) A rip from the Disney Collection rom.
Posted By: Skito Re: Fixed software lists - 01/17/10 05:48 PM
Originally Posted By etabeta78
ok, here is a link to snes+n64+gb+gbc+gba lists so that they don't get lost if my HD dies


What if people want to make new lists of systems that dont have existing lists, a2600, apple2, spectrum...?
Posted By: Haze Re: Fixed software lists - 01/17/10 06:11 PM
Originally Posted By ElBarto

Quackshot "mystery" have already been solved.
Quoting myself from the No-Intro forum :
Quote:



On the Quackshot PCB there is a 512Kb rom.
Normally all the address lines from A1 to A19 (the whole 512Kb range) tied to the chip in the right order.
But here we have the megadrive A18 line tied to the A20 pin of the rom.
This means that we have the first 256Kb of the rom followed by 0xC0000 lenght range of mirrored/zero-ed data then again the last 256Kb of the game.
I think they did that for a crappy (really crappy) copy protection.
For me, even the rom is not good, it's "good" cause it's like this that the megadrive see the cart (that's why you can play it).
None of the existant emulator can emulate quackshot with a real dump (zero-ed data trimmed or dump via eeprom dumper), maybe mess in a near future.
The only thing is to inform people about it so they won't spend hour too see what's wrong with their dump.

On a side note, I 99% sure that their is only one revision of the game worldwide, for now I only dumped three european cart and 2 jap ones. Same.
The v1.0 must be :
1) A hack of the "1.1" trimmed to 512Kb
2) A dump for a chinese pirate cart.
3) A rip from the Disney Collection rom.


It shouldn't be too hard to support the proper dump, assuming that software can call an init function like drivers (which it will need to to handle the protection on many chinese games anyway, and proper backup ram configs rather than guessing them), or, for simple cases, just with some nice use of the MAME rom load macros like with Ghouls and Ghosts.

As for the legitimacy of the other set, if it's 1.0, then maybe it was a limited release, or even a review copy, and 1.1 was the first public release, or maybe it's a hack like you say. MESS/MAME should document such things btw, not remove them. Genuine pirate dumps etc. are just as valid as legitimate games.
Posted By: ElBarto Re: Fixed software lists - 01/17/10 06:21 PM
That's where MPR is cool. All the quackshot that I've dumped (3/4 eur, one jap and two US) are all the same and the MPR doesn't finish with a 'A'.
I really think that it's a hack of the 10Mb version or a rip of the disney collection.
I still have to find a chinese pirate quackshot to be sure.
Posted By: Haze Re: Fixed software lists - 01/17/10 06:34 PM
Originally Posted By Skito
Originally Posted By etabeta78
ok, here is a link to snes+n64+gb+gbc+gba lists so that they don't get lost if my HD dies


What if people want to make new lists of systems that dont have existing lists, a2600, apple2, spectrum...?


You'll probably want to wait for eta to create them, or somebody else with an automated tool, I'd hate to try adding all the checksums by hand for those systems wink
Posted By: Duke Re: Fixed software lists - 01/18/10 09:23 AM
Originally Posted By R. Belmont
Not quite. For ROMs wider than 8 bits PC based readers/writers store the resulting 16 or 32 bit words in the dump file in Intel little-endian order (which is fine since they have no way to know the endianness of the system you're dumping).

This has the effect that if the ROMs are from a big-endian system the dump file is byte-swapped compared to what the real system (and the emulated processor in MAME) really sees so MAME has to swap them on load. Conversely dumps of 16-bit wide ROMs from little-endian systems (e.g. Namco FL) don't need swapping in MAME.

So technically the copier dumps and BIOS dumps from the Amiga itself are correct as to how the 680x0 sees memory, but you can't convince PC-based EPROM readers to do the right thing with them in many cases so we punt.

(And tempting though it would be to make everyone have to redownload every MAME set ever dumped we're not going to switch to physical byte-level accuracy).


I think theres a bit more to it. The Amiga is big-endian system, and the ROMs we have in MESS for the A3000 are loaded with "ROM_GROUPWORD | ROM_REVERSE", yet the sum16 checksum printed on the ROM chip matches:

COPYRIGHT 1990 CAI // ALL RIGHTS RESERVED // ALPHA 5 ROM 0 CS=9713

So the dump is definitely correct this way.
Posted By: etabeta78 Re: Fixed software lists - 01/18/10 10:57 AM
Originally Posted By Skito
What if people want to make new lists of systems that dont have existing lists, a2600, apple2, spectrum...?


the way I create them works as follows:

1. I take the most complete/recent set I can find and I generate the CRC/SHA list with a modified version of MESS (pre-alpha state)
2. I start improving the setnames (which is a painfully slow process) and adding some clone relationship where possible (alpha state)
3. I start adding manufacturers/years, fixing descriptions and shortnames (beta state)

Most lists in the source are now at stage 3. (wswan, svision and gamegear are still closer to 2. than 3. actually); my other lists are between 1. & 2.

as per conventions, I try to stay as close to MAME as possible:
A. parent MUST have names shorter than 8 chars and is usually the latest version (either European or US, depending on the situations)
B. clones are usually ("parent name"+ 1-3 specific chars): e.g. if we have "game" as the European parent, and we have a US release, a Japanese one, and a French one; the latter clones will be gameu, gamej and gamef respectively.
C. description should contain the region and, if needed, something to uniquely determine the version of the game
(e.g. "Game (Jpn)" and "Game (Jpn, Prototype)", and "Game (Jpn, v1.1)" etc.), but without too many info which do not really belong to the name
D. if a MAME name can be used, I use that one (e.g. most Bubble Bobble console conversions are called bublbobl; most Puzzle Bobble conversions are called pbobble; etc. ) wink


Finally, notice that it's not been established yet the exact syntax for non-cart media, but this should be something that can be taken care of later: the most time consuming part is the checksums & setnames creations smile

If you want to create a new list, feel free to contact me with some proposal (mainly to avoid duplicate efforts wink ), but be warned: it might take some time before it gets included...
Posted By: Guru Re: Fixed software lists - 01/18/10 11:41 AM
Originally Posted By R. Belmont
This has the effect that if the ROMs are from a big-endian system the dump file is byte-swapped compared to what the real system (and the emulated processor in MAME) really sees so MAME has to swap them on load. Conversely dumps of 16-bit wide ROMs from little-endian systems (e.g. Namco FL) don't need swapping in MAME.


Not sure what you mean? Are you saying the EPROM programmer doesn't read it properly? And if yes can you give me an example of an EPROM type (or arcade PCB that has those types) where an EPROM-reader would mis-read the EPROM and if copying it to another blank EPROM it wouldn't work? I'd like to test this on real hardware.
Posted By: R. Belmont Re: Fixed software lists - 01/18/10 01:57 PM
Originally Posted By Duke
Originally Posted By R. Belmont
Not quite. For ROMs wider than 8 bits PC based readers/writers store the resulting 16 or 32 bit words in the dump file in Intel little-endian order (which is fine since they have no way to know the endianness of the system you're dumping).


COPYRIGHT 1990 CAI // ALL RIGHTS RESERVED // ALPHA 5 ROM 0 CS=9713

So the dump is definitely correct this way.


Pay closer attention. As interpreted by the PC, the words are correct so a wordsum will match. However, the physical byte order in the dump file is wrong when compared to how the original system's 680x0 reads things (so e.g. text strings are scrambled). I know this is somewhat esoteric and hard to get your head around - that's why many many MAME and MESS drivers fall over on PowerPC systems.

ETA: Let me give an example. Suppose the first 4 bytes (as seen by the 68000) are 12 34 56 78. The 16-bit ROM would store those as 2 words: 1234 5678. The EPROM reader will read 1234 then 5678 (so the words are correct) but because the reader's on little-endian it writes the dump file as 34 12 78 56 and so the dump can't work for emulation without being byteswapped. (If the ROM was from a little-endian system of course things would match anyway).

When you go to burn the dump to a blank EPROM, the words get reassembled properly (1234 5678) since you're still on a little-endian system and so the EPROM ends up with the same contents as the original.

Anyway, this is off-topic now so please PM me if you still don't understand (or start a new thread).
Posted By: Duke Re: Fixed software lists - 01/18/10 02:17 PM
Yes, you are correct of course, the sum will stay the same whether its stored as big- or little-endian in the ROM image. My fault, sorry.
Posted By: gigadeath Re: Fixed software lists - 01/18/10 09:01 PM
Originally Posted By ElBarto

Quackshot "mystery" have already been solved.
Quoting myself from the No-Intro forum :

Quote:

On the Quackshot PCB there is a 512Kb rom.
Normally all the address lines from A1 to A19 (the whole 512Kb range) tied to the chip in the right order.
But here we have the megadrive A18 line tied to the A20 pin of the rom.
This means that we have the first 256Kb of the rom followed by 0xC0000 lenght range of mirrored/zero-ed data then again the last 256Kb of the game.
I think they did that for a crappy (really crappy) copy protection.
For me, even the rom is not good, it's "good" cause it's like this that the megadrive see the cart (that's why you can play it).
None of the existant emulator can emulate quackshot with a real dump (zero-ed data trimmed or dump via eeprom dumper), maybe mess in a near future.
The only thing is to inform people about it so they won't spend hour too see what's wrong with their dump.

On a side note, I 99% sure that their is only one revision of the game worldwide, for now I only dumped three european cart and 2 jap ones. Same.
The v1.0 must be :
1) A hack of the "1.1" trimmed to 512Kb
2) A dump for a chinese pirate cart.
3) A rip from the Disney Collection rom.


Yeah I remember the discussion, but I forgot the part where it was redumped via eeprom reader already. I don't know what the emulation status of the proper Quackshot dump is, but now the 10mb "1.1" dump can be removed, since it's non-existent.

Originally Posted By Haze

It shouldn't be too hard to support the proper dump, assuming that software can call an init function like drivers (which it will need to to handle the protection on many chinese games anyway, and proper backup ram configs rather than guessing them), or, for simple cases, just with some nice use of the MAME rom load macros like with Ghouls and Ghosts.

As for the legitimacy of the other set, if it's 1.0, then maybe it was a limited release, or even a review copy, and 1.1 was the first public release, or maybe it's a hack like you say. MESS/MAME should document such things btw, not remove them. Genuine pirate dumps etc. are just as valid as legitimate games.


I think we all agree on supporting legit pirate carts (confirmed by trusted sources, letting pirate/beta flags be an easy way out to flag bad dumps, as it was done in the past, is a crime - see half the MD "beta" dumps, especially the unconfirmed betas coming from very old versions of Goodgen, I'm 99,99% sure many of those are simple bad dumps / hacks).

But as ElBarto says, this Quackshot "1.1" is actually rev 1.0, because there's no revision letter at all printed on the chip. Sega was really accurate with rom labelling. MPR numbers are more accurate than headers, because MPR serials actually distinguish between those that are simple revisions and those that are entirely different releases.

See for example Revenge of Shinobi / Super Shinobi: western releases got a different MPR serial from the Japanese one, even if all four internal headers still show the classic 00-01-02-03 progression.

Super Shinobi has MPR-12675
First Revenge of Shinobi revision has MPR-12930
Second Revenge of Shinobi revision has MPR-12930A
Third Revenge of Shinobi revision has MPR-12930B

But in the headers they put 00 for Super Shinobi, then 01, 02 and 03 for the three western releases. Looking only at the headers one could think that they are revisions one of the other, but only the last three actually are (all found on western carts), while the Japanese release was singled out (probably because of the Batman/Spiderman copyright thing).

Other examples are Altered Beast / Juuouki, and Ghouls'n Ghosts / Dai Makaimura, all featuring a mix of revisions and different releases, differences that can be spotted only via MPR serial.

The only Quackshot dump actually verified, the one ElBarto is talking about, is the "1.1" one, which has MPR-14371, so it's not rev "1.1", but it's simply rev "1.0" of MPR-14371. The 01 found in the header is totally misleading and should not be taken into account.

The current "1.0":
-could be a legit dump of a different chip, having an entirely different MPR serial
-could be a pirate
-could be a hack
-could be a rip
-whatever

It shouldn't be removed for the time being of course, maybe a confirmation dump will come for it too, but there's a strong probability that it's just a manipulated dump that was never on cart. All we can do for now is to label Quackshot dumps in a rational way:
-MPR-14371 (current "1.1") has the incorrectly applied "1.1" tag removed
-current "1.0" becomes a simple (Alt)

The 1.0/1.1 relationship between them we have now is incorrect.
Posted By: ranger_lennier Re: Fixed software lists - 01/23/10 03:33 AM
I've noticed that a lot of the software lists have good documentation of undumped software, but it's all in comments. Would it be better to put these into some sort of structure in the code so that, for example, we could auto-generate missing dumps lists?
Posted By: Duke Re: Fixed software lists - 01/23/10 12:09 PM
Yes, I think known missing software should be added with a NO_DUMP flag to document them.
Posted By: Haze Re: Fixed software lists - 01/24/10 01:21 PM
Originally Posted By Duke
Yes, I think known missing software should be added with a NO_DUMP flag to document them.


I disagree, and it goes against standard MAME policy.

It will just result in people distributing fake files, then people claiming to have the games because they have fake files.

Having a romset which is entirely fake makes no sense. Once it's dumped, it's dumped, until then, it isn't. Sites like UnMAMED can take care of what's needed. Raising awareness that something isn't dumped can also increase the asking price for it and if you include known undumped protos, beta builds etc. there is an almost infinite amount of data undumped.

The comments, and details on other sites have always sufficed in MAME, and the role of the project is to document what's known to exist. Hoarded dumps also don't get documented as they can't be verified by the other developers.

NO_DUMP is useful for marking specific components as missing (graphic roms, roms which were missing on an original PCB, MCUs etc.) not entire sets. One of the problems in MESS at the moment is that you have entire bios sets which are NO_DUMP which just gives a confusing message and makes it look like somebody added and tested the dumps, but decided to not make them public so put NO_DUMP instead (and again, results in people saying they have these sets when all they have are dummy roms of the right size) I would actually vote that such cases should be cleaned up and removed.





Posted By: Duke Re: Fixed software lists - 01/24/10 01:48 PM
Originally Posted By Haze

I disagree, and it goes against standard MAME policy.

It will just result in people distributing fake files, then people claiming to have the games because they have fake files.


I'm not talking about adding them with a 0-CRC, just the ability to make an generic, automatic list with the name, description, year and manufacturer. This would be very useful for the wiki.
Posted By: Haze Re: Fixed software lists - 01/24/10 01:58 PM
Originally Posted By Duke
Originally Posted By Haze

I disagree, and it goes against standard MAME policy.

It will just result in people distributing fake files, then people claiming to have the games because they have fake files.


I'm not talking about adding them with a 0-CRC, just the ability to make an generic, automatic list with the name, description, year and manufacturer. This would be very useful for the wiki.


The NO_DUMP flag is a '0 CRC', so your two statements don't make sense together.
Posted By: Duke Re: Fixed software lists - 01/24/10 02:04 PM
Okay then, if you insist call it "SOFTWARE_NO_DUMP".
Posted By: Haze Re: Fixed software lists - 01/24/10 02:17 PM
Originally Posted By Duke
Okay then, if you insist call it "SOFTWARE_NO_DUMP".


I'm still not convinced, MAME has never seen the need to do this, and there is too much room for debate and errors "Rhinoceros Bar Command?"

Best left to the comments IMHO, with things supported when they're dumped and proven without doubt to exist. Even things like alt. regions may not really exist, many games were reviewed / sold over here even if they were only available on import. The proof that something exists is the dump, and at the point it's worth supporting.


Posted By: Umran Re: Fixed software lists - 01/24/10 03:46 PM
Sorry for a daft question but how do I use this software list?
Is it like mame where I run it through clrmame to get a list of missing software?
Posted By: R. Belmont Re: Fixed software lists - 01/24/10 03:52 PM
The software lists are not end-user accessible yet, they're still work in progress.
Posted By: Umran Re: Fixed software lists - 01/24/10 04:00 PM
ok, looking forward to it, good work everyone.
Posted By: incog Re: Fixed software lists - 01/24/10 05:58 PM
Yeah, im not fully into the NO_DUMP idea, but it is still a working solution and requires no extra work for the result ranger_lennier was looking for. The only reason I don't like it is because I also list gaps in serial numbers as comments along with known undumped carts... and those may just be gaps, not real carts.

Listing them as NO_DUMP whould be kind of stupid if they never even existed.
Posted By: Vas Crabb Re: Fixed software lists - 01/25/10 02:57 AM
Originally Posted By Haze
I'm still not convinced, MAME has never seen the need to do this, and there is too much room for debate and errors "Rhinoceros Bar Command?"

Almost all of your arguments amount to "But MAME does x." Get over yourself - MAME isn't the definition of the one, true way to do everything. You're sounding more like a broken record all the time.
Posted By: ranger_lennier Re: Fixed software lists - 01/25/10 03:22 AM
I'm not sure why this would make it more likely for someone to make a fake dump. Since there wouldn't (indeed couldn't) be a CRC listed, MESS wouldn't even recognize it as that specific game. Do people ever make fake ROM's for the incomplete dumps in MAME?

Sure, there may be some mistakes in the undumped lists, but for a lot of systems it's well documented what games came out. If we're already listing this information in the code and on the Wiki, we might as well make it easier on ourselves.
Posted By: Haze Re: Fixed software lists - 01/25/10 07:37 AM
Originally Posted By ranger_lennier
I'm not sure why this would make it more likely for someone to make a fake dump. Since there wouldn't (indeed couldn't) be a CRC listed, MESS wouldn't even recognize it as that specific game. Do people ever make fake ROM's for the incomplete dumps in MAME?


Yes, if MAME lists something people want fake files for it. It's calmed down a bit since clrmame stopped making them, but it still happens.

Creating an entirely fake set makes it look like MESS/MAME supports something, when actually it doesn't because it's never been been dumped and never been verified in the emulator. This can end up having the reverse of the desired effect in that some people might think the game is already dumped (and thus not needed) and supported because it's in the exported lists, when it actually isn't. In other cases it can increase prices, because people will see the lists and think 'Hey, MESSdev want *this specific* cart, I can charge whatever I want for it'

Overall it just creates confusion.

Posted By: Haze Re: Fixed software lists - 01/25/10 07:37 AM
Originally Posted By Vas Crabb
Originally Posted By Haze
I'm still not convinced, MAME has never seen the need to do this, and there is too much room for debate and errors "Rhinoceros Bar Command?"

Almost all of your arguments amount to "But MAME does x." Get over yourself - MAME isn't the definition of the one, true way to do everything. You're sounding more like a broken record all the time.


Well, it's a pretty good guide as to what works and what doesn't. Learning from something is never a bad idea you know? It's not like I think everything done in MAME is right, and I've disagreed with, and changed policies in the past, but certain things like the handling of these cases is spot on.

I'm sure you can find examples in my recent posts of where I've said MAME standards could be better, or don't apply to MESS (when it comes to MESS needing to provide more hand-holding support for users due to more convoluted start-up sequences for example)

Posted By: gigadeath Re: Fixed software lists - 01/25/10 02:15 PM
I don't think there an absolute "correct" way to act here.

I sure don't think the situation would be as tragic as Haze thinks it would be if those comments were explicit. After all, things like the MD undumped list I added have been public since years. That same undumped list has been, in one form or another, on No-Intro site since I started curing the dat (2004?), and there were few caring, let alone anybody creating fakes. This isn't 1999 anymore, when people maybe were proud their fakes made their way into Goodgen.

On the other hand generic "dumps we need" lists can't be rendered correctly using the NO_DUMP flag. An "undumped" list is more like a 3-in-1 list, composed of actual "undumped" list + "to be redumped" list + "I want new infos to corroborate old infos I'm almost sure about already" list.

It includes:
-undumped revisions we know for sure they exist, from trustable sources like Xacrow or Ototo (ex. World of Illusions Rev A)
-games that I strongly suggest to be redumped (ex. later Ecco Jr)
-games that we simply need redump and relative chip serial only to define correct flag and release order for the various revisions (ex. Gain Ground)

You can't render something like that with a standard do-it-all NO_DUMP flag, so those comments are a necessity.

You can argue explicit notes help exposing dumping needs to the public, but I don't think that's a crucial point yet. MESS has no MAME-like exposure yet anyway.
Posted By: etabeta78 Re: Fixed software lists - 01/25/10 02:32 PM
my opinion is that carts which need to be redumped should be marked as BAD_DUMP, but carts which are completely undumped can be documented externally from the source, e.g. on the wiki.

just my 2c
Posted By: R. Belmont Re: Fixed software lists - 01/25/10 02:37 PM
Agreed. There's already a more or less authoritative NO_DUMP list maintained by No-Intro, so we don't need to duplicate it in the source. (And I gotta agree with Haze that there are good reasons we don't NO_DUMP the heck out of MAME that also apply here).
Posted By: Haze Re: Fixed software lists - 01/25/10 02:44 PM
Originally Posted By gigadeath

-games that I strongly suggest to be redumped (ex. later Ecco Jr)


Where there is good reason to think a dump is bad, but there is no other dump of that revision available the BAD_DUMP flag should be used, pending verification of the set.
Posted By: gigadeath Re: Fixed software lists - 01/25/10 02:48 PM
I think completely undumped games sure have to be documented in the wiki, but seeing that they are so few after all, I'd keep them in the source too. I'm a strong supporter of the "the more exposure, the more returns" thinking, at least when the pros outnumber the cons. And the cons are almost nihil in terms of space used.

The "to be redumped" games issue is too complex to be solved with a BAD_DUMP flag. Point is, most of the requested redumps are not actual bad dumps, there's an 80%-vs-20% probability that they're really good dumps, it's only that we need to know for sure. That's even before discussing games that need a redump only to get the correct revisions order. If you put a BAD_DUMP flag, get a redump, and the redumper sees that 9 times out of 10 the redump actually matches the old one, the BAD_DUMP flag loses any meaning, and you could lose contributors.

Apart for clearly hacked things like the Ninja Gaiden proto and such, I still think a short, non-ambiguous comment is much better that a rigid BAD_DUMP flag.

Posted By: Haze Re: Fixed software lists - 01/25/10 02:53 PM
Originally Posted By gigadeath
I think completely undumped games sure have to be documented in the wiki, but seeing that they are so few after all, I'd keep them in the source too. I'm a strong supporter of the "the more exposure, the more returns" thinking, at least when the pros outnumber the cons. And the cons are almost nihil in terms of space used.

The "to be redumped" games issue is too complex to be solved with a BAD_DUMP flag. Point is, most of the requested redumps are not actual bad dumps, there's an 80%-vs-20% probability that they're really good dumps, it's only that we need to know for sure. That's even before discussing games that need a redump only to get the correct revisions order. If you put a BAD_DUMP flag, get a redump, and the redumper sees that 9 times out of 10 the redump actually matches the old one, the BAD_DUMP flag loses any meaning, and you could lose contributors.

Apart for clearly hacked things like the Ninja Gaiden proto and such, I still think a short, non-ambiguous comment is much better that a rigid BAD_DUMP flag.



That's why I said if there is good reason to believe a redump is needed, for cases like the Ecco it should be pretty obvious if some of the data is corrupt, it's only a revision of an existing set afterall.

If it does turn out to be the same the BAD_DUMP flag gets dropped, and a comment stating the the revision is buggy gets added.
Posted By: gigadeath Re: Fixed software lists - 01/25/10 03:01 PM
What I'm saying is that, even if you extract the comments and put them all in the wiki, at least don't let it boil down to simplistic NO_DUMP and BAD_DUMP flags like in the source. Some things NEED to be verbose.

The source can be simplified, that's fine to me, but the other info-containers have to carry the full comments, reducing everything to flags leads to loss of info.

Posted By: Haze Re: Fixed software lists - 01/25/10 04:19 PM
Originally Posted By gigadeath
What I'm saying is that, even if you extract the comments and put them all in the wiki, at least don't let it boil down to simplistic NO_DUMP and BAD_DUMP flags like in the source. Some things NEED to be verbose.

The source can be simplified, that's fine to me, but the other info-containers have to carry the full comments, reducing everything to flags leads to loss of info.



Yes, not everything can be represented with flags, although in some cases I think a few more flags would help, especially to mark known Read Protected PAL devices, MCUs with internal roms etc. Not all NO_DUMP / BAD_DUMPS are equal, even in MAME.

For more complex cases source comments are invaluable.
Posted By: judge Re: Fixed software lists - 02/17/10 04:36 PM
After some thinking and offline discussions it looks like it is more useful to not statically compile the software lists into the emulator itself but to rather build support for a more flexible xml format (more flexible than the current hashfiles anyway). Also the xml format makes it possible to keep the software and data information together instead of splitting them in rom_regions and software macros.

With the number of systems supported by MESS it is not unthinkable that the software list data could become larger than the emulator executable itself.

I have converted the currently available c lists to xml already (and added some more info to the wonderswan cartridges) here: http://www.jdg.info/mess/software_list/

The idea is that the software can be identified uniquely using <softwarelist>:<softwarename>

The emulator would look up the software information from the xml file, build up a mame rom region/rom information list dynamically and use that to eventually actually load the software or to perform a -romident operation.

At the moment software lists contain separate items all with a similar interface. Interface is used here as the hardware interface: sega master system cartridge, pokemini cartridge, etc. A driver could define a cartridge slot which takes cartridges with a sega master system interface for instance.
For the time being this means that multi floppy software gets multiple entries in the software list; not ideal so this is something I still want to look into. Improving that involves pushing the interface part into each separate software item part.

Apart from the mentioned multi part software issue, is there any (obvious) stuff I've missed in the xml files?

Posted By: AaronGiles Re: Fixed software lists - 02/17/10 07:19 PM
So basically this is the GAME() macro with the attached ROM_START/ROM_END data, wrapped in XML.

It's a fine starting point, but it seems to me you're missing a lot of functionality that ROM regions provide, including byte selection, swapping, inverting, etc. For carts with single ROM files this works fine, I guess, but it seems awfully limited.
Posted By: judge Re: Fixed software lists - 02/17/10 07:53 PM
Basically the GAME() macro and the ROM_START/ROM_END data are the most important pieces of information for defining a piece of software.

I've only added the rom region features I came across until now. Eventually only combinations should be supported. There are some examples in megadriv.xml and 32x.xml, and for multi file things bbcbc.xml and supracan.xml.

Other stuff that still needs to be added is things like mapper/pcb type so the emulation can determine how to handle banking, extra features, etc.
Posted By: ht1848 Re: Fixed software lists - 02/17/10 10:47 PM
I like the direction(from a non-technical point of view). Is there a better name for dataarea? It just sounds bad, lol.

How many "fluff" tags(maybe a TBD policy question)can/should in to the xml file?

I see at least one with <serial> which is useful but not nessesary, unlike mapper/pcb which will be required. Is the intention to store many things things like alt Language descriptions, catagorization (# of players, game type, Maws type stuff), Region (Japan,Detroit) - OR - would we keep teh mess xml files to minimum and keep those types of fluff items be kept as a reference somewhere else and just link via the softwarelist:software name key?

(advision_cart:supcobra for example I think defines a unique key right? because supcobra will be unique per that file)

I like it either way, just trying to understand.
Posted By: judge Re: Fixed software lists - 02/17/10 11:09 PM
A dataarea is basically a mame rom region. They would be used to store the data for software, not just rom, so "rom_region" doesn't make sense; but using "region" would be quite confusing. If you know of a better name, let me know wink
Posted By: Haze Re: Fixed software lists - 02/17/10 11:59 PM
so, how fast is it with romident etc. ?

what level of security does it provide to prevent people pick-and-mixing external files, and how is that implemented without hampering development?
Posted By: judge Re: Fixed software lists - 02/18/10 08:15 AM
romident is reasonably fast with the current set of xml files:
Code:
time ./messd -romident ~/Projects/Emulation/MessSoftware/neogeo_pocket/roms/0116\ SNK\ vs\ Capcom\ Card\ Fighters\ Clash\ 2\ Expand\ Edition\ \[\!\].npc

0116 SNK vs Capcom Card Fighters Clash 2 Expand Edition [!].npc= snk vs. capcom - card fighters 2 - expand edition (japan).bin  ngp:svccard2 SNK vs. Capcom - Card Fighters 2 - Expand Edition (Jpn)

real	0m0.359s
user	0m0.170s
sys	0m0.034s


Currently there is no locking down, but that could be added by signing the xml files and checking the signature when starting the emulator or when opening an xml file.
Posted By: etabeta78 Re: Fixed software lists - 02/18/10 08:21 AM
I spent some time to check the format.

overall opinion: it's good. smile

thing that can be added / changed:

1. year -> release (or date). in some cases we have more info than the year only, e.g. for Japanese games the whole date is available and it could be documented
2. I would add an optional developer field after the publisher, so that both can be used by frontends
3. it's hard to say without a look at the MESS-side of the code for support this, but would multiple dataarea be allowed to build multiple rom-regions at loading? this would be of use for AES (and possibly NES) and I think multiple floppies could go in the same software item, but loaded in separate dataarea
4. I already told you this offline, but why not add a "type" parameter to dataarea (namely, type=cart,floppy,tape,etc.)? this could be either replace the interface or be kept separate from it, but it would be valuable of its own because it would allow MESS to e.g. load a multi-floppy game starting from the first disk appearing in the software entry (i.e. the first dataarea with type=disk), and then use key-shortcuts to cycle through the other disks (with a for loop through the dataarea of the correct type only)

other comments:
- missing developer field aside, I think we have enough "non-emulation" fields for frontends
- how would you exactly launch a game from a software list, with the current approach? and how would you launch it if a system has two software lists (for different interfaces)?
Posted By: judge Re: Fixed software lists - 02/18/10 10:43 AM
Originally Posted By etabeta78
I spent some time to check the format.

overall opinion: it's good. smile

thing that can be added / changed:

1. year -> release (or date). in some cases we have more info than the year only, e.g. for Japanese games the whole date is available and it could be documented


I was thinking about the same thing; I did add actual release dates to the wonderswan xml.

Quote:

2. I would add an optional developer field after the publisher, so that both can be used by frontends


I'm not sure if we should be adding it to the software lists or whether that should be stored somewhere else.

Quote:

3. it's hard to say without a look at the MESS-side of the code for support this, but would multiple dataarea be allowed to build multiple rom-regions at loading? this would be of use for AES (and possibly NES) and I think multiple floppies could go in the same software item, but loaded in separate dataarea


Yes, multiple dataareas are supported.

Quote:

4. I already told you this offline, but why not add a "type" parameter to dataarea (namely, type=cart,floppy,tape,etc.)? this could be either replace the interface or be kept separate from it, but it would be valuable of its own because it would allow MESS to e.g. load a multi-floppy game starting from the first disk appearing in the software entry (i.e. the first dataarea with type=disk), and then use key-shortcuts to cycle through the other disks (with a for loop through the dataarea of the correct type only)


Last night I was thinking about adding a something like a "software part" which is one cartridge, disk, etc; where each software part can have more than one dataarea. The parts can be seperately identified like <swlist>:<software>:<part>.
And if you don't supply the part it just picks the first one encountered.

Quote:

other comments:
- missing developer field aside, I think we have enough "non-emulation" fields for frontends
- how would you exactly launch a game from a software list, with the current approach? and how would you launch it if a system has two software lists (for different interfaces)?


The xml lists are not tied to systems or interfaces yet. The emulation can use the interface attribute to check if a piece of software can be mounted. If they are tied then it'd be a list of lists.
Posted By: etabeta78 Re: Fixed software lists - 02/18/10 11:13 AM
Originally Posted By judge
Originally Posted By etabeta78

4. I already told you this offline, but why not add a "type" parameter to dataarea (namely, type=cart,floppy,tape,etc.)? this could be either replace the interface or be kept separate from it, but it would be valuable of its own because it would allow MESS to e.g. load a multi-floppy game starting from the first disk appearing in the software entry (i.e. the first dataarea with type=disk), and then use key-shortcuts to cycle through the other disks (with a for loop through the dataarea of the correct type only)


Last night I was thinking about adding a something like a "software part" which is one cartridge, disk, etc; where each software part can have more than one dataarea. The parts can be seperately identified like <swlist>:<software>:<part>.
And if you don't supply the part it just picks the first one encountered.


preliminary question: is there any case where a single piec of software (say a cart) needs to load different part of its content to different rom regions (and not simply to different offsets of the same rom region)?

if not, I still think it would be easier to only have a single dataarea for each component of the software (being it a cart, a floppy disk or a tape) and hence multiple dataarea when multiple disks/tapes/carts are present

if yes, I agree a <part> could be of use, and then my type="cart, floppy, tape" suggestion moves from the datarea entry to the part one

Originally Posted By judge
Originally Posted By etabeta78

other comments:
- how would you exactly launch a game from a software list, with the current approach? and how would you launch it if a system has two software lists (for different interfaces)?


The xml lists are not tied to systems or interfaces yet. The emulation can use the interface attribute to check if a piece of software can be mounted. If they are tied then it'd be a list of lists.


I'm still not sure how this translates to commands to launch a given game in MESS, but I think I will have to wait for the code to be in smile
Posted By: Haze Re: Fixed software lists - 02/18/10 11:38 AM
Originally Posted By judge
romident is reasonably fast with the current set of xml files:
Code:
time ./messd -romident ~/Projects/Emulation/MessSoftware/neogeo_pocket/roms/0116\ SNK\ vs\ Capcom\ Card\ Fighters\ Clash\ 2\ Expand\ Edition\ \[\!\].npc

0116 SNK vs Capcom Card Fighters Clash 2 Expand Edition [!].npc= snk vs. capcom - card fighters 2 - expand edition (japan).bin  ngp:svccard2 SNK vs. Capcom - Card Fighters 2 - Expand Edition (Jpn)

real	0m0.359s
user	0m0.170s
sys	0m0.034s


Currently there is no locking down, but that could be added by signing the xml files and checking the signature when starting the emulator or when opening an xml file.


That's actually a lot faster than I expected, I wonder how well it will scale.

I'm still not convinced on the security / signing side of things yet tho. The last thing I want is people reporting bugs using older xml files and set names because they prefered to not update their roms (which changes to the roms could easily have renamed sets and replaced bad dumps etc.)

The developer / publisher thing I'm not sure about. The same rom can potentially be published by different companies in different regions anyway. Mame tends to put all of that info into one field, which isn't always optimal, but there are so many variations it's impractical to do much else. (most Toaplan games have about 4 different licencees for 6+ regions, and they don't get listed)

For longer dates, it might not always be practical either.. MAME has a policy of using the year of the first published revision. Of course, except for with games where the software contains version information, or the rom labels are useful it's hard to know if the supported sets really are the first revision anyway. Of course, it would help solve 'which came first xx or yy' debates when the year is the same which comes up quite often with similar early arcade titles when they were being pumped out in the 80s.
Posted By: judge Re: Fixed software lists - 02/18/10 11:42 AM
Some exmaples would be:
Code:
1. mess megadriv -cart megadriv:columns
2. mess amiga -flop0 amiga:bigdemo1:flop1 -flop1 amiga:bigdemo1:flop2


When software lists do get tied with drivers then
Code:
mess megadriv -cart columns

could be made possible by looking for the first matching columns entry in one of the lists associated with the megadriv driver.

The code to do all this hasn't been made yet, this is just how I see things could be made working.

Posted By: Haze Re: Fixed software lists - 02/18/10 11:50 AM
Originally Posted By judge
Some exmaples would be:
Code:
1. mess megadriv -cart megadriv:columns
2. mess amiga -flop0 amiga:bigdemo1:flop1 -flop1 amiga:bigdemo1:flop2


When software lists do get tied with drivers then
Code:
mess megadriv -cart columns

could be made possible by looking for the first matching columns entry in one of the lists associated with the megadriv driver.

The code to do all this hasn't been made yet, this is just how I see things could be made working.



Again I question the -cart switch etc. These are my *main* issue with mess for basic use right now. They're ambiguous. The lists should know what each thing is, and extra switches shouldn't be needed unless you want to override the default.
Posted By: netol Re: Fixed software lists - 02/18/10 11:57 AM
and why not
Quote:
mess megadrive columns

"megadriv" doesn't sound nice
Posted By: judge Re: Fixed software lists - 02/18/10 12:02 PM
That may be an improvement for later when lists are actually usable. Not sure if it's worth all the trouble yet, most end-users will likely end up using a frontend of some kind.

I personally like to explicitly tell programs what to do. I've gotten into too many fights with programs because they decided to think for themselves.
Posted By: etabeta78 Re: Fixed software lists - 02/18/10 12:22 PM
from the way I see the possible implementation, I think the list should not be specified at command line if not necessary: i.e. each driver should have a small set of 'default' lists (say one for each device). more complex things like cross compatibility (say between coupe and spectrum, or aleste and msx carts) can be allowed by letting the user specify the gamelist to be used as a separate option (say "mess aleste -list msx_carts -cart game")

as a further step, we might allow a command like "mess megadrive columns", by browsing (default) lists in the order they have been added in the source and by launching the first match found
Posted By: etabeta78 Re: Fixed software lists - 02/18/10 12:42 PM
Originally Posted By netol
and why not
Quote:
mess megadrive columns

"megadriv" doesn't sound nice


because megadriv is the parent and parents must have names shorter than 8 chars. (and no, making genesis parent is not a great solution, if we want to match MAME habit of setting European releases as parent, being usually the latest)
Posted By: Lord Nightmare Re: Fixed software lists - 02/18/10 02:20 PM
Originally Posted By etabeta78

preliminary question: is there any case where a single piec of software (say a cart) needs to load different part of its content to different rom regions (and not simply to different offsets of the same rom region)?


NES has separate address spaces for PPU-accessible rom/vram and cpu-accessible rom/ram.

I think neogeo AES also has separate spaces for gfx roms and sound roms and code roms? (not sure about this)

LN
Posted By: R. Belmont Re: Fixed software lists - 02/18/10 02:44 PM
Yes, NES and AES both (should in the NES case, does in the AES one) load to multiple regions, and I think TI99 does as well.

The -cart/-flop/-hard switches should continue to be used only for "unlisted" media. Using them in conjunction with the lists IMO makes MESS *more* complex instead of less. And the system name prefix is unnecessary since you've already specified it. "mess genesis sonic2" or "mess snes -cart CGFM_Mode7_Demo.bin" should be fine.
Posted By: judge Re: Fixed software lists - 02/18/10 04:13 PM
You will need the system prefix if you want to mount software from a different system. Msx floppies read fine on a pc and pc dd floppies could be read on an msx (it's all microsoft..). The emulation and lists should not disallow the user from doing that; or inserting a pc floppy in an amiga floppy drive: they are all 3.5" floppies.
Posted By: etabeta78 Re: Fixed software lists - 02/18/10 04:24 PM
that's where my suggestion about having a new option

-uselist xxx_list

to tell the driver to search for games in the specified list comes into play wink

and about the switches, what if a same game was available both in tape and floppy format? being forced to have non-conflicting shortnames across different media could become a nightmare. hence, my proposal is: say we have both a bublbobl cart and a floppy one for c64; if I launch

mess c64 bublbobl

let MESS pick up the first bublbobl it finds (say in the first list we add to the driver, e.g. carts), and force me to specify

mess c64 -flop bublbobl

if I really want to access the bublbobl file available in the floppy list. personally, I will probably end up specifying the media most of the time for systems with multiple media available, but I see no reason to do the same for all systems, if we can avoid it

p.s. and thanks for the answers on multiple regions, I should have thought to that. This means we need a part entry in the xml and (possibly) a type field for the part entry. This might be handy as well for games requiring a cart+tape (e.g. a2600 + supercharger games): launching the game would create both the tape and the cart region, and MESS could access both...
Posted By: R. Belmont Re: Fixed software lists - 02/18/10 04:52 PM
Originally Posted By judge
You will need the system prefix if you want to mount software from a different system. Msx floppies read fine on a pc and pc dd floppies could be read on an msx (it's all microsoft..). The emulation and lists should not disallow the user from doing that; or inserting a pc floppy in an amiga floppy drive: they are all 3.5" floppies.


Yes, but that's an "advanced" use. IMO the software lists should be aimed squarely at giving a MAME-like experience for people who Just Want To Play Games (and also at enabling loading "correct" chip dumps from e.g. NES where it's sorely needed).

People like you or me or The Flying Ape who dig deeper into the emulated computers will end up bypassing that system anyway.
Posted By: etabeta78 Re: Fixed software lists - 02/18/10 05:02 PM
Originally Posted By R. Belmont
(and also at enabling loading "correct" chip dumps from e.g. NES where it's sorely needed).


this is being (slowly) worked on
Posted By: R. Belmont Re: Fixed software lists - 02/18/10 05:02 PM
eta: a million times no on overloading the existing media type switches. The software lists must not hurt the existing power user functionality or make it harder to use.

I suggest instead that games available on more than one media type be added to the database with conflicting shortnames. On the commandline you'd do "mess c64 pacman_cart" or "mess c64 pacman_floppy" (could be a : or | or / instead of _) and it'd figure out based on what regions/media types each entry defines which one you want. If you specified plain "mess c64 pacman" it'd load the first one found.
Posted By: etabeta78 Re: Fixed software lists - 02/18/10 05:08 PM
I was more oriented into combining the new features (lists) with the old (media switches like -flop) where possible, whithout making anything more difficult: if you launch

mess c64 -cart aaa.img

and there no aaa in the software list, it would simply load the file you pointed to.

however, if it might create conflicts (e.g. because you have both a "aaa" entry in the list and a aaa.img in the same directory as MESS), then I agree we can simply add the media at the end of the name, or even require the user to specify the right list (say "-uselist c64_floppy") to access the alternate media.

however, I would prefer not to put together different medias in the same <software> item, because it would create conflicts with games consisting of multiple media (the a2600+supercharger case comes to mind)


EDIT: I'm happy to see we agree on picking up the first item found, though wink
Posted By: JoJo Re: Fixed software lists - 02/18/10 05:41 PM
Originally Posted By etabeta78
I was more oriented into combining the new features (lists) with the old (media switches like -flop) where possible, whithout making anything more difficult: if you launch

mess c64 -cart aaa.img

and there no aaa in the software list, it would simply load the file you pointed to.


Actually, in your example, I don't see how you can confuse a software list entry with an image file: list entries (as driver names) cannot have a dot in their name - on the contrary, the current system enforces image files to have an extension chosen from a restricted set.

Originally Posted By etabeta78
However, I would prefer not to put together different medias in the same <software> item, because it would create conflicts with games consisting of multiple media (the a2600+supercharger case comes to mind)


I agree with this - I think that it's better to have different lists for different media, plus a "multiple media" list.
Posted By: judge Re: Fixed software lists - 02/18/10 06:35 PM
Originally Posted By etabeta78

however, I would prefer not to put together different medias in the same <software> item, because it would create conflicts with games consisting of multiple media (the a2600+supercharger case comes to mind)


Actually that involves two different pieces of software: firstly the supercharger cartridge, and secondly a seperate cassette with a piece of software. They were sold separately afaik.
Posted By: etabeta78 Re: Fixed software lists - 02/18/10 06:43 PM
d'oh. I didn't know that... it sounds a bit like the pce cd case, though...

any idea about how to deal with it? software items with cart+tape? or forcing people to use two media at command line?

the latter could be a good starting point, but once the code is in we would need some brainstorming about this smile

Posted By: R. Belmont Re: Fixed software lists - 02/18/10 07:28 PM
I guess I don't understand why this is complicated. As long as the lists are going to have to direct various files to various memory regions, shouldn't they also be able to direct them to various devices? For the Supercharger and PCECD cases I'd suggest something like a "required parent" where the required parent of all CD games would be the CD-ROM BIOS cartridge (and all supercharger tapes would have a parent that's the Supercharger itself).
Posted By: judge Re: Fixed software lists - 02/18/10 07:36 PM
There are different versions for the pce cd-rom bios though wink

For the supercharger case; the cassette mounting should only be allowed when there's a cartridge of type "suerpcharger" plugged in. This is actually one of those cases where plugging in a piece of hardware makes some new hardware features available. Currently the supercharger cassette port is always present in the a2600 driver.


Posted By: R. Belmont Re: Fixed software lists - 02/18/10 07:51 PM
Again, mix-and-matching CDs and BIOSes is a you-and-me feature, not a "just wanna play games" feature. People who just want to play the games should have it default to a version that works. In the PCECD case, whichever CD-ROM cart was current when the game shipped would be one reasonable default, and forcing everything to the latest (3.0?) cart would be another.
Posted By: judge Re: Fixed software lists - 02/18/10 07:55 PM
Maybe besides cloneof a required_software attribute could be added.
Posted By: Vas Crabb Re: Fixed software lists - 02/18/10 10:10 PM
People who "just wanna play games" probably won't be using MESS anyway.
Posted By: R. Belmont Re: Fixed software lists - 02/18/10 10:20 PM
Actually, a fair number of those people already do. If you're building a cabinet and want to include console or computer games, MESS is probably the easiest choice since it inherits everything that makes MAME work well in that environment.
Posted By: judge Re: Fixed software lists - 02/19/10 07:26 AM
Also for a lot of systems only windows based emulators are available.
Posted By: etabeta78 Re: Fixed software lists - 02/19/10 07:29 AM
and for some exotic systems (gamepock, supracan, vii, etc.) it is the only emu wink
Posted By: judge Re: Fixed software lists - 02/19/10 11:48 AM
I have this now:
Code:
<?xml version="1.0"?>
<!DOCTYPE softwarelist [
    <!ELEMENT softwarelist (software+)>
        <!ATTLIST softwarelist name CDATA #REQUIRED>
        <!ATTLIST softwarelist description CDATA #IMPLIED>
        <!ELEMENT software (description, releasedate?, publisher, serial?, part*)>
            <!ATTLIST software name CDATA #REQUIRED>
            <!ATTLIST software cloneof CDATA #IMPLIED>
            <!ELEMENT description (#PCDATA)>
            <!ELEMENT releasedate (#PCDATA)>
            <!ELEMENT publisher (#PCDATA)>
            <!ELEMENT serial (#PCDATA)>
            <!ELEMENT part (dataarea*)>
                <!ATTLIST part name CDATA #REQUIRED>
                <!ATTLIST part interface CDATA #REQUIRED>
                <!ELEMENT dataarea (rom*)>
                    <!ATTLIST dataarea name CDATA #REQUIRED>
                    <!ATTLIST dataarea size CDATA #REQUIRED>
                    <!ATTLIST dataarea databits (8|16|32|64) "8">
                    <!ATTLIST dataarea endian (big|little) "little">
                    <!ELEMENT rom EMPTY>
                        <!ATTLIST rom name CDATA #IMPLIED>
                        <!ATTLIST rom size CDATA #REQUIRED>
                        <!ATTLIST rom crc CDATA #IMPLIED>
                        <!ATTLIST rom md5 CDATA #IMPLIED>
                        <!ATTLIST rom sha1 CDATA #IMPLIED>
                        <!ATTLIST rom offset CDATA #IMPLIED>
                        <!ATTLIST rom status (baddump|nodump|good) "good">
                        <!ATTLIST rom loadflag (none|load16_byte|load16_word_swap|reload) "none">
]>


The old 'year' attribute has been renamed to 'releasedate'.

The 'softwarelis't tag defines a grouping of software. The 'name' attribute holds the shortname for the list; the 'description' attribute can be used to provide some more verbose information of the items in the list.

I have added 'part' to the 'software' element. Most pieces of software will only have 1 part. A 'part' can be a cartridge, a floppy, a cassette, etc. The "thing" a software part can be plugged into is defined by the 'interface' attribute of the part. This could be something like "floppy35", "megadriv_cart", "cassette", "cd".

A 'part' can have more than one 'dataarea', so we can distinguish between chr and prg roms for nes carts for instance.

I have replaced the 'dataarea' 'layout' attribute with two seperate attributes 'databits' and 'endian'.

If someone knows any better names for 'dataarea' or 'part', let me know.

Todo:
- add pcb/mapper/whatever information to 'part'.
- add required software attribute (to 'part').
- add more loadflag options
- add support for rom_copy, rom_continue, etc
- create code...
Posted By: RColtrane Re: Fixed software lists - 02/19/10 11:48 AM
Originally Posted By Vas Crabb
People who "just wanna play games" probably won't be using MESS anyway.


That's not true wink
Posted By: Barry Nelson Re: Fixed software lists - 02/19/10 06:43 PM
Originally Posted By RColtrane
Originally Posted By Vas Crabb
People who "just wanna play games" probably won't be using MESS anyway.


That's not true wink


Mess is currently one of the best, or in some cases, the only emulator for some platforms, and it runs on Mac OS X, which many other emulators do not. Face it, people ARE using the software that we are all contributing to here.

Also, FYI, there is actually NEW software being released STILL for a few legacy systems. Amiga is one. Another is platform that still gets an occaisonal new software released for it is the CoCo 3 (Tandy).

Coco3
donkey kong port ->http://www.axess.com/twilight/sock/dk/ March 31st, 2007
http://www.axess.com/twilight/sock/dk/other.html
My own patches to the real time clock drivers for Y2K for OS9
http://www.nitros9.org/
NitrOS-9 News

March 15, 2009

"Nightly" versions of NitrOS-9 are now available. You can help the project by using the latest build and reporting any bugs you find.

Amiga
http://www.amiga.org/forums/showthread.php?t=50292

AmiVNC4 v2.2.0. - (OS4) VNC server.
Added Zlib support which should speed up transfers a lot on slow networks. Now startable from WB with tooltypes. Should use less CPU and be more responsive when detecting a screen change.

BackUp v1.70 - (68k) Backup program that mirrors a directory structure.
Added an option so that you can select that files over a given size should not be copied. Added two options to prevent the setting of Amiga protection bits and Amiga file comments. Fixed a few minor bugs.

MPlayer-GUI v1.46 - (OS4) GUI for MPlayer.
It is now possible to start MPlayer-GUI with arguments from CLI or WB and files will be added to the GUI that is already opened. Added a Stop button. Added #?.flv to the default file pattern.

MultiRen v1.69 - (68k) Powerful multi-file renaming tool using MUI. Has plugin-support for MP3-renaming and more.
Fixed drag'n'drop of directories in Ambient. Added some stack to prevent crashes when opening the MUI settings on PPC systems. Minor fixes..

JoinSplitter v1.4 - (68k) Split and join files with a GUI.
It is possible to have joined files automatically deleted after being joined. Better handling of large files over 2GB, and joining of files that will result in more than 4GB.

UTF-8 v1.2 - (68k) This program lets you type UTF-8 text everywhere (like in web browser forms for example).
UTF-8 is now a commodity if started from Workbench. From Workbench it reads a few tooltypes.

http://aros.sourceforge.net/

AbiWord Ported To Amiga OS4 The package also includes a CSS/JS capable browser.



Posted By: etabeta78 Re: Fixed software lists - 02/20/10 12:00 PM
a small comment about lists, given that now also nintendo lists covering gb, gbcolor, snes and n64 have been converted to xml (with gba and nes almost ready) and because I think I know what MESSFan wants to post about...

at this stage, MESS will not use any existent rom database for its lists except as starting point (and also in this case, only for the best possible checksums).

Indeed, imho, all current sources have aspects which make them not really suitable for direct use in MESS:

1. nointro sets are wonderful, but they do not preserve pirate dumps and they list all languages for all games (something that belongs more to frontend than to embedded lists... like MAME we should use languages in the description only when they are really necessary to distinguish between two releases, e.g. for some gba games), hence they need additions and renames. also they do not use proper chip labels and split files even when these info are available
2. TOSEC and GoodTools sets are very comprehensive, but they keep listing (especially TOSEC) all bad/overdumped/underdumped files around [1]
3. sometimes sets are outdated in every existing source (e.g. for crvision) and in these cases is definitely easier to directly work with dumpers and other emu authors than to rely on available sets

given the above, we probably will go on by our own path. but of course any contribution/collaboration is more than welcome, especially if it helps fixing bad dumps or incorrect info.

an obvious example is given by gigadeath's wonderful contributions to the megadrive list (and bootgod provided me a lot of info about NES carts which I hope to use pretty soon wink )


[1] although they would become very useful, if we decide to add some demos as well in the lists, or to create some additional misfit-like lists for test programs
Posted By: R. Belmont Re: Fixed software lists - 02/20/10 12:39 PM
For the record, his deleted post was basically a diatribe against Cowering. We're *well* aware of the deficiencies of Good* lists smile
Posted By: Haze Re: Fixed software lists - 02/20/10 01:41 PM
Originally Posted By Vas Crabb
People who "just wanna play games" probably won't be using MESS anyway.


People who just want to play games are useful for testing and ensuring the emulator works well. They need to be a consideration, and need things to be easy.

There's no reason that a few years down the line MESS won't be the first choice for many systems, and the 'all under one roof' aspect is handy. MAME might not be the best emulator for all the systems it emulates at any given time, but it gets used because it's relatively easy to use, and convenient once you realise it needs specific romsets etc. Console users might not be used to this yet, but anybody familiar with MAME will be able to pick up the concept pretty quickly.

The target for all these changes should be to make MESS the 'MAME' of other systems. MAME already has a large user-base, and keeping things working in a similar way to MAME means that people familiar with MAME can support MESS users more easily, and are more likely to use MESS themselves due to the level of familiarity. As the emulation of the various systems improves it will become more and more accepted as a good way to play the games as well as a monumental achievement in the proper documentation of the systems.

For this to happen the entry point for usage needs to be as close to MAME as possible for this to happen IMHO, and it needs to be more obvious that this is MAME, but for consoles and computers. After a while people will pick up the more advanced techniques as long as they have something they can relate to in the first place, otherwise they'll just give up.

As for rants against the Good tools? well yeah, I think everybody is aware of the problems with the current lists and databases and hopefully MESS will provide an open platform for doing things in a more organized, detailed, correct and maintainable way. The 'Good' tools served people well for a long time, and while you can easily argue that they built up a mentality of collecting bad dumps etc. and even have some known bad dumps marked as verified good they did provide some method of organization and ensured that a number of dumps weren't just lost forever. Can you imagine if the 'good' project had been started by somebody who only cared about listing the latest revisions of games instead, or something equally horrific? We'd be in far more of a mess than we are now wink Believe me, there are people out there who would do that, NeoRageX development was all about hacking the roms work with the emulator, and declaring those roms good, so the good tools weren't *all* bad.

So.. they served a purpose, but moving forward things can be done in a better way, I don't think anything more needs to be said about it. Evolution happens, and I think people are over the OMG I can play MegaDrive games on my PC stage and ready to start doing things the proper way ;-)

As an aside, I'm still concerned that there aren't good methods to dump some CD formats yet, That CDi-Ready stuff sounds curious, because by the sounds if it the data track is actually an 'audio' track (the data is contained in the pregap of an audio track, like some hidden 'rewind to play' tracks on some audio cds?) Tosec don't even know how to document these properly, and the only rips that work are ones where they've been hacked to report the first track as a data track. It's a problem that's nagging me a bit, and hopefully one that MESS/CHD format can solve so that others can reference it and use properly dumped data that actually works.

Final note, was it intentional with the various recent MESS->MAME code-porting that the throttle key is no longer mapped by default? This seems a slightly odd decision as it's handy to have, but I remember it not being mapped in MESS either, it was one of those nagging inconsistencies.
Posted By: etabeta78 Re: Fixed software lists - 02/21/10 03:27 PM
a couple more thoughts on the xml structure...

looking at judge's current proposal for the software element

Code:
            <!ELEMENT description (#PCDATA)>
            <!ELEMENT releasedate (#PCDATA)>
            <!ELEMENT publisher (#PCDATA)>
            <!ELEMENT serial (#PCDATA)>
            <!ELEMENT part (dataarea*)>


I was wondering if it could be better to stay closer to MAME by only having first

Code:
            <!ELEMENT description (#PCDATA)>
            <!ELEMENT year (#PCDATA)>
            <!ELEMENT publisher (#PCDATA)>
            <!ELEMENT part (dataarea*)>


where "description" & "year" might be taken from the title screen (when available), or coming from the cart/box title and release date, when no title screen is available (this would match MAME behaviour and result quite user friendly for casual users)

+ we could have a single additional element

Code:
            <!ELEMENT frontend (developer?, releasedate?, alttitle?, serial?, 
any_other_optional_info_you_want_to_add?)>


to collect any additional info we have for games of a specific system. In this way we can

1. document alternate titles used on the game box / manual, and the precise release dates, and even other info like the supported languages in-game or the specific controllers (if any)

2. make life easier to frontend authors, since there are only 3 basic info (title+year+producer) available for all titles in all lists, while the optional info, which frontends can either use or ignore, are all collected in a single place...

of course, MESS would mainly use title+year+producer plus the software "parts", but if more info are available it could be smart to collect them altogether rather than spreading them in several places...
Posted By: judge Re: Fixed software lists - 03/02/10 12:10 PM
The 'year' element from MAME is actually intended to document when something was released. To make that intent clearer I had renamed the element to 'releasedate'.

To document alternate titles in different languages I was thinking about making it possible to have more than one 'description' element with a 'lang' attribute.

Posted By: Braille Re: Fixed software lists - 03/02/10 07:27 PM
Actually, that's not true. MAME's year is related to game's copyright (unlike others databases). This can be seen in the following discussion:
http://mametesters.org/view.php?id=3105
Posted By: judge Re: Fixed software lists - 03/02/10 07:36 PM
Hm, ok, thanks for pointing that out.
Posted By: Stiletto Re: Fixed software lists - 03/02/10 08:10 PM
Yeap, that's how it's been since I can remember.

"Official" metadata for "year" comes from copyright screen (if available) and from definitive sources (if not). If completely not available, some people used to use "197x" or "198x" to try to define a decade when known but lately these have all been converted to "????" (mostly because the policies surrounding decade declaration were arbitrary, and some people would use "197?" instead, and this may inspire more people to research the proper year if available, etc. etc.)

Actual release dates are mentioned in the sourcecode in mamedriv.c, mostly due to the exhaustive research of Yasuhiro Ogawa. However (unfortunately?) that's all the more respect that's given to them.
Posted By: judge Re: Fixed software lists - 03/02/10 08:18 PM
It makes sense to only put the copyright date there and let other external lists handle the actual dates (multiple dates are possible in different countries for instance) and other data.
Posted By: Haze Re: Fixed software lists - 03/02/10 08:36 PM
Originally Posted By Stiletto
Yeap, that's how it's been since I can remember.

"Official" metadata for "year" comes from copyright screen (if available) and from definitive sources (if not). If completely not available, some people used to use "197x" or "198x" to try to define a decade when known but lately these have all been converted to "????" (mostly because the policies surrounding decade declaration were arbitrary, and some people would use "197?" instead, and this may inspire more people to research the proper year if available, etc. etc.)

Actual release dates are mentioned in the sourcecode in mamedriv.c, mostly due to the exhaustive research of Yasuhiro Ogawa. However (unfortunately?) that's all the more respect that's given to them.


I quite like the setup, it makes mamedriv.c a handy little one-place lookup for such things but means the developers working on the drivers don't have to worry about it. It's mainly something for the people who handle trivia to debate anyway, official release dates aren't always clear, sometimes things get leaked before the official date etc. and as already pointed out, the exact same rom can be released by different companies at different times in different places.
Posted By: etabeta78 Re: Fixed software lists - 03/03/10 08:10 PM
that's why I would suggest to add it in a separate field with possible other front-end oriented info
Posted By: judge Re: Fixed software lists - 03/03/10 08:15 PM
The frontend can use the list name and software name to uniquely identify a piece of software and use that as a key/reference to find information in a separate file/database/xml maintained by the frontend. Separation of concerns kinda thing.
Posted By: raveendraa1234 Re: Fixed software lists - 03/05/10 08:16 AM
Originally Posted By judge
That may differ per system and per media.

For the epoch game pocket computer we got direct rom dumps from the chips on the cartridges, so we know those are good.

The lists will require time and effort to create and expand them. A lot of information is already out there and it makes sense to re-use that information.


Thanks a lot.........
Posted By: etabeta78 Re: Fixed software lists - 03/05/10 08:32 AM
Originally Posted By judge
The frontend can use the list name and software name to uniquely identify a piece of software and use that as a key/reference to find information in a separate file/database/xml maintained by the frontend. Separation of concerns kinda thing.


well, my point was to add an optional field that the list maintainer can use to add those info when available. frontends can decide whether to use this optional info or to rely on separate sources for these kind of information.

if you look to MAME, mamedriv.c contains a lot of real Japanese release dates and I see no reason why we could not keep these info in our xml lists, if the info are available... for one, I would love to add these info to some of the lists wink

if info are not available, we could simply stick to the year appearing on the title screen or on the package (like we do in MAME when we use the flyers or other references for old games with no title screen)
Posted By: ReadOnly Re: Fixed software lists - 03/06/10 08:32 PM
Hi R.Belmont, you stated earlier that you had nintendo databases ready. If I may ask:

1) How did you build the databases, are they source accurate?

2) Please tell me the ROMs will be split as in the PCB, I don't like combined ROMs. If needed, I can provide you a shitload of snes PCB scans, BootGot has almost all nes covered and to my knowledge, all other nintendo systems use single maskrom due to the small size of PCB (gb, gbc, gba, n64...)

3) I beg you to include these databases as soon as possible so we can start MESS sets in an underground place. The database must be internal by all means. That it increases the binaries size by 1 meg (0.00000001% of a laser disc chd) in modern systems that have sometimes several TB (mine has 5TB) should be absolutely of no concern.
Posted By: R. Belmont Re: Fixed software lists - 03/06/10 08:46 PM
I said no such thing. AFAIK Judge and etabeta are the guys doing the lifting on this.
Posted By: Haze Re: Fixed software lists - 03/06/10 10:06 PM
Originally Posted By MESSfan
Hi R.Belmont, you stated earlier that you had nintendo databases ready. If I may ask:

1) How did you build the databases, are they source accurate?


All the existing databases are built from what's out there at the moment, it would be foolish not to start from this point, as much of it is irreplaceable.

Quote:

2) Please tell me the ROMs will be split as in the PCB, I don't like combined ROMs. If needed, I can provide you a shitload of snes PCB scans, BootGot has almost all nes covered and to my knowledge, all other nintendo systems use single maskrom due to the small size of PCB (gb, gbc, gba, n64...)


correct sets will be supported and replace existing ones as they get redumped using proper methods, no sooner, no later.

Quote:

3) I beg you to include these databases as soon as possible so we can start MESS sets in an underground place. The database must be internal by all means. That it increases the binaries size by 1 meg (0.00000001% of a laser disc chd) in modern systems that have sometimes several TB (mine has 5TB) should be absolutely of no concern.


I think it's been decided that they'll be external. While I'm not a big fan of this solution (because you're going to end up with a bunch of remap tables in the source for inputs, and inits/mappers anyway in order for real code can be tied to individual sets, something which comes free with the internal database) That is what has been decided on. Link times were the main concern I believe. (and MAME/MESS store the SHA1s etc. as strings so aren't that efficient)

I've asked what the roadmap is myself, and the code simply isn't ready yet, ie, it doesn't work, not all details have been fully planned, and it isn't ready for public use.

Posted By: etabeta78 Re: Fixed software lists - 03/06/10 10:19 PM
about NES, I'm preparing the database with separate prg and chr roms (thanks to the help from bootgod) but actual implementation in MESS will require some additional work once lists are added because NES init code needs a bit of redesign to avoid having three copies of the same code (respectively for database, for .unif dumps and for .ines dumps)

it's better to spend a bit more of time at the beginning, than to have to rework everything afterwords
Posted By: ReadOnly Re: Fixed software lists - 03/07/10 07:12 AM
Originally Posted By Haze
correct sets will be supported and replace existing ones as they get redumped using proper methods, no sooner, no later.
Most of the snes set has been redumped from scratch by No-Intro dumpers. And the list of verified dumps is public and totally accurate. For a lot of these verified dumps I have access the PCB scans (most done by myself). All those dumps with a verified status and a PCB scan (and I have a lot of them) must be wiped off the list of "need proper redump", because you can't redump them more properly than we did unless you're a total paranoid wacko.

By proper redump I assume you mean dumping the mask rom directly with an eprom reader instead of reading from the cart pins, but we have already done a few eprom reader redumps and confirmed that the result is the same.

I can share my scans of course.


EDIT: Actually I just remembered that some verified No-Intro dumps are bad dumps, or more precisely underdumps. The good dump should have mirrored data and the the No-Intro staff thought they were overdumps, they remove systematically mirrored data in dumps which is ok in most cases but BootGod and I have verified that the mirrored data was burned in the maskroms in some rare cases.

The PCB scans will allow you to determine underdumps because the maskrom have their size printed on them. I have a document about how to read a maskrom size from the info printed on it, I can share that doc with you of course.
Posted By: judge Re: Fixed software lists - 03/07/10 08:09 PM
Originally Posted By Haze

I've asked what the roadmap is myself, and the code simply isn't ready yet, ie, it doesn't work, not all details have been fully planned, and it isn't ready for public use.


1. Finalize xml format (kinda done)
2. Create code to read the xml files (almost done)
3. Create code to specify software on the command line and finding the software entries and proper software parts. (some wip code exists, not good enough)
4. Add ability to specify software lists in drivers. (nothing is done yet)
5. Actually load the software parts. (nothing is done here yet)
6. Expand & improve.

Progress isn't fast; there just isn't much time left next to regular work and feeding the baby.
Posted By: R. Belmont Re: Fixed software lists - 03/08/10 04:28 PM
Related: I recently got my hands on about 750 redump.org certified offset-corrected Saturn CD rips. I'm doing some work on CHDMAN so that it can import the original bin/cue files and ensuring that all of the information is retained and can be exported again.

Once this is done these will of course be the basis for a decent Saturn list, and hopefully we'll get that SH-1 decapped before the world ends in 2012 ;-)
Posted By: ReadOnly Re: Fixed software lists - 03/12/10 07:50 PM
Originally Posted By judge

Progress isn't fast; there just isn't much time left next to regular work and feeding the baby.
Hope you are proud of your baby and work. I am.

But this needs to be done for greater good of preserving rom softwares, there needs to be a MESS software official database (and it needs to be internal at any cost). One way to get the job done could be to recruit more people to work to this. Preferably experienced dumpers such as BootGod.
Posted By: Just Desserts Re: Fixed software lists - 03/12/10 08:50 PM
Originally Posted By MESSfan
Originally Posted By judge

Progress isn't fast; there just isn't much time left next to regular work and feeding the baby.
Hope you are proud of your baby and work. I am.

But this needs to be done for greater good of preserving rom softwares, there needs to be a MESS software official database (and it needs to be internal at any cost). One way to get the job done could be to recruit more people to work to this. Preferably experienced dumpers such as BootGod.


I think you need to realize that while it obviously isn't the case for some of us, other people might have different priorities than you. It may not have been intentional, but leading off your post by acknowledging someone's job and child with one short sentence and then following it up with three sentences led off with "But" is very telling. Reading the thread, you seem to have moved from pestering devs about not dropping what they're doing and fixing Nintendo drivers to pestering devs about ROM lists.

I know it's hard to accept, but we all have our own lives, our own set of responsibilities, and I don't remember anyone posting in this thread or submitting to MESS having to sign a piece of paper saying "My soul is now beholden to the users and I will do what they demand forever more." You do a fantastic job with all of your scans, and you put in lots of hard work on your personal projects, but that doesn't mean everyone else has to completely throw themselves into this one, plain and simple.
Posted By: ReadOnly Re: Fixed software lists - 03/13/10 10:21 AM
Originally Posted By Just Desserts
Originally Posted By MESSfan
Originally Posted By judge

Progress isn't fast; there just isn't much time left next to regular work and feeding the baby.
Hope you are proud of your baby and work. I am.

But this needs to be done for greater good of preserving rom softwares, there needs to be a MESS software official database (and it needs to be internal at any cost). One way to get the job done could be to recruit more people to work to this. Preferably experienced dumpers such as BootGod.


I think you need to realize that while it obviously isn't the case for some of us, other people might have different priorities than you. It may not have been intentional, but leading off your post by acknowledging someone's job and child with one short sentence and then following it up with three sentences led off with "But" is very telling. Reading the thread, you seem to have moved from pestering devs about not dropping what they're doing and fixing Nintendo drivers to pestering devs about ROM lists.

I know it's hard to accept, but we all have our own lives, our own set of responsibilities, and I don't remember anyone posting in this thread or submitting to MESS having to sign a piece of paper saying "My soul is now beholden to the users and I will do what they demand forever more." You do a fantastic job with all of your scans, and you put in lots of hard work on your personal projects, but that doesn't mean everyone else has to completely throw themselves into this one, plain and simple.
What are you talking about? I am not pestering at all. If someone in this board is a pest it's you, did you forget your thread about Haze or how you usually treat newbies. You're a useless troll as far as the forum is concerned.

I acknowledged judge difficulties as I have the same, I have 2 kids 4 and 1 and a full time job for the government. I haven't pestered at him at all, I just suggested recruiting more members in the team and pointed out some name. How is it pestering? I would also have offered my help (I own over 1500 snes carts and am an experienced dumper) but I am too busy with real life like judge, so I can not help at all except giving some hints or providing some material that is ready to be exploited.
Posted By: Anna Wu Re: Fixed software lists - 03/13/10 10:27 AM
Calm down guys. smile
Posted By: ReadOnly Re: Fixed software lists - 03/13/10 11:09 AM
Originally Posted By Anna Wu
Calm down guys. smile
it's gal for me smile
Posted By: etabeta78 Re: Fixed software lists - 03/13/10 11:27 AM
let's try to be clear, since it seems you missed many basic points

Originally Posted By MESSfan
But this needs to be done for greater good of preserving rom softwares, there needs to be a MESS software official database


yeah, of course. and we are working on it. the point is that it's been almost 20 years of console emulation without MESS software databases. we can all survive a couple of months until more important things (like real life), get worked on.

Originally Posted By MESSfan
(and it needs to be internal at any cost).


we are waiting for your code. otherwise, please just respect our design choices. it will be external xml and it will work perfectly. period.

Originally Posted By MESSfan
One way to get the job done could be to recruit more people to work to this. Preferably experienced dumpers such as BootGod.


do you remember this is only an hobby? we don't recruit people. we collaborate with each other in our spare time.
Posted By: ReadOnly Re: Fixed software lists - 03/13/10 01:45 PM
Originally Posted By etabeta78
yeah, of course. and we are working on it. the point is that it's been almost 20 years of console emulation without MESS software databases. we can all survive a couple of months until more important things (like real life), get worked on
I realize it will be huge work and congratz for the job your doing so far (tho I haven't seen any of it, I can easily guess you did a lot of quality job). Yeah 20 years of softwares if way too much for 2 people only, especially if you're busy IRL. I think there should be more people involved in the database design, I suggest you code an open platform for this with a submission system. If such platform is created I'll use my dumpers network and ask everyone to contribute.

Originally Posted By etabeta78
Originally Posted By MESSfan
(and it needs to be internal at any cost).


we are waiting for your code. otherwise, please just respect our design choices. it will be external xml and it will work perfectly. period.
Sorry but I'm more of a webdev than a c++ dev. So I can not do it myself but Haze could probably do it. From what I could get (but I may got it all wrong) the problem isn't the code to do it, but devs just prefer an external db (I don't know the reason why). So far MESS already has an external database with the hsi format and it did not work. No one has been interested in making a MESS software sets. I am truly convinced that only an internal database will provoke the appearance of MESS software sets. So for the sake of the preservation of 20 years of home softwares, the database should be internal.


Originally Posted By etabeta78

do you remember this is only an hobby? we don't recruit people. we collaborate with each other in our spare time.
I know that but you also know that is not messdev whoever wants to be, hence the term "recruit". Anyway I take that back, even with more members working in the db it would take years if not lifetimes, the best option is an open platform.
Posted By: Haze Re: Fixed software lists - 03/13/10 02:20 PM
Originally Posted By MESSfan

Originally Posted By etabeta78


we are waiting for your code. otherwise, please just respect our design choices. it will be external xml and it will work perfectly. period.


Sorry but I'm more of a webdev than a c++ dev. So I can not do it myself but Haze could probably do it. From what I could get (but I may got it all wrong) the problem isn't the code to do it, but devs just prefer an external db (I don't know the reason why). So far MESS already has an external database with the hsi format and it did not work. No one has been interested in making a MESS software sets. I am truly convinced that only an internal database will provoke the appearance of MESS software sets. So for the sake of the preservation of 20 years of home softwares, the database should be internal.


I think the main thing is the -listxml support. If MESS can parse the lists it has and output comprehensive software lists for everything supported in an identical* format to MAME with it's -listxml command, and work with CLRMame etc. as a result then I think things will improve.

I have my reservations over the external databases for other reasons, but as a plus they'll allow other emulators to make use of them rather than the other emulators having to parse the entire -listxml output.

* of course, extra fields can be included too

If things work fine then I won't be making any extra effort to convert the database myself, unless I find it too ugly to work with, or it's too limiting. For things like MAME the external database wouldn't make sense at all (because every game is tied very closely to internal code and having a drivers for very simple systems almost entirely in a single file is rather elegant) but for MESS it stands some chance of working.

Takeup of the MESS sets will obviously depend on the users, but the key difference between this and the old hsi files is that these are more closely tied to a MAME-like system, with a direct attempt being made to document the software on the system, and assign it actual set names. The HSI lists were just a fairly basic attempt to allow some software to be recognized without any real level of integration into the -listxml output, or assigned set names to cement the fact that they were the sets you were meant to be using with the emulator.

Posted By: judge Re: Fixed software lists - 03/13/10 03:14 PM
Quote:

I realize it will be huge work and congratz for the job your doing so far (tho I haven't seen any of it, I can easily guess you did a lot of quality job). Yeah 20 years of softwares if way too much for 2 people only, especially if you're busy IRL. I think there should be more people involved in the database design, I suggest you code an open platform for this with a submission system. If such platform is created I'll use my dumpers network and ask everyone to contribute.


Keep in mind that I'm only working on creating the format and support for the format throughout the emulator, not actually creating all the lists for all the systems. Creating lists for the systems requires more in-depth knowledge of those systems, knowledge I simply don't have. Once the lists are working properly other people are free to contribute to the lists of course.

Quote:

I think the main thing is the -listxml support. If MESS can parse the lists it has and output comprehensive software lists for everything supported in an identical* format to MAME with it's -listxml command, and work with CLRMame etc. as a result then I think things will improve.


This was my intention; have the core read the software xml files and present the software information in -listxml, or possibly through an additional command. Since software lists can be re-used by different drivers it only makes sense to fully output the list once and refer to the list when it is encountered again.
Posted By: judge Re: Fixed software lists - 03/13/10 04:55 PM
I added a section on the wiki on how 'interface' will be handled and how the emulator will go looking for software in software lists: http://mess.redump.net/people:wilbert_pol#interface_check
Posted By: Stiletto Re: Fixed software lists - 03/13/10 08:00 PM
Has there been a verdict on fixed software list "signing"/"blessing" yet?
Posted By: Tetsuo55 Re: Fixed software lists - 03/13/10 10:24 PM
I was discussing a "bugs that aren't bugs" list and at some point i suggested adding the list to the "fixed software list"

It seemed to generate some postive response so i will port it here.

I suggest adding an (optional) btab section for each software.
In this section a small list of problems could be made.

This list of issues could be displayed in frontends, maybe a subsection like "warning", and when really bad even displayed as a warning screen in mess internally (like mame does with incompletely emulation). This should reduce some of the time spent on the non-bugs by both devs and end users.

I also think this would be an accurate place because its "close" to the software but still under supervision from a trusted team (mess)
Posted By: Stiletto Re: Fixed software lists - 03/13/10 10:32 PM
nah, I think a "messinfo.dat" would be good for that. MESSSOFT.DAT? And no MESS "team member" I know would want to maintain that. Better for ... well... Bugzilla seems out of place. Maybe if messtesters.org developed or something.
Posted By: Tetsuo55 Re: Fixed software lists - 03/13/10 10:53 PM
Well another benefit of using the software list is that the xml could be used to fill a wiki or webpage like MAWS.

Although a messinfo.dat or messtesters.org is a valid solution, it might be slighly more work for devs to have quick access to the btab list for each software.

Also when the xml is used there is no need for anything "extra" beyond the exising xml file.
Posted By: ReadOnly Re: Fixed software lists - 03/14/10 05:26 PM
About the software list I hope it will follow the same rules as MAME software list: only real dumps from cartridge/pcb, be it licensed or a bootleg. No homebrew softwares and homebrew hacks please, and no bad dumps. If I see 1359 super mario world hacks in the list I will raaaage. >_<
Posted By: Haze Re: Fixed software lists - 03/14/10 06:02 PM
Originally Posted By MESSfan
About the software list I hope it will follow the same rules as MAME software list: only real dumps from cartridge/pcb, be it licensed or a bootleg. No homebrew softwares and homebrew hacks please, and no bad dumps. If I see 1359 super mario world hacks in the list I will raaaage. >_<


I believe that's the plan, although personally I'd extend it to useful test software, and translations which work on real hardware (obviously not those that don't) as a fair number of people use them that way. Many such programs are _VERY_ useful for regression testing, and ensuring the emulation remains of a high level of accuracy, which is obviously an important goal for the project. They should obviously be correctly marked as such tho. (This is why I included various test programs in the HazeMD database too, as it gives me a one stop place to test specific features)

For things like Computers obviously the Public Domain scene plays a large role in their history, and so can't be ignored either. Those games were 'published' albeit in a rather different way to boxed commercial software (although many of the local stores here did sell it alongside them, with custom cases, I still have a few)

Obviously things like the 1500 Kario graphic hacks done by some 10 year old who happens to think that making mario look like a penis is hilarious have no place in the database.

The console / computer home scene has a rather different history to arcades, so it's important to consider it rather than just sticking religiously to the MAME rules.
Posted By: TheTrout Re: Fixed software lists - 03/15/10 12:02 AM
Yeah, I'm hoping that any hacks or homebrews get ignored, but that translations are an exception to that rule. Granted, it's a fine line to walk.
Posted By: Tetsuo55 Re: Fixed software lists - 03/15/10 07:59 AM
As far as hacks and translations go i don't see any reason to keep those are around as full roms if the original rom can be patched realtime.

Patches can work just like cheats, a simple on/off bolean and correct naming of the patch file.
Posted By: ReadOnly Re: Fixed software lists - 03/15/10 04:51 PM
Originally Posted By Haze
Originally Posted By MESSfan
About the software list I hope it will follow the same rules as MAME software list: only real dumps from cartridge/pcb, be it licensed or a bootleg. No homebrew softwares and homebrew hacks please, and no bad dumps. If I see 1359 super mario world hacks in the list I will raaaage. >_<


I believe that's the plan, although personally I'd extend it to useful test software, and translations which work on real hardware (obviously not those that don't) as a fair number of people use them that way. Many such programs are _VERY_ useful for regression testing, and ensuring the emulation remains of a high level of accuracy, which is obviously an important goal for the project. They should obviously be correctly marked as such tho. (This is why I included various test programs in the HazeMD database too, as it gives me a one stop place to test specific features)

For things like Computers obviously the Public Domain scene plays a large role in their history, and so can't be ignored either. Those games were 'published' albeit in a rather different way to boxed commercial software (although many of the local stores here did sell it alongside them, with custom cases, I still have a few)

Obviously things like the 1500 Kario graphic hacks done by some 10 year old who happens to think that making mario look like a penis is hilarious have no place in the database.

The console / computer home scene has a rather different history to arcades, so it's important to consider it rather than just sticking religiously to the MAME rules.
There is no difference between a home-brew hack of super mario world and a home-brew translation of <insert RPG name here>, they both fall under the home-brew hack category, and we don't want them in the list, no matter how useful they may appear to you (that's totally subjective). If you want to play a home-brew translation you can download the patch @ romhacking.net and play it in zsnes, which is the emulator they're designed for 99% of the time. Besides translations hack come out every day and I don't want an ever growing list.

If it doesn't come from a prom and if it's not commercial, we don't want it in. Most devs should agree with this as it is MAME standard and it's the most reasonable solution.

And test softwares are useful, but they don't need to be in the list, they can be used without being in the list. Again we don't want an ever growing list.
Posted By: judge Re: Fixed software lists - 03/15/10 04:54 PM
For the amiga however, it may be very worthwhile to have a list of all the demos that were released...
Posted By: R. Belmont Re: Fixed software lists - 03/15/10 04:58 PM
To expand on what Haze said, there's a QA element to these lists too - the list is software that we (eventually) will guarantee runs, or at least can have bugs filed on it. Thus, things like blargg's test programs and at least the big-name Amiga demos are important since they often hit the hardware in unique ways that can make the emulation better.
Posted By: Haze Re: Fixed software lists - 03/15/10 05:47 PM
Originally Posted By MESSfan

If it doesn't come from a prom and if it's not commercial, we don't want it in. Most devs should agree with this as it is MAME standard and it's the most reasonable solution.

And test softwares are useful, but they don't need to be in the list, they can be used without being in the list. Again we don't want an ever growing list.


Who is 'we' ? From what I gather you're not an active developer for either project?

As I've said, the console scene is very different to the arcade scene, many of the translations had wide-spread use on original hardware, a lot of them were even sold (albeit without permission of the people doing the translations) Why do you think that SNES rom has the secret 'if you bought this you were ripped off' screen in it in the first place? I've seen these things first hand, and even owned one at one point. I can't see somewhere selling PenisMario and GreenSonic!! carts. There is a magnitude of difference between them in terms of both how comparable they are to an actual licensed commerical product, and how much effort went into creating them; the translations are more akin to Public Domain software on the Amiga even if they use a commercial base.

As long as MESS properly documents what's what then I see no issue. The database won't be promoting such things as official versions, nor will they be the parent sets just because they're in English. There is no way they could ever possibly be confused with official versions as long as things are documented properly, so I don't really see your concern.

There are of course the other translations which simply don't work on original hardware at all, those should be ignored completely.

As RB says, the test programs are of enormous use for QA, MESS is more than just a database, it's primarily an emulator, and anything that helps improve and maintain the level of emulation is worth documenting. Console are very different to arcade systems in that if you break the slightest thing there could be widespread consequences. Having a good test-suite documented and at your fingertips saves weeks of debugging time in cases like that.
Posted By: ReadOnly Re: Fixed software lists - 03/15/10 07:11 PM
Originally Posted By judge
For the amiga however, it may be very worthwhile to have a list of all the demos that were released...
Yeah I have no grief with those. As licensed material they should be in.
Posted By: R. Belmont Re: Fixed software lists - 03/15/10 07:17 PM
Licensed by whom? I didn't know Fairlight, Spaceballs, and TBL had deals with Commodore smile
Posted By: ReadOnly Re: Fixed software lists - 03/15/10 07:27 PM
Originally Posted By Haze
Who is 'we' ? From what I gather you're not an active developer for either project?
Not yet, but if you try to force home-brew crap into MESS, I will be sure to join MESS OPs just to kick your arse. ;-(


Originally Posted By Haze
As I've said, the console scene is very different to the arcade scene
I know the console scene or at least the snes scene better than you I think. You didn't dump about 1500 carts as I did I believe. And I know the rom hacking scene very well too. only 1 out of about 100 translation hacks was designed in the real hardware. Actually the only translation hack developed with the real hardware I know of is Ace no Ware, a tennis game. Other translations might work on real hardware but that would be pure luck since they were developed with not so accurate emulators such as zsnes.

The last thing I want to see is people filing bug reports, bugs that don't exists on real hardware, about home-brew translation hacks and I think R.Belmont has been pretty self-explanatory about that.
Posted By: Stiletto Re: Fixed software lists - 03/15/10 07:31 PM
Originally Posted By R. Belmont
Licensed by whom? I didn't know Fairlight, Spaceballs, and TBL had deals with Commodore smile


I think she confused game demos with demoscene demos, which are more akin to homebrew music videos. smile

- Stiletto
Posted By: etabeta78 Re: Fixed software lists - 03/15/10 07:31 PM
Originally Posted By MESSfan
Originally Posted By Haze
Who is 'we' ? From what I gather you're not an active developer for either project?
Not yet, but if you try to force home-brew crap into MESS, I will be sure to join MESS OPs just to kick your arse. ;-(


as if we would care. we might add homebrew programs (e.g. very accurate blargg's test) to lists, or we might leave them in a separate databases, or we might find other solutions.

no offense, but so far you still have to offer a single useful contributions


Originally Posted By MESSfan
Originally Posted By Haze
As I've said, the console scene is very different to the arcade scene
I know the console scene or at least the snes scene better than you I think. You didn't dump about 1500 carts as I did I believe. And I know the rom hacking scene very well too. only 1 out of 100 translation hack was designed in the real hardware. Actually the only translation hack developed with the real hardware I know of is Ace no Ware, a tennis game. Other translations might work on real hardware but that would be pure luck since they were developed with not so accurate emulators such as zsnes.

The last thing I want to see is people filing bug reports, bugs that don't exists on real hardware, about home-brew translation hacks and I think R.Belmont has been pretty self-explanatory about that.


bullshit. der langrisser works on a real snes and byuu's new translation will as well. german bof2 was another translation programmed in such a way to only work properly on real hardware. you might be a very good dumper, but you don't seem so expert about translations as you claim...
Posted By: R. Belmont Re: Fixed software lists - 03/15/10 07:40 PM
Besides, the thing with this is that we aren't necessarily experts on consoles and translations and cartridge dumping and whatnot, but we're listening to people like byuu and BootGod who *are* acknowledged experts.
Posted By: Haze Re: Fixed software lists - 03/15/10 07:43 PM
Originally Posted By MESSfan
Originally Posted By Haze
Who is 'we' ? From what I gather you're not an active developer for either project?
Not yet, but if you try to force home-brew crap into MESS, I will be sure to join MESS OPs just to kick your arse. ;-(


Remember the origins of 'Homebrew' are in Home systems, and while I wouldn't advocate adding everything and anything the Public Domain and Demos scenes are where a lot of people who wrote the commercial games you so treasure built up their skills. Likewise a some of the translations were worked on by people who have worked in the industry, and wished something had been released natively but it was never considered a commercially viable project. The people involed with these things aren't exactly all amateurs.

Stuff that doesn't work on real hardware won't be supported. Byuu's stance on such things is a good one.

I believe you actually need to contribute something in the way of code to the projects to become an official developer anyway.
Posted By: ReadOnly Re: Fixed software lists - 03/15/10 08:10 PM
Originally Posted By R. Belmont
Besides, the thing with this is that we aren't necessarily experts on consoles and translations and cartridge dumping and whatnot, but we're listening to people like byuu and BootGod who *are* acknowledged experts.


If you rely on BootGod rather than Haze (as far as software lists is concerned of course), then everything should be alright. I know BootGod pretty well, and he's not interested in documenting home-brew softwares and hacks.

@etabeta: that's still weak 3 translations among hundreds

@etabeta: about collaboration, read thro this thread again, I have stated I am ok to collaborate and share all my knowledge, experience and documentation (which is A LOT of documentation) about snes dumping, you just need to request my help now. Tho my collaboration will be slow due to IRL.
Posted By: ReadOnly Re: Fixed software lists - 03/15/10 08:25 PM
Originally Posted By Stiletto
Originally Posted By R. Belmont
Licensed by whom? I didn't know Fairlight, Spaceballs, and TBL had deals with Commodore smile


I think she confused game demos with demoscene demos, which are more akin to homebrew music videos. smile

- Stiletto
As a former member of a group destroying 1337 scene demos (No-Intro) you could easily guess what I think of those demos.

Might be off topic but No-Intro has successfully imposed a new standard to the scene, which I am very proud of. You will never see another scene release with a demo again, those who tried were nuked. Only clean dumps are now validated and this is all thanks to us. smile
Posted By: etabeta78 Re: Fixed software lists - 03/15/10 08:32 PM
you still miss the point that it's not only haze who consider translations/test programs/homebrew demos worth the addition... they can offer important test cases, when they're confirmed working on the real hardware.

and nobody, neither haze nor anyone else, thinks to add *all* the available hacks and translations (those can be run in any case by using the normal loading routines)
Posted By: etabeta78 Re: Fixed software lists - 03/15/10 08:33 PM
Originally Posted By MESSfan
As a former member of a group destroying 1337 scene demos (No-Intro) you could easily guess what I think of those demos.

Might be off topic but No-Intro has successfully imposed a new standard to the scene, which I am very proud of. You will never see another scene release with a demo again, those who tried were nuked. Only clean dumps are now validated and this is all thanks to us. smile


you are talking about intros and cracks, not about demos... you should really know a bit more about what you are talking about frown
Posted By: Tetsuo55 Re: Fixed software lists - 03/15/10 08:44 PM
I still say supporting patching of games on the list is a easier solution (obviously wont work for demo's)
Posted By: Skito Re: Fixed software lists - 03/15/10 08:53 PM
Messfan, there are also systems where real dumps will never be available and nearly only cracks can be made available (apple2). Do you propose not allowing hacks for every emulated system? I can see why you'd want to keep a clean state for snes, but that just doesn't work for every system.
Posted By: Haze Re: Fixed software lists - 03/15/10 08:53 PM
Originally Posted By MESSfan
Originally Posted By Stiletto
Originally Posted By R. Belmont
Licensed by whom? I didn't know Fairlight, Spaceballs, and TBL had deals with Commodore smile


I think she confused game demos with demoscene demos, which are more akin to homebrew music videos. smile

- Stiletto
As a former member of a group destroying 1337 scene demos (No-Intro) you could easily guess what I think of those demos.

Might be off topic but No-Intro has successfully imposed a new standard to the scene, which I am very proud of. You will never see another scene release with a demo again, those who tried were nuked. Only clean dumps are now validated and this is all thanks to us. smile


I think it's a shame that the pirate games with intros aren't documented somewhere as well.... I'm not a huge fan of the "pretend the parts of history we don't like never happened" approach. That shows a strong bias for one side, rather than an attempt to document. This is one reason I think the NoIntro stuff is too aggressive, and has been responsible for destroying several bits of history rather than preserving it. I don't consider that a positive.

For the Amiga the cracked games (including their intros) are a huge, huge part of the history of the system. I don't think I can name a single person who didn't have them. Are you saying you prefer them to be erased from history simply because *you* don't like them? That's not documentation I'm afraid, it's politics. MESS needs to document facts, not paint rainbows and pretend the world was a better place.


That said, anything made for emulators alone, with no consideration for real hardware isn't worth documenting.
Posted By: ReadOnly Re: Fixed software lists - 03/15/10 08:54 PM
Originally Posted By etabeta78
Originally Posted By MESSfan
As a former member of a group destroying 1337 scene demos (No-Intro) you could easily guess what I think of those demos.

Might be off topic but No-Intro has successfully imposed a new standard to the scene, which I am very proud of. You will never see another scene release with a demo again, those who tried were nuked. Only clean dumps are now validated and this is all thanks to us. smile


you are talking about intros and cracks, not about demos... you should really know a bit more about what you are talking about frown


What's the technical difference between a demoscene demo and an scene intro, both are home-brew music videos. I know what I'm talking about.

And when did I talk about crack, do you know the difference between an intro and a crack.

===

Back to topic, how are you going to differentiate home-brew softwares designed for real hardware and those designed for emulators among the tens of thousands of home-brew softwares? Is that even possible?
Posted By: judge Re: Fixed software lists - 03/15/10 09:03 PM
Could we please get back to discussing the structure and intended operation of software lists instead of discussing the possible contents?
Posted By: R. Belmont Re: Fixed software lists - 03/15/10 09:08 PM
Quote:
I think it's a shame that the pirate games with intros aren't documented somewhere as well.... I'm not a huge fan of the "pretend the parts of history we don't like never happened" approach. That shows a strong bias for one side, rather than an attempt to document. This is one reason I think the NoIntro stuff is too aggressive, and has been responsible for destroying several bit of history rather than preserving it.


Ditto. While I applaud No-Intro for making sure we have real unaltered dumps of cartridges, I'd like to see the various crack intros and whatnot collected and preserved while they're being taken out of circulation. All of us have added games to MAME that we don't personally like, but pretending they didn't exist does a disservice to preservation.
Posted By: R. Belmont Re: Fixed software lists - 03/15/10 09:11 PM
Judge: I think there's a general consensus on those aspects as proposed, and we're all waiting to see your working code and data to rip it apart and make you cry ;-)
Posted By: judge Re: Fixed software lists - 03/15/10 09:15 PM
Nearly there, xml reading is working, selecting software and software parts from the xml files is working, just need to actually load the data; gonna have to hack around in romload.c a bit for that smile
Posted By: ReadOnly Re: Fixed software lists - 03/15/10 09:18 PM
I am terribly sorry judge blush, I don't like to bother my favorite MESSdev, I guess we should make another thread about that. Until then, just in case someone missed this:

Originally Posted By MESSfan
how are you going to differentiate home-brew softwares/hacks designed for real hardware and those designed for emulators among the tens of thousands of home-brew softwares/hacks? Is that even possible?
Posted By: R. Belmont Re: Fixed software lists - 03/15/10 09:57 PM
That's easy. 99.9% of the software we're talking about came out before 1995 when there were no emulators. And a lot of us were alive and using those computers at the time, so we *do* have some expertise there.
Posted By: Stiletto Re: Fixed software lists - 03/15/10 10:19 PM
Hell, there's a few people in the emulation community who were members of cracking groups, for that matter. shocked
Posted By: judge Re: Fixed software lists - 03/16/10 07:22 PM
Originally Posted By judge
Nearly there, xml reading is working, selecting software and software parts from the xml files is working, just need to actually load the data; gonna have to hack around in romload.c a bit for that smile


Yay, it's loading data...
Posted By: Multipass Re: Fixed software lists - 03/22/10 07:49 PM
No news for to add others softwares list in mess svn please?
Posted By: judge Re: Fixed software lists - 03/30/10 07:02 PM
Heh, I totally forgot about one of the most useful bits of information: whether the game is working or has gfx or sound issues.

That should definitely be added.
Posted By: Haze Re: Fixed software lists - 03/30/10 09:30 PM
Originally Posted By judge
Heh, I totally forgot about one of the most useful bits of information: whether the game is working or has gfx or sound issues.

That should definitely be added.


Indeed, the flags are one of the reasons why you'd want lists in the first place, so people know what the expected compatibility is, and if they should report any bugs. No point in telling me Fatal Rewind on the Genesis is broken, I know laugh

Posted By: Skito Re: Fixed software lists - 03/31/10 01:10 AM
The working state should be trinary, or at least have a "not set" value. I'm sure lots of games will be added but not all games will have been tested for workingness, these should be set to not tested.
Posted By: judge Re: Fixed software lists - 03/31/10 07:13 AM
Default is working, if the game has not been tested it should not even be in the list.
Posted By: etabeta78 Re: Fixed software lists - 03/31/10 08:00 AM
erm... for systems with thousand of games, it can take some time to test the working state...

I guess we can simply say that for the first couple of weeks the working state reported for games in the list cannot be trusted.
Posted By: judge Re: Fixed software lists - 03/31/10 01:55 PM
I'm aware of the time that would take. Just like with game/console drivers you can always add them with the game_not_working flag and remove that flag once it's proven to be working.
Posted By: Haze Re: Fixed software lists - 04/01/10 04:25 PM
Originally Posted By judge
I'm aware of the time that would take. Just like with game/console drivers you can always add them with the game_not_working flag and remove that flag once it's proven to be working.


Yeah I think eta was backing up your plans and saying tri-state isn't needed, it's just going to take some time for the lists to settle down a bit.

Keeping my fingers crossed over this stuff going in, as you can see from the SNES thread a bit of time was wasted in trying to figure out why Kale was having a problem with Super Mario Kart (J) when really his rom was just bad.

Posted By: etabeta78 Re: Fixed software lists - 04/02/10 07:29 PM
btw, for n64 list, was there a final words on the endian-ness of the carts (i.e. if files should be in .v64 or .z64 endianness)?

at a given point nointro switched from z64 to v64, but I was unable to find any exact reference to the reason. also, I seem to remember that Guru investigated the issue as well, but again I think it's been too long ago.
Posted By: etabeta78 Re: Fixed software lists - 04/03/10 06:41 AM
I knew I read about it in this same forum...

thanks to some search-skill-training, I managed to find

http://www.bannister.org/forums/ubbthreads.php?ubb=showflat&Number=34391

so "v64" format (i.e. the byteswapped one) seems to be the right one
Posted By: judge Re: Fixed software lists - 04/03/10 02:18 PM
Originally Posted By judge
Heh, I totally forgot about one of the most useful bits of information: whether the game is working or has gfx or sound issues.

That should definitely be added.


Thinking about this only adding a supported flag would be sufficient. That could be used to indicate that the software needs additional hardware emulated to get it to work properly. If supported software is experiencing graphics or sound issues then that is actually an indication of an issue with the driver. And should thus be flagged in the driver instead of the software.
Posted By: Haze Re: Fixed software lists - 04/03/10 02:39 PM
I disagree. For the megadrive 99% of games run without problems, but there are a handful I'd like to flag as not working, and imperfect graphics due to imperfections in the system emulation.

I don't think marking the entire system as imperfect graphics would be worthwhile (most of them are timing glitches) and at the same time, not having flags gives little indication of which games are actually suffering.

I'd prefer to be able to mark things on a case by case basis, as it's more useful, and for in-development drivers for complex systems (saturn and psx come to mind) there could be a large number of games which work flawlessly, and others which have significant graphic issues.

(for Saturn it's pretty easy for that case to arise, there are so many different ways the video system can be configured.. and even for SNES you'd want to flag that game which requires sub-scanline rendering as imperfect graphics as it's a known bad case, even if you were confident that the rest of the system video rendering was correct)

maybe a 'inherit system flags' flag could exist for each piece of software to be used in some cases? (ie, when the software list hasn't been extensively tested, or it's known that everything basically has the same flaws)
Posted By: judge Re: Fixed software lists - 04/03/10 02:53 PM
Still, in those cases the emulation for the system drivers is incorrect.

For known bad cases you could of course still set the supported flag to 'no' to indicate that you know the game doesn't work (because some hardware features are not implemented yet).

Posted By: etabeta78 Re: Fixed software lists - 04/03/10 03:00 PM
would it be so bad to simply re-use MAME flags? I know in some cases judge's argument is good, but given we already have the required flags we could simply stick to them...

also, how did you decide to implement the per-game init requirements?
Posted By: Haze Re: Fixed software lists - 04/03/10 04:07 PM
Originally Posted By judge
Still, in those cases the emulation for the system drivers is incorrect.

For known bad cases you could of course still set the supported flag to 'no' to indicate that you know the game doesn't work (because some hardware features are not implemented yet).



there is a huge difference between knowing that the sprite masking is incorrect for Streets of Rage, and marking the game as 'unsupported'.

AFAIK the broken sprite masking breaks 2 cases, Alien Storm, and Streets of Rage. It's not worth marking the system as 'imperfect graphics' over and it's not worth marking the games as 'unsupported' over. There is a world between a few glitches on the status bar, and a game not booting at all, or having no inputs (Fatal Rewind)

The only realistic solution is the one MAME is using now. Allowing known bad cases to be tagged.

Of course I've never been a fan of the 'no bug reports allowed for flagged games' rule that MAME has (I'd rather the issues were documented too) but I'd also rather they were flagged properly, so that people know what to expect.

The MAME flag system just works, and as I've stressed in other posts familiarity with MAME is important, so having the same set of flags help this. The changes should bring the projects closer together, not further apart.

I doubt any system in MESS is perfect, so how well the software performs is the only real useful indication in some cases.
Posted By: etabeta78 Re: Fixed software lists - 04/03/10 04:29 PM
Originally Posted By Haze
I doubt any system in MESS is perfect, so how well the software performs is the only real useful indication in some cases.


either you only refer to popular consoles, or you have never tried Thomson computers, or the old single-board computers which Curt has worked on... these are almost perfect

this, of course, does not change the fact I agree on MAME flags wink
Posted By: Haze Re: Fixed software lists - 04/03/10 05:27 PM
Originally Posted By etabeta78
Originally Posted By Haze
I doubt any system in MESS is perfect, so how well the software performs is the only real useful indication in some cases.


either you only refer to popular consoles, or you have never tried Thomson computers, or the old single-board computers which Curt has worked on... these are almost perfect

this, of course, does not change the fact I agree on MAME flags wink


No, actually I refer just as much to the older systems.

The only reason they seem perfect is probably because nothing abuses them. Take out all the c64 demos etc. and you might have a system which seems to be perfectly emulated, but isn't. This was my argument behind needing to include good test cases in the software lists, but for the more obscure systems there are unlikely to be many tests cases because there was nobody abusing the hardware. I'm not saying Curt's drivers are bad, I'm just saying I doubt he's actually encountered anything trying to push the hardware to actually give him test cases to make the emulation perfect. You can do sub-scanline timing tricks on a lot of early systems, and in most cases they'll fall apart in MESS.



Posted By: etabeta78 Re: Fixed software lists - 04/03/10 05:50 PM
and that's true for many arcade hardwares in MAME as well wink

but let's not sidetrack too much the thread (I didn't want to argue with you on this, I just wanted to say that some systems are already well emulated and fit quite well MAME/MESS architecture, as long as you don't abuse them)
Posted By: Haze Re: Fixed software lists - 04/04/10 02:47 AM
Originally Posted By etabeta78
and that's true for many arcade hardwares in MAME as well wink


of course... you only need about 10% accuracy to emulate 90% of the software on a system, it's the remaining 10% that will give you nightmares ;-)

thankfully for MAME most arcade hardware only had one or two games on it, so getting things to appear correct is easier, even if the emulation isn't actually 100% correct (and in most cases is far from it)

Posted By: judge Re: Fixed software lists - 04/04/10 10:05 AM
Originally Posted By Haze
of course... you only need about 10% accuracy to emulate 90% of the software on a system, it's the remaining 10% that will give you nightmares ;-)

thankfully for MAME most arcade hardware only had one or two games on it, so getting things to appear correct is easier, even if the emulation isn't actually 100% correct (and in most cases is far from it)


This is actually where we spent a lot of time for MESS. Fixing bugs in components that are working good enough for MAME. Consoles and especially computers tend to abuse the hardware a bit more.


Anyway, changing the supported flag to yes/partial/no should be sufficient. Going with all the mame flags feels like overkill to be honest. From an enduser point of view this should be sufficient:
yes - we have emulated the hardware and the game worked when we tested it. If you experience issues file a bug.
no - this will definitely not work. Bits and pieces of the hardware are simply missing from the emulation. Please do not bug us about it. We know.
partial - we have all the hardware emulated, this should have worked but we screwed up somewhere. You will experience issues when you run this and we know about it.
Posted By: etabeta78 Re: Fixed software lists - 04/04/10 10:39 AM
the advantage of MAME flags is that "partial" can be made more precise with IMPERFECT_VIDEO or IMPERFECT_AUDIO rather than left unspecified...
Posted By: judge Re: Fixed software lists - 04/04/10 11:18 AM
Time for a small status update. I've been testing things with the gamepock driver, a small system for which only 5 games were ever released.

This is the gamepock xml software list:
Code:
<?xml version="1.0"?>
<!DOCTYPE softwarelist [
	<!ELEMENT softwarelist (software+)>
		<!ATTLIST softwarelist name CDATA #REQUIRED>
		<!ATTLIST softwarelist description CDATA #IMPLIED>
		<!ELEMENT software (description, year?, publisher, part*)>
			<!ATTLIST software name CDATA #REQUIRED>
			<!ATTLIST software cloneof CDATA #IMPLIED>
			<!ATTLIST software supported (yes|partial|no) "yes">
			<!ELEMENT description (#PCDATA)>
			<!ELEMENT year (#PCDATA)>
			<!ELEMENT publisher (#PCDATA)>
			<!ELEMENT part (dataarea*)>
				<!ATTLIST part name CDATA #REQUIRED>
				<!ATTLIST part interface CDATA #REQUIRED>
				<!-- feature is used to store things like pcb-type, mapper type, etc. Specific values depend on the system. -->
				<!ATTLIST part feature CDATA #IMPLIED>
				<!ELEMENT dataarea (rom*)>
					<!ATTLIST dataarea name CDATA #REQUIRED>
					<!ATTLIST dataarea size CDATA #REQUIRED>
					<!ATTLIST dataarea databits (8|16|32|64) "8">
					<!ATTLIST dataarea endian (big|little) "little">
					<!ELEMENT rom EMPTY>
						<!ATTLIST rom name CDATA #IMPLIED>
						<!ATTLIST rom size CDATA #REQUIRED>
						<!ATTLIST rom crc CDATA #IMPLIED>
						<!ATTLIST rom md5 CDATA #IMPLIED>
						<!ATTLIST rom sha1 CDATA #IMPLIED>
						<!ATTLIST rom offset CDATA #IMPLIED>
						<!ATTLIST rom status (baddump|nodump|good) "good">
						<!ATTLIST rom loadflag (load16_byte|load16_word|load16_word_swap|load32_byte|load32_word|load32_word_swap|load32_dword|load64_word|load64_word_swap|reload) #IMPLIED>
]>
<softwarelist name="gamepock" description="Epoch Game Pocket Computer cartridges">
	<!-- Game 1 -->
	<software name="astrobom">
		<description>Astro Bomber</description>
		<year>1984</year>
		<publisher>Epoch</publisher>
		<part name="cart" interface="gamepock_cart">
			<dataarea name="rom" size="8192">
				<rom name="astrobom.bin" size="8192" crc="b0fd260f" sha1="453a0f3c0952ebd8e691316c39960731f1996c09" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Game 2 -->
	<software name="blockmaz">
		<description>Block Maze</description>
		<year>1984</year>
		<publisher>Epoch</publisher>
		<part name="cart" interface="gamepock_cart">
			<dataarea name="rom" size="8192">
				<rom name="blockmaz.bin" size="8192" crc="cfb3291b" sha1="50dc5736200986b326b372c17c233c4180474471" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Game 3 -->
	<software name="pokemahj">
		<description>Pokekon Mahjongg</description>
		<year>1984</year>
		<publisher>Epoch</publisher>
		<part name="cart" interface="gamepock_cart">
			<dataarea name="rom" size="16384">
				<rom name="pokemahj.bin" size="16384" crc="5c3eed48" sha1="918e1caa16cfae6b74da2026f3426d0a5818061c" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Game 4 -->
	<software name="pokereve">
		<description>Pokekon Reversi</description>
		<year>1984</year>
		<publisher>Epoch</publisher>
		<part name="cart" interface="gamepock_cart">
			<dataarea name="rom" size="8192">
				<rom name="pokereve.bin" size="8192" crc="1c461f91" sha1="ead4a4efe5439e2ec1f6befb50c350f73919da8d" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Game 5 -->
	<software name="soukoban">
		<description>Soukoban - Store Keepers</description>
		<year>1985</year>
		<publisher>Epoch</publisher>
		<part name="cart" interface="gamepock_cart">
			<dataarea name="rom" size="8192">
				<rom name="soukoban.bin" size="8192" crc="5d6f7819" sha1="61ef6483e8f9935dd8b6351fd2bdfda3af3899bd" offset="0" />
			</dataarea>
		</part>
	</software>

</softwarelist>


A software list needs to be configured in the machine driver. Optionally the interface for the cartridge slot can be specified:
Code:
static MACHINE_DRIVER_START( gamepock )
	MDRV_CPU_ADD("maincpu", UPD78C06, XTAL_6MHz)	/* uPD78C06AG */
	MDRV_CPU_PROGRAM_MAP( gamepock_mem)
	MDRV_CPU_IO_MAP( gamepock_io)
	MDRV_CPU_CONFIG( gamepock_cpu_config )

	MDRV_MACHINE_RESET( gamepock )

	MDRV_SCREEN_ADD("screen", LCD)
	MDRV_SCREEN_REFRESH_RATE( 60 )
	MDRV_SCREEN_FORMAT( BITMAP_FORMAT_INDEXED16 )
	MDRV_SCREEN_SIZE( 75, 64 )
	MDRV_SCREEN_VISIBLE_AREA( 0, 74, 0, 63 )
	MDRV_DEFAULT_LAYOUT(layout_lcd)

	MDRV_PALETTE_LENGTH( 2 )
	MDRV_PALETTE_INIT(black_and_white)
	MDRV_VIDEO_UPDATE( gamepock )

	/* sound hardware */
	MDRV_SPEAKER_STANDARD_MONO("mono")
	MDRV_SOUND_ADD("speaker", SPEAKER, 0)
	MDRV_SOUND_ROUTE(ALL_OUTPUTS, "mono", 0.50)

	/* cartridge */
	MDRV_CARTSLOT_ADD("cart")
	MDRV_CARTSLOT_INTERFACE("gamepock_cart")
	MDRV_CARTSLOT_EXTENSION_LIST("bin")
	MDRV_CARTSLOT_NOT_MANDATORY
	MDRV_CARTSLOT_START(gamepock_cart)
	MDRV_CARTSLOT_LOAD(gamepock_cart)

	/* Software lists */
	MDRV_SOFTWARE_LIST_ADD("gamepock")
MACHINE_DRIVER_END


The software list and interface settings are part of the listxml output:
Code:
<mess build="0.137.1 (Apr  4 2010)" debug="yes" mameconfig="10">
	<machine name="gamepock" sourcefile="gamepock.c">
		<description>Game Pocket Computer</description>
		<year>1984</year>
		<manufacturer>Epoch</manufacturer>
		<rom name="egpcboot.bin" size="4096" crc="ee1ea65d" sha1="9c7731b5ead721d2cc7f7e2655c5fed9e56db8b0" region="maincpu" offset="0"/>
		<chip type="cpu" tag="maincpu" name="uPD78C06" clock="6000000"/>
		<chip type="audio" tag="speaker" name="Speaker"/>
		<display type="lcd" rotate="0" width="75" height="64" refresh="60.000000" pixclock="288000" htotal="75" hbend="0" hbstart="75" vtotal="64" vbend="0" vbstart="64" />
		<sound channels="1"/>
		<input players="1" buttons="4">
			<control type="joy8way"/>
		</input>
		<driver status="good" emulation="good" color="good" sound="good" graphic="good" savestate="unsupported" palettesize="2"/>
		<device type="cartridge" tag="cart" interface="gamepock_cart">
			<instance name="cartridge" briefname="cart"/>
			<extension name="bin"/>
		</device>
		<softwarelist name="gamepock">
			<software name="astrobom">
				<description>Astro Bomber</description>
				<year>1984</year>
				<publisher>Epoch</publisher>
				<part name="cart" interface="gamepock_cart">
					<dataarea name="rom" size="2000">
						<rom name="astrobom.bin" size="8192" crc="b0fd260f" sha1="453a0f3c0952ebd8e691316c39960731f1996c09" offset="0" />
					</datearea>
				</part>
			</software>
			<software name="blockmaz">
				<description>Block Maze</description>
				<year>1984</year>
				<publisher>Epoch</publisher>
				<part name="cart" interface="gamepock_cart">
					<dataarea name="rom" size="2000">
						<rom name="blockmaz.bin" size="8192" crc="cfb3291b" sha1="50dc5736200986b326b372c17c233c4180474471" offset="0" />
					</datearea>
				</part>
			</software>
			<software name="pokemahj">
				<description>Pokekon Mahjongg</description>
				<year>1984</year>
				<publisher>Epoch</publisher>
				<part name="cart" interface="gamepock_cart">
					<dataarea name="rom" size="4000">
						<rom name="pokemahj.bin" size="16384" crc="5c3eed48" sha1="918e1caa16cfae6b74da2026f3426d0a5818061c" offset="0" />
					</datearea>
				</part>
			</software>
			<software name="pokereve">
				<description>Pokekon Reversi</description>
				<year>1984</year>
				<publisher>Epoch</publisher>
				<part name="cart" interface="gamepock_cart">
					<dataarea name="rom" size="2000">
						<rom name="pokereve.bin" size="8192" crc="1c461f91" sha1="ead4a4efe5439e2ec1f6befb50c350f73919da8d" offset="0" />
					</datearea>
				</part>
			</software>
			<software name="soukoban">
				<description>Soukoban - Store Keepers</description>
				<year>1985</year>
				<publisher>Epoch</publisher>
				<part name="cart" interface="gamepock_cart">
					<dataarea name="rom" size="2000">
						<rom name="soukoban.bin" size="8192" crc="5d6f7819" sha1="61ef6483e8f9935dd8b6351fd2bdfda3af3899bd" offset="0" />
					</datearea>
				</part>
			</software>
		</softwarelist>
	</machine>
</mess>


I can start a game by using:
Code:
./messd gamepock -cart astrobom


The romident command will also search through software lists:
Code:
./messd -romident /path/to/epoch_game_pocket/roms/Astro\ Bomber\ \(epoch-24\).bin 
Astro Bomber (epoch-24).bin= astrobom.bin          gamepock:astrobom Astro Bomber


The file structure is as follows:
Code:
roms/gamepock.zip
roms/gamepock/astrobom.zip

Posted By: etabeta78 Re: Fixed software lists - 04/04/10 11:28 AM
> The software list and interface settings are part of the listxml output:

part of a system specific xml or part of the full MESS one? because SNES + GameBoy (mono, color, advance) lists would make mess.xml size to rapidly become HUGE...
Posted By: judge Re: Fixed software lists - 04/04/10 11:31 AM
the full mess one.
Posted By: etabeta78 Re: Fixed software lists - 04/04/10 11:38 AM
do we really want this? I mean, we could simply mention that a gamelist is supported without fully including it, given that it's just a copy of the external xml list...
Posted By: judge Re: Fixed software lists - 04/04/10 11:42 AM
How else would a frontend know which games are supported. It does not know beforehand where the hash file folders are (where the xml files are stored).
Posted By: etabeta78 Re: Fixed software lists - 04/04/10 11:53 AM
reading the list folder from mess.ini and then parsing system.xml if it exists?

while now it makes more sense to include it in xml (I hadn't thought to frontends), it still seems a bit silly to include a 1:1 copy of each gamelist xml into mess.xml...
Posted By: judge Re: Fixed software lists - 04/04/10 12:00 PM
I had actually been thinking about adding a -listsoftware command which would output the software lists and the -listxml would only contain references to the lists.
Posted By: R. Belmont Re: Fixed software lists - 04/04/10 12:05 PM
Either a "-listsoftware a2600" command or including the full path and hash of each system's XML database in -listxml would serve frontends without things getting out of hand.
Posted By: judge Re: Fixed software lists - 04/04/10 01:53 PM
Done.

Now for the other commands. I think -romident should remain expanded to also search through the software lists, since the user does not know beforehand if a file is a system rom or a piece of software. For verifying software it makes sense to add a -verifysoftware <gamename> command instead of integrating it all into the -verifyroms command.
Posted By: etabeta78 Re: Fixed software lists - 04/04/10 01:54 PM
agreed
Posted By: etabeta78 Re: Fixed software lists - 04/04/10 03:00 PM
current svn does not compile due to load_software_part_region used in softlist.c but not declared...
Posted By: judge Re: Fixed software lists - 04/04/10 03:03 PM
Fixed, sorry about that.
Posted By: Haze Re: Fixed software lists - 04/04/10 03:10 PM
Originally Posted By judge
Done.

Now for the other commands. I think -romident should remain expanded to also search through the software lists, since the user does not know beforehand if a file is a system rom or a piece of software. For verifying software it makes sense to add a -verifysoftware <gamename> command instead of integrating it all into the -verifyroms command.


yeah -romident needs to look through everything

likewise -listxml needs to list everything, for every system by default. (the -listsoftware will be handy for individual systems, but to do sites like MAWS you need one big database outputting somewhere, otherwise it's much more hassle, and people will just ignore the minor systems if they're not part of one large output IMHO.)

I'd have
-listxml (everything, just like mame)
-listbios (lists all bios roms - the current mess behavior?)
-listsoftware (all software)
-listsoftware <system> (all software for given system)

Again this builds on what MAME does, and produces the expected behavior (-listxml showing everything) rather than implement the same commands, but giving different behavior, which is confusing. One of the most irritating things about MESS has always been the subtle but annoying differences between it and MAME. I find the complete 'this is what we support' list created by -listxml to be important

(I'm aware I could just load the invidivual system xmls, but having one big file to search through, and ident commands that can identify anything is easier and more desirable in some cases)
Posted By: etabeta78 Re: Fixed software lists - 04/04/10 03:27 PM
well, one could parse -lx output and for each softlist entry found in the xml launch a second command to parse the correspondent softlists.

I keep considering silly to include a copy of the (huge) lists in -lx output

also, a frontend which would like to ignore softlists could be started way faster if he has only to parse the old xml file than if he has to also parse megabytes of lists
Posted By: Haze Re: Fixed software lists - 04/04/10 03:33 PM
Originally Posted By etabeta78

I keep considering silly to include a copy of the (huge) lists in -lx output


It's consistency between projects, and it's essential. The lack of it shown by some software is one of my pet hates. As a user, player, and a developer you expect things to work in the same way, not have different behavior 'just because'. There is no good reason to NOT provide the same behavior. The only thing you achieve by not providing the same behavior is irritating people who have used MAME and expect -listxml to output everything then find it doesn't (I'd consider that a bug myself)

It would be like if you bought a racing game, and found that the brake was on the left trigger and the accelerator on the right... It's just not something you do.

Using a familiar format will also make it easier for existing frontends to support, without having to write new code to scan for and parse other xmls.
Posted By: R. Belmont Re: Fixed software lists - 04/04/10 03:39 PM
I still think MESS's problem is that it's too much like MAME. That causes people to think it should work exactly like MAME (as Haze has fallen victim to) when in fact arcade emulation and computer/console should be and historically have been very different.

That having been said, if you're going this route you may as well export 5 gigabytes for -listxml. It'll crash most/all existing frontends, but if you're claiming to have MAME commandline options, they should work exactly as MAME commandline options do.
Posted By: Haze Re: Fixed software lists - 04/04/10 03:42 PM
Originally Posted By R. Belmont
I still think MESS's problem is that it's too much like MAME. That causes people to think it should work exactly like MAME (as Haze has fallen victim to) when in fact arcade emulation and computer/console should be and historically have been very different.


Why?

I see no reason for them to be very different at all, aside from management of extra devices etc.

It's put me off using MESS, and developing for MESS. Half the differences are differences for the sake of differences; they have no benefit.

A 5 gig -listxml output is what I'd expect to see if there was 5 gig of data to export.

The way a perfectly good idea is being reengineered again you may as well have just stuck with the existing hash files, because you're losing half of the proposed benefits anyway. It's starting to look like you're just reinventing what you already have.

MAME and MESS share huge amount of code, there are many common arcade and console/computer platforms. Expecting the projects to work in the same way, so that things can be developed together to ensure that no bugs are introduced in either project is important. For things which are so close anyway there is no reason to have the projects work in completely different ways, giving a completely different working environment. When you're quickly switching between two projects to work on the same thing that's the LAST thing you want.

Expanding on existing behavior and adding extra functionality is fine. Replacing existing behavior with unexpected behavior is annoying. MESS can easily do the former, without having to do the latter. This is why I propose that -listxml works like MAME, and MESS can have extra options to list / filter specific platforms, or bios roms. The -listxml works as expected, and if it's not actually what you desire, you have other options which may be more useful to you.
Posted By: Cpt. Pugwash Re: Fixed software lists - 04/04/10 04:00 PM
Originally Posted By etabeta78

while now it makes more sense to include it in xml (I hadn't thought to frontends), it still seems a bit silly to include a 1:1 copy of each gamelist xml into mess.xml...


Since the output is XML can't we declare the software lists as entities and include the existing XML files that way ?

Posted By: etabeta78 Re: Fixed software lists - 04/04/10 04:03 PM
Originally Posted By Haze
Why?

I see no reason for them to be very different at all, aside from management of extra devices etc.

It's put me off using MESS, and developing for MESS. Half the differences are differences for the sake of differences; they have no benefit.


but MESS is consistent with MAME. the difference is that for MESS systems (being consoles or computers) are the equivalent items to MAME games, not software...

software (and softlists) adds a brand new layer in MESS which is absent in MAME. so there cannot be an expected behavior for softlists wink

even if we manage to further simplify command line (in order to remove the need of the "cart" switch), we will always have

mess xxxx

to start system xxxx and

mess xxxx yyyy

to start yyyy software to be run in system xxxx. hence, MAME similarities stop at the system level...
Posted By: Haze Re: Fixed software lists - 04/04/10 04:10 PM
Originally Posted By etabeta78
but MESS is consistent with MAME. the difference is that for MESS systems (being consoles or computers) are the equivalent items to MAME games, not software...


Sorry, I don't buy this at all. It creates a confusing divide. eg. the Radica stuff would be output as a system, and be fully playable without 'software' dsepite the fact that it does have software (because the bios is the software)

Yes, there might be some subtle differences in the underlying implementations in MESS, but having options which work in the same way is important. There is no fundamental difference between a NeoGeo and a Megadrive, and aside from the slight differences in how they are launched (which could easily be part of MAME anyway) I don't think the user needs to be exposed to internal differences. There IS an expected behavior, even if your developer-centric mentality doesn't want to admit it*.

-listxml, by definition is a 'list all' option (it's non-specific, therefore you expect everything). It should be a dump of everything to xml, unless you decide you want to filter it further. Any other behavior IS confusing. The filtered options might end up being used more, but that doesn't give reason to strip out the expected full output.

* This isn't meant to be an insult, but it's the impression I get from a lot of your changes, both to MAME and MESS. You seem to forget that users just expect things to work in a certain way, whereas you're more obsessed with implementation details and working against that. (Hence me having to clean up all the default eeprom stuff so that the games worked again because the masses simply considered them 'broken' when they wouldn't boot anymore..It was also irritating to me because I usually work with a clean tree and clean folders, and having to init every clone of every game I want to test if I make driver changes was annoying. It also made it clear that nobody was playing the games with more complex init procedures at all, Cybersled was completely broken even if you did calibrate it, yet marked as working, and other things only required 5 minute fixes to make significant improvements to. I can't for the life of me work out the proper calibration technique for Hard Drivin' either..)
Posted By: R. Belmont Re: Fixed software lists - 04/04/10 04:18 PM
I'm curious - are there any developers other than Haze that refuse to touch MESS right now because of the way it works or would prefer it to be a more MAME-exact experience?

I'm not trying to be an @$$ here or anything, I'm really curious. I understand and support the end-user benefits of software lists, but I'd love to also know the benefits on the dev side.
Posted By: Haze Re: Fixed software lists - 04/04/10 04:28 PM
Originally Posted By R. Belmont
I'm curious - are there any developers other than Haze that refuse to touch MESS right now because of the way it works or would prefer it to be a more MAME-exact experience?

I'm not trying to be an @$$ here or anything, I'm really curious. I understand and support the end-user benefits of software lists, but I'd love to also know the benefits on the dev side.


I'd also look at things the other way. If MAME and MESS are similar it could make it easier for MESS developers to work on MAME. There are a number of drivers in MAME (MPU4 Video with all it's funky serial comms etc. comes to mind) where I think the experience of MESSdevs would be of great benefit. (Not to mention all the computer/console based systems in MAME.. Marcusz got that SexyBoom / PuzzleStar MSX thing booting, but the colours really are a mess, might be a useful test case unless it's just a banking problem)

So I think it's also worth considering if changes would make it easier for developers who are primarily MESS developers to contribute to MAME, because MAME does need more developers right now, especially ones willing to give the attention to detail that's often needed to computer systems.

(That's why I've often proposed simply merging the projects.. It increases the scope for what developers can work in, things that catch their interest which might otherwise be missed because they're only looking at / working within one project, devices which are already emulated in one project are less likely to be reimplemented in the other because people don't realise they exist)
Posted By: etabeta78 Re: Fixed software lists - 04/04/10 04:42 PM
Originally Posted By Haze
Originally Posted By etabeta78
but MESS is consistent with MAME. the difference is that for MESS systems (being consoles or computers) are the equivalent items to MAME games, not software...


Sorry, I don't buy this at all. It creates a confusing divide. eg. the Radica stuff would be output as a system, and be fully playable without 'software' dsepite the fact that it does have software (because the bios is the software)


and indeed we should split radica from megadrive and run them separately as systems with a single set of games. there is already some system working like this (e.g. the batman game sharing vii CPU)

same work should be done probably for some famiclones, if we ever manage to get proper dumps...

Originally Posted By Haze
* This isn't meant to be an insult, but it's the impression I get from a lot of your changes, both to MAME and MESS. You seem to forget that users just expect things to work in a certain way, whereas you're more obsessed with implementation details and working against that. (Hence me having to clean up all the default eeprom stuff so that the games worked again because the masses simply considered them 'broken' when they wouldn't boot anymore..It was also irritating to me because I usually work with a clean tree and clean folders, and having to init every clone of every game I want to test if I make driver changes was annoying)


actually, I care about end-users: as soon as you mentioned that MARP was in need of default eeproms for konami games, I never opposed anymore their inclusion, because it would have meant to lose a MAME feature wink
(and, btw, eeprom init hacks were removed by Aaron and per-Aaron's request, it was not an idea of mine... check the svn logs wink )

I don't see the same gain in replacing systems with software... or making -lx output to blow up in size.

Originally Posted By R. Belmont
I'm not trying to be an @$$ here or anything, I'm really curious. I understand and support the end-user benefits of software lists, but I'd love to also know the benefits on the dev side.


well, lists give you a reliable set of testcases to avoid false bug reports. also, for obscure ex-ddr or japanese computers is not so easy to collect software and a softlist can help preservation to a large extent.
Posted By: Haze Re: Fixed software lists - 04/04/10 04:47 PM
Originally Posted By etabeta78

and indeed we should split radica from megadrive and run them separately as systems with a single set of games. there is already some system working like this (e.g. the batman game sharing vii CPU)

same work should be done probably for some famiclones, if we ever manage to get proper dumps...


my point that was from a users point of view you're creating an artificial divide. some actual playable software will be listed because it's a 'system' and others won't. I wasn't arguing that they should / shouldn't be systems. From a developers point of view it makes a bit more sense, but not everybody is a developer.

Originally Posted By etabeta78

(and, btw, eeprom init hacks were removed by Aaron and per-Aaron's request, it was not an idea of mine... check the svn logs wink )


fair enough, I've often had the same feeling about Aaron's changes. It felt like a job half done.

At least it did highlight some other cases, and I fixed a few bugs while going over everything adding the defaults. As I said, the 'simple*' roadblock of having to calibrate things like CyberSled, CyberCommando, Final Lap R and Speed Racer was clearly leading to nobody even bothering to run / test them at all.

* as I said, I still haven't figured out how to Calibrate Hard Drivin' and it took me about 20 minutes to actually get Speed Racer correct..

Originally Posted By etabeta78

well, lists give you a reliable set of testcases to avoid false bug reports. also, for obscure ex-ddr or japanese computers is not so easy to collect software and a softlist can help preservation to a large extent.


... and a large -listxml output helps with the same thing IMHO. Otherwise people will just list the systems they know, and ignore the rest. If MAME didn't list mahjong and other gambling games in the default -lx output it would reduce the size a lot, but it wouldn't be the right choice. Not creating a giant xml full of obscure systems might reduce the size, but it wouldn't be the right choice either IMHO.

Posted By: R. Belmont Re: Fixed software lists - 04/04/10 04:51 PM
Yeah, a full-flavor -listxml is vital for harnessing the PokeROMs to our own evil ends smile

Quote:
I'd also look at things the other way. If MAME and MESS are similar it could make it easier for MESS developers to work on MAME.


That's very true. Nearly all of the major unemulated hardware remaining for MAME in the future is PC or console-derived.
Posted By: Haze Re: Fixed software lists - 04/04/10 05:02 PM
Originally Posted By R. Belmont
Yeah, a full-flavor -listxml is vital for harnessing the PokeROMs to our own evil ends smile


Well I was thinking more in terms of Database sites like MAWS too. A giant site with the entire MESS output would be useful. For many games on obscure systems, even if they are emulated, the only reference you can find ANYWHERE is a low quality screenshot, or a youtube video of somebody running it on their desktop.

For such a purpose you wouldn't really want filtered data.
Posted By: etabeta78 Re: Fixed software lists - 04/04/10 05:27 PM
look, I don't know at all how MAWS or ProgettoEMMA translate the xml into a website (we should ask to cutebutwrong or s_bastian), but I don't think is so hard to
* create mess.xml with -lx
* parse it as usual
* extract all the <softwarelist> items (they are already present in xml, as if they were devices)
* for each of them, run -listsoftware to create the software subpages

it seems to me that it can be made as automated as only parsing the -lx output...
but at the same time, if I want to only extract the hardware info without the software ones, I still would be able to...
Posted By: etabeta78 Re: Fixed software lists - 04/04/10 05:33 PM
@judge: windows build (clean) fails to compile here:

Code:
Compiling src/osd/windows/d3d9intf.c...
In file included from src/mess/image.h:17,
                 from src/mess/messopts.h:10,
                 from src/mess/mess.h:13,
                 from src/emu/emu.h:95,
                 from src/osd/windows/d3d9intf.c:48:
src/mess/softlist.h:22: error: expected unqualified-id before 'struct'
src/mess/softlist.h:22: error: expected ';' before 'struct'
In file included from src/mess/image.h:17,
                 from src/mess/messopts.h:10,
                 from src/mess/mess.h:13,
                 from src/emu/emu.h:95,
                 from src/osd/windows/d3d9intf.c:48:
src/mess/softlist.h:53: error: expected ',' or '...' before 'struct'
make: *** [obj/windows/fastmess/osd/windows/d3d9intf.o] Error 1
Posted By: Haze Re: Fixed software lists - 04/04/10 05:48 PM
Originally Posted By etabeta78
look, I don't know at all how MAWS or ProgettoEMMA translate the xml into a website (we should ask to cutebutwrong or s_bastian), but I don't think is so hard to
* create mess.xml with -lx
* parse it as usual
* extract all the <softwarelist> items (they are already present in xml, as if they were devices)
* for each of them, run -listsoftware to create the software subpages

it seems to me that it can be made as automated as only parsing the -lx output...
but at the same time, if I want to only extract the hardware info without the software ones, I still would be able to...


but WHY?

There hasn't been a single compelling reason given to be different other than 'because we can' and 'people can work around the differences, so they don't matter' You're not giving any good reason for the differences to exist in the first place.

The XML lists are there, what's wrong with allowing MESS to output all the data, in an identical format to MESS, but with a 'platform' flag for each entry. It's an established and understood, standard format.

Otherwise you may as well not have a -listxml at all, and just tell people to directory list the folder and load the xml dats directly.

This is what I mean by differences for the sake of differences and change for the sake of change.

If MAME/MESS do things in a similar way it's also healthier in the long-term, there is less temptation to start introducing things like the gross hack that is/was the ingame Windows-specific menubar if the mantra is to be equal where possible, and only differ where there is no real choice.

Note, I'm not saying MAME is always right and MESS is always wrong, but the projects should be able to influence each other, not end up going in opposite directions which will eventually cause nothing but usability and code-sharing problems. (need I mention pinMAME which rendered itself unreabable, and unmaintainable, and unable to benefit from any real MAME improvements by taking such a path? Come the day MAME implements proper votrax speech they simply won't be able to benefit without a complete rewrite; likewise the PinMAME test cases can no longer be used for development of such a core because the codebase is so incompatible, a situation MESS/MAME should always avoid.)
Posted By: Justin Re: Fixed software lists - 04/04/10 11:32 PM
Originally Posted By R. Belmont
Yeah, a full-flavor -listxml is vital for harnessing the PokeROMs to our own evil ends smile


This. MESS already doesn't have much "market power", requiring anything fancy to get at the software lists is shooting ourselves in the foot IMO. (Look how many ROM managers support MESS' CRC / HSI files, which have been around since the beginning....) If it's in the -listxml then ROM managers and frontends more or less have to do something with it, and it becomes much more of a clear-cut "MESS set" for hoarder types.

Yeah, it'll make the output stupidly big, but then MAME's is already pretty huge, it's not like anything but automated tools will be looking at it anyway.

The software lists as they exist in SVN now are pretty spartan anyway, if there was a bunch of extra junk in there about the Swahili release date or the developer's favourite colour then there would be more of an argument for keeping it out IMO, but as it stands any real front-end or ROM manager is going to need all the information in there anyway so why make them work for it.
Posted By: Curt Coder Re: Fixed software lists - 04/06/10 06:46 AM
Can the ROM sizes/lengths be specified in hexadecimal? Much easier to read than decimal...
Posted By: etabeta78 Re: Fixed software lists - 04/06/10 06:52 AM
I think judge followed the (weird) format of MAME listxml output. I agree it would be easier to always use hex (maybe adding "0x" in front, to help frontends and clrmame to keep backward compatibility?)
Posted By: Anna Wu Re: Fixed software lists - 04/06/10 07:01 AM
etabeta78 check your PM (2x)
Posted By: Haze Re: Fixed software lists - 04/06/10 07:06 AM
Originally Posted By etabeta78
I think judge followed the (weird) format of MAME listxml output. I agree it would be easier to always use hex (maybe adding "0x" in front, to help frontends and clrmame to keep backward compatibility?)


Well sticking to a compatible format (decimal) in the -listxml format is ok, and a good idea. The output of that should mirror MAME as closely as possible.

I'd use hexidecimal in the actual software list files tho, to mirror the internal ROM loading macros they mimic. It's easier for human reading, as Curt says.
Posted By: Anna Wu Re: Fixed software lists - 04/06/10 07:25 AM
OK, i posted a message on clrmamepro board.
Posted By: Curt Coder Re: Fixed software lists - 04/06/10 08:27 AM
Another suggestion is that the DTD should be an external file in the hash directory so it doesn't need to be at the top of each system's xml.
Posted By: etabeta78 Re: Fixed software lists - 04/06/10 09:40 AM
it depends if we want the "feature" field to require a default value or not

"feature" is system specific and it's where we store the mapper/config options to accordingly choose the banking/handlers installation/driver init at start
Posted By: judge Re: Fixed software lists - 04/06/10 01:51 PM
I tried to follow the output of -listxml for the rom declarations since that is what people are familiar with and I do not want to do deviate too much from the mame output.

There is an odd thing in the listxml output: size is decimal and offset is hexdecimal. I asked on the list for the reasoning behind this, but haven't received a reply yet.


Posted By: Haze Re: Fixed software lists - 04/06/10 02:32 PM
Originally Posted By judge
I tried to follow the output of -listxml for the rom declarations since that is what people are familiar with and I do not want to do deviate too much from the mame output.

There is an odd thing in the listxml output: size is decimal and offset is hexdecimal. I asked on the list for the reasoning behind this, but haven't received a reply yet.




It sounds to me like it should be fixed in MAME, so that both output hexdecimal.
Posted By: R. Belmont Re: Fixed software lists - 04/06/10 02:53 PM
Agreed. We always use hex in the source, there's no reason not to output it that way too.
Posted By: Anna Wu Re: Fixed software lists - 04/06/10 04:26 PM
Originally Posted By Anna Wu
OK, i posted a message on clrmamepro board.


I hope my answer is correct.
Posted By: judge Re: Fixed software lists - 04/06/10 05:27 PM
Perhaps you could add that 2 lines were added to the DTD.
Posted By: Anna Wu Re: Fixed software lists - 04/06/10 06:04 PM
Originally Posted By judge
Perhaps you could add that 2 lines were added to the DTD.


done, thanks smile
Posted By: Robbbert Re: Fixed software lists - 04/06/10 09:44 PM
Quote from Roman:

Quote:
Actually it's not related to the software list tags but to some other line limitations. Will be fixed in next version.
However if you try to load the xml file into cmpro (and not using a direct import from the mess binary) you need to define a new engine.cfg entry like

engine (
name MESS_DATFILE
gamelist mess
cachefile _messdat
replace machine game
)

...but even with that you currently run into some xml detection line limitation.....as mentioned...this will get fixed.


So by next version all should be good.
Posted By: Anna Wu Re: Fixed software lists - 04/07/10 11:36 PM
Originally Posted By robbbert
Quote from Roman:

Quote:
Actually it's not related to the software list tags but to some other line limitations. Will be fixed in next version.
However if you try to load the xml file into cmpro (and not using a direct import from the mess binary) you need to define a new engine.cfg entry like

engine (
name MESS_DATFILE
gamelist mess
cachefile _messdat
replace machine game
)

...but even with that you currently run into some xml detection line limitation.....as mentioned...this will get fixed.


So by next version all should be good.


Roman released clrmamepro 3.133

- fixed: some line limit detection fixes on xml dat parser (MESS > .137 issue)

and some more
Posted By: judge Re: Fixed software lists - 04/08/10 05:51 AM
great
Posted By: etabeta78 Re: Fixed software lists - 04/08/10 05:55 AM
now we have to see if he wants to add support for the software lists, and possibly find a way to handle them easily (e.g. should cmpro run -lx and extract the <softlist> items? or should the user specify in advance the system? we need Roman's input probably...)
Posted By: etabeta78 Re: Fixed software lists - 04/28/10 08:30 AM
-update status on software lists-

as you may have noticed, as of current svn, MESS can launch software list items for some more systems. however, more supported lists means that we need some help to set properly the supported/unsupported flag. the current status is:

1. Systems with software list and verified emulation status (i.e. the supported field is correctly set in the xml)
Code:
* scv (I think judge took care of flags for this)
* ibmpcjr (I think judge took care of flags for this)
* gamepock (I think judge took care of flags for this)
* pasogo (all games work)
* advision (turtles is not working)
* gmaster (I tested all the games)
* vboy (no game is playable to my knowledge)


2. Systems with software list and unverified emulation status
Code:
* bbcbc
* pv1000
* supracan
* gamecom


3. Systems with software list and emulation status believed to be correct
Code:
* pokemini
* ngp / ngpc
* sg1000
* megaduck
* arcadia


for systems in 2., I'm going to test gamecom later today, but for bbcbc, pv1000 and supracan I don't have the required software. hence, you are welcome to report which games work and which do not, so that we can update the xml file accordingly

for systems in 3., I tested some of the images and they were working fine. Therefore, I left all games as supported. However, if you know of any game for those systems which is not working, please report it here and we will fix the xml accordingly.

thanks in advance for the help.
Posted By: Robbbert Re: Fixed software lists - 04/28/10 08:45 AM
Do you mean the games work in any emulator, or MESS in particular?

BBCBC all the games work.

GAMECOM none work mostly because of cpu issues due to lack of documentation

POKEMINI all work, but the sound isn't emulated

PASOGO doesn't work at all, so how do we know the games work?
Posted By: etabeta78 Re: Fixed software lists - 04/28/10 09:05 AM
in MESS, of course. we are talking about adding
Code:
supported="no"
supported="partial"

to the MESS xmls, so it's the MESS status which matters

pasogo: I was able to start all 3 games in pasogo.xml and they worked to some extent (inputs were recognized in the menus, and I was able to start some go game). that's how I know that they work, even if I don't know exactly the go rules. wink

gamecom: I was able to play frogger yesterday... that's why I haven't added the unsupported flag to all the games as I had done with vboy
Posted By: Robbbert Re: Fixed software lists - 04/28/10 10:15 AM
Frogger and Centipede will "run", but only while keys are being pressed. They freeze while you don't do anything. That is one of the major problems. Another example is the inbuilt Solitare game, you will notice the timer only works when you do something.

I've tried all games under the debugger, and most of them do appear to want to run (I have some never-before seen snapshots). But, MESS-wise, none really work.


Thanks for the info about Pasogo. I've never seen anything apart from a black screen, so far. I'll give it a proper test.

VBOY: I do have some demos etc that seem to "work", but i'm not sure if the inputs responded, it's been a while since i looked at that system.

ADVISION: I'm missing one game, and the 4 i have do work.
Posted By: etabeta78 Re: Fixed software lists - 04/28/10 10:30 AM
thanks for the info about gamecom. I will flag the games as unsupported for the time being.

about vboy, we don't have demos in the xml gamelist, so there is not much to flag as supported (inputs seem to be working to some extent: Virtual Boy Wario World recognize Start pressure, but then freezes)
Posted By: Firewave Re: Fixed software lists - 04/28/10 10:53 AM
I might look into adding softlist support to mame_regtest, that should help with the testing.
Posted By: ranger_lennier Re: Fixed software lists - 04/28/10 11:51 AM
The Pasogo system should probably be classified as working. PeT listed it as not working when he wrote the driver, probably because he didn't test it very much, but I've never seen any problems with it. That said, I can't say with complete confidence since I know virtually nothing about Japanese and Go and as such have basically just randomly pressed buttons. smile
Posted By: Anna Wu Re: Fixed software lists - 04/28/10 12:08 PM
Originally Posted By robbbert
Frogger and Centipede will "run", but only while keys are being pressed. They freeze while you don't do anything. That is one of the major problems. Another example is the inbuilt Solitare game, you will notice the timer only works when you do something.

I've tried all games under the debugger, and most of them do appear to want to run (I have some never-before seen snapshots). But, MESS-wise, none really work.


Thanks for the info about Pasogo. I've never seen anything apart from a black screen, so far. I'll give it a proper test.

VBOY: I do have some demos etc that seem to "work", but i'm not sure if the inputs responded, it's been a while since i looked at that system.

ADVISION: I'm missing one game, and the 4 i have do work.


Which game ?
Posted By: etabeta78 Re: Fixed software lists - 04/28/10 12:16 PM
Originally Posted By ranger_lennier
The Pasogo system should probably be classified as working. PeT listed it as not working when he wrote the driver, probably because he didn't test it very much, but I've never seen any problems with it. That said, I can't say with complete confidence since I know virtually nothing about Japanese and Go and as such have basically just randomly pressed buttons. smile


yes, pasogo could probably be re-flagged to some IMPERFECT emulation, rather than completely NOT_WORKING.

back to gamelists, I'm going to add soon preliminary[1] support for most of the remaining lists. then, we can start working on the missing features with a good number of available test cases...

and, who knows, we might succeed in attracting haze to work on some MESS systems (e.g. lynx or vboy...) wink

[1] preliminary means that probably sram won't work, and instead of having per-game init routines, I will re-use the current generic handling as much as possible. at a later stage, the loading code will have to be cleaned up a bit to better exploit softlist features
Posted By: Robbbert Re: Fixed software lists - 04/28/10 12:48 PM
I should make a correction in regard to the GameCom; the game "Lights Out" does work.
Posted By: ht1848 Re: Fixed software lists - 04/28/10 06:48 PM
I have been out of pocket for a little while and am very excited to see this progressing...Sorry if this is a duplicate question..

Has anyone talked with Roman about what a software scan/fix/check might look like in clrmame based on the new listxml output?

I would love to just configure a top level software folder and have it check all the software/<systemname> folders against the software list, if present, and standard system name. With a couple of clicks I could validate all the roms (that I dumped) and software (that I own of course) against the MESS standard. That would be some power.
Posted By: etabeta78 Re: Fixed software lists - 04/29/10 12:34 PM
check for progresses on clrmame supporting our lists at http://www.emulab.it/forum/index.php?topic=300.0

philosophically, I think it would be better to handle separately the lists, but I'm awaiting for some feedback from Roman, then we can discuss about it smile

in the meanwhile, current svn have preliminary supports all softlists but svision (I think it requires rom mirroring), crvision (which requires some more work on the loading routines) and 32x (which requires *A LOT* of work on the loading routines)

also, I updated jaguar and gamecom lists to report as not working all the games.

soon-ish there will be gb, gbc, gba, snes and n64 lists added (some additional cleanup is needed), and some more images will be added to current lists, for dumps which have surfaced after list creation...

then, we can start thinking about sram and any other problems which might surface

the road is still quite long, but at least we are moving
Posted By: Haze Re: Fixed software lists - 04/29/10 01:24 PM
Originally Posted By etabeta78
check for progresses on clrmame supporting our lists at http://www.emulab.it/forum/index.php?topic=300.0

philosophically, I think it would be better to handle separately the lists, but I'm awaiting for some feedback from Roman, then we can discuss about it smile


Just handle them all, and let users uncheck ones they don't want, like they do for sets etc. at the moment. My concern is that if things get excluded by default people simply won't bother to use the lists to scan obscure systems at all which are the strongest area of MESS. Sure, scanning everything might sound overkill, but aside from CHDs it's not that insane of a prospect to full scan everything somebody might own, and the others can be disabled ;-)

Loading one list and unchecking a few boxes is far easier than having to load, setup paths for dat, scan, load, setup paths, scan, load setup paths, scan etc. for every single supported system. That's why some of the other rom managers around (not clrmame) basically have the option to load all lists at once anyway, because it's easier that way, especially if you're dealing with 30 systems where there are only a handful of small titles on each system; in such a case you're talking a matter of minutes to scan everything vs. a good hour or two to load, configure and scan each time. It's also much easier to get an overall idea of everything that's been changed / added in a new release if everything is scanned / included by default.

Guess which method has the most benefits?
Posted By: incog Re: Fixed software lists - 04/29/10 02:44 PM
MAME has a bunch of verified NeoGeo AES cart dumps in it, maybe we should borrow those for a software list.
Posted By: etabeta78 Re: Fixed software lists - 04/29/10 02:51 PM
feel free to create such a list if you like, but mind that supporting it in MESS will require some time (adding the code to load those carts will require some efforts).

also, would you mind checking supracan and pv1000 lists to see if there are games which should be marked as supported="no"? I have just a few games for those systems, so the lists are basically untested.
Posted By: incog Re: Fixed software lists - 04/29/10 02:58 PM
Sure, just let me build and i'll test everything I have smile

(Edit: done, I noticed an A'can dump missing from the list so I added that too)
Posted By: Kaylee Re: Fixed software lists - 04/29/10 03:03 PM
Hi checked the casio pv-1000.

Almost Everything runs, but Pachinko-Ufo stops at title. Says push start and does nothing.

Checked mine against No-intro and scans ok.
Posted By: etabeta78 Re: Fixed software lists - 04/30/10 04:09 PM
as of rev.7946, gameboy (gb, gbc & gba), snes and n64 lists have been included.

they still need a lot of work (years, publishers, spelling fixes etc.), but at least I don't have to fear hd failures anymore wink

I hope to finish the NES list as well, soon-ish, and to have some smart idea for SRAM handling (so that loading from software lists can be considered as good as loading from absolute paths)
Posted By: Tetsuo55 Re: Fixed software lists - 04/30/10 08:43 PM
Originally Posted By Haze
Just handle them all, and let users uncheck ones they don't want, like they do for sets etc. at the moment.
I agree with Haze completely.
Posted By: k1w1 Re: Fixed software lists - 04/30/10 08:57 PM
I have been keenly following this thread since it began and am looking forward to being fully able to use the xml software lists.

However i am having trouble using them at the moment and suspect it could be a problem because of the Windows 7, 64bit OS i am using.

I have ensured that everything is setup as etabeta's instructions to roman in the pevious thread, i.e: -

1) i have cleaned out the old hsi files from the hash directory and replaced them with the xml files from the supplied link.
2)created a new mess ini file that has hashpath pointing to the hash directory.
3)i have downloaded the latest svn build from bobz site.

yet when i run the command "mess ibmpcjr -listsoftware" i get this: -

E:\mess>mess ibmpcjr -listsoftware
<?xml version="1.0"?>
<!ELEMENT softwarelists (softwarelist*)
<!ELEMENT softwarelist (software+)>
<!ATTLIST softwarelist name CDATA #REQUIRED>
<!ATTLIST softwarelist description CDATA #IMPLIED>
<!ELEMENT software (description, year?, publisher, part*)>
<!ATTLIST software name CDATA #REQUIRED>
<!ATTLIST software cloneof CDATA #IMPLIED>
<!ATTLIST software supported (yes|partial|no) "yes">
<!ELEMENT description (#PCDATA)>
<!ELEMENT year (#PCDATA)>
<!ELEMENT publisher (#PCDATA)>
<!ELEMENT part (dataarea*)>
<!ATTLIST part name CDATA #REQUIRED>
<!ATTLIST part interface CDATA #REQUIRED>
<!ATTLIST part feature CDATA #IMPLIED>
<!ELEMENT dataarea (rom*)>
<!ATTLIST dataarea name CDATA #REQUIRED>

<!ATTLIST dataarea size CDATA #REQUIRED>

<!ATTLIST dataarea databits (8|16|32|64)
"8">
<!ATTLIST dataarea endian (big|little) "
little">
<!ELEMENT rom EMPTY>
<!ATTLIST rom name CDATA #IMPLIE
D>
<!ATTLIST rom size CDATA #REQUIR
ED>
<!ATTLIST rom crc CDATA #IMPLIED
>
<!ATTLIST rom md5 CDATA #IMPLIED
>
<!ATTLIST rom sha1 CDATA #IMPLIE
D>
<!ATTLIST rom offset CDATA #IMPL
IED>
<!ATTLIST rom status (baddump|no
dump|good) "good">
<!ATTLIST rom loadflag (load16_b
yte|load16_word|load16_word_swap|load32_byte|load32_word|load32_word_swap|load32
_dword|load64_word|load64_word_swap|reload) #IMPLIED>
<softwarelists>
<softwarelist name="ibmpcjr_cart">
</softwarelist>
</softwarelists>

no xml software data at all.

I know that there have been some issues with Win7 64bit and mess in the past and was wondering if any one else was having this problem with reading the xml files.


k1w1
Posted By: R. Belmont Re: Fixed software lists - 04/30/10 09:13 PM
There are zero problems with Win7 64 bit and MAME or MESS - both emulators have always been fully compatible with that operating system (and Vista 64-bit before it) when properly configured. Please stop spreading FUD.
Posted By: etabeta78 Re: Fixed software lists - 04/30/10 09:27 PM
Originally Posted By Tetsuo55
Originally Posted By Haze
Just handle them all, and let users uncheck ones they don't want, like they do for sets etc. at the moment.
I agree with Haze completely.


Well, implementation is mainly up to Roman.

Once more, I realize that my needs are pretty different from those of average users [1] , and hence I simply leave other people to find a solution which satisfies most of the users. smile

In the meanwhile, rev 7947 adds software list handling to svision and crvision. And I decided that I need someone to help me with 32x carts, or the list won't be hooked up for the time being.

the point is that the cart needs to be loaded twice

Code:
	ROM_REGION16_BE( 0x400000, "gamecart", ROMREGION_ERASE00 ) /* 68000 Code */
	ROM_CART_LOAD("cart", 0x000000, 0x400000, ROM_NOMIRROR)

	ROM_REGION32_BE( 0x400000, "gamecart_sh2", ROMREGION_ERASE00 ) /* Copy for the SH2 */
	ROM_CART_LOAD("cart", 0x000000, 0x400000, ROM_NOMIRROR)


and you can see that it needs to be loaded once as 16bit_BE and once as 32bit_BE. can anyone provide me with the necessary code to handle the two processes (I guess it all reduces to a couple of for loops, but I failed to properly implement it)...


[1] I need no flashing UIs, I prefer verbose command line options to shortcuts, etc.
Posted By: etabeta78 Re: Fixed software lists - 04/30/10 09:34 PM
Originally Posted By k1w1
I know that there have been some issues with Win7 64bit and mess in the past and was wondering if any one else was having this problem with reading the xml files.


you've been probably misinformed: MAME and MESS have probably been the only emulators which have had no problems with the passage to new Windows OS and to 64 bit (it helps to have the head of the project working at MS)

that said, Roman fixed his problems, so I guess you have to find what you are doing wrong. As I said on clrmame forum, try open the xml file and check if the contents matches this file

http://git.redump.net/cgit.cgi/mess/plain/hash/ibmpcjr_cart.xml

lists have been reported as working on basically every OS, so I cannot see a good reason why they should fail on your setup wink
Posted By: Robbbert Re: Fixed software lists - 04/30/10 10:32 PM
I checked ibmpcjr and had the same problem, then realised I had been caught by this before. Checking the hash path in ini\ibmpcjr.ini showed it to be incorrect. Fixing it fixed the problem.



Is it necessary to output the attribute list (!ATTLIST, !ELEMENT, etc) at the start?
Posted By: k1w1 Re: Fixed software lists - 05/01/10 12:37 AM
I remember reading in this forum some issue with the n64 emulation and Win 7 64bit. I did not intend to spread any FUD, just thought i was helping to spot a possible problem. Sorry for any misunderstanding.

Anyhow, i have found the problem.

I copied the files from the git site using "save target as.." and did not realise that it had copied the files across in htm format.

Lack of sleep will do that to you.

Thanks for the help anyway.

k1w1
Posted By: R. Belmont Re: Fixed software lists - 05/01/10 01:05 AM
Ahh, yes. The N64 emulation *only* works on 64-bit OSes, which is pretty much the opposite of a problem with it smile
Posted By: etabeta78 Re: Fixed software lists - 05/01/10 05:46 AM
Originally Posted By robbbert
]Is it necessary to output the attribute list (!ATTLIST, !ELEMENT, etc) at the start?


yes, because any use of the xml output with a xml parser would require the dtd declaration
Posted By: Robbbert Re: Fixed software lists - 05/01/10 07:41 AM
Ah, ok.
Posted By: etabeta78 Re: Fixed software lists - 05/01/10 09:35 AM
Originally Posted By R. Belmont
Ahh, yes. The N64 emulation *only* works on 64-bit OSes, which is pretty much the opposite of a problem with it smile


I tried to re-apply rev 7700 to make 32bit MESS to work again with n64, but it keeps calling the drc code rather than using the non-drc code... any idea about why? shouldn't there be a way to let the driver decide if it has to use the drc version or the non-drc version of the cpu?
Posted By: R. Belmont Re: Fixed software lists - 05/01/10 01:04 PM
Yeah, don't do that, let JD handle it.
Posted By: etabeta78 Re: Fixed software lists - 05/01/10 01:23 PM
I asked him if I could take care of this (since he seems busy) and he said "Sure, go ahead."

however, it seems I'm not capable of fixing it... hence, it's up to JD to find the proper solution
Posted By: etabeta78 Re: Fixed software lists - 05/07/10 08:36 AM
update on the clrmame support for software lists: Roman has added preliminary support for them

Originally Posted By Roman
They are handled as any other datfile and you as an user have to care about how to setup rompaths.
If you now drop a snes.xml, it will show you a datfile in the profiler
name="snes" description="Nintendo SNES cartridges"
and it holds the single sets, 2020bb, 2020bbj, 96zenko etc....in this case these are 1-rom-sets.

I guess any further support beyond that is something for later releases....if at all..


in other words, for the moment you will have to create a xml list for each system with

Code:
mess system -listsoftware >system.xml


then load it in clrmame, setup the directory to rebuild/scan and finally use clrmame to handle the romsets. I plan to write a small guide to software lists before 0.138 comes out... stay tuned smile
Posted By: Haze Re: Fixed software lists - 05/07/10 11:51 AM
Originally Posted By etabeta78
update on the clrmame support for software lists: Roman has added preliminary support for them

Originally Posted By Roman
They are handled as any other datfile and you as an user have to care about how to setup rompaths.
If you now drop a snes.xml, it will show you a datfile in the profiler
name="snes" description="Nintendo SNES cartridges"
and it holds the single sets, 2020bb, 2020bbj, 96zenko etc....in this case these are 1-rom-sets.

I guess any further support beyond that is something for later releases....if at all..


in other words, for the moment you will have to create a xml list for each system with

Code:
mess system -listsoftware >system.xml


then load it in clrmame, setup the directory to rebuild/scan and finally use clrmame to handle the romsets. I plan to write a small guide to software lists before 0.138 comes out... stay tuned smile


Sounds like a complete nightmare. I won't be using it that way, I have 40 odd games for 40 odd systems, there is _no way_ you're going to get me to create and load 40 xml files for each new update / release. It's a non-starter, dead in the water, utterly useless. You don't appear to be designing this around the needs of people who are going to use it.

It *needs* to be one-stop (or at least have that option). That is a baseline requirement of the MESS output, or the CLRMame processing, and more likely to happen by having that option in MESS than forcing Roman to do something special.

I'm not trying to troll you here, but seriously, the current proposal is unusable for my perfectly normal needs, and presents no benefits over the existing mess of dats etc. for the individual systems.

There is nothing wrong with *allowing* the output of individual software lists, that could be considered a useful feature. *Forcing* the output of individual software lists OTHO is a hindrance, an anti-feature.
Posted By: etabeta78 Re: Fixed software lists - 05/07/10 12:07 PM
did you bother to read the linked discussion at clrmame thread?

after Roman was able to add this basic support (with separated profiles), I suggested an additional feature, based on the way MESS expects games to be stored: the user sets only his roms directory (the top of the tree) and then clrmame sets automatically the path as subfolders of this directory and creates the folder if there are none. i.e. you only set a thing, and clrmame deals with the rest (with no additional changes needed in MESS).

Roman's answer was: "I guess any further support beyond that is something for later releases....if at all..".

feel free to convince him to add it right now.

I prefer to look the positive side: we have some basic support in MESS (but much more is needed to be worked on) and we have some basic support in clrmame = we have moved the first step in the right direction before 0.138.
Posted By: etabeta78 Re: Fixed software lists - 05/07/10 12:11 PM
Originally Posted By Haze

There is nothing wrong with *allowing* the output of individual software lists, that could be considered a useful feature. *Forcing* the output of individual software lists OTHO is a hindrance, an anti-feature.


did you bother to try MESS lately? you already can output all the lists together, by simply calling the -listsoftware option with no system (you get a huge xml with all the lists one after the other)... it's just clrmame which cannot handle them atm, and therefore you have to rely on the above method I mentioned...

I'm not forcing anything, and I don't like to have creating ~30 lists either, but it is the only way that works at this stage... frown
Posted By: Haze Re: Fixed software lists - 05/07/10 12:14 PM
Originally Posted By etabeta78
did you bother to read the linked discussion at clrmame thread?

after Roman was able to add this basic support (with separated profiles), I suggested an additional feature, based on the way MESS expects games to be stored: the user sets only his roms directory (the top of the tree) and then clrmame sets automatically the path as subfolders of this directory and creates the folder if there are none. i.e. you only set a thing, and clrmame deals with the rest (with no additional changes needed in MESS).

Roman's answer was: "I guess any further support beyond that is something for later releases....if at all..".

feel free to convince him to add it right now.

I prefer to look the positive side: we have some basic support in MESS (but much more is needed to be worked on) and we have some basic support in clrmame = we have moved the first step in the right direction before 0.138.


So, there is an easy solution.

1) Output the whole lot
2) Add an extra field in the XML for each entry to indicate the base 'system' name. ClrMAME can ignore this for now
3) Allow MESS to look in more places for the Roms (and Bios) this should be done anyway, the forced directory structure is beyond annoying.

That way, sure, it might not look pretty if I have all my roms for all my systems in the same folder, but it's simple, and it works.

Roman can expand CLRMAME in the future to acknowledge the system path if he wants, and if having multiple software with the same name on different systems triggers a clrmame bug, he can fix that (I don't care if ghouls.zip contains both the snes, md and nes versions to be honest)

This is an expandable solution which requires minimal work from you, minimal work from Roman, and doesn't place a huge burden on anybody actually wanting to use the system.

Sometimes the simple ideas are the best ones...
Posted By: Haze Re: Fixed software lists - 05/07/10 12:17 PM
Originally Posted By etabeta78
Originally Posted By Haze

There is nothing wrong with *allowing* the output of individual software lists, that could be considered a useful feature. *Forcing* the output of individual software lists OTHO is a hindrance, an anti-feature.


did you bother to try MESS lately? you already can output all the lists together, by simply calling the -listsoftware option with no system (you get a huge xml with all the lists one after the other)... it's just clrmame which cannot handle them atm, and therefore you have to rely on the above method I mentioned...

I'm not forcing anything, and I don't like to have creating ~30 lists either, but it is the only way that works at this stage... frown


Is there a specific issue, that Roman can't work around? If the XML gets dumped out in the same format as MAME the only concerns I can see are

a) Can CLRMAME ignore unused fields? (should be an easy fix)
b) Can CLRMAME deal with duplicate roms with the same name (messier, but I'm sure easy enough to workaround, either by using unique names for now, or a kludge in clrmame until things can be fixed)

None of these are showstoppers, and surely more workable than the current solution without having to make any huge changes anywhere.

I still think you're overdesigning things here, and everything can be introduced in a much more streamlined way, which suits both developers, users, and external contributors like Roman, by allowing things to evolve over time.

The current system burdens at the very least, users and Roman, requiring more immediate changes from him, and avoidable work for the users.

If Roman expands CLRMAME later to add support for software paths that can be a feature, not a requirement. If prior to that sets have to have unique (and uglier) names, so be it; the later renaming can be a feature / improvement once the external support is there. If I still have to scan BIOS/Software in different scans, it's annoying, but not as much as the current situation. Progress comes in steps, the first of those is to make things work with what already exists!

Posted By: etabeta78 Re: Fixed software lists - 05/07/10 12:22 PM
Originally Posted By Haze
That way, sure, it might not look pretty if I have all my roms for all my systems in the same folder, but it's simple, and it works.


it's easier to have bublbobl name for all bubble bobble games and have them in separate folders, than to have to remember 10 different names for the same game. so, imho, it would be a bad thing (and very confusing for the user) to allow for this.

moreover, usually people already collect games for diff systems in diff folders (check common request on emuloader boards) and not in a big messy folder. with our "forced" structure, you only have to rename the folder to the MESS system name and add the folder above as rompath. definitely easier than what you suggest.

if you just have spare files in the same directory, you rebuild them with clrmame (or you use mess -romident, which scans all the lists in a breath) and the programs put them in the right folder for the right system and then you launch MESS.


EDIT: I hadn't noticed this part

Originally Posted By Haze
3) Allow MESS to look in more places for the Roms (and Bios) this should be done anyway, the forced directory structure is beyond annoying.


bios can go in any rompath (as romsets in MAME). software goes in a system/ subfolder of one of the rompath. I currently have 4-5 rompaths on 3 different HD and software is spread in subfolder of these paths. it's not that you have to put everything in a single location...
Posted By: etabeta78 Re: Fixed software lists - 05/07/10 12:30 PM
Originally Posted By Haze
]Is there a specific issue, that Roman can't work around?


is he busy? is he lazy? is there any issue? if you read the thread @ clrmame board, you know as much as I know about it, which is: we now have this support, and anything more will have to wait (if it will happen ever)
Posted By: Haze Re: Fixed software lists - 05/07/10 12:43 PM
Originally Posted By etabeta78
Originally Posted By Haze
That way, sure, it might not look pretty if I have all my roms for all my systems in the same folder, but it's simple, and it works.


it's easier to have bublbobl name for all bubble bobble games and have them in separate folders, than to have to remember 10 different names for the same game. so, imho, it would be a bad thing (and very confusing for the user) to allow for this.



Easier for you, but the external tools (clrmame) don't support this yet. Working within the limits of what's available is better than producing a system that doesn't work. As for rom paths and locations, allowing choice is always better than forced behavior. I agree, the ultimate goal should be to allow the same set name across different systems, but it should be *my* choice where I put those files even then. Mess should of course look in sub-folders, but it should also look in the base folder.

Originally Posted By etabeta78

moreover, usually people already collect games for diff systems in diff folders (check common request on emuloader boards) and not in a big messy folder. with our "forced" structure, you only have to rename the folder to the MESS system name and add the folder above as rompath. definitely easier than what you suggest.


So let those people output the individual lists. That's easier for them, until clrmame is improved to support it automatically. It's not easier for me, it's about 40x more work every single time.

I fail to see how there can be any issues. If I add a bunch more systems to HazeMD then ClrMAME still works. I designed it within the limits of what already existed, it worked, and it worked well. I could paste in the SNES and N64 lists, and it would still work.

I also fail to see how you've managed to completely break this with MESS, unless you've decided to compeltely rewrite established standards rather than evolving them, slowly, together with the tools that support them. At that point it becomes your problem, working outside of established limits.

It was perfectly possible to create something that was functional, with existing tools, even if it wasn't that pretty yet.

It was perfectly possible to produce this system in a way whereby the external tools could be improved, and support the output in a better way, without breaking existing compatibility.

It was perfectly possible to produce this system in such a way that once the external tools were improved any special support (eg naming conventions) for the legacy versions could be removed.

I consider it a failure that you haven't managed this and see it as the result of either arrogance or incompetence.


In other words:

You could have made everything work with the current build of ClrMAME in some form.

You could have said "hey Roman, it would be cool if you acknowledged the new 'system folder' data we output in the XML"

People could have used things as they were for now with the current tools, picking the use case which suited them based. (one dat with everything crammed into one folder, or loading several dats depending on if they only cared about a handful of systems, rather than a handful of games on many systems)

Roman could have made improvements in his own time, allowing for 'cleaner' organization of the rom folders automatically.

That would have been the easy route...
Posted By: etabeta78 Re: Fixed software lists - 05/07/10 01:14 PM
Originally Posted By Haze
I fail to see how there can be any issues. If I add a bunch more systems to HazeMD then ClrMAME still works. I designed it within the limits of what already existed, it worked, and it worked well. I could paste in the SNES and N64 lists, and it would still work.


mmm... you still seem not to see that the equivalent of MAME/HazeMD drivers in MESS are system bios not games in the software lists. accordingly, if we would add 10 thousand of new home computers, there would be absolutely no problem with clrmame, rompaths or whatever.

the equation (MAME drivers = MESS systems) has been like this from day 1, it's not something I made up to displease you (and I also repeated it many of times, even if you seem to keep misunderstanding what I say).


Originally Posted By Haze
I also fail to see how you've managed to completely break this with MESS, unless you've decided to compeltely rewrite established standards rather than evolving them, slowly, together with the tools that support them. At that point it becomes your problem, working outside of established limits.

It was perfectly possible to create something that was functional, with existing tools, even if it wasn't that pretty yet.


except that this is mainly judge's work and not mine (I mainly worked on list building, not on the core support), the point of this discussion was to add software lists (which are definitely useful) without rebuilding the whole emulator from scratch. if you want to change the whole core, feel free to send the correspondent patch, but do not count on me because rewriting the core is far beyond my skills.

Originally Posted By Haze
I consider it a failure that you haven't managed this and see it as the result of either arrogance or incompetence.


don't get me wrong, but from now on, I'm waiting for your patch which very easily adds everything that was necessary without breaking any other part of MESS.


Posted By: Haze Re: Fixed software lists - 05/07/10 01:32 PM
Originally Posted By etabeta78

don't get me wrong, but from now on, I'm waiting for your patch which very easily adds everything that was necessary without breaking any other part of MESS.


I already released it, it was called HazeMD. It worked. I could take all the console systems from MESS and expand that if I really wanted, then add extra info to the xml, then move the software lists to be external, then get it to look in additional rom paths automatically, then hope Roman adds features to ClrMAME to make management cleaner. Following on from that, I could have made the set names cleaner because the external tools were now better. Evolution. I'm more and more tempted to just take this path...

You're caring _far_ too much about the semantics (Mess software vs. MAME etc.) and not enough about creating something that works.

I'm not proposing that there is no difference, I'm proposing you create an output that can ignore those differences (but provides the neccessary data to use it later) and works with existing software, which can then evolve. A compatibility mode, something which OSI seems to forget exists.

There is no fundamental reason why this couldn't have been done. Instead you've basically created a system which is incompatible with what already exists, and makes things difficult for the sake of being difficult.
Posted By: Just Desserts Re: Fixed software lists - 05/07/10 01:36 PM
It's very easy to criticize other people without lifting a finger to expend a shred of effort, isn't it? Either put up or shut the fuck up, and no, HazeMD doesn't count.
Posted By: incog Re: Fixed software lists - 05/07/10 01:38 PM
I can't even read his replies anymore and take them seriously, he's trollin'.

You dont demand a fundemental change, have it happen with no help from yourself and then get to complain about minor details like having to type -cart or piping all the xml's together.

Yes there is no fundamental reason why it doesn't work they way you want it to, just like there is no fundamental reason stopping you from fixing it or just plain forking MESS.
Posted By: Haze Re: Fixed software lists - 05/07/10 01:39 PM
Originally Posted By Just Desserts
It's very easy to criticize other people without lifting a finger to expend a shred of effort, isn't it? Either put up or shut the fuck up, and no, HazeMD doesn't count.


It's also very easy to design a system which doesn't work, if you don't listen to the needs of people who will be using it, and I'd say HazeMD very much *does* count, it was a 'this can work' prototype. I believe we had a discussion about this in the shoutbox just the other day, and you were agreeing with me. In this case, consider me the artist.

Unfortunately all that I'm going to get are the usual stream of Open Source type replies where any level of critisism against the project is written off as trolling now. That's why such things are doomed.

Posted By: incog Re: Fixed software lists - 05/07/10 01:44 PM
This isn't just critisism, that is a false dichotomy. I'm also fully aware of the irony of me replying when I just called you out as a troll.

This thread just smacks of you trying desperately trying to find reasons not to work on MESS after having the fixed software lists, which you wanted, added.
Posted By: etabeta78 Re: Fixed software lists - 05/07/10 01:47 PM
Originally Posted By Haze
There is no fundamental reason why this couldn't have been done. Instead you've basically created a system which is incompatible with what already exists, and makes things difficult for the sake of being difficult.


there is the fundamental reason that the core would not easily support what you suggest (HazeMD has no concept of system + software command line options, it only launches every game as if a separate system; following the same path in MESS would have broken any software loading in home computer or required a lot of rewrites).
the current core structure is also the main reason why we currently only have lists for carts (no floppies and no tapes) and we have some problems with nvram (but these should not be a big deal). if you want to fully rewrite this feel free, the source is available.

if you prefer, I easily admit my incompetence on the subject: that's why I worked mainly on the lists and not on the core (except for adding some validation code)
however, as a user, I also find definitely simple and intuitive the current implementation and therefore I appreciate a lot the work judge has done on the core side. I don't think to be the only one, but time will tell...
Posted By: Haze Re: Fixed software lists - 05/07/10 01:50 PM
Originally Posted By incog
This isn't just critisism, that is a false dichotomy. I'm also fully aware of the irony of me replying when I just called you out as a troll.

This thread just smacks of you trying desperately trying to find reasons not to work on MESS after having the fixed software lists, which you wanted, added.


Not at all. I'm just banging my head on the desk over how something which should have been so simple to produce a first-pass working version of has turned into something so illogical and difficult from an end-user point of view rather, seemingly over minor technical details which could have been set aside during this stage of development.

I'm *glad* MESS is documenting some software now, that much is a big step in the right direction. I still plan on looking forward to working on MESS, I've said to Kale that once the implementation is solid enough, that will happen. (and likewise suggested we look at Lynx, Jaguar etc.) Once we're all on the same page that will be easy.

Posted By: etabeta78 Re: Fixed software lists - 05/07/10 01:54 PM
Originally Posted By Haze
It's also very easy to design a system which doesn't work, if you don't listen to the needs of people who will be using it


haven't you said since day 1 that you do not use MESS? I heard a lot of people being happy with the improvements of the past year in MESS...
Posted By: Haze Re: Fixed software lists - 05/07/10 01:55 PM
Originally Posted By etabeta78
there is the fundamental reason that the core would not easily support what you suggest (HazeMD has no concept of system + software command line options, it only launches every game as if a separate system; following the same path in MESS would have broken any software loading in home computer or required a lot of rewrites).
the current core structure is also the main reason why we currently only have lists for carts (no floppies and no tapes) and we have some problems with nvram (but these should not be a big deal). if you want to fully rewrite this feel free, the source is available.


You don't need a concept of systems + software commandline options to have a compatible 'xml/dat' output. You simply need to look in the base folder. You can output the extra info, and let ClrMAME use it when Roman adds support. That's the key point I'm making here. It places minimal burden on anybody else but allows expansion when they're ready.

I could have added an extra field 'system' to the GAME macro in HazeMD, output that in the XML, changed the rom loading to additionally look in that folder, and asked Roman to give the option to rebuild using that folder name. It would have worked! Yes HazeMD isn't MESS, Yes MESS has a seperation between systems and software, but for the purpose of a first pass compatible output, it's irrelevant.

I don't know how this 'breaks' computer systems, but you claim they are 'broken' anyway, so it sounds like an issue with MESS. Lists for console systems, using the existing architecture would have been a good base. If it's just a case of folder organization and 'compatible systems' it simply sounds like those could have come in time, once ClrMAME was improved.

Originally Posted By etabeta78

if you prefer, I easily admit my incompetence on the subject: that's why I worked mainly on the lists and not on the core (except for adding some validation code)
however, as a user, I also find definitely simple and intuitive the current implementation and therefore I appreciate a lot the work judge has done on the core side. I don't think to be the only one, but time will tell...


As above, I'm glad this is being done, but seriously, things are being made a lot more difficult than they need to be.
Posted By: Haze Re: Fixed software lists - 05/07/10 01:58 PM
Originally Posted By etabeta78
Originally Posted By Haze
It's also very easy to design a system which doesn't work, if you don't listen to the needs of people who will be using it


haven't you said since day 1 that you do not use MESS? I heard a lot of people being happy with the improvements of the past year in MESS...


and reasons like this are basically the reason it's so annoying to use / learn. things which should have been simple and logical being designed 'against the grain'. Don't get me wrong, some parts of the improvements are good, other parts seem to exist just to make things difficult.

my plan is to pick up MESS once this is done. There is nothing I can do for MAME anymore.
Posted By: etabeta78 Re: Fixed software lists - 05/07/10 02:04 PM
Originally Posted By Haze
Originally Posted By etabeta78
there is the fundamental reason that the core would not easily support what you suggest (HazeMD has no concept of system + software command line options, it only launches every game as if a separate system; following the same path in MESS would have broken any software loading in home computer or required a lot of rewrites).
the current core structure is also the main reason why we currently only have lists for carts (no floppies and no tapes) and we have some problems with nvram (but these should not be a big deal). if you want to fully rewrite this feel free, the source is available.


You don't need a concept of systems + software commandline options to have a compatible 'xml/dat' output. You simply need to look in the base folder. You can output the extra info, and let ClrMAME use it when Roman adds support. That's the key point I'm making here.

I don't know how this 'breaks' computer systems, but you claim they are 'broken' anyway, so it sounds like an issue with MESS. Lists for console systems, using the existing architecture would have been a good base. If it's just a case of folder organization it simply sounds like those could have come in time, once ClrMAME was improved.

Originally Posted By etabeta78

if you prefer, I easily admit my incompetence on the subject: that's why I worked mainly on the lists and not on the core (except for adding some validation code)
however, as a user, I also find definitely simple and intuitive the current implementation and therefore I appreciate a lot the work judge has done on the core side. I don't think to be the only one, but time will tell...


As above, I'm glad this is being done, but seriously, things are being made a lot more difficult than they need to be.


computers are not broken, but they definitely require different devices to be supported to work (say you want to play a 8 disk floppy games which also requires a DOS disk in floppy drive 2, you can simply not load all the disks and DOS into memory... it's not like the system worked and you would break accuracy). hence, yes, there is need of the concept of system+software and comparing this to HazeMD, which only emulates consoles, does not bring us anywhere.

software lists for computers simply require more work in the softlist core before tapes and disks can be added, but the current implementation is capable of such a support (in addition to the good old non-softlist loading), while I don't see your approach to be as flexible. feel free to prove me wrong, but with some code and not only with your strong belief.
Posted By: Haze Re: Fixed software lists - 05/07/10 02:09 PM
Originally Posted By etabeta78

computers are not broken, but they definitely require different devices to be supported to work (say you want to play a 8 disk floppy games which also requires a DOS disk in floppy drive 2, you can simply not load all the disks and DOS into memory... it's not like the system worked and you would break accuracy). hence, yes, there is need of the concept of system+software and comparing this to HazeMD, which only emulates consoles, does not bring us anywhere.

software lists for computers simply require more work in the softlist core before tapes and disks can be added, but the current implementation is capable of such a support (in addition to the good old non-softlist loading), while I don't see your approach to be as flexible. feel free to prove me wrong, but with some code and not only with your strong belief.


I still don't really see how a great deal of this prevents a compatible output list being created, especially not when there are no computer lists yet anyway. The actual software lists can still contain any needed information, the 'compatible mode' lists output don't need to.

If it's a huge problem then limit the compatible lists to consoles only, it's still far easier, and more compatible with what already exists than having to process 40 xmls if I have software for 40 consoles. (and likewise, far easier for existing MAME frontends to use too)

Computers are a special case, they always will be. Consoles can be handled in the output with the existing structures, they should be.

It just seems like MESS is trying to jump 4 steps ahead, in the name of things which aren't done yet, when nobody around is ready to support that, or likely to be in the near future. I think this will hinder progress.

Maybe it's just becamse I'm used to working within already established constraints, with available APIs etc. that I see this as a big deal, but it just seems like common sense to make something which .. works...


Posted By: etabeta78 Re: Fixed software lists - 05/07/10 02:15 PM
but compatible output lists are already created (or better: they will be compatible once the new clrmame version is out).
you simply cannot directly use the all-in-one list in clrmame due to the same-name-different-game problem. nothing is being prevented.

the problems against your ideal approach is that MESS needs devices (for computer to work) and that MESS could not load the right bublbobl if all the roms are in the same folder.
Posted By: etabeta78 Re: Fixed software lists - 05/07/10 02:19 PM
Originally Posted By Haze
It just seems like MESS is trying to jump 4 steps ahead, in the name of things which aren't done yet, when nobody around is ready to support that, or likely to be in the near future. I think this will hinder progress.


actually, me, judge and Micko are interested to push lists towards computers, and I think Curt was interested as well. This makes 4 out of around 10 active devs... not too bad.

probably, with no real life, judge and Micko would have already made most of the work. like things are, I prefer to make console lists more stable and complete before moving to the next task (i.e. computers)

also, as soon as computers are part of the game, recognized software will jump to several thousands of units, making it impossible to be dealt with in a single directory or as separate systems as in MAME
Posted By: Haze Re: Fixed software lists - 05/07/10 02:20 PM
Originally Posted By etabeta78
but compatible output lists are already created (or better: they will be compatible once the new clrmame version is out).
you simply cannot directly use the all-in-one list in clrmame due to the same-name-different-game problem. nothing is being prevented.


Then it would be much more logical to have unique names until the time ClrMAME can support things better! It's not 'ideal' but it works, it works a lot better than the current approach. It can change in the future when ClrMAME is more mature and can handle it. It's all about working within existing limits....

Quote:

the problems against your ideal approach is that MESS needs devices (for computer to work) and that MESS could not load the right bublbobl if all the roms are in the same folder.


This sounds like a problem in MESS then? Likewise, it's caused by computers, which again will need some afterthought.

Why can't MESS identify which files are important? It has a database, you're even being pedantic about forcing device types on the command-line. At worst you end up with an extra 'which file?!' menu in MESS.
Posted By: Haze Re: Fixed software lists - 05/07/10 02:26 PM
Originally Posted By etabeta78
Originally Posted By Haze
It just seems like MESS is trying to jump 4 steps ahead, in the name of things which aren't done yet, when nobody around is ready to support that, or likely to be in the near future. I think this will hinder progress.


actually, me, judge and Micko are interested to push lists towards computers, and I think Curt was interested as well. This makes 4 out of around 10 active devs... not too bad.

probably, with no real life, judge and Micko would have already made most of the work. like things are, I prefer to make console lists more stable and complete before moving to the next task (i.e. computers)

also, as soon as computers are part of the game, recognized software will jump to several thousands of units, making it impossible to be dealt with in a single directory or as separate systems as in MAME


My point was external systems aren't really ready for it. ClrMAME, Frontends etc. You can have all the on-project devs you want working on it, but you're also relying on external developers. The existing frontends will almost certainly never support computers fully, but giving them something compatible for consoles would work much better than the current approach which just breaks everything.

Posted By: R. Belmont Re: Fixed software lists - 05/07/10 02:26 PM
Uhh, if duplicate names are a problem, just prepend a system identifier to every set like MAME does with PC10 and MegaTech/MegaPlay. It can always be removed when ClrMAME is fixed.
Posted By: etabeta78 Re: Fixed software lists - 05/07/10 02:28 PM
Originally Posted By Haze
Originally Posted By etabeta78
but compatible output lists are already created (or better: they will be compatible once the new clrmame version is out).
you simply cannot directly use the all-in-one list in clrmame due to the same-name-different-game problem. nothing is being prevented.


Then it would be much more logical to have unique names until the time ClrMAME can support things better! It's not 'ideal' but it works, it works a lot better than the current approach. It can change in the future when ClrMAME is more mature and can handle it.


so you first would create lists with all different names, then you would rewrite them all when clrmame is capable of supporting them better. i.e. you do the work twice to make easier the use of a rommanager? it's nonsense.

I think it is more productive for MESS to create lists in the best possible way, and simply say to the users: well, at the moment rommanagers still cannot do all the work for you. hence, if you really want to use our lists be patient and do some additional work. if you don't want, please keep using for a couple of versions the old generic loading commands and only use the -romident function of the lists when you want to report a bug so that we are sure you are not using a bad dump.
Posted By: etabeta78 Re: Fixed software lists - 05/07/10 02:30 PM
Originally Posted By R. Belmont
Uhh, if duplicate names are a problem, just prepend a system identifier to every set like MAME does with PC10 and MegaTech/MegaPlay. It can always be removed when ClrMAME is fixed.


are you really suggesting people to keep 10 thousands of roms of 15 different systems in the same directory?
Posted By: Haze Re: Fixed software lists - 05/07/10 02:32 PM
Originally Posted By etabeta78

so you first would create lists with all different names, then you would rewrite them all when clrmame is capable of supporting them better. i.e. you do the work twice to make easier the use of a rommanager? it's nonsense.


Yes, I would do that. The external tools require that at the moment. It's less nonsense than making me load 40 lists for 40 systems!

you only have to rename sets which conflict across systems, it's not an insane amount of work, sets get renamed in MAME all the time. MESS can prompt you of the set names if you're not sure sometimes, it works with the existing systems, it can be changed in the future when external support is better, everybody wins!
Posted By: Haze Re: Fixed software lists - 05/07/10 02:33 PM
Originally Posted By etabeta78
Originally Posted By R. Belmont
Uhh, if duplicate names are a problem, just prepend a system identifier to every set like MAME does with PC10 and MegaTech/MegaPlay. It can always be removed when ClrMAME is fixed.


are you really suggesting people to keep 10 thousands of roms of 15 different systems in the same directory?


I'm proposing they have that option, and that when Roman improves ClrMAME they also have the option to move them to the system folders that were handily output in the xml, and everything 'just works'. Choice. (Just like right now I can put a CHD in a Zip if I really want, it's pointless in many cases, but I can do it, it works, there is minimal cost in supporting that, and it almost makes sense for sub 200meg CHDs)

Posted By: R. Belmont Re: Fixed software lists - 05/07/10 02:34 PM
Originally Posted By etabeta78
are you really suggesting people to keep 10 thousands of roms of 15 different systems in the same directory?


No, I'm suggesting the following:

1) If users want to do that, they should be able to (even if in the final form that means that bublbobl.zip contains versions for 20 different systems)

2) You stated that ClrMAME and the core's inability to handle duplicate names was a blocking issue. I offered an easy, tested solution that could be applied and removed by automated tools.
Posted By: etabeta78 Re: Fixed software lists - 05/07/10 02:35 PM
Originally Posted By Haze
My point was external systems aren't really ready for it. ClrMAME, Frontends etc. You can have all the on-project devs you want working on it, but you're also relying on external developers. The existing frontends will almost certainly never support computers fully, but giving them something compatible for consoles would work much better than the current approach which just breaks everything.


frontends which supports MESS already support also computers: adding -flop or -cass followed by a filename is not more difficult than parsing mame.ini. the problem is that supporting softlist will require some work no matter which way you implement it: frontends which supported MESS will have to add the handling of the software list; frontends which never supported MESS will have to add the handling of system+software commands (because you will never launch "mess bublbobl" as in MAME, you will always need to specify the system to use...)

your argument does not work well...
Posted By: Haze Re: Fixed software lists - 05/07/10 02:38 PM
Originally Posted By etabeta78
Originally Posted By Haze
My point was external systems aren't really ready for it. ClrMAME, Frontends etc. You can have all the on-project devs you want working on it, but you're also relying on external developers. The existing frontends will almost certainly never support computers fully, but giving them something compatible for consoles would work much better than the current approach which just breaks everything.


frontends which supports MESS already support also computers: adding -flop or -cass followed by a filename is not more difficult than parsing mame.ini. the problem is that supporting softlist will require some work no matter which way you implement it: frontends which supported MESS will have to add the handling of the software list; frontends which never supported MESS will have to add the handling of system+software commands (because you will never launch "mess bublbobl" as in MAME, you will always need to specify the system to use...)

your argument does not work well... given that


True, they would need to add support for the extra commandline parameter, but it's significantly less work. They can extract the system name from the extra XML field, chuck it at the command-line and be done with it. Minimal extra coding needed. ClrMAME would 'just work', and could be improved to support multiple paths later.
Posted By: R. Belmont Re: Fixed software lists - 05/07/10 02:43 PM
Yup. ClrMAME might even eventually add a MESS-specific "supermerge" option that flattens like-named software across systems into a single zip.
Posted By: etabeta78 Re: Fixed software lists - 05/07/10 02:44 PM
okay, since I'm fine with the current code, and I'm neither capable of adding the kind of support you suggest nor to make frontend authors' life easier, I will simply sit and wait for cool things to happen.
when core improvements are in, I will check if the lists I work on are still fully compatible, and I will fix any possible regression.
Posted By: etabeta78 Re: Fixed software lists - 05/07/10 02:50 PM
Originally Posted By R. Belmont
1) If users want to do that, they should be able to (even if in the final form that means that bublbobl.zip contains versions for 20 different systems)


except the same file could probably not work when loaded with the old command line switches (since it would not load by crc).

overall, I would not suggest this to users, but I'm not the average user.
Posted By: Haze Re: Fixed software lists - 05/07/10 02:52 PM
Originally Posted By etabeta78
okay, since I'm fine with the current code, and I'm neither capable of adding the kind of support you suggest nor to make frontend authors' life easier, I will simply sit and wait for cool things to happen.
when core improvements are in, I will check if the lists I work on are still fully compatible, and I will fix any possible regression.


A good start would be simply allowing MESS to look in the base ROMs folder for both software and bios roms after checking the system specific folders. I honestly doubt that would be much code, and would be much quicker for somebody who already has SVN access to change.
Posted By: Haze Re: Fixed software lists - 05/07/10 02:55 PM
Originally Posted By etabeta78
Originally Posted By R. Belmont
1) If users want to do that, they should be able to (even if in the final form that means that bublbobl.zip contains versions for 20 different systems)


except the same file could probably not work when loaded with the old command line switches (since it would not load by crc).

overall, I would not suggest this to users, but I'm not the average user.


yes, but the point of the new system is that you've created 2 different ways to use MESS anyway.

1) The database, in addition to everything else it uses and supports real sets (NES Prg+Char dumps etc.) that don't stand a chance of working with the generic loader anyway. This is good for documentation, good for bug reports and such. Stuff in the database will be in the best known format for each piece of software (most complete / accurate formats etc.)

2) the legacy system. This is useful if you're doing your own development etc. The CRCs etc. don't matter, it loads what you give it the best it can. It loads legacy formats, snapshots, you name it. It doesn't document anything, it can guess NVRams from headers, hacks, or whatever else, it doesn't check anything, it gives no guarantees that anything will work, but if you need it, it's there.

Chances are if you're using system #2 then you're not going to be using the database and ClrMAME anyway, nor merging everything into one zip. You're probably not going to care, because you have very specific needs in the first place. By supporting 'real' dumps in the database you're already making it impossible to load those sets with method 2. I don't see that as an issue, they're not legacy formats and not designed to be loaded with that method.

Posted By: R. Belmont Re: Fixed software lists - 05/07/10 03:01 PM
Aye. You'd play listed media via the list system and everything else via the current options. I don't see why you'd want to -cart something that the list system could already load.
Posted By: etabeta78 Re: Fixed software lists - 05/07/10 03:55 PM
Originally Posted By Haze
A good start would be simply allowing MESS to look in the base ROMs folder for both software and bios roms after checking the system specific folders. I honestly doubt that would be much code, and would be much quicker for somebody who already has SVN access to change.


This might be done, I think (like MAME allows for CHDs being in the same directory as the roms [1]). unfortunately, I have no idea where the various pieces of image code are, after the conversion of image.c to be a device, so I can't help...


[1] in fact, I think this is inconsistent as well, but iirc either OG or smf use them like that...
Posted By: Haze Re: Fixed software lists - 05/07/10 04:42 PM
Originally Posted By etabeta78

[1] in fact, I think this is inconsistent as well, but iirc either OG or smf use them like that...


It saves time if you're quickly adding / checking something, just like as I said, you can zip them up too. I have them on my portable drive like that.

Again, it's just giving people the choice, and preventing people doing it is nothing more than an artificial limitation, and why I hate Apple so much for example.
Posted By: etabeta78 Re: Fixed software lists - 05/07/10 05:09 PM
further reading the code, I now fear it is not easily doable: I think currently MESS creates at loading new rom_regions corresponding to the dataarea in the xml list. once this is done, the emulator can only look for the data where it would look for the correspondent bios, i.e. either inside a folder/zipfile named as the system or in a folder/zipfile with the name of the parent.

I see no easy way to change this without heavily change the way bioses (and MAME roms) are handled, except by having to duplicate romload code... or, if a way exists, is by far beyond my skills.
Posted By: Haze Re: Fixed software lists - 05/07/10 05:14 PM
Originally Posted By etabeta78
further reading the code, I now fear it is not easily doable: I think currently MESS creates at loading new rom_regions corresponding to the dataarea in the xml list. once this is done, the emulator can only look for the data where it would look for the correspondent bios, i.e. either inside a folder/zipfile named as the system or in a folder/zipfile with the name of the parent.

I see no easy way to change this without heavily change the way bioses (and MAME roms) are handled, except by having to duplicate romload code... or, if a way exists, is by far beyond my skills.


It sounds like the original code has been badly designed then. It should build on and expand MAME, which would make this trivial, it sounds like it's replaced MAME with it's own weird systems. Kinda what I've been getting at with the whole subject.
Posted By: etabeta78 Re: Fixed software lists - 05/07/10 05:20 PM
from what I see it's the opposite: instead of adding a completely additional infrastructure, it uses the MAME code everywhere (relying on emu/romload.c to do all the work, rather than duplicating its functionalities).

if you can find a better design, with working code, I'll be happy to extensively test it.
Posted By: Haze Re: Fixed software lists - 05/07/10 05:24 PM
Originally Posted By etabeta78
from what I see it's the opposite: instead of adding a completely additional infrastructure, it uses the MAME code everywhere (relying on emu/romload.c to do all the work, rather than duplicating its functionalities).

if you can find a better design, with working code, I'll be happy to extensively test it.


Well, I should probably look at the code a bit then, it just seems bizzare that it would lose any context of what the base path was if it was well designed.

While in a perfectly organized system you'd have bios roms in bios, and system/game roms in their individual folders this is a far from ideal situation if you're just trying to rapidly check something away from your main system, where it's far easier to just dump a bunch of correctly named files in a rom folder and know they'll work. Only looking in paths (subfolders) you didn't explicitly specify is just irritating in such a case.

Posted By: etabeta78 Re: Fixed software lists - 05/10/10 11:34 AM
Originally Posted By Haze
this is a far from ideal situation if you're just trying to rapidly check something away from your main system, where it's far easier to just dump a bunch of correctly named files in a rom folder and know they'll work.


after some more investigations, I still don't see where to fix this. for the time being, I suggest the following workaround: to use softlists when you have files in the correct folders and to fall back to generic loading (i.e. with full path) when you want rapid tests, with -romident to be a good resource when you want to check if the rom you use is a correct one.

even if it's not ideal, it can work until a better fix is found.
Posted By: ht1848 Re: Fixed software lists - 05/10/10 08:19 PM
Thanks for all your effort on this MESS enhancement etabeta. Much appreciated.
Posted By: mizapf Re: Fixed software lists - 05/11/10 01:05 PM
I have not thoroughly followed the discussion about the software lists, so could anyone sum up in two/three sentences what it is about or how far things have come?

As far as I understood I can put the roms of an application program or game in the rom folder and start up the system as "mess <system> <software>", and the specific information to mount the media are retrieved from the emulator where they have been compiled into.

So this targets those systems that have a power-on startup of application programs (i.e. automatically start the contents of a plugged-in cartridge) and where the user wishes to run just this software for this session?

But we still retain the possibility to explicitly mount and swap cartridges/disks/harddisks, don't we? I'm just asking so that there won't be surprises later. smile I had a look at the wikis, but I did not found more explanations beyond that what is scattered in this thread.

Michael
Posted By: etabeta78 Re: Fixed software lists - 05/11/10 02:23 PM
I'm working on a post for my blogs which will explain a bit more the work of the past month about this new feature. However, I will summarize a bit what this thread is about, because you have got most things wrong wink

1. (which I think is what you mostly are interested) explicitly mount/unmount devices will never be removed. nor multicart and rpk support has been modified. current svn (and next 0.138) has simply an additional option for users which was not available earlier.

2. (back to the new option) to document which original software was available for (some of) the emulated systems we have added lists of software in xml format to the hash/ directory (containing e.g. shortnames like MAME and checksums, check one of the xml files if you want more details). Notice that the info is not compiled into the executable, but it is a separate thing which MESS scans when requested. Also, we mainly aim to actual releases (both official and pirate) but not to homebrew programs nor to tech demos, atm.

3. to launch MESS with the software recognized in the list, you put the software inside a roms/list_name/ folder and you type

mess system -cart shortname

multiple files to be loaded at multiple addresses are supported.
also, only commandline works at the moment, but newui and MESSUI support will be added eventually.

4. if you have some spare .bin file in C:\my\spare\roms\ you have forgotten which system is for, you can try to use

mess -romident C:\my\spare\roms\

to verify if the file is in one of the xml lists


5. all the above currently only applies to the cartslot device, but the code has been thought in such a way that floppies and tapes lists will follow

Finally,

Quote:
So this targets those systems that have a power-on startup of application programs (i.e. automatically start the contents of a plugged-in cartridge) and where the user wishes to run just this software for this session?


no, this possibly aims all the systems in MESS and the goal is to provide documentation of the available software (not always trivial for some obscure systems like e.g. supracan, scv, gmaster) and of which dumps are good (which becomes quite important when dealing with mainstream consoles like e.g. nes, snes, sms and megadriv whose available dumps are not always verified as good)

also notice that, while this new feature partially overlaps the multicart feature (in the sense that both of them offers a solution to the multi-file loading), they are not really conflicting: e.g. I plan to add rpk support to NES after the softlist is ready, to handle demos, hacks and translations which do not fit in the xml list...
Posted By: mizapf Re: Fixed software lists - 05/11/10 06:05 PM
Thanks, etabeta! So that clears up a bit. But this also shows that the DevWikis need to be updated better sooner than later, and not only for the software lists but also for the new features of the devices.

...

Actually, my thoughts go a bit further than just concerns about using the software lists. I'm currently thinking about a way to enhance the extensibility.

For the TI there is a collection of expansion cards like different disk controllers, SCSI controllers, RAM expansions, serial interfaces etc. Currently, some of those cards are hardwired into the emulation, selectable by dip switches. The ROMs that are required for those cards are put into the system zip file as optional roms. But this also means that every newly created card emulation will require to update the system zip file. If you don't have the card ROMs you'll keep getting warnings about missing optional ROMs. Indeed, this concept does not seem right to me. The roms are not *optional* for the console, but they are parts of extensions.

So my idea is: what about handling them like cartridges? This would require to introduce another switch like "-ext" (to avoid mixing up with "-cart") and another instance of the multicart system. Would be pretty easy to implement. These are still early thoughts; nothing has been done by me in that direction.

The question right now is: Should I design the extension cards as RPKs, or is that better handled by the software lists (maybe not now but in a later release)?

Michael
Posted By: etabeta78 Re: Fixed software lists - 05/11/10 06:16 PM
software list documentation will appear soon on my blog and, as soon as it is not going to change too much, in the User Manual.

About the remaining part of your post, I think you would better discuss it with Aaron: optional roms implementation probably needs to be fixed (given that optional roms are not currently optional at all!); and before adding "extensions", we might want to see if devices can be updated to be mounted/unmounted and to internally support things like dipswitches.

Not that I dislike your proposal, but probably parts of it can be merged into Aaron's current plans for devices (which I fear only Aaron knows). Hence, I would not suggest you to start *any* design without prior discussion with Aaron (even if he seems to be really busy these days)
Posted By: etabeta78 Re: Fixed software lists - 05/11/10 07:05 PM
Originally Posted By etabeta78
software list documentation will appear soon on my blog...


all you wanted to know about softlists but were afraid to ask:

http://mamedev.emulab.it/etabeta/?p=188


Originally Posted By Haze
[snip]Sounds like a complete nightmare. I won't be using it that way, I have 40 odd games for 40 odd systems, there is _no way_ you're going to get me to create and load 40 xml files for each new update / release. It's a non-starter, dead in the water, utterly useless. You don't appear to be designing this around the needs of people who are going to use it.

It *needs* to be one-stop (or at least have that option). [snip]


It turned out that the one-stop procedure is already available thanks to batchrun capabilities of clrmame (see the instructions for multiple lists handling in http://mamedev.emulab.it/etabeta/?p=188#Q12 ) and it will only require Roman to release the next version of its rommanager which will add support for our xml lists (both the ones generated with the -listsoftware command and the ones bundled with MESS in the hash/ directory). Of course, on the long term it would be nice to better integrate clrmame with our lists, but I think that on the short term Roman's solution is a good workaround compared to setting up more than 40 lists manually...
Posted By: netol Re: Fixed software lists - 05/11/10 11:18 PM
Thank you for that entry in your blog etabeta. Everything is more clear now.
Posted By: franciscohs Re: Fixed software lists - 05/12/10 02:35 AM
etabeta, after reading your post in the blog, it seems that in the end there is no mechanism to march the softlists to the mess version that is being used.

Is this still a planned feature or was dropped?, I think that as it's now, and lists being moving targets, in the sense that I guess they will be updated very often, as the mame ones, wouldn't there be a big problem at bug reporting? (which you list as the first benefit)

I could report a certain game not working on the latest mess version but have an old softlist, etc... softlists don't even have any version numbers as far as I see, so you can't even identify what list is the user is using.

Or I'm missing something?
Posted By: Robbbert Re: Fixed software lists - 05/12/10 03:26 AM
Perhaps the SVN revision number could be included in the header (eg r7914), but only for doco purposes, since there is no way for MESS to determine its own revision.
Posted By: etabeta78 Re: Fixed software lists - 05/12/10 05:11 AM
Originally Posted By franciscohs
etabeta, after reading your post in the blog, it seems that in the end there is no mechanism to march the softlists to the mess version that is being used.


you are correct. at the moment there is no way to determine which list version you are using. I think we will have to add some sort of version number to the header (and to print it out when you use the -listsoftware command)

thanks for pointing out the problem which could give with bug reporting, though. I hadn't thought to that side of the question. something will be done. smile
Posted By: Anna Wu Re: Fixed software lists - 05/12/10 05:57 AM
Thanks eta, your software list documentation is very helpful. smile
Posted By: R. Belmont Re: Fixed software lists - 05/12/10 06:13 AM
The idea is that MESS will not load lists at random - it must know exactly what lists it wants to load (which AFAIK is already true) and what the correct SHA1 for each list is (which AFAIK isn't). When you update the list in SVN, you would also update the SHA1 in the source. These hashes should also be included in -list* output. And obviously we should not enable any form of list support without a mechanism like this. Version numbers would not actually provide any form of developer protection.
Posted By: etabeta78 Re: Fixed software lists - 05/12/10 07:26 AM
well, given that lists are currently only accessible from command line (i.e. they probably won't get great exposure anytime soon) and that some of the larger lists still need a lot of work before being complete (they mostly miss years, publishers, supported status...), I prefer working on the lists themselves than on implement list checksum verification.

but if you throw in some validation code, I will readily update lists to comply.
Posted By: incog Re: Fixed software lists - 05/13/10 04:54 PM
Hi, I'm wanting to rip my collection of ps1 games to CHD so I can start on a fixed list, but I have no idea if the CHD format is now final and to start now or to wait. Anybody have any idea?
Posted By: etabeta78 Re: Fixed software lists - 05/13/10 04:58 PM
IIRC, Arbee stated some major update was in progress for CHDCD (when suggesting Kale to wait for a Dreamcast list)...
Posted By: R. Belmont Re: Fixed software lists - 05/13/10 06:29 PM
Ideally fixed lists for CD media will be sourced from redump.org or similar "S&M-style" rippers with offset correction. The next version of CHDMAN (which I'm holding back to avoid potential breakage in the 0.138 release) will support bin/cue for input and output (bin/toc will still work as well) and will preserve pregap information. This allows 100% preservation of the redump.org Saturn collection in the CHD format, and if I can get my hands on more redump.org rips I will extend CHDMAN again if necessary.

Contrary to what I originally thought the changes will be back-compatible, but discs with audio tracks will need to be re-ripped so accurate pregap information is included. (Single-track data-only discs like CPS3, 573 Digital, and some PS1 games can remain as-is).
Posted By: incog Re: Fixed software lists - 05/13/10 07:30 PM
Aye aye sir!

I'll rip to redump.org standards in future.
Posted By: Haze Re: Fixed software lists - 05/13/10 07:43 PM
Originally Posted By R. Belmont
Ideally fixed lists for CD media will be sourced from redump.org or similar "S&M-style" rippers with offset correction. The next version of CHDMAN (which I'm holding back to avoid potential breakage in the 0.138 release) will support bin/cue for input and output (bin/toc will still work as well) and will preserve pregap information. This allows 100% preservation of the redump.org Saturn collection in the CHD format, and if I can get my hands on more redump.org rips I will extend CHDMAN again if necessary.

Contrary to what I originally thought the changes will be back-compatible, but discs with audio tracks will need to be re-ripped so accurate pregap information is included. (Single-track data-only discs like CPS3, 573 Digital, and some PS1 games can remain as-is).


Have you discussed the CD-I Ready stuff and how it will work with this with CDIFan or so? From what I can see the Data is in an audio-pregap, and gets recognized as audio, because it's the same 'track' as the first audio track (which is a real audio track..) It sounds horribly ugly to support properly. AFAIK Redump/Tosec have no way to deal with these yet, and the CDI driver very much needs to work with them properly for 'The Apprentice' (if we're to avoid hacking the cuesheets)

Posted By: R. Belmont Re: Fixed software lists - 05/13/10 07:56 PM
The situation remains the same with those 2 discs: nobody at redump or TOSEC has any idea how to handle that, a lot of commercial ripping software won't even touch it, and since those guys are all a lot smarter about CDs than I am I'm not going to attempt to touch it. The revised format has a provision for data blocks in the pregap so we don't have to change the format again but at this time I have no answers on how that should be implemented.

In the meantime I understand we have a working CHD of the Apprentice even without this support so perhaps everyone is over-complicating it.
Posted By: Haze Re: Fixed software lists - 05/13/10 08:03 PM
Originally Posted By R. Belmont
The situation remains the same with those 2 discs: nobody at redump or TOSEC has any idea how to handle that, a lot of commercial ripping software won't even touch it, and since those guys are all a lot smarter about CDs than I am I'm not going to attempt to touch it. The revised format has a provision for data blocks in the pregap so we don't have to change the format again but at this time I have no answers on how that should be implemented.

In the meantime I understand we have a working CHD of the Apprentice even without this support so perhaps everyone is over-complicating it.


well it 'works' because the pregap has been converted into a data track and all the audio tracks shuffled up by one, so I'm surprised it plays the right music at all. Of course, that might just be how a real CDI sees the discs.

It also hangs with the CDDA emulation, but that's MGs problem afaik, it floods the CPU with too many interrupts during CD playback from what I can see and thus gets stuck with only enough cpu time to play the music, not service other interrupt requests.

That's why I said it might be worth talking to CDiFan about, he seems to know more about the format / discs / what the CDi sees than most people.

I think for the sake of sanity dumps which haven't been dumped with approved methods will have to be supported and marked as 'BAD DUMP' until they're dumped properly, for many systems there are games out there which simply aren't going to get dumped properly anytime soon; these being a prime example, as well as prototype discs etc. (some only exist as ISO/MP3, which can be converted to a bin/cue/chd, but urgh, why did people do that.. they're still important for testing drivers until they get better dumps tho)
Posted By: Just Desserts Re: Fixed software lists - 05/13/10 08:45 PM
Originally Posted By Haze
It also hangs with the CDDA emulation, but that's MGs problem afaik, it floods the CPU with too many interrupts during CD playback from what I can see and thus gets stuck with only enough cpu time to play the music, not service other interrupt requests.


???
Posted By: Haze Re: Fixed software lists - 05/13/10 09:30 PM
Originally Posted By Just Desserts
Originally Posted By Haze
It also hangs with the CDDA emulation, but that's MGs problem afaik, it floods the CPU with too many interrupts during CD playback from what I can see and thus gets stuck with only enough cpu time to play the music, not service other interrupt requests.


???


The Apprentice hasn't worked properly since you added CDDA. I told you this at the time, it takes 3-4 attempts to get ingame, and if you manage that it usually hangs once you finish a level. (and yes, the music is playing properly, it just hangs, doing nothing except playing the music)

Unless you have a different CHD, but the one I was using behaves like that, and works fine if you remove the CDDA related stuff. I'd know if there were software-lists so everybody knew exactly what other people were using...

Posted By: Just Desserts Re: Fixed software lists - 05/13/10 09:57 PM
Oh, okay - that was totally not what I thought you meant. I thought you meant it was hanging while playing audio CDs, and I was like, "Haha, what, I've never even added support for audio CDs." smile
Posted By: mahlemiut Re: Fixed software lists - 05/14/10 06:30 AM
Originally Posted By R. Belmont
Ideally fixed lists for CD media will be sourced from redump.org or similar "S&M-style" rippers with offset correction. The next version of CHDMAN (which I'm holding back to avoid potential breakage in the 0.138 release) will support bin/cue for input and output (bin/toc will still work as well) and will preserve pregap information.

This will be awesome for FM Towns also, as its BIOS expects the first track to be at 02:00 (the CD controller uses MSF format for disc locations), and should allow be to get CD-DA working correct for games developed by Ving (because they like to hard-code the audio track locations). I can get rid of the hack that I use to read the data track, hopefully. grin
Posted By: Stiletto Re: Fixed software lists - 05/14/10 03:27 PM
Supposedly this is how to rip a CD-i, but I haven't checked to see if it mangles the tracks like Haze says: http://www.cdinteractive.co.uk/cic/images/cd-i_burning_guide_v2.pdf
Posted By: R. Belmont Re: Fixed software lists - 05/14/10 03:53 PM
That guide is for making pirate backups. It's useless for what Haze is discussing, unfortunately.
Posted By: Stiletto Re: Fixed software lists - 05/15/10 11:43 PM
Okay, thanks...
Posted By: etabeta78 Re: Fixed software lists - 05/16/10 10:09 AM
FWIW, MESS User Manual has been updated with preliminary instructions for software lists: minor (but important) improvement compared to my blog entry is that I found out lists are handled by image.c and therefore are not limited to cartslot, even if only carts have been listed so far...


I plan to eventually integrate the instructions about lists with the ones about generic loading (probably ending up with some rewrite of the Manual here and there), but for the time being I hope the instructions are good enough for brave users which want to experiment with this.
Posted By: Robbbert Re: Fixed software lists - 05/17/10 07:36 AM
I found an issue with this, in Jaguar. I used the list to load a cart (from the command line obviously). Then I quit. Then I started messui, selected Jaguar, chose a quickload, and received an error. It wanted to load the cart and it was somehow mandatory. The only thing I could do was exit messui, go to command line and unload the cart. Not only that, it screwed around with jaguar.ini, restoring the rom path back to the default, thereby preventing the load of anything via the software list. Most annoying.
Posted By: etabeta78 Re: Fixed software lists - 05/17/10 07:58 AM
it seems a problem in ini/cfg savings or in MESSUI (or a combination of both). do you happen to know which code is responsible for saving the path of the last mounted image (I don't have here my eeepc so I cannot experiment with the windows layer)?
Posted By: Robbbert Re: Fixed software lists - 05/17/10 09:05 AM
I did know once, but I've since forgotten. I'll take a look.

At one time I did also make it that MESSUI saved the complete ini file instead of the differences, however something must have changed. Another thing to look at.

I'd like to know why a softwarelist cart load is mandatory though, that needs to be gotten rid of.
Posted By: etabeta78 Re: Fixed software lists - 05/17/10 09:16 AM
it's not mandatory indeed from command line (I keep alternating list and old-style loading for testing purposes). but I guess the UI somehow misinterprets the ini options that get saved and thinks you have to reload from the list...
Posted By: Robbbert Re: Fixed software lists - 05/17/10 12:26 PM
Checked newui and it all works fine there, so it is something messui-specific. I'll look now, was just fixing something else..
Posted By: etabeta78 Re: Fixed software lists - 05/17/10 12:44 PM
on the subject of your cassette changes, can I suggest that 60 seconds seem a bit too much? while I agree 1 second does not help, 1 minutes seems overkill... especially in view of the fact no original tape drive was able to skip 60 seconds at once (to further speed up the emulated tape, Insert makes wonder, but directly skipping so many seconds in the core seems not a good idea) wink
Posted By: JoJo Re: Fixed software lists - 05/17/10 12:59 PM
Hmmm... I'd like to have FFwd/Rew implemented, at least in the oldui, with a sort of auto-repeat i.e. as long as you keep ENTER pressed the tape counter advances or decreases, and when you release the key it stops.
Posted By: Robbbert Re: Fixed software lists - 05/17/10 01:13 PM
In newui, there isn't really much choice, unless you want to keep calling up the menu and clicking rewind. It was faster to unmount the tape then mount it again.

Oldui is another matter, it is separate, and 1 sec is quite viable there, just as long as enter key can be made to autorepeat. If you can fix the key, then i'd certainly agree 1 sec is fine.
Posted By: Robbbert Re: Fixed software lists - 05/17/10 02:30 PM
Yes, messui certainly has issues, the printf i added shows it writing files that it shouldn't touch. Need to look more tomorrow.
Posted By: incog Re: Fixed software lists - 05/17/10 05:19 PM
new clrmamepro
3.133b
added: support for mess software lists (either as hash/*.xml file or via -listsoftware output)

Woo! Now it just needs to be hooked into the UI.
Posted By: etabeta78 Re: Fixed software lists - 05/17/10 05:30 PM
I've spent the past weekend rebuilding sets and checking that all sizes/crc/sha1 were correct (and that all included sets were available).

the support is pretty good, as long as you first batch create the directories and run the first scan/rebuild as explained at my blog and in the MESS User Manual (otherwise you have to manually create and setup a lot of folders) smile
Posted By: etabeta78 Re: Fixed software lists - 05/17/10 08:11 PM
Originally Posted By JoJo
Hmmm... I'd like to have FFwd/Rew implemented, at least in the oldui, with a sort of auto-repeat i.e. as long as you keep ENTER pressed the tape counter advances or decreases, and when you release the key it stops.


this requires quite a lot of work: MAME only allows for Left&Right to be repeated (not Enter), and the Enter behavior has to be treated differently depending you are acting on REW/FFW or Play/Stop (where you don't want Enter repeats to be allowed)

too much to be done before 0.138 (with the risk of breaking a lot of things in the attempt)
Posted By: Duke Re: Fixed software lists - 05/17/10 10:04 PM
Why not use left for rewind and right for fast forward then?
Posted By: Robbbert Re: Fixed software lists - 05/18/10 08:55 AM
Etabeta: I've fixed the messui issue. Also, should the "mandatory" issue strike again (perhaps due to software being moved), you can go into the device view and unload the software there.
Posted By: ht1848 Re: Fixed software lists - 05/18/10 08:46 PM
I got the new clrmame last night and was rockin' with the MESS software list. It took a little bit to setup each folder but overall it worked pretty well. I didn't find any show stoppers.
Posted By: etabeta78 Re: Fixed software lists - 05/18/10 10:15 PM
ehm, the folder setup is automatic if you select an empty folder and you let clrmame to create the folder structure as described in the MESS User Manual.
Posted By: F1ReB4LL Re: Fixed software lists - 05/22/10 07:53 PM
Originally Posted By R. Belmont
The situation remains the same with those 2 discs: nobody at redump or TOSEC has any idea how to handle that


Because noone wants to donate us such discs or to make some test dumps on specific drives to let us examine the results.

Originally Posted By mahlemiut
This will be awesome for FM Towns also, as its BIOS expects the first track to be at 02:00 (the CD controller uses MSF format for disc locations


Hmm? AMSF 00:02:00 equals LBA 0, every disc image starts from that sector, sectors 00:00:00 to 00:01:74 (LBA -150 to -1) contain pregap which is usually omitted, while dumping.
Posted By: incog Re: Fixed software lists - 05/22/10 08:32 PM
What kind of discs do you need lending? I have many Pippin, Playdia discs, hundreds of PS1/PS2 discs, Mega-CD, Saturn, Dreamcast but only one CD-I game.

As my only working optical drive is a USB2 external drive (apparently this is bad for archival), I would be happy to come to some arrangement where I send spools of original discs to a dumper and have them sent back.

Should I start compiling lists of what I have?
Posted By: F1ReB4LL Re: Fixed software lists - 05/22/10 08:49 PM
Originally Posted By incog
What kind of discs do you need lending? I have many Pippin, Playdia discs, hundreds of PS1/PS2 discs, Mega-CD, Saturn, Dreamcast but only one CD-I game.


CD-i Ready format discs were mentioned in that post, we don't have any skilled dumpers with them yet. As for the other discs, you should be able to dump most of them by yourself. You can deal with Pippin and Playdia already, PS1/PS2 ones shouldn't be too hard, Mega-CD and Saturn ones require a drive capable to overread into lead-out to dump the last audio track completely, DC ones are the toughest, especially if you want to dump them properly on a PC drive instead of using a console.
Posted By: mahlemiut Re: Fixed software lists - 05/22/10 10:39 PM
Originally Posted By F1ReB4LL
Hmm? AMSF 00:02:00 equals LBA 0, every disc image starts from that sector, sectors 00:00:00 to 00:01:74 (LBA -150 to -1) contain pregap which is usually omitted, while dumping.

Nope, the FM Towns specifically asks to load the data track from 00:02:00. The CD-ROM controller may well adjust that, but since it's not dumped, who knows what it does exactly.
Posted By: R. Belmont Re: Fixed software lists - 05/22/10 10:46 PM
Wouldn't dumping DC discs on a PC instead of the real Sega/Panasonic hardware cause a different track offset compared to the real life hardware?

Barry: as he said, 00:02:00 is sector 0 of track 1 when track 1 is data. The Saturn does the same thing. If anything tries to read before that is where it would get interesting (nothing does on Saturn).
Posted By: F1ReB4LL Re: Fixed software lists - 05/23/10 01:26 AM
Originally Posted By R. Belmont
Wouldn't dumping DC discs on a PC instead of the real Sega/Panasonic hardware cause a different track offset compared to the real life hardware?


All the offsets should be compensated while dumping, same for the CD-titles. Those dumps from the real h/w are shifted, yes (and no gaps, IIRC). As I've said already, you can emulate the offset in the emulator itself, but it shouldn't be in the dump.
Posted By: R. Belmont Re: Fixed software lists - 05/23/10 02:27 AM
Yes, but if the correct offsets from real h/w are in the dump the emulator doesn't have to do anything to be perfectly accurate smile
Posted By: F1ReB4LL Re: Fixed software lists - 05/23/10 10:06 AM
And if the decrypted CPS-2/Neo Geo/whatever games are what's inside the PCB after the decrypting process - why to bother with emulating it? Leave everything decryted, the gameplay will be exactly the same. Remember: all these offset issues happen only because all the CD-media dumps are incomplete. You can either try to dump the entire user area from the first byte upto the last or you can just try to dump "some data" using your PC drives, console drives or whatever else. We dump CD and GD medias themselves, not the random console outputs, I don't know what MESS is going to dump, it's on your own.
Posted By: judge Re: Fixed software lists - 05/23/10 10:10 AM
Originally Posted By F1ReB4LL

Originally Posted By mahlemiut
This will be awesome for FM Towns also, as its BIOS expects the first track to be at 02:00 (the CD controller uses MSF format for disc locations


Hmm? AMSF 00:02:00 equals LBA 0, every disc image starts from that sector, sectors 00:00:00 to 00:01:74 (LBA -150 to -1) contain pregap which is usually omitted, while dumping.


Too bad that those sectors never get dumped: the TOC lives in there. Having those sectors would remove the need of having toc information supplied with a dumped cd.
Posted By: F1ReB4LL Re: Fixed software lists - 05/23/10 10:18 AM
TOC lives waay before those sectors (somewhere around LBA -10000)and each drive has a command to read the TOC in RAW form, so it's not really a problem. TOC is located in the subchannels area, btw, so it's not an easy task to read those "as is" even on the drives that are able to.
Posted By: etabeta78 Re: Fixed software lists - 05/26/10 07:18 AM
Back to the main list topic, I just noticed that there is no code to load roms from the parent set.

This would be useful for AES and (especially) for NES whose cart revisions often differ only for PRG or CHR and using split sets like MAME would help

I guess the code should go into image_load_internal in image.c, but I have been unable to find how to implement it.

any suggestions?
Posted By: Firewave Re: Fixed software lists - 05/26/10 01:45 PM
I think the logic might be identical to the code in romload.c, maybe at some point it might even be possible to use the same code.
Posted By: judge Re: Fixed software lists - 05/26/10 05:56 PM
The logic is indeed the same, I just havent gotten around to make the code in romload.c a bit less dependant on the machine driver structure to make it reusable for both machine driver rom loading and software list rom loading.
Posted By: Haze Re: Fixed software lists - 05/27/10 01:52 PM
any chance you can make it look for everything in the rom folder as a 'last place to look' fallback while you're at it? eta mentioned he didn't know how to handle that, and it can't be that badly coded that it's impossible I'm sure.

I'd still rather have the choice to simply dump files in the roms folder if I want to rather than messing around putting some in bios folders, creating sub-folders etc. etc.
Posted By: etabeta78 Re: Fixed software lists - 05/27/10 02:08 PM
as a temporary workaround you can either run

mess system -cart C:\place_where_I_threw_my_file\gamename.zip

or name the folder you have dropped your files in as "system" and launch

mess system -cart gamename -rp C:\folder_above_the_folder_with_roms\

it's not pleasant, but it can work if you only want to test one or two files...
Posted By: rogerjowett Re: Fixed software lists - 05/27/10 02:34 PM
hi mess?
sam coupe support?
does it read actual compact flash cards from an atom lite and allow me to use the software thereon?
does it support zx spectrum 48/128 mb-02+ dma support?
ula+
is there an art package for the cpc+ - does the + make any difference to the art other than 4096 colours and would the incredibly small 16x16 sprites with their own 15 colours be usable? is there no interlace mode?
have tyou seen any russian spectrum clone with 14mhz processor running spectrum 128 or 48 software using te faster processor pelase?
can you replay rzx files in mess? can we accelerate the processor to speed up replay like in zx spin please?
can you choose processor speed for playback? could the processor speed emualte the functionality of the msx turbo r800 processor?
does mess support general sound neo for russian clones? - does any software use the z84 (they told me it was 24mhz - although zilog dont even make such a chip - nor is there any info on a dma at 24mhz either - what a shame zilog didnt make a z84 24mhz with dma and r800 core compatiblity!

if you ahve made mess to be compatible with so many machines and their individual video configurations how long before you can create a hardware version of mess compatible with most home computer systems? surely a vitrual video environment could emualte most home computer video chip modes adn you then only need the device to support the z80 or 6502 instruction sets as these were pretty much the only processors in most micros - with a faster version z84 24mhz or is there a 20mhz 6502? surely support for both is possible and accelerators could replay rzx style files faster then youtuibte can replay low quality uncompressed video files! - or crash when you try to install flash player into a 64bit browser or crash when you select one of the tv tuner video inputs as a webcam in the settings

below files are for fuse emulator video screen playback with ay 8912 hopefully soon to be converted to sam - but we could do with mode 4 16 colours and maybe interlaced if atom lite+ or velesoft datagear dma or mayhem quazar or vleesoft accelerators can handle it
Posted By: Robbbert Re: Fixed software lists - 05/27/10 02:56 PM
my, my... thx to RB for leaving that there.. laugh
Posted By: ASH Re: Fixed software lists - 05/27/10 07:02 PM
wow

How do I find what he's smoking cool
Posted By: Firewave Re: Fixed software lists - 06/01/10 05:47 PM
I started looking at the softlists to implement support in mame_regtest and found a problem with associating a "software" entry with a device.

This is an entry from sms.xml:

Code:
<software name="20embr">
	<description>20 em 1 (Bra)</description>
	<year>1995</year>
	<publisher>Tec Toy</publisher>
	<part name="cart" interface="sms_cart">
		<dataarea name="rom" size="40000">
			<rom name="20 em 1 (brazil).bin" size="262144" crc="f0f35c22" sha1="2012763dc08dedc68af8538698bd66618212785d" offset="0"/>
		</dataarea>
	</part>
</software>


But a device entry in -listxml output actually looks like this:

Code:
<device type="cartridge" tag="cart16">
	<instance name="cartridge16" briefname="cart16"/>
	<extension name="sms"/>
	<extension name="bin"/>
</device>


IMO the "name" attribute in the "part" node should have the same name as the device type it is supposed to be used with. So it should be "cartridge" instead of "cart". The latter is just the brief name used for command-line parameters and device tags in the code and not the actual type.
Posted By: etabeta78 Re: Fixed software lists - 06/01/10 07:13 PM
I do not agree. the matching should only require the image and the device to have the same interface.

the name attribute of part could be used at a later stage to handle games which comes with e.g. a tape and a cart, but I'm not 100% sure we should force the part name to match the device type...

judge?
Posted By: judge Re: Fixed software lists - 06/01/10 09:22 PM
The code matches on interface type, if no interface is set for the cartridge slot it just picks the first entry. Unless you specify the full software name: softlist:softname:partname of course.
Posted By: Firewave Re: Fixed software lists - 06/01/10 10:58 PM
So how do I know exactly what entry is for which device? How would somebody who is creating a front-end know what software list to show? Since I have to run the driver with

Code:
mess sms -cart 20embr


When a system has tapes and carts listed I would need to implement some kind of translation code.

And how are entries, that have multiple entries, supposed to be loaded? If it has a cart and a tape I can't really use "-cart" as parameter.
Posted By: etabeta78 Re: Fixed software lists - 06/02/10 05:08 AM
Originally Posted By Firewave
So how do I know exactly what entry is for which device? How would somebody who is creating a front-end know what software list to show? Since I have to run the driver with

Code:
mess sms -cart 20embr


the interface is the key.

if you look at the -lx output, you will see that cartslot device as a interface attribute. in the list xml, part has an interface attribute as well.

when you load from software lists, MESS run through the list in search of a game with the correct name and the correct interface. if you have correct name but no <part> with the correct interface, the game is not loaded.

in principle you might load games from any list, but loading is successful only if the interface matches

a frontend author might let the user specify a list for each media/device of a system and then run through the chosen list showing in the interface only the entries of that list with the proper interface.

the frontend author I'm working with to support lists, though, will use a different approach (which I haven't fully understood, given it's strictly linked to the way the frontend parses xml) and I'm pretty curious to test it... more on this when there is some working sample to share wink

Originally Posted By Firewave
When a system has tapes and carts listed I would need to implement some kind of translation code.

And how are entries, that have multiple entries, supposed to be loaded? If it has a cart and a tape I can't really use "-cart" as parameter.


first, there is no answer yet, because only carts have an interface so far and because no system with multiple entries has lists so far.

second, you are right, I took the wrong example. cart + tape should be handled only relying on the interface attribute (even if since we support no tape + cart combo, things may change)

for sure you will always need to load

mess system -cart name -cass name

the point is where the "name" checksums are defined.

possibility one: we have separate lists for tape vs. cart and the system looks in the different lists for items with tape_intf vs. cart_intf

possibility two: a single lists, and for each game with a combo cart+tape we have two parts with two different interface attributes

in both cases, the name attribute of part is not used.

what you might need the name for, is for games with multiple floppies/tapes: e.g. you could use it to introduce a sort of hierarchy where "disk1" part goes in the first floppy drive, "disk2" goes in the second drive, etc., if the user does not specify differently.

but again, there is list support only for carts at the moment, so it's a bit early to say what we might want to use name for (let's say the attribute is there to allow for extensions in the future).
Posted By: Firewave Re: Fixed software lists - 06/02/10 09:45 AM
Originally Posted By etabeta78

the interface is the key.

if you look at the -lx output, you will see that cartslot device as a interface attribute. in the list xml, part has an interface attribute as well.

when you load from software lists, MESS run through the list in search of a game with the correct name and the correct interface. if you have correct name but no <part> with the correct interface, the game is not loaded.

in principle you might load games from any list, but loading is successful only if the interface matches

a frontend author might let the user specify a list for each media/device of a system and then run through the chosen list showing in the interface only the entries of that list with the proper interface.

the frontend author I'm working with to support lists, though, will use a different approach (which I haven't fully understood, given it's strictly linked to the way the frontend parses xml) and I'm pretty curious to test it... more on this when there is some working sample to share wink


Oh, I see it now. Unfortunately I chose a confusing sample. The interface is just set for the first entry, but IMO it should be set for all of them. The extension information is also redundant for all and it might be possible, that there are different cart slots on a machine (not in this case, but still).
Posted By: etabeta78 Re: Fixed software lists - 06/02/10 09:55 AM
Originally Posted By Firewave
The interface is just set for the first entry, but IMO it should be set for all of them. The extension information is also redundant for all and it might be possible, that there are different cart slots on a machine (not in this case, but still).


that has been a mistake at my end, because when I added softlist at sms.c I hadn't tried the display multislot unit (actually, I didn't even want to add list to it...)

each cartslot is a separate device, hence it should have its own interface of course, and I'm going to amend this. similarly, different slot might support different extensions (think in 2020 the Nintendo DS driver with a DS slot and a GBA slot), so there is no redundancy

about different slots with different interfaces, snesbsx will require that (but it's currently not working, so I haven't even thought if I will split their cart from snes.xml or if I will simply change the interface where needed)

EDIT: otoh, at the moment only one single list can be added to each driver. maybe, this will change in future...

EDIT2: smssdisp is fixed in rev.8199. each cartslot can load from xml list now. smile
Posted By: incog Re: Fixed software lists - 06/02/10 01:23 PM
How will cross-compatibility work?

There are oddities like the gamegear to sms converter, the amstrad mega pc that has a meagdrive cart slot and many others. Do we have to create a redundent list for these or is there some way we can use some compatibility flag pointing to the main list of the compatible sofware?
Posted By: Duke Re: Fixed software lists - 06/02/10 02:00 PM
We don't need a flag. If you have a matching interface between cartslot device and software list, it will work automatically.
Posted By: etabeta78 Re: Fixed software lists - 06/02/10 02:03 PM
Originally Posted By incog
How will cross-compatibility work?

There are oddities like the gamegear to sms converter, the amstrad mega pc that has a meagdrive cart slot and many others. Do we have to create a redundent list for these or is there some way we can use some compatibility flag pointing to the main list of the compatible sofware?


you haven't read my blog entry carefully, then wink

you launch games from other lists by using e.g.

mess gbcolor -cart gameboy:XXXX

this might work easily for mega pc, as long as its cartslot has the megadrive interface, and might mimic pretty well compatibilities through adapters. alternatively, for the megapc only, the whole megadrive list can be added to that specific driver. no need of a separate list.
Posted By: judge Re: Fixed software lists - 06/02/10 03:42 PM
Great, then we get closer to adding the creditcard sized cartridge slot to some sms models and also to the smssdisp driver (yes, it had 16 regular cartridge slots and 16 creditcard size slots).
Posted By: JoJo Re: Fixed software lists - 06/02/10 04:29 PM
Originally Posted By etabeta78

EDIT: otoh, at the moment only one single list can be added to each driver. maybe, this will change in future...



It would definitely make sense - consider for example the Commodore 64 and 128: the C128 can load all the C64 cartridges, but the C64 can't load C128 specific cartridges (all the two of them wink ) even if the interface is the same - therefore you need two lists: c64_cart and c128_cart, with the c128 cartslot supporting both of them.
Posted By: judge Re: Fixed software lists - 06/02/10 05:57 PM
Actually, you can add a list of lists to a driver. With
Code:
MDRV_SOFTWARE_LIST_CONFIG(1,"list2")
MDRV_SOFTWARE_LIST_CONFIG(2,"list3")

you can add more entries.

Not the most elegant solution, but it works with having to instantiate the device.
Posted By: etabeta78 Re: Fixed software lists - 06/02/10 09:39 PM
good to know (even if I still had no need of multiple list, it might become handy with home computers too, to have separate lists for tapes, floppies and carts... and separate folders, of course, to keep my romsets in order wink )
Posted By: judge Re: Fixed software lists - 06/02/10 09:43 PM
it may come in useful for the gx4000 or the zemina msx clone which only took cartridges.
Posted By: JoJo Re: Fixed software lists - 06/02/10 10:27 PM
Found a little bug: -listsoftware doesn't print the "description" attribute of the "softwarelist" element.
Posted By: etabeta78 Re: Fixed software lists - 06/03/10 04:43 AM
I never thought it was a bug. is it necessary to print it?
Posted By: JoJo Re: Fixed software lists - 06/03/10 08:42 AM
Not really necessary (unless for frontends or rom tools), but since the field is defined in the .xml file I guessed it should be present in -listsoftware output.
Posted By: judge Re: Fixed software lists - 06/03/10 09:11 AM
I think Duke added the description to the list struct, so it possible to output it now. I never needed it for the core functionality so I wasn't storing it in memory.
Posted By: Micko Re: Fixed software lists - 06/04/10 12:12 PM
As you may all already saw there is now software list support done in UI, it is still quite preliminary, but it works now.

In list all software items from all lists for specific driver will be shown with details. Thing is that with clicking on specific item driver will start that software. For not there is no auditing and listing only available games, but it is planned.

Please suggest how can we improve it, and/or feel free to update code.
Posted By: etabeta78 Re: Fixed software lists - 06/04/10 07:22 PM
the way it works is quite useful, but in my opinion is not optimal.

take e.g. atomeb and smssdisp: they have multiple cartslots and each slot can load a software list item, while in the current implementation only the first slot can be used.

while this can be fine with smssdisp (mainly because the display unit does not really work), in atomeb it is quite important to load multiple expansions.

a possible solution would be to somehow link the Software tab to the Mount/Unmount functionality of the Device View tab: sort of
having three options, "Mount from path", "Mount from list", "Unmount"

This would also allow to only show the items with the proper interface expected by that specific slot (if I put an entry in snes.xml with interface="gameboy_cart", command line MESS would simply not load it, while I think MESSUI would currently list it in the Software Tab).

Additionally, this might also solve the huge slowdown when scrolling up/down through e.g. Game Boy consoles (since every time the whole list has to be parsed)
Posted By: k1w1 Re: Fixed software lists - 06/05/10 07:14 AM
Glad to see the software tab is hooked up and working.

One little problem however. I have the new software directory's sitting under a directory called software like this: -

c:\mess\software\32x\

I prefer this than placing the software directory's in the rom folder like this :-

c:\mess\roms\32x\

Here is the problem. I have the rompath set up in the mess.ini file and can run the software from the picker tab.

I can't do this from the software tab unless i put the software directory's (32x, aes, etc...) in the rom folder. It's as if the software tab is not seeing the rom path in th mess.ini file. The rompath is set like this: -

rompath roms;c:\mess\software

Is there some way i can point the software tab to my prefered directory structure.

k1w1
Posted By: etabeta78 Re: Fixed software lists - 06/05/10 08:08 AM
it should work fine if you add the second rompath in the general Directory properties of MESSUI (here it worked perfectly). I guess MESSUI only reads the rompath from its ini file and not from the main mess.ini. not sure if this is by design or a mistake, though.
Posted By: k1w1 Re: Fixed software lists - 06/05/10 09:25 AM
Thanks again etabeta.

I think i have defined the problem a little more.

I have all of the cartridge software for all of the systems covered with the xml's. These have been verified with clrmamepro.

It appears only a few sytems can run from the software tab. Some of these are: -

advision
aes
arcadia
pce

to name a few. Most others just report "not found", systems like: -

snes
gameboy
jaguar

One of the strange thing is that although pce runs it's complete software list, it's subsystems sgx & tg16 don't run theirs.

I can't figure out why this is happening. They run from the picker tab.

OK i have just dropped jaguar into my roms directory and it is now recognised by the software tab.

So even though i have the rompaths setup correctly only some systems run from my software directory and others like jaguar have to be placed in my roms directory.

I'm confused!

k1w1

Hang on, even more confusing is that jaguarcd loads and runs the jaguar software list from my software directory, but not jaguar.
Posted By: etabeta78 Re: Fixed software lists - 06/05/10 09:53 AM
nope. they all load fine here from the secondary path: jaguar, tg16, pce, gameboy and snes.

check that you have the right rompath in Options->Directories and/or try to delete any jaguar.ini (and snes.ini, etc.) old file which might conflict with the new one
Posted By: k1w1 Re: Fixed software lists - 06/05/10 10:11 AM
Thats got it, I have erased all the files in the INI directory, and things are working as they should.

Thanks
Posted By: ASH Re: Fixed software lists - 06/07/10 10:18 AM
Struggling here with these software lists etc

a couple of questions

1.am I right in thinking the Software list will check your (roms/tapes/disc's etc) and add info against the ones that match MESS's internal list (if that's correct then cool smile )

and this will be shown in the PICKER (its working already and is fine smile )


2.What is the Software Tab for and how do I get it to display anything (I was reading K1W1's post's above but I still don't get it frown )


3.How do I get a software list into clrmamepro
please supply idiot guide as I only use the GUI so if I have to use CMD prompts what exactly do I type (don't use *typenewnamegamehere* ) as I will type typenewnamegamehere :0

4.If I get a software list into clrmamepro will it then rebuild aes roms that will load in mess I have some but mess won't see them.

sorry for being a thicky on this....

be patient with me. smile

ASH2
Posted By: etabeta78 Re: Fixed software lists - 06/07/10 10:37 AM
point 1: not yet. the MESSUI support is only preliminary.
point 2: MESSUI (& MESS) when loading from software lists does not look for software in the software path you set for the picker. you have to put the items (renamed by clrmame) in a subfolder of one of the rompaths (by default roms/ ). e.g. gba games in roms/gba/ and aes games in roms/aes

points 3-4 read the User Manual

http://mess.redump.net/mess:howto#software_lists

or my blog entry

http://mamedev.emulab.it/etabeta/?p=188
Posted By: ASH Re: Fixed software lists - 06/07/10 11:44 AM
*goes off to read the docs blush*

EDIT

OK found the problem smile

the reason I am struggling here is that there are no xml files in the hash folder with v138

and trying to create an xml with mess arcadia -listsoftware > arc.xml produces a corrupt file so I had to goto http://git.redump.net/cgit.cgi/mess/plain/hash/arcadia.xml

copy it: paste it in notepad and save it in the hash folder as xml it now works (well the arcadia does)...so reading the manual is not as helpful as you make out frown


is there a way of grabbing the xml files? or will I have to copy:paste them all?
Posted By: Duke Re: Fixed software lists - 06/07/10 12:30 PM
You don't need to copy & paste, just use "File -> Save As" in your browser. But those files should be included with 0.138.
Posted By: ASH Re: Fixed software lists - 06/07/10 12:39 PM
@Duke : thanks yeah that's faster blush

well I have just checked the zip of Mess0138b (hash folder) and they are not there...honest smile

But on a side note when it works it's fantastic just created the aes.xml put it in clrmamepro pointed it to mame roms and its makeing me some neogeo games thankyouverymuch.

smile

EDIT so clrmamepro has created 33 aes roms but when I mount a game.

'kof95.zip'

I get this error

Quote:
Error Loading Multicart: No pcb found


any ideas?



Posted By: etabeta78 Re: Fixed software lists - 06/07/10 01:54 PM
you don't mount these games, you double click the game in the Software TAB to launch them


moreover, here -listsoftware creates a perfectly fine arcadia list... not sure what problem did you encounter
Posted By: ASH Re: Fixed software lists - 06/07/10 02:50 PM
But that does not work as MESS says

Quote:
System 'Neo-Geo AES' requires that device cartridge must have image to load


so the only way I can do that is to mount the game?

or am I missing something else crazy
Posted By: etabeta78 Re: Fixed software lists - 06/07/10 02:57 PM
I cannot test because I don't have an updated compile with MESSUI and I don't have here my eeepc. it works fine from command line, but it might have some problem from MESSUI for systems which have mandatory carts... can you launch a sega master system game in sg1000m3 from the Software TAB or does it give you the same error?

EDIT: where did you rebuild your roms with clrmame? you must have them in roms/aes/ for MESS to be able to load them...
Posted By: R. Belmont Re: Fixed software lists - 06/07/10 03:10 PM
AES lists work fine from the commandline on latest. No idea about newui.
Posted By: ASH Re: Fixed software lists - 06/07/10 03:18 PM
I rebuilt my roms from my MAME romset

these are in my mess/software folder (I have added mess/software dir to the Roms: directories and also copied my aes to the roms folder so I have covered all bases. smile

and I also get the same error on the sg1000

what would be the command line for kof95 for aes?
Posted By: R. Belmont Re: Fixed software lists - 06/07/10 03:22 PM
mess aes -cart kof95
Posted By: ASH Re: Fixed software lists - 06/07/10 03:33 PM
yep the cmd line works so it's not the roms its the gui

Posted By: etabeta78 Re: Fixed software lists - 06/07/10 03:41 PM
Originally Posted By ASH
I rebuilt my roms from my MAME romset

these are in my mess/software folder (I have added mess/software dir to the Roms: directories and also copied my aes to the roms folder so I have covered all bases. smile


notice that they have to be in

mess/software/aes/

if they are, then it's a MESSUI problem and I cannot help any more for a few days, not having a windows computer here

EDIT: I missed the last message. so it's a UI problem, probably related to systems with mandatory carts...

Posted By: ASH Re: Fixed software lists - 06/07/10 03:45 PM
ok


and yes it's in mess/roms/aes

and also mess/software/aes.

Posted By: ASH Re: Fixed software lists - 06/08/10 07:59 AM
Ok after getting the xml files and putting them in the hash folder

I have found that the vc4000.xml crashes the GUI if you select any system associated with it I.E. 1292apvs +clones, vc4000 etc.

Posted By: k1w1 Re: Fixed software lists - 06/12/10 09:52 PM
etabeta,

I would like to have a go at compiling a software list for the Atari 400/800 & XL/XE. Is that alright or would you rather keep all the list creation with you at present?

Secondly, the Atari 8-bit cartridges don't load in MESS at the moment. I seem to remember they did at one stage but there must have been some regression introduced.

k1w1
Posted By: Anna Wu Re: Fixed software lists - 06/12/10 10:17 PM
Originally Posted By k1w1
etabeta,

I would like to have a go at compiling a software list for the Atari 400/800 & XL/XE. Is that alright or would you rather keep all the list creation with you at present?

Secondly, the Atari 8-bit cartridges don't load in MESS at the moment. I seem to remember they did at one stage but there must have been some regression introduced.

k1w1


You are right, most carts are not working. Some are working like DigDug ...
Posted By: R. Belmont Re: Fixed software lists - 06/12/10 10:23 PM
AFAIK the Atari carts don't work because there is no mapper support, something that would be part of adding software list support.
Posted By: k1w1 Re: Fixed software lists - 06/20/10 02:10 AM
I have just spent the last week putting together all of the Atari 8-bit cartridges available and have divided them into 2 groups: -

1) Atari 400/800 (260 files). Most of these cartridges run on the XL/XE as well and were released prior to 1986.

2) Atari XEGS (36 files). These were released specifically for the XE/XEGS and are banked cartridges greater than 16 meg in size.

I have also gone through them all and removed any headers and tested them all on the "Atari800Win Plus v.4" emulator to ensure they run o.k.

I am about to create a MESS softwarelist for these and was wondering if any tools had been created that would help automate the compilation of the XML files.

k1w1
Posted By: judge Re: Fixed software lists - 06/20/10 08:40 AM
notepad or vi laugh

I created the first batch of xml lists by taking the attempts at lists in C, parse them with a bit of perl and made it write out the xml. Also small updates to the format were done that way.

There are probably other ways to achieve the same result.

Posted By: etabeta78 Re: Fixed software lists - 06/20/10 10:25 AM
I found quite useful to create a dat with clrmame and then to write a perl script to make a xml list out of it

then I have to adjust small bits by hand (e.g creating shortnames), but at least checksums are 100% correct this way
Posted By: k1w1 Re: Fixed software lists - 06/21/10 08:58 AM
Thanks for the info guy's.

I am compiling the XEGS xml now. There are only 36 files so it is a small one to start with.

One question though, what should i put in the offset field?

I am guessing it is where the carts load in memrory. However do i leave it at 0 or nominate a region. For example: -

Ballblazer.rom is 64k

This bank-switched cartridge occupies 16 KB of address space between $8000 and $BFFF. The cartridge memory is divided into 8 banks, 8 KB each. Bank H (the last one) is always mapped to $A000-$BFFF. Three lowest bits of a byte written to $D500-$D5FF select the bank mapped to $8000-$BFFF.

What do i need to do?

k1w1
Posted By: Duke Re: Fixed software lists - 06/21/10 09:12 AM
Use "0" for now.
Posted By: etabeta78 Re: Fixed software lists - 06/21/10 09:56 AM
and then take care in DEVICE_LOAD_IMAGE of the memory location it has to be loaded at wink
Posted By: k1w1 Re: Fixed software lists - 06/22/10 10:26 AM
Etabeta,

Check your PM.

I have completed the first of the Atari Cartridge xml's.

k1w1
Posted By: mizapf Re: Fixed software lists - 06/26/10 06:44 PM
@etabeta: I've recently studied the descriptions to software lists a bit more throughly, and also checked the DTD.

I guess that with more and more systems adopting the lists and less using the RPKs, I (or whoever) will not have another option but to implement them for the TI family as well in not-so-distant future. Still, there are some points which are missing or for which I need some more clarifications:

DTD:

- <ram> element for defining NVRAM in the cartridge (easy, just means to add this to the DTD and the parser)

- <pcb>: seems to be covered by <feature>, but one should consider to have this as an own (optional) element as it is a very important information for the cartridge, and hiding it in a generic element is not a good way to go (since the pcb is a mandatory information for my carts)

Usage:

- I need the possibility to handle multiple cartridges at the same time (-cart1, -cart2, ...). This is not per se a problem with the software lists, and no problem on the command line, but rather how this should be presented in the UI. A simple software selection will not do; I still need to specify where to mount the cartridge (just as possible now).

- I see that cartridge contents are stored in subdirectories as separate files. Can I also drop zip files there without unpacking (like the system zips in the roms folder)? I don't intend to tell users to download the zips, unpack them in the right location and repack them when sending them to someone else. That would be a big leap backwards.

I suppose that there will be more features requests over the time for the lists (not only by me smile ). It does not make sense to mention them somewhere in mailing lists or forums where they become lost. What about a section on suggestions or feature requests on the Wiki (like the "Discussion" page in MediaWiki)?

Michael
Posted By: etabeta78 Re: Fixed software lists - 06/27/10 10:33 AM
I'm traveling fro work, so I will only be able to give you a complete and detailed answer after monday/tuesday

however, a few points:

1. rpk is not going to disappear, if I can avoid it. for one, I'd love to make NES able to use it (it would allow to remove nes.hsi without compromise support of .nes files)
2. I disagree on the <ram> & <pcb> items: the <pcb> should be implemented by

<feature name="pcb_type" value="pcb_name">

it works like a charm for NES which have around ~300 pcb types
and ram should be in a separate dataarea with no <rom>

<dataarea name="ram" size="8192">
</dataarea>

properly handled at loading time. I spent some time thinking about the DTD, but I think that we should keep it as simple as possible given most of the things can be already covered
Posted By: mizapf Re: Fixed software lists - 06/27/10 12:38 PM

Originally Posted By etabeta78
rpk is not going to disappear, if I can avoid it. for one, I'd love to make NES able to use it (it would allow to remove nes.hsi without compromise support of .nes files)


Seems as if there is not a unique position among the developers, since the current RPK mechanisms are now called LEGACY_DEVICEs. Which may become a bit confusing in the case of ti99cart, since I already have a "legacy" system (the .bin files) there. So there are now two flavors of legacy, the newer and the older legacy.

Michael

Posted By: R. Belmont Re: Fixed software lists - 06/27/10 12:45 PM
LEGACY_DEVICE means "using an older API and will need to be updated", not "Micko wants this to go away".
Posted By: etabeta78 Re: Fixed software lists - 06/28/10 08:24 AM
yeah, many MAME components use that as well. it was just impossible to convert all the old devices to the new C++ implementation, so most of them use the legacy implementation while being (slowly) updated wink
Posted By: Anna Wu Re: Fixed software lists - 07/05/10 05:50 PM
Originally Posted By R. Belmont
AES lists work fine from the commandline on latest. No idea about newui.


No problem with the actual SVN builds. Just the carts nam1975 and gpilots in combination with MESSUI and the Software tab tested. smile
Posted By: incog Re: Fixed software lists - 07/05/10 08:36 PM
Just finished the Atari 5200 softlist, I was going to start a 7800 list but all the dumps I can locate are all odd sizes, probably some gross headers. Anybody know the deal with 7800 roms?
Posted By: ASH Re: Fixed software lists - 07/06/10 09:15 AM
the vc4000.xml is still crashing my UI anybody else getting this problem (I have to remove the xml from my hash folder for it to work)

I don't expect a lot of problems with this as only a few people have xml's at the moment but when you release them in v0.139...

(the UI crashes of I select any affiliated system i.e. 1292apvs , rwtrntcs , vc4000 etc)

Windows Vista 32
Posted By: Duke Re: Fixed software lists - 07/06/10 09:32 AM
It's not crashing here. Are you sure your xml file isn't damaged?
Posted By: Anna Wu Re: Fixed software lists - 07/06/10 10:03 AM
Originally Posted By Duke
It's not crashing here. Are you sure your xml file isn't damaged?


ditto smile
Posted By: ASH Re: Fixed software lists - 07/06/10 10:38 AM
Originally Posted By Duke
It's not crashing here. Are you sure your xml file isn't damaged?


Dunno confused

Code:

<?xml version="1.0"?>
<!DOCTYPE softwarelist SYSTEM "softwarelist.dtd">

<softwarelist name="vc4000" description="Interton VC 4000 cartridges">

	<!-- Cassette 1 -->
	<software name="carraces">
		<description>Autorennen / Car Races / Course Automobile</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="carraces.bin" size="2048" crc="5c7f11e0" sha1="0d1cd6f7da36660f19e0b0e9c4312b1c56d5fbc6" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 2 -->
	<software name="blckjack">
		<description>Blackjack</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="blckjack.bin" size="2048" crc="ec2a91b3" sha1="ba4553d116cffa74a71ef10fbffa4c28c779c836" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 3 -->
	<software name="paddleg">
		<description>Ballspiele / Paddle Games / Jeux De Balle</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="paddleg.bin" size="2048" crc="682b876a" sha1="ecc0146c40daa79a13934b8884bc8fd0f96461c1" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 4 -->
	<software name="tbattle">
		<description>Panzerschlacht/Luftkampf / Tank Battle/Air Battle / Bataille De Blindes/Combat Aerien</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="tbattle.bin" size="2048" crc="468e4779" sha1="191d0a1bf399a09525b57dbbca92d83de18003fd" offset="0" />
			</dataarea>
		</part>
	</software>
	
	<!-- Cassette 5 -->
	<software name="mathema1">
		<description>Mathematik I / Mathematics I / Mathematique I</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="mathema1.bin" size="2048" crc="e38e0c03" sha1="d48c8ca8811fa9418d257c92365b88bba0cdffb2" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 6 -->
	<software name="mathema2">
		<description>Mathematik II / Mathematics II / Mathematique II</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="mathema2.bin" size="2048" crc="66c5975e" sha1="efa4a5ebe540efdedec226c0af238c2652ad5b5a" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 7 -->
	<software name="airseaba">
		<description>Luftkampf/Seegefecht / Air/Sea Battle / Combat Aerien/Bataile Navale</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="airseaba.bin" size="2048" crc="f3d37699" sha1="922b9469b2262eb1a135defd5dbc11f33094df5c" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 8 -->
	<software name="memory1">
		<description>Memory I/Motivsuche / Memory I/Flag Capture / Memory I/Recherche De Motifs</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="memory1.bin" size="2048" crc="ea8f717f" sha1="a383ced230178c9c829b82b2793cb036411ccb5d" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 9 -->
	<software name="intel1">
		<description>Intelligenz I / Intelligence I</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="intel1.bin" size="2048" crc="4d3f9cfb" sha1="b8cba1672f5b7286e8895fc6e6c46bbb54fb3f46" offset="0" status="baddump" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 10 -->
	<software name="wintersp">
		<description>Wintersport / Winter Sports / Sports D'Hiver</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="4096">
				<rom name="wintersp.bin" size="4096" crc="bdd652b7" sha1="aa3af1df1095aadf9d7428a59847fbef3505d93d" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 11 -->
	<software name="hippodro">
		<description>Hippodrom / Hippodrome</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="hippodro.bin" size="2048" crc="aaea290f" sha1="0c4dd4f31adccdf02ba783a23ca9debe2c4cee02" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 12 -->
	<software name="hunting">
		<description>Jagd / Hunting / Chasse</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="hunting.bin" size="2048" crc="31978ad4" sha1="68a21765ae57357c7e5385e57b3caa60f796c0cc" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 13 -->
	<software name="chess">
		<description>Schach / Chess / Les Echecs</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<!-- 1k internal ram -->
			<dataarea name="rom" size="4096">
				<rom name="chess.bin" size="4096" crc="e9c53288" sha1="468188fd58aefb9f8e17e72ea4494f4e8219338a" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 14 -->
	<software name="motocrss">
		<description>Motocross</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="4096">
				<rom name="motocrss.bin" size="4096" crc="55011f0a" sha1="a461b66f6cff37e4038e80da0a205a816abe35d4" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 15 -->
	<software name="intel2">
		<description>Intelligenz II / Intelligence II</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="intel2.bin" size="2048" crc="8ab96827" sha1="58c7cabd14e3cd0505f8ea27aaf08cc8045bc590" offset="0" status="baddump" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 16 -->
	<software name="intel3">
		<description>Intelligenz III / Intelligence III</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="intel3.bin" size="2048" crc="52b8c6e7" sha1="b83fd9679912b0888df38f708dbea80dfe3b8193" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 17 -->
	<software name="circus">
		<description>Circus / Cirque</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="circus.bin" size="2048" crc="0ab80f3e" sha1="cff0b7066669aad1e35803767fd98ecef4ea162d" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 18 -->
	<software name="boxing">
		<description>Boxkampf / Boxing Match / Match De Boxe</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="4096">
				<rom name="boxing.bin" size="4096" crc="922c9f0d" sha1="b49a37dd6d0272f6c71d778ffada6bc7c90f8348" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 19 -->
	<software name="outersc">
		<description>Krieg im Weltraum / Outer Space Combat / Guerre Dans L'Espace</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="outersc.bin" size="2048" crc="b1c31f9a" sha1="8a2b29af42051f61a22a033c7dd140705cf7bc6e" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 20 -->
	<software name="memory2">
		<description>Memory II / Melodie/Melody / Simon</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="memory2.bin" size="2048" crc="d68da80a" sha1="4cf0f74b4a0d3305c9ffa830c067a054fab25419" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 21 -->
	<software name="reversi">
		<description>Intelligenz IV/Attacke (Reversi) / Intelligence IV/Attack (Reversi) / Intelligence IV/Attaque (Reversi)</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="reversi.bin" size="2048" crc="619c07c9" sha1="30c595f60e464a93c69579ae9bffdd42ee2f73e7" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 22 -->
	<software name="chess2">
		<description>Schach II / Chess II / Les Echecs II</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<!-- 1k internal ram -->
			<dataarea name="rom" size="6144">
				<!-- redump needed, should be a 4k and a 2k file -->
				<rom name="chess2.bin" size="6144" crc="1b948aeb" sha1="0da5c4e865ce4a6b0293e51949eb0fb525881cf0" offset="0" />
			</dataarea>
		</part>
	</software>
	
	<!-- Cassette 23 -->
	<software name="pinball">
		<description>Flipper / Pinball</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="pinball.bin" size="2048" crc="add99c6e" sha1="85b904e5bc37e5a44c88c000307401438285910a" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 24 -->
	<software name="soccer">
		<description>Fussball / Soccer / Football</description>
		<year>1979</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="soccer.bin" size="2048" crc="05af8228" sha1="fd4212b955688aec985102d84575cae55bc651eb" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 25 -->
	<software name="bowling">
		<description>Bowling/Kegeln / Bowling/Ninepins / Yeux Quilles</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="bowling.bin" size="2048" crc="a47fe1bf" sha1="e637787ed9d868f85339a413bbf2989a785fa2a7" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 26 -->
	<software name="draughts">
		<description>Dame / Draughts / Dames</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<!-- 1k internal ram -->
			<dataarea name="rom" size="4096">
				<rom name="draughts.bin" size="4096" crc="7b868473" sha1="2557172b07c0ebaffe18a345cfa88ef24febecb0" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 27 -->
	<software name="golf">
		<description>Golf</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="4096">
				<rom name="golf.bin" size="4096" crc="d399ce07" sha1="41b6c7d3bc8ea2ef9777066dfd74ec2c5932ddb6" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 28 -->
	<software name="cockpit">
		<description>Cockpit</description>
		<year>19??</year>
		<publisher>?</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="4096">
				<rom name="cockpit.bin" size="4096" crc="11dd7f0d" sha1="456e9c5d1d48877e8c5eb5e5499c911a73e30584" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 29 -->
	<software name="metropol">
		<description>Metropolis / Hangman</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="4096">
				<rom name="metropol.bin" size="4096" crc="a640a330" sha1="3b9b1b5c35d3e2ef909e1498f96026cfb437ea80" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 30 -->
	<software name="solitair">
		<description>Solitr / Solitaire</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="solitair.bin" size="2048" crc="08209d98" sha1="bf96ceb173f17017b99cf8d38ed55e66b5320260" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 31 -->
	<software name="casino">
		<description>Casino</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="casino.bin" size="2048" crc="50fd6a18" sha1="7d2549a3a3373592d2eb88c3a5c9918b32fddafd" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 32 -->
	<software name="invaders">
		<description>Weltrauminvasoren / Invaders / Invaseur De L'Espace</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="invaders.bin" size="2048" crc="9497204a" sha1="1fba78476ee93f09b6db033ec4d5332498ebc4d0" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 33 -->
	<software name="superinv">
		<description>Super Invaders / Super Invaders / Super Envahisseurs</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="superinv.bin" size="2048" crc="a46c2f9e" sha1="bf851dbd92a539232c89fe2f6c4e2759b61ef3d4" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 34 (unreleased) -->
	<!-- Cassette 35 (unreleased) -->

	<!-- Cassette 36 -->
	<software name="backgamm">
		<description>Backgammon / Jacquet</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<!-- 1k internal ram -->
			<dataarea name="rom" size="4096">
				<rom name="backgamm.bin" size="4096" crc="08cd8135" sha1="942f65a68e4116b1da66df57c09829f9d9b62659" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 37 -->
	<software name="mnstrman">
		<description>Monster-Man</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="4096">
				<rom name="mnstrman.bin" size="4096" crc="27cce96f" sha1="f55798c634be020b46699f2e3e71301230db86d2" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 38 -->
	<software name="hyperspa">
		<description>Hyperspace / Hyperespace</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="2048">
				<rom name="hyperspa.bin" size="2048" crc="b3ed1129" sha1="4c0f862f6fa4850e5e7305b90df71f7b899f0c27" offset="0" />
			</dataarea>
		</part>
	</software>

	<!-- Cassette 39 (unreleased) -->
	
	<!-- Cassette 40 -->
	<software name="suprspac">
		<description>Super-Space</description>
		<year>19??</year>
		<publisher>Interton</publisher>
		<part name="cart" interface="vc4000_cart">
			<dataarea name="rom" size="4096">
				<rom name="suprspac.bin" size="4096" crc="306e37bb" sha1="b11e9fed7700d5192b05dfabeca7b23feca64540" offset="0" />
			</dataarea>
		</part>
	</software>

</softwarelist>
Posted By: Duke Re: Fixed software lists - 07/06/10 10:47 AM
Well that isn't very useful. Get a fresh one from SVN and compare them (or just use the new one).
Posted By: sminty Re: Fixed software lists - 07/06/10 11:02 AM
A7800 cart info can be found on Dan Boris' page

http://atarihq.com/danb/7800cart/a7800cart.shtml

Check pm too
Posted By: ASH Re: Fixed software lists - 07/06/10 11:13 AM
Originally Posted By Duke
Well that isn't very useful. Get a fresh one from SVN and compare them (or just use the new one).


Yes that works now, I am using a browser (Chrome) that dosn't download and save xml so I copied and pasted the xml to notepad and saved it through that ...WRONG... eek

used IE and save.as.. and that is the correct one

sorry guys...my bad...doh.
Thanks. blush

Posted By: Kaylee Re: Fixed software lists - 07/06/10 02:33 PM
etabeta78 you've got mail
Posted By: Kaylee Re: Fixed software lists - 07/06/10 04:50 PM
@etabeta you've got mail
Posted By: judge Re: Fixed software lists - 07/06/10 05:59 PM
Originally Posted By incog
Just finished the Atari 5200 softlist, I was going to start a 7800 list but all the dumps I can locate are all odd sizes, probably some gross headers. Anybody know the deal with 7800 roms?


Just strip off the first 128 bytes; that's the header.

The text "ACTUAL CART DATA STARTS HERE" in the header was a useful hint wink
Posted By: Mitch Re: Fixed software lists - 07/06/10 06:00 PM
I actually started working on a 7800 list a while ago but never finished it. One thing to note is that quite a few of the ROMs for 7800 protos have been tweaked to make them work and are not original dumps.

Mitch
Posted By: etabeta78 Re: Fixed software lists - 07/06/10 06:07 PM
Originally Posted By Kaylee
@etabeta you've got mail


whenever you write me, a flashy message appear at the top of the forum next time I login. it's done in purpose and it works perfectly.
hence, please STOP posting here every time you send me a PM: you're just spamming the thread...
Posted By: Just Desserts Re: Fixed software lists - 07/06/10 07:55 PM
Kaylee being an annoying poster? Why, I never!
Posted By: incog Re: Fixed software lists - 07/06/10 09:28 PM
Originally Posted By judge
Originally Posted By incog
Just finished the Atari 5200 softlist, I was going to start a 7800 list but all the dumps I can locate are all odd sizes, probably some gross headers. Anybody know the deal with 7800 roms?


Just strip off the first 128 bytes; that's the header.

The text "ACTUAL CART DATA STARTS HERE" in the header was a useful hint wink


Haha, now I feel stupid for not even checking them out. Thankfully I was directed to an archive of headerless dumps and I'm now creating a softlist, though I doubt any of them will load.
Posted By: judge Re: Fixed software lists - 07/06/10 09:38 PM
Maybe you can try using some of the information on dan boris' page to at least tell which kind of cartridge pcb they oughta be using.
Posted By: etabeta78 Re: Fixed software lists - 07/06/10 10:18 PM
yup! a good <feature name="cart_type" value="xxxxx"/> would do wonders to make easier loading them later in the driver...
Posted By: Kaylee Re: Fixed software lists - 07/08/10 02:15 PM
Hi

I would like to know.

I would like to do the Einstein TC-01 xml file.

It uses floppydisk and would I use the following
<part name="flop1" interface="einstein_flop">
Posted By: etabeta78 Re: Fixed software lists - 07/08/10 02:33 PM
floppy lists are still under discussion, so please wait until Micko adds one of two working samples wink

EDIT: also, a couple of guidelines:
- put multi-disk games inside a single <software> entry of the list, as separate <part>s (this might change later, but it is the current plan). same goes for different sides of a tape
- avoid multigame compilations (especially when they are homemade, coverdisks might be eventually added, still to be decided)
- when you have several alt versions (say 5-10 alt vers) of a same game from a same publisher, which are not different revisions, maybe try to play the game itself to see if you can spot differences: the goal is to find the best copy available with these format, not to include every single bit variation which could be due to ruined media...

EDIT2: the "flop_interface" part is fine, for the "flop1" (and on) wait for micko's example / instructions before sending the list smile
Posted By: judge Re: Fixed software lists - 07/08/10 03:01 PM
For interface, I'd prefer to see something about floppy dimensions in there. Things like 'floppy_3.5', 'floppy_3', 'floppy_8', etc with the driver disk drive settings also updated to reflect the kind of floppy the drive supports.
Posted By: incog Re: Fixed software lists - 07/08/10 03:20 PM
I've always wondered what the convention was for multiple games on a cart/disk/whatever.

Previously I've used the convention snagged from MAME of using a slash to seperate the game names (see: http://maws.mameworld.info/maws/romset/alphaho). I've also seen this used on games which have multiple titles, usually in different languages (see: hash/vc4000.xml).

It's only a minor niggle but I can see confusion being generated over the collision of conventions.
Posted By: etabeta78 Re: Fixed software lists - 07/08/10 03:37 PM
well, I have often preferred to use a '~' for multiple titles rather than a '/' but it's fine both ways, I'd say.

for multiple games on a single cart, I used a '+' in gba.xml, but again I think '/' is not a bad choice...

finally, in the case of pirate multicarts with tons of games, a generic "XX in 1" name is fine, I think

my point about multiple software in a single tape/disk was that if you're going to include the image of a floppy disk which was created by my cousin to give me a couple of tiny games, then this kind of image can be thrown away

I've specified this because for some exotic computer a few games appear to be available only inside multigame floppies which do not represent at all a real piece of software and, as such, is not really worth the inclusion...
Posted By: Duke Re: Fixed software lists - 07/08/10 03:51 PM
Just to clear some confusion maybe, unlike the cartridge software lists, the floppy software lists will have only a few interface types. Like judge said, something like floppy_35, floppy_525. Simple example: You can insert and use (or well, should be able to in the future) IBM PC floppies in the Amiga driver.
Posted By: Haze Re: Fixed software lists - 07/08/10 03:56 PM
Originally Posted By etabeta78
floppy lists are still under discussion, so please wait until Micko adds one of two working samples wink

EDIT: also, a couple of guidelines:
- put multi-disk games inside a single <software> entry of the list, as separate <part>s (this might change later, but it is the current plan). same goes for different sides of a tape
- avoid multigame compilations (especially when they are homemade, coverdisks might be eventually added, still to be decided)
- when you have several alt versions (say 5-10 alt vers) of a same game from a same publisher, which are not different revisions, maybe try to play the game itself to see if you can spot differences: the goal is to find the best copy available with these format, not to include every single bit variation which could be due to ruined media...

EDIT2: the "flop_interface" part is fine, for the "flop1" (and on) wait for micko's example / instructions before sending the list smile


coverdisks and things like the well known 'assassins' PD collections should *definitely* be included. There were quite a few exclusive releases on coverdisks (level editors, unique games, things like that), same with covertapes on the speccy, various magazines had game design competitions with the winners going on the tape and most of them were commercial quality. Also keep in mind that the multi-game compilations were often rereleases of the games, that might have had bug fixes etc. and often used different copy protection schemes to the original releases.

For a good example of coverdisks see 'Gravity Power' (an exclusive beefed up version of Gravity Force II) as well as a level editor for said game on a later issue, exclusive add-on disks for games like Skidmarks and Super Skidmarks, also some demos for games which never ended up being released like Putty Squad could only be found on magazines.

These were major parts of the system's software library, I'd say 75% of the software I owned on the systems was covertapes / disks / public domain.

Some piece of junk I recorded for my friend back in the day is less important of course.
Posted By: Micko Re: Fixed software lists - 07/08/10 04:19 PM
Think that we would need to split softlist for popular systems into more files, since it will be quite hard to find needed things. Something like TOSEC collection do, we can make games, applications, demos,.. lists for same system.
Posted By: incog Re: Fixed software lists - 07/08/10 05:03 PM
I don't think we need to split the softlists down further if we are already excluding garbage like overdumps, baddumps, partial translations and other crappy hacks. The lists are already split thanks to their physical format (many computers have cartslots, diskdrives and casette decks), spliting them down further feels artificial.

The MESS GUI also has a search function (although that only works on the main driver list at the moment) so finding what you need on the list shold be pretty easy.
Posted By: R. Belmont Re: Fixed software lists - 07/08/10 05:04 PM
I actually agree with Micko. As Haze said, coverdisks and PD collection series disks should be included, and on the Amiga there were how many Fred Fish disks, for instance? smile
Posted By: incog Re: Fixed software lists - 07/08/10 05:17 PM
I agree with the PD sets, demos, documentation disks and coverdisks too but how should we split? With the latest news coming from The Software Preservation Society should we start listing IPF or stick with ADF?
Posted By: etabeta78 Re: Fixed software lists - 07/08/10 05:18 PM
I'm not against coverdisks at all, but I would suggest to start with properly released software as priority target (especially for external contributions)
Posted By: etabeta78 Re: Fixed software lists - 07/08/10 05:20 PM
Originally Posted By incog
I agree with the PD sets, demos, documentation disks and coverdisks too but how should we split? With the latest news coming from The Software Preservation Society should we start listing IPF or stick with ADF?


IPF is still closed source, so how would we test if MESS cannot support it?
if anything, I'd go with DRAFT, rather than IPF, but I'm not even sure the specs are finalized

hence, I think we should start with .adf
Posted By: R. Belmont Re: Fixed software lists - 07/08/10 05:23 PM
IMO for standard non-protected disks there's no reason not to use existing formats like ADF. Protected commercial disks absolutely should be in DRAFT or a derivative low-level format and the disk emulation modified as necessary to cope with them.
Posted By: incog Re: Fixed software lists - 07/08/10 05:27 PM
Does anybody have a copy of Haze's patches to HazeMD that added CD-i? should be trivial enough to turn that software list into an xml.
Posted By: JoJo Re: Fixed software lists - 07/08/10 05:45 PM
Originally Posted By etabeta78
floppy lists are still under discussion, so please wait until Micko adds one of two working samples wink

EDIT: also, a couple of guidelines:
- put multi-disk games inside a single <software> entry of the list, as separate <part>s (this might change later, but it is the current plan). same goes for different sides of a tape


I agree that multi-tape or multi-disk games should have a separate part for each physical support, but different "sides" of the same support should be grouped together in a new, deeper level of the specification's hierarchy: aside from better documentation, the internal GUI could take advantage of this fact by implementing two different mechanisms to cycle between the images in opposition to "flip" the already mounted image.
Posted By: Haze Re: Fixed software lists - 07/08/10 05:48 PM
they were converted with old versions of CHDMAN, as per RB's recommendations they should probably be reconverted rather than used as-is. Also, I didn't convert every version of every game nor digital video games etc. (seemed rather pointless given i already knew they wouldn't work)

http://mamedev.emulab.it/haze/misc/tinycdi_281009.zip

I thought I had a newer version posted than this, but, can't find it ATM, but seriously, it's worthless for creating the software lists due to format changes.
Posted By: Haze Re: Fixed software lists - 07/08/10 05:53 PM
Originally Posted By etabeta78
Originally Posted By incog
I agree with the PD sets, demos, documentation disks and coverdisks too but how should we split? With the latest news coming from The Software Preservation Society should we start listing IPF or stick with ADF?


IPF is still closed source, so how would we test if MESS cannot support it?
if anything, I'd go with DRAFT, rather than IPF, but I'm not even sure the specs are finalized

hence, I think we should start with .adf


I'd include the IPFs etc. but put ADFs alongside them until somebody comes along to improve the emulation well enough to support the ADFs. Once the IPFs are supported the ADFs can be deprecated and removed if they _match_. For pirate versions there is little point in having anything better than ADFs, because they're unprotected, and along the same lines you won't find ADFs for protected original games because the format is incapable of supporting them.

While it might be a lot of work these databases will just have to evolve with time, like things in MAME do. It's more work right now because you're playing catch-up, and don't really have automated tools to deal with batch conversions or cross-format image comparisons.

Do the software lists have the capability to list additional files which aren't required for emulation, but are known useful references? It would probably be a good idea to CRC/MD5 the various manual scans etc. for reference, even if they're not required to run MESS. A lot of the Amiga/Speccy software is unusable without them, it's rather different to the arcade scene.
Posted By: incog Re: Fixed software lists - 07/08/10 05:54 PM
I agree with JoJo on the flipping of discs, this also elegantly solves a few other odd corner-cases, like the bizarre Xonox double ender carts (http://www.google.com/images?hl=en&q...sa=N&tab=wi) and the easter egg in Karateka that displayed the game upside down if the disc was inserted the wrong way.
Posted By: ASH Re: Fixed software lists - 07/08/10 06:09 PM
so just to clarify something...

the soft list for floppy's and tapes...
will they auto load these files I.E. on the spectrum you select Manic Miner.tzx and M.E.S.S. will insert the tape : auto type LOAD "" [enter] : start tape

or will we still have to learn that bit?
Posted By: Haze Re: Fixed software lists - 07/08/10 06:11 PM
Originally Posted By ASH
so just to clarify something...

the soft list for floppy's and tapes...
will they auto load these files I.E. on the spectrum you select Manic Miner.tzx and M.E.S.S. will insert the tape : auto type LOAD "" [enter] : start tape

or will we still have to learn that bit?


For now I believe you'll still have to learn that bit. I think it's important to get the lists implemented first, thoughts on any kind of scripted automation can come later, once things are more stable, and it will have to be done at a different level to standard inps which are too prone to breaking between versions.
Posted By: etabeta78 Re: Fixed software lists - 07/08/10 06:16 PM
@JoJo and incog: probably you are right about the dual side media (like fds floppies and some tape): then we should use a single <part> and two separate <dataarea> one for side A and one for side B... Ideally, each <part> should represent a separate item (be it a cart, a disc or a cassette) and each <dataarea> should represent a separate component of that specific items (like different sides, or different chips inside the cart)... the dual interface cart example might or might not fit in this settings, it depends on what the system needs to identify which ends of the cart is connected...

@haze: you can list whatever you want in the xml lists as separate <part>s and clrmame will look for it inside the zip, while MESS will only look for whatever the loading routine requires.
I already used this for dipswitches inside NES carts, something which we cannot use right now, but that it was necessary to support (even if I used a <dataarea>, given that they're part of the cart and not a separate thing)
Posted By: etabeta78 Re: Fixed software lists - 07/08/10 06:18 PM
and yeah, I fear automated inputs will have to wait lists to stabilize a bit... I hope whoever hooks up the lists to the driver will post the correspondent instructions here in the forum or on the wiki smile
Posted By: ASH Re: Fixed software lists - 07/08/10 06:30 PM
Originally Posted By Haze
Originally Posted By ASH
so just to clarify something...

the soft list for floppy's and tapes...
will they auto load these files I.E. on the spectrum you select Manic Miner.tzx and M.E.S.S. will insert the tape : auto type LOAD "" [enter] : start tape

or will we still have to learn that bit?


For now I believe you'll still have to learn that bit. I think it's important to get the lists implemented first, thoughts on any kind of scripted automation can come later, once things are more stable, and it will have to be done at a different level to standard inps which are too prone to breaking between versions.


Ok

Just checking smile
Posted By: ht1848 Re: Fixed software lists - 07/08/10 09:48 PM
Originally Posted By Haze
For now I believe you'll still have to learn that bit. I think it's important to get the lists implemented first, thoughts on any kind of scripted automation can come later, once things are more stable, and it will have to be done at a different level to standard inps which are too prone to breaking between versions.


I agree later sounds good, but could the cheat system be able to support something like an "auto-load" cheat where it types a fixed command if enabled? Just thinking out loud. Software list is fist victory.
Posted By: R. Belmont Re: Fixed software lists - 07/08/10 10:01 PM
Cheats can't type, that would be dreadful.
Posted By: Haze Re: Fixed software lists - 07/08/10 10:42 PM
yeah, it's going to need some thought, typing is the least of the problems when you move on to some obscure Japanese mouse driven systems.

likewise, how to deal with Windows based games, I'm assuming there will be some factory install images associated with each set, for at least Windows, installing the entire OS each time for every game you need to test when making changes doesn't make sense from a testing perspective.

Lots of problems to be solved, eventually, but for now good progress can be made by implementing software lists for cases that CAN be handled.
Posted By: ranger_lennier Re: Fixed software lists - 07/08/10 11:20 PM
Originally Posted By etabeta78

my point about multiple software in a single tape/disk was that if you're going to include the image of a floppy disk which was created by my cousin to give me a couple of tiny games, then this kind of image can be thrown away

I've specified this because for some exotic computer a few games appear to be available only inside multigame floppies which do not represent at all a real piece of software and, as such, is not really worth the inclusion...


Do you mean a case where commercial software was pirated onto a compilation, and now only this compilation, not the original, has been archived? In that case, I would want the compilation to be properly documented as the best available copy of the game. It could be removed later if an original was found.
Posted By: mahlemiut Re: Fixed software lists - 07/08/10 11:24 PM
Originally Posted By etabeta78
EDIT: also, a couple of guidelines:
- put multi-disk games inside a single <software> entry of the list, as separate <part>s (this might change later, but it is the current plan). same goes for different sides of a tape


How do you then select each individual image within that piece of software? Currently it only tries to load the first image. That would be rather necessary when it comes to disk swapping.
Posted By: etabeta78 Re: Fixed software lists - 07/09/10 05:41 AM
Originally Posted By mahlemiut
How do you then select each individual image within that piece of software? Currently it only tries to load the first image. That would be rather necessary when it comes to disk swapping.


MIcko has some plan. that's why I suggested this line.

If the plan does not come together or it turns out not to be easy to use, we will split the files in the lists wink

Originally Posted By Haze
likewise, how to deal with Windows based games, I'm assuming there will be some factory install images associated with each set, for at least Windows, installing the entire OS each time for every game you need to test when making changes doesn't make sense from a testing perspective.


I'm not 100% sure we should have any factory install... or you would end up with 'formats' like hdi o whd which are very friendly but do not document how the original software was distributed... MESS might support this kind of formats with the old-style loading but I'm torn about adding them to lists.

while the OS is not a problem (the first time you load up a driver you simply have to install an Operative System into it, then you keep the CHD in your computer to avoid the need of reinstalling it), I don't have a ready answer for the games requiring installation.

at first, the easy solution is to install them the first time you play them, and then not to remove the CHD from your computer to avoid the need of re-installing it... like you would have done with the HD on your old computer smile

Originally Posted By ranger_lennier
Do you mean a case where commercial software was pirated onto a compilation, and now only this compilation, not the original, has been archived? In that case, I would want the compilation to be properly documented as the best available copy of the game. It could be removed later if an original was found.


while I agree about e.g. preserving tape->floppy user-made copies as the best-known image for some games until a proper tape can be preserved, a few days ago I stumbled into some user made disk collections for Japanese computers which consist of 1 or 2 .d88 files containing ~10 games each (many of them taken from tapes). Among these games there are a bunch of them which are not available separately, but I don't think we should include these compilations in our lists: they still remain available in sets like TOSEC if someone badly wants to play the game to see the quality of the conversion, but the software list should only mention the game as undumped (and wanted)

on this topic, I plan to eventually add to Wiki some more list of undumped non-proto carts (and tapes and disks later on) that I'm currently documenting into xml (see e.g. tutor of sordm5)
Posted By: Haze Re: Fixed software lists - 07/09/10 08:25 AM
Originally Posted By etabeta78

at first, the easy solution is to install them the first time you play them, and then not to remove the CHD from your computer to avoid the need of re-installing it... like you would have done with the HD on your old computer smile


It's not an amazingly practical solution tho, is it? There comes a point of making compromises, otherwise we may as well be telling people to mount a bunch of roms on a PCIe card to run anything in MAME, because the original games required physical ROM chips to run etc.

Sometimes if you're blitzing through things wanting to test them out you need a ready-made, stable, usable, reproducable test environment, which usually means clearing out all your CHD diffs, NVrams etc. and in a case like this would result in having to reinstall everything for every game.

In the real world, that doesn't work and you'll just find many things getting broken because nobody has the time to go through that process to test them. It's something that MESS will have to deal with eventually, even it it means some hierarchy of clean OS CHDs and installed-game diffs to test with as a listed alternative to the install media.
Posted By: etabeta78 Re: Fixed software lists - 07/09/10 09:50 AM
"eventually" is the key word wink

especially because no i286-i386 is really working to the point to install a fully featured OS (except maybe for X68000 driver), and the other drivers which comes close to this, i.e. the early IBM PC drivers, have other issues with HDs since a few months and you cannot basically mount any CHD.

you can see this as a long way towards the 'perfectly usable' emulation of PC-like home computers: a journey with many intermediate steps, in my opinion

1. the HD issues mentioned above for early IBM PC shall be ironed out

2. once we have working emulation of hard disk for more DOS-compatible machines, we might want to discuss how to handle HDs:

- on one side we definitely want to have writable (and hence uncompressed) CHD for non-gaming emulation
- on the other side, there are situation when we might want to make a CHD not writable anymore, compress it and start having the changes saved as diffs.

Ideally, chdman might have an option to switch between the two.

3. The final step, which I really have not many ideas about at this stage, would be to think about how to simplify the life to people which only want to play or to devs which want to rapidly test ingame issues

in any case, until step 3, we should simply preserve images of the bootable floppy and CD-ROM rather than hacking up an incomplete CHD-alike format containing pre-installed games...
Posted By: ASH Re: Fixed software lists - 07/09/10 06:20 PM
well here's a suggestion

could we make the software list's region specific

I.E. Snes Pal (Euro) , Snes NTSC (Jpn,USA)

this would help when loading carts instead of the (this cart is designed for use with pal systems only)
Posted By: Duke Re: Fixed software lists - 07/09/10 08:01 PM
Yeah, the US SNES should have a different software list to the EU or Japan SNES (or at least a different interface).
Posted By: etabeta78 Re: Fixed software lists - 07/09/10 11:09 PM
my original plan was to change them to use a different interfaces, but I still have to find the time to do it (as well as to add the other thousands of serial numbers/manufacturers/release dates which waits to be added to nintendo lists wink )
Posted By: ranger_lennier Re: Fixed software lists - 07/10/10 12:01 AM
Originally Posted By etabeta78


Originally Posted By ranger_lennier
Do you mean a case where commercial software was pirated onto a compilation, and now only this compilation, not the original, has been archived? In that case, I would want the compilation to be properly documented as the best available copy of the game. It could be removed later if an original was found.


while I agree about e.g. preserving tape->floppy user-made copies as the best-known image for some games until a proper tape can be preserved, a few days ago I stumbled into some user made disk collections for Japanese computers which consist of 1 or 2 .d88 files containing ~10 games each (many of them taken from tapes). Among these games there are a bunch of them which are not available separately, but I don't think we should include these compilations in our lists: they still remain available in sets like TOSEC if someone badly wants to play the game to see the quality of the conversion, but the software list should only mention the game as undumped (and wanted)


If practical, I don't see a problem with separating an otherwise unarchived game found on an unofficial compilation into a single-game image. This would be no less accurate than the compilation itself, and would avoid lots of repeated games.
Posted By: Haze Re: Fixed software lists - 07/10/10 01:36 AM
Originally Posted By ranger_lennier
Originally Posted By etabeta78


Originally Posted By ranger_lennier
Do you mean a case where commercial software was pirated onto a compilation, and now only this compilation, not the original, has been archived? In that case, I would want the compilation to be properly documented as the best available copy of the game. It could be removed later if an original was found.


while I agree about e.g. preserving tape->floppy user-made copies as the best-known image for some games until a proper tape can be preserved, a few days ago I stumbled into some user made disk collections for Japanese computers which consist of 1 or 2 .d88 files containing ~10 games each (many of them taken from tapes). Among these games there are a bunch of them which are not available separately, but I don't think we should include these compilations in our lists: they still remain available in sets like TOSEC if someone badly wants to play the game to see the quality of the conversion, but the software list should only mention the game as undumped (and wanted)


If practical, I don't see a problem with separating an otherwise unarchived game found on an unofficial compilation into a single-game image. This would be no less accurate than the compilation itself, and would avoid lots of repeated games.


If there is no better available version the compilations should be supported. Test cases are vital as far as accurate emulation is concerned, and sometimes you just have to make use of what's available. The ultimate goal is to produce the best possible, most compatible, most accurate emulator. Anything that helps the cause should be considered. Purposefully avoiding doing so just makes life more difficult.
Posted By: etabeta78 Re: Fixed software lists - 07/10/10 06:26 AM
except that then half of the people which might have the original tape/floppy would simply think that the game is already available and he would not share its (wanted) item...

I don't know... we will eventually see case per case, I guess (and in general if this happens for a system which already have hundreds of dumps available, I don't think that leaving out ~20 more games makes a difference for testing purposes wink )
Posted By: etabeta78 Re: Fixed software lists - 07/10/10 09:12 AM
also, I strongly believe that rather than jumping directly to batch converting existent collections and filling up our lists of new sets for the simple sake of it, we should spend some time improving the new collections *before* adding them to MESS

if a new list simply is a 1:1 copy of an existing set (e.g. taken from TOSEC), then we are not really adding anything neither to the preservation aspect (the collection we based upon is already out there) nor to the emulation aspect because we are possibly adding lots of duplicate sets without knowing exactly what is the best copy available, if any exists...

of course, if any dev is interested in a particular system, then he's more than welcome to add a full list for that system and then to go through the sets by removing/fixing items

however, I don't see a great value in adding to svn 100s of lists which an exact copy of the available sets with some bad dumps removed.

on the other hand, external contributions like k1w1's xegs list and Kaylee's tutor list (and incog's lists as well, of course wink ) show how one can add a lot of valuable info by starting from an existent set and manually improving it based on other existent sources

otherwise, we would simply end up first spreading out tons of unverified images (without anyone actively working on verification) and then having to go back re-doing the work again and again...

and this is why, as of late, I'm spending more time going through the list I added for consoles to add more available info, even if for consoles it was easier since nointro has already a lot of dump verifications at the source wink
Posted By: Curt Coder Re: Fixed software lists - 07/10/10 12:54 PM
Shouldn't multi-floppy images have only 1 entry in the list? Re: MS-DOS 5.0 in r8463.
Posted By: incog Re: Fixed software lists - 07/10/10 01:33 PM
Originally Posted By Curt Coder
Shouldn't multi-floppy images have only 1 entry in the list? Re: MS-DOS 5.0 in r8463.


I will change once this is confirmed, right now I'm dumping all the floppies in my house and noticed that half of them are now sadly unreadable. That is the case with SVGA Utilities and Drivers (Disk 1) for the Amstrad Mega PC, I could only get the second disk to read frown
Posted By: etabeta78 Re: Fixed software lists - 07/10/10 02:50 PM
Originally Posted By incog
I will change once this is confirmed,


who should confirm this? let me know and I will ask him to personally confirm the fact to you wink

otherwise, guidelines are guidelines (and I wrote them after discussing with Micko) wink
Posted By: incog Re: Fixed software lists - 07/10/10 02:52 PM
Where are these guidelines? I was hoping you would confirm curt coders concerns.
Posted By: etabeta78 Re: Fixed software lists - 07/10/10 02:58 PM
http://www.bannister.org/forums/ubbthreads.php?ubb=showflat&Number=63277#Post63277

Originally Posted By etabeta78
also, a couple of guidelines:
- put multi-disk games inside a single <software> entry of the list, as separate <part>s (this might change later, but it is the current plan). same goes for different sides of a tape
- avoid multigame compilations (especially when they are homemade, coverdisks might be eventually added, still to be decided)
- when you have several alt versions (say 5-10 alt vers) of a same game from a same publisher, which are not different revisions, maybe try to play the game itself to see if you can spot differences: the goal is to find the best copy available with these format, not to include every single bit variation which could be due to ruined media...


later amended by Duke

Originally Posted By Duke
Just to clear some confusion maybe, unlike the cartridge software lists, the floppy software lists will have only a few interface types. Like judge said, something like floppy_35, floppy_525. Simple example: You can insert and use (or well, should be able to in the future) IBM PC floppies in the Amiga driver.


and by yourself and JoJo wink

Originally Posted By etabeta78
@JoJo and incog: probably you are right about the dual side media (like fds floppies and some tape): then we should use a single <part> and two separate <dataarea> one for side A and one for side B... Ideally, each <part> should represent a separate item (be it a cart, a disc or a cassette) and each <dataarea> should represent a separate component of that specific items (like different sides, or different chips inside the cart)... the dual interface cart example might or might not fit in this settings, it depends on what the system needs to identify which ends of the cart is connected...

Posted By: incog Re: Fixed software lists - 07/10/10 03:01 PM
Thanks, I'm not arguing for single disks over everything in a zip, I must have just skimmed over that post smile I'll fix the xml in a jiffy.

I've just finished dumping all the floppies I own. Lots of dead ones frown
Posted By: incog Re: Fixed software lists - 07/10/10 04:48 PM
Just added a tiny software list of the floppies I dumped today (90% of them wont read, these are the only ones that still worked) all from original media. These are all 3.5" disks, should a second .xml be made for the 5.25" disk images?

It feels bizarre listing stuff from 2000 for a system in 1984, just shows how long the technology has endured.
Posted By: etabeta78 Re: Fixed software lists - 07/11/10 09:42 PM
time to discuss a bit more the homebrew argument: the einstein list which got created by Kaylee, could include a few titles developed by the creator of the Einstein Reborn website (I have temporarily commented them, but they're still there).

should we include them?
and which publisher should be used for those games (which never got actually published)?
Posted By: Haze Re: Fixed software lists - 07/11/10 10:48 PM
are they useful? will they help with the emulation? do they provide useful test cases?
Posted By: etabeta78 Re: Fixed software lists - 07/12/10 01:47 PM
who knows? the driver has been cleaned up by Duke, but I have no idea if he can be considered the maintainer

and who should decide if some homebrew is useful for orphaned drivers?
Posted By: Haze Re: Fixed software lists - 07/12/10 02:07 PM
Originally Posted By etabeta78
who knows? the driver has been cleaned up by Duke, but I have no idea if he can be considered the maintainer

and who should decide if some homebrew is useful for orphaned drivers?


Good question, however, I doubt you'll find a definitive answer to it.

As long as things are clearly marked I think they can be supported, if somebody wants to pick up a driver, especially an orphaned one with which they're not familiar, then every little helps (unless it's stuff that doesn't work on original hardware, but that's another debate entirely - that stuff definitely should not be supported)

If actively working on a driver it might be worth making a note of some of the useful test cases in the software lists, so that if the driver does become orphaned later people can refer to those too. (For example, some of the SNES edge cases might be worth making a comment about, usually those would go in the source, but with the database being external I guess they belong there, although XML is ugly to read and work with, so I'm not sure.... but you know I never liked the external idea in the first place ;-))

You could easily add a switch to MESS to filter out / ignore anything marked as homebrew completely, as long as it's appropriately tagged in the lists.
Posted By: Kaylee Re: Fixed software lists - 07/14/10 03:01 PM
Did someone change something in the software list?

The Einstein & Tutor Doesn't want to load in clrmamepro. The PV2000 list loads.
Posted By: etabeta78 Re: Fixed software lists - 07/14/10 03:42 PM
the einstein list problem is due to a clrmame issue. I'm waiting for Roman's decision about changing his tool.

the tutor problem should be fixed now (there was some encoding problem at the top of the file) and if it's not, it is due to the same problem in clrmame
Posted By: Curt Coder Re: Fixed software lists - 07/14/10 04:09 PM
Originally Posted By incog
(90% of them wont read) These are all 3.5" disks


Yeah, 3.5" disks are utter crap. I've bought dozens of C64 floppy games on eBay and only 1 or 2 have been broken beyond repair.
Posted By: Quantum Leaper Re: Fixed software lists - 07/16/10 12:08 AM
Originally Posted By Curt Coder
Originally Posted By incog
(90% of them wont read) These are all 3.5" disks


Yeah, 3.5" disks are utter crap. I've bought dozens of C64 floppy games on eBay and only 1 or 2 have been broken beyond repair.


I have to agree with you there, I have 2 small boxes of FD2000 disks for the Commodore 64 and everyone I checked has errors on it. It's a shame, because one of the those disks I really wanted to get the data off. I do have most of the data on some 5.25" disks, so I will just have to find those. Other people I have talked too on the internet, have had no problems with the Commodore drive 3.5" disks but I have a funny feeling they did check all the files on the disk, like I did for my FD2000 disks. I do wonder about other 3.5" disks, like Amiga, Mac, PC and others. I wonder how they have held up over the years.
My 5.25" have almost no errors, which is nice.
Posted By: R. Belmont Re: Fixed software lists - 07/16/10 12:29 AM
I haven't had any problem with my Apple IIgs 3.5" disks. The Sony drive mechs from back then that Apple used were pretty solid.
Posted By: Duke Re: Fixed software lists - 07/16/10 07:18 AM
Same with Amiga, I can read my old disks just fine. But I've had lots and lots of problems with PC 3.5" disks, they are very fragile.
Posted By: Haze Re: Fixed software lists - 07/16/10 10:43 AM
guess it depends how they were kept too.. Most of my amiga disks are dead, many were dead 10 years ago ;-)
Posted By: Quantum Leaper Re: Fixed software lists - 07/16/10 09:25 PM
Originally Posted By Haze
guess it depends how they were kept too.. Most of my amiga disks are dead, many were dead 10 years ago ;-)


Maybe but my 3.5" disks were dieing even when I was still using the drive, I didn't buy cheap bulk disks, but name brands, since I never went though as many. My 3.5" were kept in a plastic hard sided with cover 3.5" disk box, unlike my 5.25" which were kept in wooden boxes without a lid. All the 3.5" disks I still read the directory, but when I got to access the data, and programs, I get errors. I used to regularly clean the drive heads also, since I had access to about 20 cleaning kits from my former Commodore Club. Even the utilities disk that came with the FD2000 drive died.
The only 5.25" that I found that had error was my updated Buddy 128 Assembler which I got from the Author but the error is only in the file editor for sequential source code files, if I remember correctly.
Posted By: Anna Wu Re: Fixed software lists - 07/17/10 04:43 PM
Barry, I add the game Xevious in the software list (fm77av)
because I like the game. smile

Code:
	
  <software name="xevious">
		<description>Xevious</description>
		<year>1985</year>
		<publisher>Dempa</publisher>
		<part name="disk1" interface="fm7_flop">
			<dataarea name="flop" size="380448">
				<rom name="xevious.d77" size="380448" crc="abd96ccc" sha1="1c6bb9fad36730654fbfd11fa3e06d24fd63a1bc" 
offset="000000" />
      </dataarea>
    </part>
  </software> 



Posted By: mahlemiut Re: Fixed software lists - 07/17/10 11:33 PM
It's not FM-77AV only, as far as I know. I'm going to do a separate list for FM-7 disk software (I'm working on a cassette list currently, there's reasonable amount of them).
Posted By: Anna Wu Re: Fixed software lists - 07/18/10 06:10 AM
Originally Posted By mahlemiut
It's not FM-77AV only, as far as I know. I'm going to do a separate list for FM-7 disk software (I'm working on a cassette list currently, there's reasonable amount of them).


Agree, it make more sense.
If you need any help, please tell me.
Posted By: Anna Wu Re: Fixed software lists - 07/19/10 12:24 PM
X1
Ancient Ys Vanished Omen
Just a test and it is working.

Code:
<?xml version="1.0"?>
<!DOCTYPE softwarelist SYSTEM "softwarelist.dtd"> 


<!--
	FIXME: crc / sha1 info
-->


<softwarelist name="x1" description="Sharp X1 disk images">
	<software name="177">
		<description>177</description>
		<year>1986</year>
		<publisher>Macadamia</publisher>
		<part name="disk1" interface="x1_flop">
			<dataarea name="flop" size="327680">
				<rom name="177.2d" size="327680" crc="0" sha1="0" offset="000000" />
			</dataarea>
		</part>
	</software>

	<software name="1942">
		<description>1942</description>
		<year>198?</year>
		<publisher>Capcom?</publisher>
		<part name="disk1" interface="x1_flop">
			<dataarea name="flop" size="327680">
				<rom name="1942.d88" size="348848" crc="0" sha1="0" offset="000000" />
			</dataarea>
		</part>
	</software>
	
		<software name="ys1">
		<description>Ancient Ys Vanished Omen</description>
		<year>1987</year>
		<publisher>Falcom</publisher>
		<part name="disk1" interface="x1_flop">
			<dataarea name="flop" size="415840">
				<rom name="ys_x1a.d88" size="415840" crc="b439fdc2" sha1="1fe34ce088740b348145008997dd65dd8050110f" offset="000000" />
			</dataarea>
		</part>
		<part name="disk2" interface="x1_flop">
			<dataarea name="flop" size="415840">
				<rom name="ys_x1b.d88" size="415840" crc="04bcaf77" sha1="26a6a06af93108a1d67fe40d22497825764e6a48" offset="000000" />
			</dataarea>
		</part>
	</software>
	
</softwarelist>
Posted By: etabeta78 Re: Fixed software lists - 07/19/10 03:48 PM
a few words on the x1 lists (and on the pc8801 one which should be ready during this week): I had started them to mainly put some order in the various folders I collected over time... descriptions are known to be inaccurate, but there might also be bigger issues, like multiple disks for a given game listed as separate entries or different games merged into a single entry, and some alt versions might be simply d88->2d conversions, or images which only differ for the data which are written by emulators on the disks...

please feel free to report any mistake

some more X1 disks will be added when I arrive at home tonight, while I don't plan to add user-made gamepacks...
Posted By: Anna Wu Re: Fixed software lists - 07/19/10 04:35 PM
I think a complete software list for the PC 8801 will be a long term project. smile

Certainly a lot of disk images exist, which you can not find in the TOSEC sets.
Posted By: etabeta78 Re: Fixed software lists - 07/19/10 04:42 PM
yeah, I have tons of files in addition to TOSEC (even if not as many as for the pc9801 wink ), but also the list is quite advanced.

it depends on how much work I'll have...
Posted By: Anna Wu Re: Fixed software lists - 07/19/10 04:50 PM
Originally Posted By etabeta78
yeah, I have tons of files in addition to TOSEC (even if not as many as for the pc9801 wink ), but also the list is quite advanced.

it depends on how much work I'll have...


Yes, the amount of the pc9801 software is incredible. wink
Additional, I made for my self a lot of Harddisk images (Win31/Win95, Games), Bootdisks for CD images etc.)



Posted By: Haze Re: Fixed software lists - 07/19/10 05:08 PM
Originally Posted By etabeta78
yeah, I have tons of files in addition to TOSEC (even if not as many as for the pc9801 wink ), but also the list is quite advanced.

it depends on how much work I'll have...


That's why it's important for MESS to document everything, even if it takes some time.

The MESS lists also have the huge advantage over Tosec in that they can be leveraged by others, and describe things the software needs making support / implementation for others easier too :-) Good to see lists for some of the more obscure things being added, will help a lot I think.
Posted By: etabeta78 Re: Fixed software lists - 07/19/10 05:48 PM
/me hopes so smile
Posted By: Anna Wu Re: Fixed software lists - 07/19/10 05:52 PM
x1_flop

eta, are you sure the disk range for ys is correct ?
The save disk (disk1) is not bootable.

3 > 1
2
1 > 3

from:

Code:
<software name="ys">
		<description>Ancient Ys Vanished Omen</description>
		<year>1987</year>
		<publisher>Falcom</publisher>
		<part name="disk1" interface="x1_flop">
			<dataarea name="flop" size="348848">
				<rom name="user.d88" size="348848" crc="97e1d16a" sha1="cfcf4d849b90fe99c4f300d03ca9b9faeae0ad97" offset="0" />
			</dataarea>
		</part>
		<part name="disk2" interface="x1_flop">
			<dataarea name="flop" size="415840">
				<rom name="ys_a.d88" size="415840" crc="b439fdc2" sha1="1fe34ce088740b348145008997dd65dd8050110f" offset="0" />
			</dataarea>
		</part>
		<part name="disk3" interface="x1_flop">
			<dataarea name="flop" size="415840">
				<rom name="ys_b.d88" size="415840" crc="04bcaf77" sha1="26a6a06af93108a1d67fe40d22497825764e6a48" offset="0" />
			</dataarea>
		</part>
	</software>


to:

Code:
<software name="ys">
		<description>Ancient Ys Vanished Omen</description>
		<year>1987</year>
		<publisher>Falcom</publisher>
		<part name="disk1" interface="x1_flop">
			<dataarea name="flop" size="415840">
				<rom name="ys_a.d88" size="415840" crc="b439fdc2" sha1="1fe34ce088740b348145008997dd65dd8050110f" offset="0" />
			</dataarea>
		</part>
		<part name="disk2" interface="x1_flop">
			<dataarea name="flop" size="415840">
				<rom name="ys_b.d88" size="415840" crc="04bcaf77" sha1="26a6a06af93108a1d67fe40d22497825764e6a48" offset="0" />
			</dataarea>
		</part>
		<part name="disk3" interface="x1_flop">
			<dataarea name="flop" size="348848">
				<rom name="user.d88" size="348848" crc="97e1d16a" sha1="cfcf4d849b90fe99c4f300d03ca9b9faeae0ad97" offset="0" />
			</dataarea>
		</part>
	</software>
Posted By: etabeta78 Re: Fixed software lists - 07/19/10 06:05 PM
it can be, I will change the order when I add the few remaining disks I have...

to workaround the problem, you can also load the proper file in TAB > File Manager > Floppy Disk by selecting the correct disk image inside the zipfile

Notice that there might more multi-disk games which suffer of the same problem (I caught a few while testing, but I haven't tested all the games): all of these will be fixed by Micko's future improvements to disk handling, but in the meanwhile I will fix the order for the games which will be reported (so that we don't need to wait for Micko wink )
Posted By: Anna Wu Re: Fixed software lists - 07/19/10 06:07 PM
Originally Posted By etabeta78
it can be, I will change the order when I add the few remaining disks I have...

to workaround the problem, you can also load the proper file in TAB > File Manager > Floppy Disk by selecting the correct disk image inside the zipfile

Notice that there might more multi-disk games which suffer of the same problem (I caught a few while testing, but I haven't tested all the games): all of these will be fixed by Micko's future improvements to disk handling, but in the meanwhile I will fix the order for the games which will be reported (so that we don't need to wait for Micko wink )


No need, I modified my xml file already and it works. smile
Posted By: judge Re: Fixed software lists - 07/19/10 06:11 PM
Depending on the game it could be that the save disk only contains save data and no actual game data.
If that's the case then they should not be in the list imo, the user should be able to create them him/herself.
Posted By: Anna Wu Re: Fixed software lists - 07/19/10 06:14 PM
Originally Posted By judge
Depending on the game it could be that the save disk only contains save data and no actual game data.
If that's the case then they should not be in the list imo, the user should be able to create them him/herself.


I absolute agree. Save data disks are not neccessary and the user can make for his own.
Posted By: Haze Re: Fixed software lists - 07/19/10 06:17 PM
Originally Posted By Anna Wu
Originally Posted By judge
Depending on the game it could be that the save disk only contains save data and no actual game data.
If that's the case then they should not be in the list imo, the user should be able to create them him/herself.


I absolute agree. Save data disks are not neccessary and the user can make for his own.


Some games shipped with save disks tho. Frontier Elite II on the Amiga did, it had several premade saves on it, although everybody just used their own save disk anyway ;-)
Posted By: Anna Wu Re: Fixed software lists - 07/19/10 06:22 PM
Originally Posted By Haze
Originally Posted By Anna Wu
Originally Posted By judge
Depending on the game it could be that the save disk only contains save data and no actual game data.
If that's the case then they should not be in the list imo, the user should be able to create them him/herself.


I absolute agree. Save data disks are not neccessary and the user can make for his own.


Some games shipped with save disks tho. Frontier Elite II on the Amiga did, it had several premade saves on it, although everybody just used their own save disk anyway ;-)


If a software producer add a save data disk, which is/was very seldom, it is more convenient but not really neccessary.
Posted By: etabeta78 Re: Fixed software lists - 07/19/10 06:29 PM
when info is available they can be added (if dumped) or removed accordingly...
Posted By: Anna Wu Re: Fixed software lists - 07/20/10 07:45 AM
Barry, maybe it is better to call the FM7 cassette software list > fm7_cass.xml and the coming FM7 disk software list > fm7_flop.xml ?
Posted By: mahlemiut Re: Fixed software lists - 07/20/10 08:35 AM
I was thinking that you could have more than one software list in an XML file, but I could certainly keep them separate if that is preferred.
Posted By: Anna Wu Re: Fixed software lists - 07/20/10 08:39 AM
FM7.xml

Barry, pls can you check ...

<software name="blacko">

<rom name="azusa.T77" size="16" crc="24B4D6C4" sha1="0F397F343CC74993777E10DEFFE6F9579713FB8E"

<rom name="azusa.T77" size="42132" crc="E4459A46" sha1="53903CD0F6C8BB578B12A2A4F4B0A5D57AF11C46"

Twice the same rom name, can this work ?
Posted By: etabeta78 Re: Fixed software lists - 07/20/10 08:40 AM
Originally Posted By mahlemiut
I was thinking that you could have more than one software list in an XML file, but I could certainly keep them separate if that is preferred.


they can be kept in the same xml file, but in that case you cannot use the same shortname or description for floppies vs. tapes versions of the same game (or MESS validation checks would complain)

that's the only reason I prefer to keep them separated, rather than together
Posted By: etabeta78 Re: Fixed software lists - 07/20/10 08:41 AM
Originally Posted By Anna Wu
FM7.xml

Barry, pls can you check ...

<software name="blacko">

<rom name="azusa.T77" size="16" crc="24B4D6C4" sha1="0F397F343CC74993777E10DEFFE6F9579713FB8E" offset="000000" />

<rom name="azusa.T77" size="42132" crc="E4459A46" sha1="53903CD0F6C8BB578B12A2A4F4B0A5D57AF11C46"

Twice the same rom name, can this work ?


well, it can work of course, there is no general reason why it couldn't... but in this case there is probably something wrong (maybe the 16bytes file was a readme?) wink
Posted By: Anna Wu Re: Fixed software lists - 07/20/10 08:43 AM
Originally Posted By etabeta78
Originally Posted By Anna Wu
FM7.xml

Barry, pls can you check ...

<software name="blacko">

<rom name="azusa.T77" size="16" crc="24B4D6C4" sha1="0F397F343CC74993777E10DEFFE6F9579713FB8E" offset="000000" />

<rom name="azusa.T77" size="42132" crc="E4459A46" sha1="53903CD0F6C8BB578B12A2A4F4B0A5D57AF11C46"

Twice the same rom name, can this work ?


well, it can work of course, there is no general reason why it couldn't... but in this case there is probably something wrong (maybe the 16bytes file was a readme?) wink


The 16 byte as data file exist.
Content: XM7 TAPE IMAGE 0

Maybe a data file for the XM7 emulator.
Posted By: mahlemiut Re: Fixed software lists - 07/20/10 01:02 PM
Many of the tape zips that I have have numerous levels of encoded gibberish folders in them (they are converted from different archive formats), so it's certainly possible there are multiple files with the same filename. Thankfully, software lists ignore folders in zips. I didn't attempt to do too much with zips that have multiple images since software lists don't implement using them yet. The list was initially created with a simple C# app that I wrote which went through a folder containing zips, and created entries for those zips that had T77 files in them, with CRC/SHA1 added for each image found. Then I went through each item, adding the description/title/shortname/manufacturer where possible.
Posted By: Anna Wu Re: Fixed software lists - 07/21/10 09:07 AM
Barry please check the game " salada " in the FM7 software list.

Beside the obscure file name " 1&#9577;NAf[^.T77 " (even I use a capable japanese font, it not display correct the characters), the range in the software list is not correct.



Quote:
<!-- &#12469;&#12521;&#12480;&#12398;&#22269;&#12398;&#12488;&#12510;&#12488;&#23019;.zip -->
<software name="salada">
<description>&#12469;&#12521;&#12480;&#12398;&#22269;&#12398;&#12488;&#12510;&#12488;&#23019;</description>
<year>19??</year>
<publisher>Unknown</publisher>
<part name="cass1" interface="fm7_cass">
<dataarea name="cass" size="34030">
<rom name="1&#9577;NAf[^.T77" size="34030" crc="BB3AD189" sha1="1A959989704B3616E9E9EA8112595A86E6E2936C" offset="000000" />
</dataarea>
</part>
<part name="cass2" interface="fm7_cass">
<dataarea name="cass" size="4538496">
<rom name="salada1.t77" size="4538496" crc="8A378BEC" sha1="00FB9BD58CACA28669F8249E8A84621A41A711DE" offset="000000" />
</dataarea>
</part>
<part name="cass3" interface="fm7_cass">
<dataarea name="cass" size="4628654">
<rom name="salada2.t77" size="4628654" crc="9F71B75D" sha1="289C5CBD54649ABA25616E2F53E2CE3CDAF10B2F" offset="000000" />
</dataarea>
</part>
</software>

The file for cass1 > 1&#9577;NAf[^.T77 ( I call it data.t77) is only a save data image (created for using on the XM7 emulator) and content not a loader. So it can not work at once on MESS.

The range shall be :
1 > 3
2 > 1
3 > 2

I changed the range and it works.



PS: Here you can see the name , publisher and release date.
Posted By: mahlemiut Re: Fixed software lists - 07/21/10 09:36 PM
Yeah, I didn't check multi-image zips much, since they aren't yet supported through software lists. I plan to look at multi-image software at some point when some decision is made on how they'll be handled.

Should probably fix up the filenames, though. smile
Posted By: Kaylee Re: Fixed software lists - 07/22/10 09:14 AM
FM7 list gives error in cmp:

Code:
Blacko:

<part name="cass1" interface="fm7_cass">
			<dataarea name="cass" size="16">
				<rom name="azusa.T77" size="16" crc="24B4D6C4" sha1="0F397F343CC74993777E10DEFFE6F9579713FB8E" offset="000000" />
			</dataarea>
		</part>
<part name="cass9" interface="fm7_cass">
			<dataarea name="cass" size="42132">
				<rom name="azusa.T77" size="42132" crc="E4459A46" sha1="53903CD0F6C8BB578B12A2A4F4B0A5D57AF11C46" offset="000000" />
			</dataarea>
		</part>

Denzeni:
		<part name="cassa" interface="fm7_cass">
			<dataarea name="cass" size="34030">
				<rom name="savedisk.T77" size="34030" crc="5901992E" sha1="8BC20F85750E9FC2C322EE313398F741A62D9A32" offset="000000" />
			</dataarea>
		</part>

		<part name="cass4" interface="fm7_cass">
			<dataarea name="cass" size="68044">
				<rom name="savedisk.T77" size="68044" crc="9BC582F5" sha1="3E83BF18ADA35E17CB8B45673D879B55EF32066F" offset="000000" />
			</dataarea>
		</part>
Posted By: etabeta78 Re: Fixed software lists - 07/22/10 09:34 AM
ignore it for the moment... clrmame will simply add a _0 to the second file
Posted By: etabeta78 Re: Fixed software lists - 07/26/10 04:15 PM
just to keep record of the progresses: all lists are currently hooked up but

- a800
- ibm5170
- lisa
- megapc
- pc8001_cass
- pc8001_flop

did I miss any?
Posted By: ASH Re: Fixed software lists - 07/30/10 11:54 AM
I am starting to implement the software list (I.E. renaming my software using clrmamepro and putting them in my roms folder)

Is there a chance to add a checker...

by this I mean that the software list also has a (Found or Missing) next to each rom/tape/disc so we know if we are able to load that file. and we would also know if we have some missing roms/tapes/floppies. smile

the found or missing could also be anything from a X or faint italics
up to you lot but with a large amount of software it would stop you selecting a program you don't have (which it allows at the moment)

Posted By: k1w1 Re: Fixed software lists - 08/04/10 10:42 AM
Originally Posted By etabeta78
just to keep record of the progresses: all lists are currently hooked up but

- a800
- ibm5170
- lisa
- megapc
- pc8001_cass
- pc8001_flop

did I miss any?


The a7800 is not hooked up either.

k1w1
Posted By: Anna Wu Re: Fixed software lists - 08/08/10 09:37 AM
NEC PC-6001/mk2/mk2SR/6601/6601SR softwarelists

I suggest to add a new line, if it is possible.

Example for the pc6001m2_cass (RAM snaphot/Cartridge/whatever) list:

Quote:
<software name="thundfrc">
<description>Thunder Force</description>
<year>1983</year>
<publisher>Tecno-Soft</publisher>
<mode>basic mode="5" page="3"</mode>
<part name="xxxxx" interface="xxx_xxxx">
<dataarea name="xxx" size="xxxxxx">
<rom name="xxx.xxx" size="xxxxxx" crc="xxxxxxxx"
sha1="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
offset="000000" />
Posted By: etabeta78 Re: Fixed software lists - 08/08/10 10:15 AM
at the moment all the info necessary to launch a game are added as comments (see e.g. vic1001_cart.xml and the FM7 lists)

Indeed there is no way for MESS to automatically setup anything just from reading software lists: it's always the end user which has to take care for that
Posted By: Anna Wu Re: Fixed software lists - 08/08/10 10:28 AM
I see, so better as normal comment like

<!-- Enter "Mode 5" and "Page 3" to run ..... -->
Posted By: Drucifer MESSUI issues with software list fields - 08/21/10 03:17 PM
When I customize the fields in the Software list through the GUI, the next time MESSUI is launched there are some anomalies with the fields. For example, when I removed the List field, the publisher field was shown twice (or more). The default settings for mess_column_shown in MESSUI.ini include quotes around the settings. It seems that when the fields are changed, the quotes are lost and some extra values are duplicated. Here's the settings in MESSUI.ini.

Before I removed the List field(default settings):
mess_column_order 0,1,2,3,4,5,6,7
mess_column_shown "1, 1, 1, 1, 1, 0, 0, 0"

After I removed the List field:
mess_column_order 0,2,3,4,4,6,7,1
mess_column_shown 1,0,1,1,1,0,0,0

As you can see, CRC (5) is no longer available through the GUI, and Publisher (4) is listed and displayed twice.
Posted By: Kale Re: MESSUI issues with software list fields - 09/01/10 12:17 AM
Guess that at this point I can express my thoughts about the software list stuff, specifically:

1) Why the size is expressed in decimal units?
2) Why nothing moans if I put a deliberately bad image (for crc / sha1 checking)? I would imagine that no crc / sha1 is checked at all?
3) Last but not least, I still don't basically get why we should do things different than MAME in this specific area ... MAME rom loading system isn't even that rocket science, why for example:

Code:
	<software name="excitebk">
		<description>Excite Bike</description>
		<year>1985</year>
		<publisher>Nintendo / Hudson Soft</publisher>

		<part name="disk1" interface="mz2500_flop">
			<dataarea name="flop" size="1">
				<rom name="excite bike.d88" size="98624" crc="657367ed" sha1="1c15ffa591c0de997694da691a5f3acb301463c0" offset="0" />
			</dataarea>
		</part>
	</software>


Is really that easy compared to ...

Code:
	ROM_LOAD( "excite bike.d88", 0x00000, 98624, CRC(657367ed) SHA1(1c15ffa591c0de997694da691a5f3acb301463c0) )

GAME( 1985, excitebk, "Nintendo / Hudson Soft", "Excite Bike")


... I mean, it's the second natural thing that you learn from MAME (first one being the input sub-system) ...

So, with these doubts in mind, I would at least like to know what kind of system I should use to create a fast xml list ...
Originally Posted By Kale
... I mean, it's the second natural thing that you learn from MAME (first one being the input sub-system) ...


Uh, because the plan is to make it *extensible*, and a fixed macro like in MAME absolutely is not?
Posted By: Kale Re: MESSUI issues with software list fields - 09/01/10 01:55 AM
But that doesn't mean the xml is user friendly with current hookup, which is basically my pov ...
It's significantly too late to complain about the basics of the format at this point. Haze had his input back when it mattered (and given your close relationship with Haze presumably his opinions are yours as well), and the format's now been set for something like a year.

If you truly find it too hard, let other people create the lists or create an editor/tool that makes it more friendly (e.g. import/export .csv software lists would be very useful).
xml allows lists to be fully external which has several positive sides:
- faster linking
- smaller exe size
- you can update lists without updating the exe (e.g. if you stick to the last full release)

nothing of this was possible with rom macro like MAME

Quote:
1) Why the size is expressed in decimal units?


this is the same in mame -lx output. Curt (or Judge or both) complained on the MAME list last winter, but I don't remember if there was any reason to keep it, except for backward compatibility with frontends which are not updated anymore

Quote:
2) Why nothing moans if I put a deliberately bad image (for crc / sha1 checking)? I would imagine that no crc / sha1 is checked at all?


I thought it is checked, but it seems it's not. as soon as Judge finds time to fully merge list loading with internal rom loading we should get this for free
Posted By: Haze Re: MESSUI issues with software list fields - 09/01/10 12:18 PM
Originally Posted By R. Belmont
It's significantly too late to complain about the basics of the format at this point. Haze had his input back when it mattered (and given your close relationship with Haze presumably his opinions are yours as well), and the format's now been set for something like a year.

If you truly find it too hard, let other people create the lists or create an editor/tool that makes it more friendly (e.g. import/export .csv software lists would be very useful).


Well I can't say I was a huge fan of the xml thing from the outset, xml leads to ugly, bloated-looking hard-to-read data files which are much harder for the human eye to scan than the MAME macros. However, the argument to keep them internal was already killed off by that point, so there wasn't much choice.

I guess if it was in my hands, then something which looked like the MAME macros, but got compiled into XML by a custom tool would probably have been my preferred option if things had to stay external. That would have bridged the gap between MAME's easy to read macros, and the supposed advantages ETA points out (the last of which I find highly debatable, and given the close tie between exe and lists I'd consider a _major_ disadvantage instead. The lists represent compatibility for a given exe version, if something gets marked as working due to core improvements then it's not going to work if you didn't update the exe...)

Anyway, I'm past arguing on this side of things now, you have what you have. I'm more concerned that basic functionality is still severely lacking (eg. as Kale points out, it doesn't really validate anything, and as I've said many times you have no best-match system and the forced folder structure is still damn annoying if you just want to do something quick)
Posted By: mizapf Re: MESSUI issues with software list fields - 09/01/10 01:33 PM
@Kale: It's true that the example you pointed out is not that convincing. I guess the reason for the XML structure goes back to the rompack structure (RPK) for cartridges whose design I was also involved in. Software lists were realized some months later.

In the RPKs, a layout.xml file specifies the set of ROM dumps to be loaded, how they are assignment to memory regions, also whether there is RAM in the cartridge, how large, NVRAM or normal, and the PCB type. According to the PCB type you can quite different setups with different numbers of ROMs or RAMs. In the early design, more options like locale settings or different configurations for one cartridge were envisaged, but did not find their way into the known format.

As I pointed out some time ago, not all features have been ported to software lists yet; instead some other metadata was included like manufacturer, year and so on. This means that in some cases, when not all of the original features are required, it may not be obvious why the information in the list could not be represented by a macro in a more concise way.

Although I use to be sceptical about XML formats, at least for the RPKs it is really useful; you can plan for later amendments and also include system-specific information without breaking the whole format.

It's not a question of ease in my view.

Michael
Posted By: ReadOnly Re: MESSUI issues with software list fields - 09/11/10 09:10 AM
For some systems such as the super famicom or mega drive, will the MESS software list use split roms as in the pcbs or joined roms. If you're going to use split roms then I might help you by providing pcb scans.

BTW, stick this thread please.
Posted By: Duke Re: MESSUI issues with software list fields - 09/11/10 11:57 AM
It will use whatever is on the real PCB. See the NES software list for an example.
Posted By: ReadOnly Re: MESSUI issues with software list fields - 11/05/10 06:09 PM
Originally Posted By Duke
It will use whatever is on the real PCB. See the NES software list for an example.
I just checked the snes xml and it's nothing like the real PCB, more like a rip of goodsnes joint format, and I see the same underdumps from goodsnes too, some 12MB roms should be 16MB for example. I own about 1500 snes cartridges and have scanned hundreds of PCBs, so trust me I know what I'm talking about.

Also what is this Jpn and Euro crap, you are too lazy to type Japan or Europe? =______=

Even the chaos in MAME's lack of naming convention is looking better than that.

I am just going to have to ignore these MESS software lists if things don't improve.
Originally Posted By MESSfan
Originally Posted By Duke
It will use whatever is on the real PCB. See the NES software list for an example.
I just checked the snes xml and it's nothing like the real PCB, more like a rip of goodsnes joint format, and I see the same underdumps from goodsnes too, some 12MB roms should be 16MB for example. I own about 1500 snes cartridges and have scanned hundreds of PCBs, so trust me I know what I'm talking about.


If I would have no real work and no private life, you would already have much better lists for non-NES nintendo consoles. Since I have both, lists are slightly down in my priorities (but still at top of my hobbyist projects) and current xml lists for gb, gbc, gba and snes are mostly imported from nointro (in sync until 2 months ago, now a bit outdated)

Originally Posted By MESSfan
Also what is this Jpn and Euro crap, you are too lazy to type Japan or Europe?

Even the chaos in MAME's lack of naming convention is looking better than that.


No laziness. I just liked more (and I still prefer) to use only 3 letters for countries (e.g. like country codes in Naomi carts or at Olympic games) and Alt for alternate dumps. Titles are already long enough without adding tons of "(notes)"

Originally Posted By MESSfan
I am just going to have to ignore these MESS software lists if things don't improve.


or you could try to help to improve lists... we are definitely interested to improve the SNES pcb documentation
Posted By: Haze Re: MESSUI issues with software list fields - 11/05/10 10:00 PM
Originally Posted By MESSfan
Originally Posted By Duke
It will use whatever is on the real PCB. See the NES software list for an example.
I just checked the snes xml and it's nothing like the real PCB, more like a rip of goodsnes joint format, and I see the same underdumps from goodsnes too, some 12MB roms should be 16MB for example. I own about 1500 snes cartridges and have scanned hundreds of PCBs, so trust me I know what I'm talking about.


When this information is available, and proper dumps are made, they will be use. Ignoring what already exists is suicidal as far as giving a working set of test material is concerned. It's called evolution, it's how emulators develop.

Originally Posted By MESSfan

Also what is this Jpn and Euro crap, you are too lazy to type Japan or Europe? =______=


Preference? It's easy to understand exactly like that.

Originally Posted By MESSfan

Even the chaos in MAME's lack of naming convention is looking better than that.


MAME's naming system comes down to the individual devs and drivers, unsurprisingly so does the MESS one.

Originally Posted By MESSfan

I am just going to have to ignore these MESS software lists if things don't improve.


Ignore them then, if you have no real constructive input.

They exist to document the software, and help the developers.

You're probably going to be horrified when I batch CHD the PCE-CD and Sega-CD dumps I have, because AFAIK none of them are tosec verified or anything else. Again, these can be replaced when better dumps or available, I don't have the better dumps, so I'll use the ones I have to make the initial list. Again, evolution.
Originally Posted By MESSfan
I own about 1500 snes cartridges and have scanned hundreds of PCBs, so trust me I know what I'm talking about.

Are you able to dump those SNES carts? If so, that would be a great start to getting the SNES list updated with proper dumps. As is it, you've gotta start somewhere...
Posted By: mock Re: MESSUI issues with software list fields - 11/06/10 01:46 AM
Could the partial dump of Sonic Spinball that drx posted be added to megadriv.xml? It's not playable but it's important for preservation.

The "unknown" ROM seems to be number 4 incidentally.
Posted By: Haze Re: MESSUI issues with software list fields - 11/06/10 12:31 PM
Originally Posted By mock
Could the partial dump of Sonic Spinball that drx posted be added to megadriv.xml? It's not playable but it's important for preservation.

The "unknown" ROM seems to be number 4 incidentally.


It's debatable really.. vital code roms missing on a one-of-a-kind prototype make it a little bit useless.. for anything? It's never going to run, and they're never going to be found, and it probably only ever existed on a single board somewhere. You can't even really analyse much in the way of what changed in that revision because important parts of the code are missing.

It could be added.. but.. is it worth it for a dump in that state? There are things that never got added to MAME for similar reasons.
Posted By: Anna Wu Re: Fixed software lists - 11/11/10 04:47 PM
Dear Micko,

you released a CDI softlist as example for adding CHD's.
I know, its a example only but please let me tell you. smile

The CRC is for the Frog Feast Demo only. The full version must have a different checksum.

Description: Frog Feast (Demo)
Publisher: Rastersoft
Year: 2007

Posted By: ReadOnly Re: MESSUI issues with software list fields - 11/12/10 03:39 PM
Originally Posted By Haze
When this information is available, and proper dumps are made, they will be use.
What do you believe you know a proper dump is? Please answer carefully, and consider there are probably many things I know and you don't about Nintendo proms.
Posted By: Haze Re: MESSUI issues with software list fields - 11/12/10 04:08 PM
Originally Posted By MESSfan
Originally Posted By Haze
When this information is available, and proper dumps are made, they will be use.
What do you believe you know a proper dump is? Please answer carefully, and consider there are probably many things I know and you don't about Nintendo proms.


Please cut the bullshit.

MAMEdev (and MESSdev) are quite aware of what proper dumps are.

Your "I'm better than you" attitude will just lead to everybody ignoring you, starting with me, right now.

Originally Posted By MESSfan
Please answer carefully, and consider there are probably many things I know and you don't about Nintendo proms.


Do you ever not behave like a petulant child?
Originally Posted By MESSfan
What do you believe you know a proper dump is? Please answer carefully, and consider there are probably many things I know and you don't about Nintendo proms.


start sharing the PCB informations, with info about
* which games use which PCBs,
* which chips are present on the various pcbs (including sizes and content compared to the available dumps),
* which variants exist (I expect some games exist in multiple chips size, depending from manufacturing cost and chip availability in the factory) [1]

and we will add the info to the xml lists as soon as we can, as we have done with NES & megadrive (if you look at megadriv.xml, you can find f22a, fifa97, jb007, jpond2, lotust, nhl94, pga2a, riserbot and splat2j which all come from real carts, dumped chip by chip and in the correct byte order).

of course, we will need a long time to fill all the lists with correct dumps, but we want to do it

[1] in this latter case, we might start documenting a single dump, mentioning in the source that alternate versions exist.
Posted By: Anna Wu Re: Fixed software lists - 11/12/10 05:04 PM
Be more constructive and objective please, MESSfan.
Posted By: ReadOnly Re: Fixed software lists - 11/12/10 05:26 PM
@etabeta
I am working on it
If you are interested I have also infos about
1. how to figure out the size of the chip from what is written on it
2. prom dumps tests results (as opposed to cart dump) very interesting to
a. verify the ROM image size from PCB scans (and fix rampant underdumps)
b. rebuild a "proper dump" from existing dumps with PCB scans (all the known variants at least, I have seen up to three variants)

@Anna
You see nothing wrong with Haze attitude?
Posted By: Haze Re: Fixed software lists - 11/12/10 05:58 PM
Originally Posted By MESSfan
@Anna
You see nothing wrong with Haze attitude?


If you're going to keep talking down on a team of people who have been doing this for 15+ years, coming from a scene where traditionally nobody has given a shit about doing things correctly... (and had they, we wouldn't have this mess in the first place)

You're the one who started insulting the team(s), questioning their ability, and talking down on them, and believe me, I wasn't just speaking for myself with that comment.

For CDs and floppies, I concede there are people who can handle it better than the current dev teams, for ROMs? It's not rocket science, and with time things will be done correctly.

I get stuff done, and I expect others to do the same. I don't have time for egos and such. There are ways you can be productive, and ways you can't, you weren't being productive.

Getting things done is more important than endless bureaucracy and treading water with pointless changes and delaying things to the point where no actual progress is made. Evolution takes care of the rest.

If you know things about Nintendo ROMs, post it. If you don't, shut up. In the meantime we'll do what we can.
Posted By: etabeta78 Re: Fixed software lists - 11/12/10 06:17 PM