Home Page

Great news for Genesis/MD

Posted By: R. Belmont

Great news for Genesis/MD - 11/09/05 05:52 AM

"Project 2612" is now open with 75 VGM rips and more are coming. For those who don't know the difference, VGM has more precise note timing than GYM and thus reproduces the sound much more faithfully. Depending on the game the differences can be dramatic. (AO supports both formats, of course). Check out the site at: http://project2612.bandwidthmonkey.net/

(Even though they only say to use AO on the Mac. Grr. Arrgh.)

ETA: Doh. Their files don't work with current AO, at least on the PC. Looks like we maybe can't handle .vgz in a .rar or something.
Posted By: Slick Mandela

Re: Great news for Genesis/MD - 11/09/05 07:20 AM

Beat me to it! Yeah, this site has a lot of great Genesis VGM files, and it looks like updates will be very frequent. Sure, it sucks that the file sizes are EXTREMELY large, even more so than GYMs, but for accuracy, it's worth it.

And just a quick bug to Belmont & Bannister: Their files don't work on the Mac either; it gives the standard "file format not yet supported" dialog box. It works if you unzip the rars and re-zip them, though.
Posted By: Tetrafish

Re: Great news for Genesis/MD - 11/09/05 07:50 AM

So for now it's only 'Okay news' :p

I'll keep my eyes on this. smile
Posted By: R. Belmont

Re: Great news for Genesis/MD - 11/09/05 08:38 AM

Well, if you unrar the files manually they do play smile But we'll sort out the problem too.
Posted By: Slick Mandela

Re: Great news for Genesis/MD - 11/09/05 11:25 AM

While we're talking about VGMs, can I address a bug? On some extremely short VGMs (the ones that are only one second long), Audio Overload seems to not play them, and instead buffers for a really long time. Is that fixable?
Posted By: R. Belmont

Re: Great news for Genesis/MD - 11/09/05 11:52 AM

Sure, point us to one that does it.
Posted By: Slick Mandela

Re: Great news for Genesis/MD - 11/09/05 12:09 PM

Kid Chameleon on that website you linked to has a bunch of those. Just listen to most of the transformation snippets.

Also Beavis & Butt-Head has a few tracks that don't play, such as the stage intros. That should be enough to get you started. Let me know if you need more. wink
Posted By: Richard Bannister

Re: Great news for Genesis/MD - 11/09/05 02:16 PM

Quote:
Originally posted by R. Belmont:
Their files don't work with current AO, at least on the PC. Looks like we maybe can't handle .vgz in a .rar or something.
That is supported on the Mac via that godawful hack you've not implemented (ui_attempt_decompress, expanding to a tmp file).
Posted By: Richard Bannister

Re: Great news for Genesis/MD - 11/09/05 02:17 PM

Quote:
Originally posted by Speedy Boris:
While we're talking about VGMs, can I address a bug? On some extremely short VGMs (the ones that are only one second long), Audio Overload seems to not play them, and instead buffers for a really long time. Is that fixable?
Yes - it's already done; I caught that one a few weeks ago. The change will be in the next update.
Posted By: Richard Bannister

Re: Great news for Genesis/MD - 11/09/05 02:38 PM

Okay, it turns out that this was broken on the Mac too. I'm working on fixing it properly right now.
Posted By: Richard Bannister

Re: Great news for Genesis/MD - 11/09/05 02:59 PM

And it's now fixed. I'll send you an update RB as soon as I've cleaned up the code a bit smile
Posted By: Richard Bannister

Re: Great news for Genesis/MD - 11/09/05 03:23 PM

...and done! I'd guess we should aim for a release sooner rather than later, since that web page tells people to use a player that currently doesn't work smile
Posted By: Tetrafish

Re: Great news for Genesis/MD - 11/10/05 05:39 AM

Quote:
Originally posted by Richard Bannister:
...and done! I'd guess we should aim for a release sooner rather than later, since that web page tells people to use a player that currently doesn't work smile
He-he. laugh That sounds like a good strategy for getting Developers to release updates sooner. Veeery Sneaky. wink

Cue the great news. cool
Posted By: Slick Mandela

Re: Great news for Genesis/MD - 11/10/05 06:34 AM

On the topic of that webpage, I really hope they start making a list of "What's New" on the front page. It'd be slightly annoying to have to comb through the VGM list to see what was recently added.
Posted By: R. Belmont

Re: Great news for Genesis/MD - 11/10/05 06:43 AM

Yeah - they say they're already adding stuff, but damned if I can remember everything that was there before smile
Posted By: blargg

Re: Great news for Genesis/MD - 11/10/05 10:14 AM

I'm going to be updating my VGM library to handle this, and while I do that I'll probably add a way to transparently access gzipped files in RAR archives using my Rar_Extractor (which Audio Overload uses). The comment on the VGM file spec page about gzip being "trivial to support" doesn't take into account the lunacy of putting a gzipped file in an RAR archive.

... oh my, the files are huge. My modem will make these painful to acquire. I might have to work on a more efficient format that stores the samples separately.
Posted By: Richard Bannister

Re: Great news for Genesis/MD - 11/10/05 02:50 PM

Quote:
Originally posted by blargg:
I'm going to be updating my VGM library to handle this, and while I do that I'll probably add a way to transparently access gzipped files in RAR archives using my Rar_Extractor (which Audio Overload uses). The comment on the VGM file spec page about gzip being "trivial to support" doesn't take into account the lunacy of putting a gzipped file in an RAR archive.
It's only about 20 lines of code to undo a gzip buffer in RAM. I'll send you the code by e-mail. For what it's worth, Audio Overload supports LZH and GZIP archives, which can be inside CAB, RAR, or ZIP; I don't think gzip functionality inside Rar_Extractor would be particularly useful to us as we'd need to duplicate it elsewhere anyway.
Posted By: Slick Mandela

Re: Great news for Genesis/MD - 11/10/05 08:11 PM

Well shut my mouth, they started posting what's new on the front page. Outstanding.
Posted By: R. Belmont

Re: Great news for Genesis/MD - 11/10/05 09:17 PM

blargg: One of the emulators (Gens?) does support another format that stores samples seperately. We really need an emulated rip format - I'm told someone has one worked out complete with some rips, but they haven't released it yet.
Posted By: Richard Bannister

Re: Great news for Genesis/MD - 11/10/05 10:35 PM

CGFM2 did so a good while back, I believe...
Posted By: Vlcice

Re: Great news for Genesis/MD - 11/11/05 12:10 AM

This is slightly off-topic, but any chance, Richard, of VGM logging being implemented in one of your Genesis emulators? I'd be intererested in logging a few games to submit to this site.
Posted By: Richard Bannister

Re: Great news for Genesis/MD - 11/11/05 03:13 AM

Neither has accurate enough sound support at this time to make good quality VGM rips.
Posted By: R. Belmont

Re: Great news for Genesis/MD - 11/11/05 04:24 AM

Yeah, even on the PC only Kega Fusion is considered accurate enough to make really good VGM rips.
Posted By: blargg

Re: Great news for Genesis/MD - 11/11/05 05:31 AM

Yesterday I looked into the VGM format and found that it has significant problems with PCM samples. Basically the format logs writes to the sound chips. So far so good. But then it rounds the times of the writes to 1/44100 second precision. In the console they occur at something like 1/4000000 second precision. This isn't such a big deal for writes to the FM and PSG chip, but for PCM samples this causes the spacing between them to be ragged, for example 3, 3, 3, 2, 3, 4, 3, 3, etc.

I did a test of encoding a waveform with varying PCM rate and got a lot of aliasing when converting it to VGM. By simply increasing the precision of the VGM format 256 times what it normally is, all the aliasing went away. In my opinion, the format is broken and shouldn't be used for yet more sound chips. The fix is simple: record the time of writes in CPU clocks, thus no information is lost. That it's currently at the (common) output sampling rate is not proof of its accuracy; with sound chips you can arrange it so that a write one CPU clock later will cause a vastly different effect.

As for another format, I'm going to try making a simple convertor that takes a VGM/GYM and outputs one with each sample stored only once. It will be lossless; conversion from VGM to it and back should result in an identical file. Update: After twenty minutes of coding, using Sonic - Green Hill Zone as a test, I was able to reduce in-memory size from 2.4MB to 868K, and gzipped on-disk size from 590K to 28K. The next step is to eliminate the duplicate samples, which will affect in-memory size most.
Posted By: blargg

Re: Great news for Genesis/MD - 11/11/05 10:15 AM

I did a full size test using the Sonic 1 set as a test. First off, if they simply eliminated the gzip compression, they'd make the RAR archive 5.5 times smaller! I'll use this new sized archive as a more realistic baseline.

Using my new format, the set uses 1/3 the memory while playing, and an RAR archive of them is four times smaller. Merely gzipping the individual files (no RAR) makes them 2.5 times smaller than the baseline RAR, or 15 times smaller than the originals only gzipped. Here are the raw numbers:

* 5046 KB VGZ.RAR
*23029 KB VGM (no compression)
* 7855 KB MD (my new format)
* 887 KB VGM.RAR (no gzip)
* 247 KB MD.RAR
* 338 KB MD.GZ (each file individually gzipped)

I didn't even bother removing the redundant samples in this test, since gzip and RAR do fine at that (this is all based on the quick algorithm I wrote). At best, completely eliminating the sample data would halve memory requirements, and barely affect the compressed size (gzip or RAR).

As for logging accurate Sega Genesis/Mega Drive music, this should be easy to add to an emulator (and do it right), since it just involves logging sound chip writes.

(Sorry for posting what might not be very interesting to others. I just hate bloated file formats, especially when they're replacing a previously bloated file format.)
Posted By: -DJSW-

Re: Great news for Genesis/MD - 11/11/05 11:44 AM

Quote:
Originally posted by blargg:
Using my new format, the set uses 1/3 the memory while playing, and an RAR archive of them is four times smaller. Merely gzipping the individual files (no RAR) makes them 2.5 times smaller than the baseline RAR, or 15 times smaller than the originals only gzipped. Here are the raw numbers:
Wow. Certainly would save us bandwidth, wouldn't that? laugh You should contact Maxim, the maintainer of the VGM format over at SMS Power . He's been meaning to add that kind of feature to the format for a while but never bothered since he's more into the 8-bit stuff. I'd love to see it added to the official spec. wink
Posted By: Richard Bannister

Re: Great news for Genesis/MD - 11/11/05 03:44 PM

To be clear in my mind, this alteration you've made is completely lossless, i.e. you can convert back and forth between VGM and this?
Posted By: blargg

Re: Great news for Genesis/MD - 11/11/05 07:47 PM

Correct, you can recover the original VGM from the optimized version. I've just successfully run the optimizer on each track from Sonic the Hedgehog, Ristar, Pulseman, and Langraisser II, and every one verified. The "unoptimizer" used for verification could also be used as a quick way to make a current VGM player work with the format.

The change is trivial: replace the extremely common PCM output sequence 52 2A XX 61 0Y 00, where XX is PCM data and Y is a short delay, with the new command 8Y, and add XX to a PCM area after the logged data. Gzip is then able to compress PCM data greatly.

The huge Pulseman set goes from 29.5MB to 2MB. Memory use while playing is more than halved. I'll be getting this in motion.
Posted By: Richard Bannister

Re: Great news for Genesis/MD - 11/11/05 08:24 PM

OK. Let me know when you've got the changes into the official spec and I'll implement it.
Posted By: Slick Mandela

Re: Great news for Genesis/MD - 11/13/05 11:48 AM

Can I assume that I can't access the VGM site at the moment because they're making the changes that blargg addressed? smile
Posted By: R. Belmont

Re: Great news for Genesis/MD - 11/13/05 12:09 PM

Looks like normal technical/connectivity problems actually.
Posted By: Dark Pulse

Re: Great news for Genesis/MD - 11/13/05 02:34 PM

Yes, it was a little downtime, and regrettably not for the changes, though those might be coming soon. Everything's back to normal now.

Sorry to disappoint, people! :p
Posted By: Dark Pulse

Re: Great news for Genesis/MD - 11/15/05 02:46 AM

Sorry the site's been down the last few days, folks... had a hardware failure! It should be back up within a few hours of this post.
Posted By: Slick Mandela

Re: Great news for Genesis/MD - 11/15/05 11:12 PM

Looks like it's back up. So how is this going to work? Is Blargg going to send you replacement VGM files that are smaller, or is he going to have his own website for the smaller files?
Posted By: R. Belmont

Re: Great news for Genesis/MD - 11/16/05 01:25 AM

Blargg will get the official guy to update the standard, send them a converter, and then Richard and I will post the only player that works with the new files. Profit! ;-)
Posted By: Dark Pulse

Re: Great news for Genesis/MD - 11/16/05 01:27 AM

Quote:
Originally posted by R. Belmont:
Blargg will get the official guy to update the standard, send them a converter, and then Richard and I will post the only player that works with the new files. Profit! ;-)
RB hit it on the head. smile
Posted By: Slick Mandela

Re: Great news for Genesis/MD - 11/17/05 11:21 AM

Another quick question about VGM: I know that in the current AO release, the only option is for the song to play to the looping point and stop. Is there a method to get these kinds of music files to loop like SPCs, NSFs, and GBSs do?
Posted By: Richard Bannister

Re: Great news for Genesis/MD - 11/17/05 01:52 PM

Not at this time, though it is something I will look at in combination with the changes to VGM expected shortly.
Posted By: Dark Pulse

Re: Great news for Genesis/MD - 11/19/05 03:05 PM

We might have something ready this weekend, guys. wink
Posted By: Dark Pulse

Re: Great news for Genesis/MD - 11/20/05 06:05 PM

OK, I've gotten the tool to shrink our sets and a RC of in_vgm 0.33. Assuming no bugs are found, the plugin should be available in the next several hours, with the site updating itself through today and possibly tomorrow.

New archives will come in Zip, by the way, as it's more cross-platform. Sure Richard will be happy to hear that. wink
Posted By: Slick Mandela

Re: Great news for Genesis/MD - 11/20/05 09:52 PM

Apparently it sounds like the new, smaller format isn't yet supported by the current Audio Overload. Is that correct?
Posted By: blargg

Re: Great news for Genesis/MD - 11/20/05 10:02 PM

Correct, Audio Overload will need an update to support the smaller Sega Genesis/Mega Drive VGMs. The changes should be minor (I added support for the format to two other players and it was simple).
Posted By: Richard Bannister

Re: Great news for Genesis/MD - 11/21/05 03:26 AM

Honestly, I'm not worried about ZIP or RAR; both formats work equally well in Audio Overload.
Posted By: Dark Pulse

Re: Great news for Genesis/MD - 11/21/05 04:02 AM

I know, but some people brought it up and it's probably nicer. smile

The plugin is out; The new versions of the sets will start going up tonight and possibly into early tomorrow, so RB and Richard, get crackin'. :p
Posted By: Dark Pulse

Re: Great news for Genesis/MD - 11/21/05 06:48 AM

Site is updated, so now it's up to RB and Richard. :p
Posted By: Richard Bannister

Re: Great news for Genesis/MD - 11/21/05 01:58 PM

Your timing is flawless, landing first thing on a Monday morning! It'll be mid week at least before I get to this.
Posted By: Dark Pulse

Re: Great news for Genesis/MD - 11/21/05 02:27 PM

Blame Maxim and Blargg, not me, sir. ;p
Posted By: Slick Mandela

Re: Great news for Genesis/MD - 11/21/05 09:42 PM

That's excellent news, R.B.! I eagerly look forward to the new version. smile

I take it you don't compile Audio Overload with CodeWarrior then? I saw in that one topic how it crapped out on you.
Posted By: Richard Bannister

Re: Great news for Genesis/MD - 11/21/05 09:55 PM

I do. I'm hoping it'll be fixed by then.
Posted By: Richard Bannister

Re: Great news for Genesis/MD - 11/22/05 05:43 AM

Okay, I've just sacrificed an hour of my sleep tonight to get this to work.

Audio Overload didn't support VGM newer than 1.0, so it required a fair bit of hacking to get things going again. I'll clean it up tomorrow night smile
Posted By: Dark Pulse

Re: Great news for Genesis/MD - 11/22/05 01:13 PM

1.0? How the heck did it play the old rips then? Those were 1.01...

In fact, Kega Fusion will still log to 1.01...
Posted By: Richard Bannister

Re: Great news for Genesis/MD - 11/22/05 01:32 PM

Hmm, maybe it was 1.01. Whatever the case, it didn't support the more recent header values like the specific clock rate for YM2612.
Posted By: Richard Bannister

Re: Great news for Genesis/MD - 11/25/05 04:19 AM

Okay, I'm happy with my VGM implementation now. I've just sent the code to R. Belmont to test on the Windows and Linux ports.

I'd anticipate a release in the next few days.
Posted By: Dark Pulse

Re: Great news for Genesis/MD - 11/25/05 07:36 PM

Sounds good! We'll let the public know AO is updated as soon as we get word.
Posted By: R. Belmont

Re: Great news for Genesis/MD - 11/27/05 10:44 PM

Windows, Linux, and Linux x86-64 binaries have been built, tested, and sent to Richard.
Posted By: Richard Bannister

Re: Great news for Genesis/MD - 11/27/05 11:12 PM

Richard was about to go to the Advent Carols service at his local church , but elected to be a few minutes late in order to get a release out now. He has decided also to start a new thread in this forum; this one has gone on long enough! smile
Posted By: Dark Pulse

Re: Great news for Genesis/MD - 11/29/05 12:41 AM

Thanks a bunch, fellas, we'll update our site; users of AO will no doubt be pleased.

BTW; thanks for the link, Arbee. smile
© 2020 Forums