Forums Calendar Active Threads
Previous Thread
Next Thread
Print Thread
ZIP Corruption Caused by QMC2 v0.71 ROMAlyzer
#109009 02/26/17 03:30 PM
Joined: Jul 2010
Posts: 41
Usually London, sometimes Pemb...
P
Praxis Offline OP
Member
OP Offline
Praxis
Member
P
Joined: Jul 2010
Posts: 41
Usually London, sometimes Pemb...
Hi there, while doing this month's round of updates to MAME and QMC2 0.183, I discovered a problem caused by the previous version of QMC2, v0.71. I'm reporting it in bceause I couldn't see any fixes related to the problem (ZIP handling?) mentioned in the most recent QMC2 Changelog. So this might still be a bug in the current version.
Here's what happened:
During the month before the most recent MAME and QMC2 versions, I had a go at familiarising myself with the ROMAlyzer. Specifically the ROM Collection Rebuilder, to try and learn how it works. The challenge I set myself was to try getting QMC2 to create or update a few ROM sets based on what I threw at it from a certain directory.
Fast-forward to last weekend and I start this month's updates of MAME and QMC2, and also a full scan of my collection using ClrMAMEPro. The CMP scan complains that three ROM sets are corrupt. So I take a closer look and all three were the victims of ROMAlyzer in QMC v0.71.

Here's the first one, as reported by 7Zip:
[Linked Image]
free upload image

Here's the second one, as reported by WinRAR:
[Linked Image]
gifs upload

And here's the third one, as reported by WinRAR and including the comment (also present on both of the above) which tells us that QMC2 v0.71 was somehow involved:
[Linked Image]
free image host

Other relevant stuff? This didn't happen with every ROM set that I experimented. Only these three of the dozen or so.

Hopefully it's just a simple fix for you of updating the external ZIP handling libaray that QMC2 uses. Or maybe you fixed it in the current version without me even noticing!
Either way, I thought I should let you know about this.
Thanks
Re: ZIP Corruption Caused by QMC2 v0.71 ROMAlyzer
Praxis #109013 02/26/17 05:28 PM
Joined: May 2008
Posts: 4,930
Likes: 24
Germany
Q
qmc2 Offline
Very Senior Member
Offline
qmc2
Very Senior Member
Q
Joined: May 2008
Posts: 4,930
Likes: 24
Germany
I haven't seen any issues like this. QMC2 uses zlib / minizip or libarchive to create ZIPs. If they produce wrong headers occasionally, then that's probably an issue upstream, but again, I haven't seen this yet.

All of them look good here, I just recently recreated my complete collection with QMC2 0.183:

Code
rene@marvin:~/src/qmc2> unzip -vl /home/games/mame/roms/cbombers.zip
Archive:  /home/games/mame/roms/cbombers.zip
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
  524288  Defl:N    47515  91% 2017-02-19 14:59 b9f48284  d83_39.ic17
  524288  Defl:N    97998  81% 2017-02-19 14:59 a2f4c8be  d83_41.ic4
  524288  Defl:N    53321  90% 2017-02-19 14:59 b05f59ea  d83_40.ic3
  524288  Defl:N   105601  80% 2017-02-19 14:59 0a10616c  d83_38.ic16
  131072  Defl:N    43101  67% 2017-02-19 14:59 4f49b484  d83_26.ic37
  131072  Defl:N    53746  59% 2017-02-19 14:59 2aa1a237  d83_27.ic38
  131072  Defl:N    17894  86% 2017-02-19 14:59 06328ef7  d83_28.ic26
  131072  Defl:N     4360  97% 2017-02-19 14:59 771b4080  d83_29.ic27
 2097152  Defl:N   450973  79% 2017-02-19 14:59 79f36cce  d83_04.ic8
 2097152  Defl:N   456468  78% 2017-02-19 14:59 7787e495  d83_05.ic7
 2097152  Defl:N  1099150  48% 2017-02-19 14:59 4b71944e  d83_06.ic28
 2097152  Defl:N   996271  53% 2017-02-19 14:59 29861b61  d83_07.ic30
 2097152  Defl:N   806640  62% 2017-02-19 14:59 a0e81e01  d83_08.ic32
 2097152  Defl:N   490088  77% 2017-02-19 14:59 7e4dec50  d83_09.ic45
 1048576  Defl:N   344322  67% 2017-02-19 14:59 a790e490  d83_11.ic41
 1048576  Defl:N   312832  70% 2017-02-19 14:59 2f237b0d  d83_12.ic29
 1048576  Defl:N   248671  76% 2017-02-19 14:59 c2cceeb6  d83_13.ic31
 1048576  Defl:N   151660  86% 2017-02-19 14:59 8b6f4f12  d83_14.ic44
 2097152  Defl:N   456037  78% 2017-02-19 14:59 36c440a0  d83_10.ic43
 1048576  Defl:N   163590  84% 2017-02-19 14:59 1b71175e  d83_15.ic42
 1048576  Defl:N   352408  66% 2017-02-19 14:59 d364cf1e  d83_16.ic19
 1048576  Defl:N   349958  67% 2017-02-19 14:59 0ffe737c  d83_17.ic5
 1048576  Defl:N   164264  84% 2017-02-19 14:59 87979155  d83_18.ic6
  262144  Defl:N    27608  90% 2017-02-19 14:59 85c37961  d83_31.ic10
  262144  Defl:N    60952  77% 2017-02-19 14:59 b0db2559  d83_32.ic11
  262144  Defl:N     7642  97% 2017-02-19 14:59 eb86dc67  d83_30.ic9
 2097152  Defl:N  1284093  39% 2017-02-19 14:59 912799f4  d83_01.ic40
 2097152  Defl:N  1144085  45% 2017-02-19 14:59 2abca020  d83_02.ic39
 2097152  Defl:N  1384375  34% 2017-02-19 14:59 1b2d9ec3  d83_03.ic18
     128  Defl:N       40  69% 2017-02-19 14:59 9f7368f4  eeprom-cbombers.bin
--------          -------  ---                            -------
32768128         11175663  66%                            30 files

Code
rene@marvin:~/src/qmc2> unzip -vl /home/games/mame/roms/thndzone.zip
Archive:  /home/games/mame/roms/thndzone.zip
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
  131072  Defl:N    24049  82% 2017-02-19 17:40 20250da6  gz01-1.a15
  131072  Defl:N    43540  67% 2017-02-19 17:40 3595fad0  gz03-1.a17
  131072  Defl:N    23294  82% 2017-02-19 17:40 b7277175  gt00.a14
  131072  Defl:N    35282  73% 2017-02-19 17:40 cde31e35  gt02.a16
  131072  Defl:N    30867  77% 2017-02-19 17:40 811d86d7  gz10-1.a12
  131072  Defl:N    61842  53% 2017-02-19 17:40 8f61ab1e  gz08-1.a9
  131072  Defl:N    12813  90% 2017-02-19 17:40 80cb23de  gt11-1.a14
  131072  Defl:N    50305  62% 2017-02-19 17:40 0a8fa7e1  gt09-1.a11
   65536  Defl:N    30378  54% 2017-02-19 17:40 81c29ebf  gt04.f18
   65536  Defl:N    32188  51% 2017-02-19 17:40 0aae996a  gt05.h11
   65536  Defl:N    30168  54% 2017-02-19 17:40 4efdf03d  gt06.h12
 1048576  Defl:N   486245  54% 2017-02-19 17:40 383bbc37  maj-02.h14
 1048576  Defl:N   545943  48% 2017-02-19 17:40 9840a204  maj-01.c18
 1048576  Defl:N   369268  65% 2017-02-19 17:40 87ea8d16  maj-00.c17
  524288  Defl:N   176428  66% 2017-02-19 17:40 36e49b19  maj-04.r1
  524288  Defl:N   184353  65% 2017-02-19 17:40 80fc71cc  maj-05.r2
  524288  Defl:N   188480  64% 2017-02-19 17:40 2e7a684b  maj-06.r3
  524288  Defl:N   194121  63% 2017-02-19 17:40 3acc1f78  maj-07.r5
  524288  Defl:N   132195  75% 2017-02-19 17:40 1958a36d  maj-08.s6
  524288  Defl:N   133723  75% 2017-02-19 17:40 c21087a1  maj-09.s8
  524288  Defl:N   151006  71% 2017-02-19 17:40 a02fa641  maj-10.s9
  524288  Defl:N   131600  75% 2017-02-19 17:40 dabe9305  maj-11.s11
  131072  Defl:N    65188  50% 2017-02-19 17:40 9a86a015  gt12.n1
  131072  Defl:N    56481  57% 2017-02-19 17:40 f4709905  gt13.n2
  131072  Defl:N    45842  65% 2017-02-19 17:40 750fc523  gt14.n3
  131072  Defl:N    31952  76% 2017-02-19 17:40 f14edd3d  gt15.n5
  131072  Defl:N   122417   7% 2017-02-19 17:40 750b7e5d  gt07.h15
  524288  Defl:N   436041  17% 2017-02-19 17:40 31dcfac3  maj-03.h16
    2048  Defl:N      260  87% 2017-02-19 17:40 bde780a2  mb7128y.10m
    2048  Defl:N     1299  37% 2017-02-19 17:40 c44d2751  mb7128y.16p
     260  Defl:N       41  84% 2017-02-19 17:40 199e83fd  pal16l8b.7d
--------          -------  ---                            -------
 9769220          3827609  61%                            31 files

Code
rene@marvin:~/src/qmc2> unzip -vl /home/games/mame/roms/topland.zip
Archive:  /home/games/mame/roms/topland.zip
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
  131072  Defl:N    28811  78% 2017-02-19 17:41 28264798  b62_41.43
  131072  Defl:N    52764  60% 2017-02-19 17:41 db872f7d  b62_40.14
  131072  Defl:N    35360  73% 2017-02-19 17:41 1bd53a72  b62_25.42
  131072  Defl:N    85839  35% 2017-02-19 17:41 845026c5  b62_24.13
  131072  Defl:N    30123  77% 2017-02-19 17:41 ef3a971c  b62_23.41
  131072  Defl:N    76547  42% 2017-02-19 17:41 94279201  b62_22.12
   65536  Defl:N    26442  60% 2017-02-19 17:41 389230e0  b62-42.34
    8192  Defl:N      972  88% 2017-02-19 17:41 5f38460d  b62-21.35
    8192  Defl:N     1491  82% 2017-02-19 17:41 a4afe958  b62-20.6
  131072  Defl:N    17825  86% 2017-02-19 17:41 38786867  b62-33.39
  131072  Defl:N    19196  85% 2017-02-19 17:41 4259e76a  b62-36.48
  131072  Defl:N    20112  85% 2017-02-19 17:41 efdd5c51  b62-29.27
  131072  Defl:N    20382  84% 2017-02-19 17:41 a7e10ca4  b62-34.40
  131072  Defl:N    20601  84% 2017-02-19 17:41 cba7bac5  b62-35.47
  131072  Defl:N    20131  85% 2017-02-19 17:41 30e37cb8  b62-30.28
  131072  Defl:N    19134  85% 2017-02-19 17:41 3feebfe3  b62-31.29
  131072  Defl:N    17784  86% 2017-02-19 17:41 66806646  b62-32.30
  131072  Defl:N    98418  25% 2017-02-19 17:41 36447066  b62-17.5
  131072  Defl:N   108292  17% 2017-02-19 17:41 203a5c27  b62-16.4
  131072  Defl:N   101427  23% 2017-02-19 17:41 e35ffe81  b62-15.3
  131072  Defl:N    96075  27% 2017-02-19 17:41 617948a3  b62-14.2
  131072  Defl:N    97761  25% 2017-02-19 17:41 b37dc3ea  b62-13.1
  131072  Defl:N   106197  19% 2017-02-19 17:41 3a4e687a  b62-18.31
    8192  Defl:N      838  90% 2017-02-19 17:41 c4be68a6  b62-28.22
     324  Defl:N       58  82% 2017-02-19 17:41 c43ab9d8  pal20l8b-b62-02.bin
     324  Defl:N       67  79% 2017-02-19 17:41 904753fa  pal20l8b-b62-03.bin
     324  Defl:N       79  76% 2017-02-19 17:41 80512abc  pal20l8b-b62-04.bin
     260  Defl:N       39  85% 2017-02-19 17:41 6c1e3fc4  pal16l8a-b62-10.bin
--------          -------  ---                            -------
 2712784          1102765  59%                            28 files

Is there anything in the collection rebuilder's log that would indicate a problem?

Also, does disabling "Add ZIP comment" change anything?

Last edited by qmc2; 02/26/17 05:32 PM.
A mind is like a parachute. It doesn't work unless it's open. [Frank Zappa]
Re: ZIP Corruption Caused by QMC2 v0.71 ROMAlyzer
Praxis #109014 02/26/17 05:38 PM
Joined: Jul 2010
Posts: 41
Usually London, sometimes Pemb...
P
Praxis Offline OP
Member
OP Offline
Praxis
Member
P
Joined: Jul 2010
Posts: 41
Usually London, sometimes Pemb...
Thanks for taking a look!
It's not a big problem for me, I just wanted to let you know about it.
Good idea to check the logs.
Opening up the ROM collection rebuilder in ROMAlyzer just shows a fresh, empty log.
Enable log is checked with 10000 lines. Is it saved to a file somewhere so I can find what happened with those three ROMs? Or does that refer to the log area of the window which gets cleared on close and open?
Re: ZIP Corruption Caused by QMC2 v0.71 ROMAlyzer
Praxis #109015 02/26/17 05:44 PM
Joined: May 2008
Posts: 4,930
Likes: 24
Germany
Q
qmc2 Offline
Very Senior Member
Offline
qmc2
Very Senior Member
Q
Joined: May 2008
Posts: 4,930
Likes: 24
Germany
This log is not saved. You have to rebuild a set to get some log output from this operation.

Then it should look something like this:

Code
18:45:40.254: rebuilding started
18:45:40.296: updating hash cache
18:45:46.305: hash cache updated - 128957 hashes loaded
18:45:47.283: set rebuilding started for 'viprp1'
18:45:47.283: required ROM: name = 'seibu1.211', CRC = 'e5caf4ff', SHA-1 = '7c87a4e8e8dacfb7cc0be8f778352bce2801e59b', database status = 'available'
18:45:47.284: required ROM: name = 'seibu2.212', CRC = '688a998e', SHA-1 = '0c48374b6800cd00e3ee96c0fb12119a680b091d', database status = 'available'
18:45:47.284: required ROM: name = 'seibu3.210', CRC = '990fa76a', SHA-1 = '7619a631d6f83b3677eb47f984aff684e9518d6d', database status = 'available'
18:45:47.284: required ROM: name = 'seibu4.29', CRC = '13e3e343', SHA-1 = 'aac0c7450059847f53b5081e4abf26303a50f999', database status = 'available'
18:45:47.284: required ROM: name = 'seibu5.u0413', CRC = '5ece677c', SHA-1 = 'b782cf3296f866f79fafa69ff719211c9d4026df', database status = 'available'
18:45:47.284: required ROM: name = 'seibu6.u048', CRC = '44844ef8', SHA-1 = 'bcbe24d2ffb64f9165ba4ab7de27f44b99b5ff5a', database status = 'available'
18:45:47.284: required ROM: name = 'v_bg-11.415', CRC = '6fc96736', SHA-1 = '12df47d8af2c1febc1bce5bcf3218766447885bd', database status = 'available'
18:45:47.285: required ROM: name = 'v_bg-12.415', CRC = 'd3c7281c', SHA-1 = '340bca1f31486609b3c34dd7830362a216ff648e', database status = 'available'
18:45:47.285: required ROM: name = 'v_bg-21.410', CRC = 'd65b4318', SHA-1 = '6522970d95ffa7fa2f32e0b5b4f0eb69e0286b36', database status = 'available'
18:45:47.285: required ROM: name = 'v_bg-22.416', CRC = '24a0a23a', SHA-1 = '0b0330717620e3f3274a25845d9edaf8023b9db2', database status = 'available'
18:45:47.285: required ROM: name = 'v_obj-1.322', CRC = '3be5b631', SHA-1 = 'fd1064428d28ca166a9267b968c0ba846cfed656', database status = 'available'
18:45:47.285: required ROM: name = 'v_obj-2.324', CRC = '924153b4', SHA-1 = 'db5dadcfb4cd5e6efe9d995085936ce4f4eb4254', database status = 'available'
18:45:47.285: required ROM: name = 'v_obj-3.323', CRC = 'e9fb9062', SHA-1 = '18e97b4c5cced2b529e6e72d8041c6f78fcec76e', database status = 'available'
18:45:47.285: required ROM: name = 'v_pcm.215', CRC = 'e3111b60', SHA-1 = 'f7a7747f29c392876e43efcb4e6c0741454082f2', database status = 'available'
18:45:47.286: required ROM: name = 'flash0_blank_regionbe.u1053', CRC = 'a4c181d0', SHA-1 = '0aeea4cac4030f60ee77d62deca6b67c318c0866', database status = 'available'
18:45:48.485: creating new ZIP archive '/home/games/mame/export/viprp1.zip'
18:45:48.560: reading 'seibu1.211' from ZIP archive '/home/games/mame/roms/viprp1.zip' (uncompressed size: 512,00 KB)
18:45:48.574: writing 'seibu1.211' to ZIP archive '/home/games/mame/export/viprp1.zip' (uncompressed size: 512,00 KB)
18:45:48.595: reading 'seibu2.212' from ZIP archive '/home/games/mame/roms/viprp1.zip' (uncompressed size: 512,00 KB)
18:45:48.598: writing 'seibu2.212' to ZIP archive '/home/games/mame/export/viprp1.zip' (uncompressed size: 512,00 KB)
18:45:48.618: reading 'seibu3.210' from ZIP archive '/home/games/mame/roms/viprp1.zip' (uncompressed size: 512,00 KB)
18:45:48.621: writing 'seibu3.210' to ZIP archive '/home/games/mame/export/viprp1.zip' (uncompressed size: 512,00 KB)
18:45:48.642: reading 'seibu4.29' from ZIP archive '/home/games/mame/roms/viprp1.zip' (uncompressed size: 512,00 KB)
18:45:48.645: writing 'seibu4.29' to ZIP archive '/home/games/mame/export/viprp1.zip' (uncompressed size: 512,00 KB)
18:45:48.662: reading 'seibu5.u0413' from ZIP archive '/home/games/mame/roms/viprp1.zip' (uncompressed size: 128,00 KB)
18:45:48.663: writing 'seibu5.u0413' to ZIP archive '/home/games/mame/export/viprp1.zip' (uncompressed size: 128,00 KB)
18:45:48.669: reading 'seibu6.u048' from ZIP archive '/home/games/mame/roms/viprp1.zip' (uncompressed size: 64,00 KB)
18:45:48.671: writing 'seibu6.u048' to ZIP archive '/home/games/mame/export/viprp1.zip' (uncompressed size: 64,00 KB)
18:45:48.672: reading 'v_bg-11.415' from ZIP archive '/home/games/mame/roms/viprp1.zip' (uncompressed size: 2,00 MB)
18:45:48.684: writing 'v_bg-11.415' to ZIP archive '/home/games/mame/export/viprp1.zip' (uncompressed size: 2,00 MB)
18:45:48.805: reading 'v_bg-12.415' from ZIP archive '/home/games/mame/roms/viprp1.zip' (uncompressed size: 1,00 MB)
18:45:48.815: writing 'v_bg-12.415' to ZIP archive '/home/games/mame/export/viprp1.zip' (uncompressed size: 1,00 MB)
18:45:48.854: reading 'v_bg-21.410' from ZIP archive '/home/games/mame/roms/viprp1.zip' (uncompressed size: 1,00 MB)
18:45:48.860: writing 'v_bg-21.410' to ZIP archive '/home/games/mame/export/viprp1.zip' (uncompressed size: 1,00 MB)
18:45:48.925: reading 'v_bg-22.416' from ZIP archive '/home/games/mame/roms/viprp1.zip' (uncompressed size: 512,00 KB)
18:45:48.928: writing 'v_bg-22.416' to ZIP archive '/home/games/mame/export/viprp1.zip' (uncompressed size: 512,00 KB)
18:45:48.950: reading 'v_obj-1.322' from ZIP archive '/home/games/mame/roms/viprp1.zip' (uncompressed size: 4,00 MB)
18:45:48.973: writing 'v_obj-1.322' to ZIP archive '/home/games/mame/export/viprp1.zip' (uncompressed size: 4,00 MB)
18:45:49.092: reading 'v_obj-2.324' from ZIP archive '/home/games/mame/roms/viprp1.zip' (uncompressed size: 4,00 MB)
18:45:49.114: writing 'v_obj-2.324' to ZIP archive '/home/games/mame/export/viprp1.zip' (uncompressed size: 4,00 MB)
18:45:49.215: reading 'v_obj-3.323' from ZIP archive '/home/games/mame/roms/viprp1.zip' (uncompressed size: 4,00 MB)
18:45:49.236: writing 'v_obj-3.323' to ZIP archive '/home/games/mame/export/viprp1.zip' (uncompressed size: 4,00 MB)
18:45:49.338: reading 'v_pcm.215' from ZIP archive '/home/games/mame/roms/viprp1.zip' (uncompressed size: 1,00 MB)
18:45:49.345: writing 'v_pcm.215' to ZIP archive '/home/games/mame/export/viprp1.zip' (uncompressed size: 1,00 MB)
18:45:49.382: reading 'flash0_blank_regionbe.u1053' from ZIP archive '/home/games/mame/roms/viprp1.zip' (uncompressed size: 1,00 MB)
18:45:49.385: writing 'flash0_blank_regionbe.u1053' to ZIP archive '/home/games/mame/export/viprp1.zip' (uncompressed size: 1,00 MB)
18:45:49.392: done (creating new ZIP archive '/home/games/mame/export/viprp1.zip')
18:45:49.392: set rebuilding finished for 'viprp1'
18:45:49.452: rebuilding finished - total rebuild time = 00:00:09.183, sets processed = 1, missing ROMs = 0, missing disks = 0
18:45:49.454: waiting for work

Last edited by qmc2; 02/26/17 05:46 PM.
A mind is like a parachute. It doesn't work unless it's open. [Frank Zappa]
Re: ZIP Corruption Caused by QMC2 v0.71 ROMAlyzer
Praxis #109017 02/26/17 05:57 PM
Joined: Jul 2010
Posts: 41
Usually London, sometimes Pemb...
P
Praxis Offline OP
Member
OP Offline
Praxis
Member
P
Joined: Jul 2010
Posts: 41
Usually London, sometimes Pemb...
In that case I'll have to go by memory... I don't remember seeing anything amiss when the log scrolled up while working on those three ROM sets.
I can't think of anything else to check at my end, so this mystery is going to remain unsolved for now.
Thanks for the quick reponses today!

Moderated by  qmc2 

Link Copied to Clipboard
Who's Online Now
1 members (Pernod), 14 guests, and 2 robots.
Key: Admin, Global Mod, Mod
ShoutChat
Comment Guidelines: Do post respectful and insightful comments. Don't flame, hate, spam.
Forum Statistics
Forums9
Topics9,170
Posts120,096
Members5,039
Most Online1,283
Dec 21st, 2022
Our Sponsor
These forums are sponsored by Superior Solitaire, an ad-free card game collection for macOS and iOS. Download it today!

Superior Solitaire
Forum Rules · Mark All Read Contact Us · Forum Help · Home Page
Forum hosted by www.retrogamesformac.com