Best tool for building subset collections

noah

Distinguished
Apr 12, 2004
22
0
18,510
Archived from groups: alt.games.mame (More info?)

I'm trying to fill a single CD with the golden age classics.
I want to select games published before a given date
and when added together will use less than a given
amount of disk space. ... The problem is that it seems that ROMs
in one ZIP file depend on ROMs in parent ZIP files.
It gets rather confusing. I'm having a hard time just dragging
and dropping files from the master source directory. I have to
identify a game that I want, then figure out the name of the zip file,
then drag and drop it into a working directory. Finally, when this is
all done I plan to check the ROMs in the working directory using
AdvanceScan
http://advancemame.sourceforge.net/scan-readme.html
and then add any missing required ZIP files.

This is tedius -- especially the part where I look up
the ZIP filename for a given game and then copy it by hand.
Is there a smarter way?

Yours,
Noah
 
Archived from groups: alt.games.mame (More info?)

noah@noah.org wrote in news:1113005768.695091.326810
@o13g2000cwo.googlegroups.com:

> I'm trying to fill a single CD with the golden age classics.
> I want to select games published before a given date
> and when added together will use less than a given
> amount of disk space. ... The problem is that it seems that ROMs
> in one ZIP file depend on ROMs in parent ZIP files.
> It gets rather confusing. I'm having a hard time just dragging
> and dropping files from the master source directory. I have to
> identify a game that I want, then figure out the name of the zip file,
> then drag and drop it into a working directory. Finally, when this is
> all done I plan to check the ROMs in the working directory using
> AdvanceScan
> http://advancemame.sourceforge.net/scan-readme.html
> and then add any missing required ZIP files.
>
> This is tedius -- especially the part where I look up
> the ZIP filename for a given game and then copy it by hand.
> Is there a smarter way?
>
> Yours,
> Noah
>
>

Well, I don't know how accurate it will be... but

Sort your rom files by size (ascending order). All older roms will be
small. Double Dragon 2 which came out in 1987, and has pretty good
graphics for that time is only 732KB. You should be able to put nearly
all of the roms from about 1990 and earlier, and MAME, and a front end on
one DVD with room to spare. If you're careful, you'll be able to get
just about any rom you want (prior to 1990 release) on 1 CD.
Good Luck!!!
 
Archived from groups: alt.games.mame (More info?)

noah@noah.org wrote:
> I'm trying to fill a single CD with the golden age classics.
> I want to select games published before a given date
> and when added together will use less than a given
> amount of disk space. ... The problem is that it seems that ROMs
> in one ZIP file depend on ROMs in parent ZIP files.
> It gets rather confusing. I'm having a hard time just dragging
> and dropping files from the master source directory. I have to
> identify a game that I want, then figure out the name of the zip file,
> then drag and drop it into a working directory. Finally, when this is
> all done I plan to check the ROMs in the working directory using
> AdvanceScan
> http://advancemame.sourceforge.net/scan-readme.html
> and then add any missing required ZIP files.
>
> This is tedius -- especially the part where I look up
> the ZIP filename for a given game and then copy it by hand.
> Is there a smarter way?
>
> Yours,
> Noah
>

Hmmmmm..... Roman could probably tell you best how to do this, but here
is an idea (as age seems to be inversely proportional to size):

1) Start up Explorer, set view to 'Details', and point to your
MAME roms folder
2) Sort by size
3) Select about 20-30% more roms than your space will allow.
4) copy to temp1 folder
5) Start up ClrMamePro
6) Rebuild 'Non-Merged' sets in temp1 folder to temp2 folder
7) Remove leftovers in temp1 folder
8) Remove games you don't want/clones from temp2 folder
9) Rebuild 'Merged' or 'Split' sets in temp2 folder to temp1 folder
10) Add roms from temp1 folder to your CD image until it is full.
(Adding extras will take a bit of fiddling.)

Voila! One 'MAME Classics' CD created.

--
Thnik about it!
Dead_Dad
 
Archived from groups: alt.games.mame (More info?)

This is a pretty good suggestion, but what I finally
ended up doing is writing a Python script to make subsets for me.
All I needed was the game list XML dump from MAME:
xmame -listxml
My script reads the XML list and creates an index of file names
and sizes sorted by the year the game was published.
Then the script copies the ROMs and extra files (snap, flyers,
samples, etc) to a target directory until all of the file sizes add up
to the given target size. In my case, I choose 600 MB since that
will fit on a 650MB CDR with plenty of extra for MAME32.

In case anyone else finds themselves needing to do something similar
I am attaching the script here. If you know Python it should be
easy to modify to do other MAME housekeeping chores.

Thanks for the help!
Yours,
Noah Spurrier

<pre>
#######################################################################
# This script scans a ROM path and copies ROMs based on rules.
# In this case, we select ROMs ordered by year util
# a given total target size is filled.
# Noah Spurrier 2005
# No copyright. Public Domain.

from xml.dom import pulldom
import csv, stat, os, shutil

MEGABYTE = 1048576
TARGET_SIZE = 600 * MEGABYTE
GAMELIST_XML = "gamelist.xml"
SOURCE_ROM_PATH = r"C:\roms"
SOURCE_SAMPLE_PATH = r"C:\mame_extras_95\samples"
SOURCE_SNAP_PATH = r"C:\mame_extras_95\snap"
SOURCE_ARTWORK_PATH = r"C:\mame_extras_95\artwork"
SOURCE_CABINETS_PATH = r"C:\mame_extras_95\cabinets"
SOURCE_FLYERS_PATH = r"C:\mame_extras_95\flyers"
TARGET_ROM_PATH = r"C:\mame\roms"
TARGET_SAMPLE_PATH = r"C:\mame\samples"
TARGET_SNAP_PATH = r"C:\mame\snap"
TARGET_ARTWORK_PATH = r"C:\mame\artwork"
TARGET_CABINETS_PATH = r"C:\mame\cabinets"
TARGET_FLYERS_PATH = r"C:\mame\flyers"

#######################################################################
def get_text (nodelist):
"""This returns all the text data from an XML element."""
rc = ""
for node in nodelist:
if node.nodeType == node.TEXT_NODE:
rc = rc + node.data
return rc

def file_size (filepath):
"""This returns the size of the file. This is a wrapper
around os.stat. Any OS error returns -1."""
try:
return os.stat (filepath)[stat.ST_SIZE]
except OSError:
return -1

def dir_size (top):
"""This returns the size of a directory and all files in it."""
size = 0
for (dirpath, dirnames, filenames) in os.walk (top):
for filename in filenames:
size = size + file_size (os.path.join (dirpath, filename))
return size

#######################################################################

# The real work starts here.

#
# Parse the gamelist XML to build up a database of rom names and sizes.
# Also build an index of games by year.
#
db = {}
year_index = {}
xml_file = open(GAMELIST_XML, "rb")
events = pulldom.parse(xml_file)
count = 0
for (event, node) in events:
if event == pulldom.START_ELEMENT:
if node.tagName == "game":
count = count + 1

name = node.getAttribute("name")
romof = node.getAttribute("romof")
size = file_size (os.path.join (SOURCE_ROM_PATH, name +
".zip"))

events.expandNode(node)
try:
year =
get_text(node.getElementsByTagName("year")[0].childNodes)
except:
# Some game nodes don't have <year> for some reason...
year = None

db[name] = dict (romof=romof, year=year, size=size)

if year is not None:
if year not in year_index:
year_index[year] = [name]
else:
year_index[year].append(name)
#
# Copy each game in order of year until the target size is reached.
# This copies roms, samples, artwork, cabinets, flyers, and snaps.
#
total_size = 0
count = 0
years = year_index.keys()
years.sort()
for year in years:
if total_size >= TARGET_SIZE:
break
for name in year_index[year]:
if db[name]['size'] > 0 and total_size <= TARGET_SIZE:
# Copy the ROM and all the extras.
total_size = total_size + db[name]['size']
shutil.copyfile (os.path.join (SOURCE_ROM_PATH, name +
".zip"), os.path.join (TARGET_ROM_PATH, name + ".zip"))
if os.path.isdir(os.path.join (SOURCE_SAMPLE_PATH, name)):
shutil.copytree(os.path.join (SOURCE_SAMPLE_PATH,
name), os.path.join (TARGET_SAMPLE_PATH, name))
total_size = total_size + dir_size (os.path.join
(SOURCE_SAMPLE_PATH, name))
if os.path.isdir(os.path.join (SOURCE_ARTWORK_PATH, name)):
shutil.copytree(os.path.join (SOURCE_ARTWORK_PATH,
name), os.path.join (TARGET_ARTWORK_PATH, name))
total_size = total_size + dir_size (os.path.join
(SOURCE_ARTWORK_PATH, name))
if os.path.isfile(os.path.join (SOURCE_CABINETS_PATH, name
+ ".png")):
shutil.copyfile (os.path.join (SOURCE_CABINETS_PATH,
name + ".png"), os.path.join (TARGET_CABINETS_PATH, name + ".png"))
total_size = total_size + file_size (os.path.join
(SOURCE_CABINETS_PATH, name + ".png"))
if os.path.isfile(os.path.join (SOURCE_FLYERS_PATH, name +
".png")):
shutil.copyfile (os.path.join (SOURCE_FLYERS_PATH, name
+ ".png"), os.path.join (TARGET_FLYERS_PATH, name + ".png"))
total_size = total_size + file_size (os.path.join
(SOURCE_FLYERS_PATH, name + ".png"))
if os.path.isfile(os.path.join (SOURCE_SNAP_PATH, name +
".png")):
shutil.copyfile (os.path.join (SOURCE_SNAP_PATH, name +
".png"), os.path.join (TARGET_SNAP_PATH, name + ".png"))
total_size = total_size + file_size (os.path.join
(SOURCE_SNAP_PATH, name + ".png"))

print name, "-", year, "-", total_size, "BYTES - ",
total_size/float(MEGABYTE), "MB"
count = count + 1

print "Total ROMs copied:", count
### END OF SCRIPT #####################################################
</pre>
 
Archived from groups: alt.games.mame (More info?)

Noah wrote:
> This is a pretty good suggestion, but what I finally
> ended up doing is writing a Python script to make subsets for me.
> All I needed was the game list XML dump from MAME:
> xmame -listxml
> My script reads the XML list and creates an index of file names
> and sizes sorted by the year the game was published.
> Then the script copies the ROMs and extra files (snap, flyers,
> samples, etc) to a target directory until all of the file sizes add up
> to the given target size. In my case, I choose 600 MB since that
> will fit on a 650MB CDR with plenty of extra for MAME32.
>
> In case anyone else finds themselves needing to do something similar
> I am attaching the script here. If you know Python it should be
> easy to modify to do other MAME housekeeping chores.
>
> Thanks for the help!
> Yours,
> Noah Spurrier
>
> <pre>
> #######################################################################
> # This script scans a ROM path and copies ROMs based on rules.
-----8<-----
> ### END OF SCRIPT #####################################################
> </pre>
>

This is a wooooonderful idea. Unfortunately, it doesn't seem to work on
my computer (XP Pro w/SP2 & Python 2.3). Here is the error I get:

<pre>
Traceback (most recent call last):
File "D:\Emulators\MAME\makecd.py", line 121, in ?
shutil.copyfile (os.path.join (SOURCE_ROM_PATH, name + ".zip"),
os.path.join
(TARGET_ROM_PATH, name + ".zip"))
File "C:\Program Files\Python 2.3\Lib\shutil.py", line 39, in copyfile
fdst = open(dst, 'wb')
IOError: [Errno 2] No such file or directory:
u'D:\\temp\\mame\\roms\\galaxian.zip'
</pre>

My Python skills aren't up to debugging this yet. Could you please tell
me what I need to change?

BTW, how do I get a repeated ".", to print '.....' instead of '. . . . . '?

--
Thnik about it!
Dead_Dad
 
Archived from groups: alt.games.mame (More info?)

Send me your script. It looks like your
destination directory (\temp\mame\roms\)
does not already exist. When the script
copies a file from a source directory to
a destination directory the destination
directory must already exist.

Yours,
Noah