What config file format to use for user-friendly strings of arbitrary bytes?

问题

So I made a short Python script to launch files in Windows with ambiguous extensions by examining their magic number/file signature first:

https://superuser.com/a/317927/13889
https://gist.github.com/1119561

I'd like to compile it to a .exe to make association easier (either using bbfreeze or rewriting in C), but I need some kind of user-friendly config file to specify the matching byte strings and program paths. Basically I want to put this information into a plain text file somehow:

magic_numbers = {
# TINA
'OBSS': r'%PROGRAMFILES(X86)%\DesignSoft\Tina 9 - TI\TINA.EXE',

# PSpice
'*version': r'%PROGRAMFILES(X86)%\Orcad\Capture\Capture.exe', 
'x100\x88\xce\xcf\xcfOrCAD ': '', #PSpice?

# Protel
'DProtel': r'%PROGRAMFILES(X86)%\Altium Designer S09 Viewer\dxp.exe', 

# Eagle
'\x10\x80': r'%PROGRAMFILES(X86)%\EAGLE-5.11.0\bin\eagle.exe',
'\x10\x00': r'%PROGRAMFILES(X86)%\EAGLE-5.11.0\bin\eagle.exe',
'<?xml version="1.0" encoding="utf-8"?>\n<!DOCTYPE eagle ': r'%PROGRAMFILES(X86)%\EAGLE-5.11.0\bin\eagle.exe',

# PADS Logic
'\x00\xFE': r'C:\MentorGraphics\9.3PADS\SDD_HOME\Programs\powerlogic.exe', 
}

(The hex bytes are just arbitrary bytes, not Unicode characters.)

I guess a .py file in this format works, but I have to leave it uncompiled and somehow still import it into the compiled file, and there's still a bunch of extraneous content like { and , to be confused by/screw up.

I looked at YAML, and it would be great except that it requires base64-encoding binary stuff first, which isn't really what I want. I'd prefer the config file to contain hex representations of the bytes. But also ASCII representations, if that's all the file signature is. And maybe also regexes. :D (In case the XML-based format can be written with different amounts of whitespace, for instance)

Any ideas?

回答1:

You've already got your answer: YAML.

The data you posted up above is storing text representations of binary data; that will be fine for YAML, you just need to parse it properly. Usually you'd use something from the binascii module; in this case, likely the binascii.a2b_qp function.

magic_id_str = 'x100\x88\xce\xcf\xcfOrCAD '
magic_id = binascii.a2b_qp(magic_id_str)

To elucidate, I will use a unicode character as an easy way to paste binary data into the REPL (Python 2.7):

>>> a = 'Φ'  
>>> a  
'\xce\xa6'  
>>> binascii.b2a_qp(a)  
'=CE=A6'  
>>> magic_text = yaml.load("""  
... magic_string: '=CE=A6'  
... """)  
>>> magic_text  
{'magic_string': '=CE=A6'}  
>>> binascii.a2b_qp(magic_text['magic_string'])  
'\xce\xa6'

回答2:

I would suggest doing this a little differently. I would decouple these two settings from each other:

Magic number signature ===> mimetype
mimetype ==> program launcher

For the first part, I would use python-magic, a library that has bindings to libmagic. You can have python-magic use a custom magic file like this:

import magic
m = magic.Magic(magic_file='/path/to/magic.file')

Your users can specify a custom magic file mapping magic numbers to mimetypes. The syntax of magic files is documented. Here's an example showing the magic file for the TIFF format:

# Tag Image File Format, from Daniel Quinlan (quinlan@yggdrasil.com)
# The second word of TIFF files is the TIFF version number, 42, which has
# never changed.  The TIFF specification recommends testing for it.
0       string          MM\x00\x2a      TIFF image data, big-endian
!:mime  image/tiff
0       string          II\x2a\x00      TIFF image data, little-endian
!:mime  image/tiff

The second part then is pretty easy, since you only need to specify text data now. You could go with an INI or yaml format, as suggested by others, or you could even have just a simple tab-delimited file like this:

image/tiff         C:\Program Files\imageviewer.exe
application/json   C:\Program Files\notepad.exe

回答3:

I've used some packages to build configuration files, also yaml. I recommend that you use ConfigParser or ConfigObj.

At last, the best option If you wanna build a human-readable configuration file with comments I strongly recommend use ConfigObj.

ConfigObj
Brief ConfigObj tutorial
ConfigParser
Brief ConfigParser tutorial

Enjoy!

Example of ConfigObj

With this code:

You can use ConfigObj to store them too. Try this one: import configobj

def createConfig(path):
    config = configobj.ConfigObj()
    config.filename = path
    config["Sony"] = {}
    config["Sony"]["product"] = "Sony PS3"
    config["Sony"]["accessories"] = ['controller', 'eye', 'memory stick']
    config["Sony"]["retail price"] = "$400"
    config["Sony"]["binary one"]= bin(173)
    config.write()

You get this file:

[Sony]
product = Sony PS3
accessories = controller, eye, memory stick
retail price = $400
binary one = 0b10101101

来源：https://stackoverflow.com/questions/9687841/what-config-file-format-to-use-for-user-friendly-strings-of-arbitrary-bytes

标签

python

configuration-files

binary-data

magic-numbers