How to serialize a scandir.DirEntry in Python for sending through a network socket?

问题

I have server and client programs that communicate with each other through a network socket.

What I want is to send a directory entry (scandir.DirEntry) obtained from scandir.scandir() through the socket.

For now I am using pickle and cPickle modules and have come up with the following (excerpt only):

import scandir, pickle

s = scandir.scandir("D:\\PYTHON")
entry = s.next()
data = pickle.dumps(entry)

However, I am getting the following error stack:

File "untitled.py", line 5, in <module>
  data = pickle.dumps(item)
File "C:\Python27\Lib\pickle.py", line 1374, in dumps
  Pickler(file, protocol).dump(obj)
File "C:\Python27\Lib\pickle.py", line 224, in dump
  self.save(obj)
File "C:\Python27\Lib\pickle.py", line 306, in save
  rv = reduce(self.proto)
File "C:\Python27\Lib\copy_reg.py", line 70, in _reduce_ex
  raise TypeError, "can't pickle %s objects" % base.__name__

TypeError: can't pickle DirEntry objects

How can I get rid of this error?

I have heard of using marshall or JSON. UPDATE: JSON is not dumping all the data within the object.

Is there any completely different way to do so to send the object through the socket?

Thanks in advance for any help.

回答1:

Yes, os.DirEntry objects are intended to be short-lived, not really kept around or serialized. If you need the data in them to be serialized, looks like you've figured that out in your own answer -- serialize (pickle) a dict version of the attributes you need.

To deserialize into an object that walks and quacks like an os.DirEntry instance, create a PseudoDirEntry class that mimics the things you need.

Note that you can directly serialize the stat object already, which saves you picking the fields out of that.

Combined, that would look like this:

class PseudoDirEntry:
    def __init__(self, name, path, is_dir, stat):
        self.name = name
        self.path = path
        self._is_dir = is_dir
        self._stat = stat

    def is_dir(self):
        return self._is_dir

    def stat(self):
        return self._stat

And then:

>>> import os, pickle
>>> entry = list(os.scandir())[0]
>>> pickled = pickle.dumps({'name': entry.name, 'path': entry.path, 'is_dir': entry.is_dir(), 'stat': entry.stat()})
>>> loaded = pickle.loads(pickled)
>>> pseudo = PseudoDirEntry(loaded['name'], loaded['path'], loaded['is_dir'], loaded['stat'])
>>> pseudo.name
'.DS_Store'
>>> pseudo.is_dir()
False
>>> pseudo.stat()
os.stat_result(st_mode=33188, st_ino=8370294, st_dev=16777220, st_nlink=1, st_uid=502, st_gid=20, st_size=8196, st_atime=1478356967, st_mtime=1477601172, st_ctime=1477601172)

回答2:

Well I myself have figured out that for instances of non-standard classes like this scandir.DirEntry, the best way is to convert the class member data into a (possibly nested) combination of standard objects like (list, dict, etc.).

For example, in the particular case of scandir.DirEntry, it can be done as follows.

import scandir, pickle

s = scandir.scandir("D:\\PYTHON")
entry = s.next()

# first convert the stat object to st_
st = entry.stat()
st_ = {'st_mode':st.st_mode, 'st_size':st.st_size,\
   'st_atime':st.st_atime, 'st_mtime':st.st_mtime,\
   'st_ctime':st.st_ctime}

# now convert the entry object to entry_
entry_ = {'name':entry.name, 'is_dir':entry.is_dir(), \
 'path':entry.path, 'stat':st_}

# one may need some other class member data also as necessary

# now pickle the converted entry_
data = pickle.dumps(entry_)

Although for my purpose, I only require the data, after the unpickling in the other end, one may need to reconstruct the unpickled entry_ to unpickled scandir.DirEntry object 'entry'. However, I am yet to figure out how to reconstruct the class instance and set the data for the behaviour of methods like is_dir(), stat().

来源：https://stackoverflow.com/questions/39757325/how-to-serialize-a-scandir-direntry-in-python-for-sending-through-a-network-sock

标签

python

sockets

serialization

scandir