How do you write a full struct to shared memory in python?

问题

There are plenty of examples showing how to write single variables or even individual members of a struct to shared memory, but is there a way to put the whole struct into shared memory so that you can simply manipulate the struct to update shared memory?

This is an example what I'm doing so far (in my actual program - there are over 50 fields in the struct - possibly 100+ by the time I'm done). It updates shared memory with x,y,z coordinates every 0.05 seconds. While it works as it sits, it's packing up a new struct at every step and writing the whole thing to shared memory - which seems inefficient to me.

import mmap
import struct
import ctypes
import time
import random

class GenericData(ctypes.Structure):
    _pack_ = 4
    _fields_ = [
        ('PosX', ctypes.c_float),
        ('PosY', ctypes.c_float),
        ('PosZ', ctypes.c_float),
    ]

# fake getters:
def getX():
    return random.random()*10
getZ = getY = getX

def main():
    buff = mmap.mmap(0, ctypes.sizeof(GenericData), "$MyTag$")
    data = GenericData()
    fmt = ''.join([f[1]._type_ for f in data._fields_])

    while (1):
        data.PosX = getX()
        data.PosY = getY()
        data.PosZ = getZ()

        print "Setting %f, %f, %f " % (data.PosX, data.PosY, data.PosZ)
        struct.pack_into(fmt, buff, 0, *[getattr(data,field) for field,typ in data._fields_])
        time.sleep(0.05)

if __name__ == "__main__":
    main()

I am aware that I could create a mapping of variables to locations in the shared memory file, but with so many fields, that's a little unwieldy.

I would like to think that the struct could be the buffer (or mapped to the buffer) and by simply setting data.PosX, shared memory is updated. Is this possible? Is there any way to make this more efficient? The struct.pack_into line is the one that concerns me.

I would like to think that something like this could be done:

buff = mmap.mmap(0, ctypes.sizeof(GenericData), "$MyTag$")
data = from_buffer(buff, GenericData)
while (1):
    data.posX = getX()
    data.posY = getY()
    data.posZ = getZ()
    time.sleep(0.05)

...which would then update the shared memory. Possible?

回答1:

As @eryksun pointed out in the first comment to the question, you can use ctypes.from_buffer to share the ctypes structure GenericData with the mmap buffer buff. @eryksun also pointed out that while Windows allows 0 as a file descriptor to map anonymous memory, -1 is the correct value - as is mentioned in the docs.

With that, here's a working example, adjusted to include @eryksun's answer:

import ctypes
import mmap
import time
import math

class GenericData(ctypes.Structure):
    _pack_ = 4
    _fields_ = [
        ('PosX', ctypes.c_float),
        ('PosY', ctypes.c_float),
        ('PosZ', ctypes.c_float),
    ]

# fake getters:
def getX():
    return random.random()*10
getZ = getY = getX

def main():
    buff = mmap.mmap(-1, ctypes.sizeof(GenericData), "$MyTag$")

    data = GenericData.from_buffer(buff)

    for fname, ftype in data._fields_:
        setattr(data, fname, 0)

    count = 0

    while (1):
        data.PosX = getX()
        data.PosY = getY()
        data.PosZ = getZ()

        print ("Setting %f, %f, %f " % (data.PosX, data.PosY, data.PosZ))
        count += 1
        time.sleep(0.05)


if __name__ == "__main__":
    main()

来源：https://stackoverflow.com/questions/45202481/how-do-you-write-a-full-struct-to-shared-memory-in-python

标签

python

ctypes

shared-memory