Problems using multipart_encode (poster library)

Question

I am trying to upload a file using multipart_encode to realize the MIME process. However, I met the following error AttributeError: multipart_yielder instance has no attribute '__len__'. Below are is my approach, I really appreciate if anyone can give me some suggestions.

url = "https://pi-user-files.s3-external-1.amazonaws.com/"           
post_data = {}
#data is a dict
post_data['AWSAccessKeyId']=(data['ticket']['AWSAccessKeyId'])
post_data['success_action_redirect']=(data['ticket']['success_action_redirect'])
post_data['acl']=(data['ticket']['acl'])
post_data['key']=(data['ticket']['key'])
post_data['signature']=(data['ticket']['signature'])
post_data['policy']=(data['ticket']['policy'])
post_data['Content-Type']=(data['ticket']['Content-Type'])

#I would like to upload a text file "new 2"
post_data['file']=open("new  2.txt", "rb")

datagen, headers = multipart_encode(post_data)
request2 = urllib2.Request(url, datagen, headers)
result = urllib2.urlopen(request2)

Answer 1

The problem is that in httplib.py, the generator is not detected as such and is treated instead like a string that holds the full data to be sent (and therefore it tries to find its length):

if hasattr(data,'read') and not isinstance(data, array): # generator 
    if self.debuglevel > 0: print "sendIng a read()able"
    ....

A solution is to make the generator act like a read()able:

class GeneratorToReadable():
    def __init__(self, datagen):
        self.generator = datagen
        self._end = False
        self.data = ''

    def read(self, n_bytes):
        while not self._end and len(self.data) < n_bytes:
            try:
                next_chunk = self.generator.next()
                if next_chunk:
                    self.data += next_chunk
                else:
                    self._end = True
            except StopIteration:
                self._end = True
        result = self.data[0:n_bytes]
        self.data = self.data[n_bytes:]
        return result

and use like so:

datagen, headers = multipart_encode(post_data)
readable = GeneratorToReadable(datagen)
req = urllib2.Request(url, readable, headers)
result = urllib2.urlopen(req)

Answer 2

If you want to send a file you should wrap other parameters with a MultipartParam object, example code for creating a send file request:

from poster.encode import multipart_encode, MultipartParam
import urllib2

def postFileRequest(url, paramName, fileObj, additionalHeaders={}, additionalParams={}):
    items = []
    #wrap post parameters
    for name, value in additionalParams.items():
        items.append(MultipartParam(name, value))
    #add file
    items.append(MultipartParam.from_file(paramName, fileObj))
    datagen, headers = multipart_encode(items)
    #add headers
    for item, value in additionalHeaders.iteritems():
        headers[item] = value
    return urllib2.Request(url, datagen, headers)

Also I think you should execute register_openers() once at the beginning. Some details you can find in docs