Django Azure upload file to blob storage

Question

I\'ve got an application that needs to upload a file to an Azure Blob Storage container. I\'ve been trying for a very long time and followed every tutorial to no avail.

Answer 1

I reviewed the source code azure_storage.py of jschneier/django-storages for Azure Storage and the document for Azure Storage, your issue was caused by the incorrect value of AZURE_LOCATION in settings.py which should be "" or a prefix string of blob as subfolder name in a container like <container name>/<prefix string as AZURE_LOCATION, such as A/B/..../ ><filename>.

The conclusion is based on the code analysis below.

The _get_file function atLINE 34 below used self._storage.service.get_blob_to_stream method of Storage SDK, and the value of parameter blob_name is self._path which appeared self._path = storage._get_valid_path(name) at LINE 32 inside __init__ function for AzureStorageFile:

#LINE 32 inside def __init__(self, name, mode, storage):
    self._path = storage._get_valid_path(name)

# LINE 34
def _get_file(self):
    if self._file is not None:
        return self._file

    file = SpooledTemporaryFile(
        max_size=self._storage.max_memory_size,
        suffix=".AzureStorageFile",
        dir=setting("FILE_UPLOAD_TEMP_DIR", None))

    if 'r' in self._mode or 'a' in self._mode:
        # I set max connection to 1 since spooledtempfile is
        # not seekable which is required if we use max_connections > 1
        self._storage.service.get_blob_to_stream(
            container_name=self._storage.azure_container,
            blob_name=self._path,
            stream=file,
            max_connections=1,
            timeout=10)
    if 'r' in self._mode:
        file.seek(0)

    self._file = file
    return self._file

Then, the storage.__get_valid_path function with its dependent function _path at LINE 173:

#LINE 147
location = setting('AZURE_LOCATION', '')

#LINE 173
def _path(self, name):
    name = _clean_name_dance(name)
    try:
        return safe_join(self.location, name)
    except ValueError:
        raise SuspiciousOperation("Attempted access to '%s' denied." % name)

def _get_valid_path(self, name):
    # Must be idempotent
    return _get_valid_path(self._path(name))

def _open(self, name, mode="rb"):
    return AzureStorageFile(name, mode, self)

def get_valid_name(self, name):
    return _clean_name_dance(name)

def get_available_name(self, name, max_length=_AZURE_NAME_MAX_LEN):
    """
    Returns a filename that's free on the target storage system, and
    available for new content to be written to.
    """
    name = self.get_valid_name(name)
    if self.overwrite_files:
        return get_available_overwrite_name(name, max_length)
    return super(AzureStorage, self).get_available_name(name, max_length)

So you can see the code return blob_name from _path to _get_valid_path which joined AZURE_LOCATION with name. If use {AZURE_ACCOUNT_NAME}.blob.core.windows.net as location, the blob name for downloading will different from the blob name uploaded, not one to one correspondence.

Answer 2

Figured out the problem. I followed a tutorial for Amazon S3 and then applied the exact same principals to the Azure scenario. After all, I'm sure the developer of this class wanted to keep everything uniform.

mysite/custom_azure.py <-- just put it in the same folder as your settings.py file

from storages.backends.azure_storage import AzureStorage

class AzureMediaStorage(AzureStorage):
    location = 'media'
    file_overwrite = False

mysite/settings.py

STATICFILES_DIRS = [
    os.path.join(BASE_DIR, '<directory that houses the static files>/static'),
]

AZURE_ACCOUNT_NAME = '<azure container name>'
AZURE_ACCOUNT_KEY = '<azure account key for this container>'
AZURE_CUSTOM_DOMAIN = f'{AZURE_ACCOUNT_NAME}.blob.core.windows.net'
AZURE_LOCATION = '<blob container name>'
AZURE_CONTAINER = '<blob container name>'

STATIC_LOCATION = 'static'
STATIC_URL = f'https://{AZURE_CUSTOM_DOMAIN}/{STATIC_LOCATION}/'

STATICFILES_STORAGE = 'storages.backends.azure_storage.AzureStorage'
DEFAULT_FILE_STORAGE = 'mysite.custom_azure.AzureMediaStorage'

Also, note that if you have the following in your mysite/urls.py from some other tutorial or something:

if settings.DEBUG:
    urlpatterns += static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
    urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

you need to remove the MEDIA line:

if settings.DEBUG:
    urlpatterns += static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)

from the model, remove any reference to 'storage' and just leave the 'upload_to' option like this:

thumbnail = models.ImageField(default='default.jpg', upload_to='video_thumbs')

it all just worked.

don't for get to do the following to check yourself:

python3 manage.py collectstatic

hope this helps other people!!!