Most effective way to manage multiple tenant storage in Azure?

Question

We\'re creating a multi-tenant application that must segregate data between tenants. Each tenant will save various documents, each of which can fall into several different docum

Answer 1

I am just going to put my thought on the topic, and it do have some redundant information to Gaurav Mantri's answer. This is based on a design that I came up with after doing something very similar at my current work.

Azure Blob storage

1. Randomly select a pod from pod pool when tenant is created and store its namespace along with the tenant information.

2. Provide an api for creating containers where container names are composite of tenant id Guid::ToString("N") + . You dont need to sell the to your users as containers, i can be folders, worksets or filebox, you find a name.

3. Provide an api for maintaining documents within these containers.

This means that you can just increase the pod pool if getting more tenants, ect remove those pods that is getting filled up.

The benefits of this is that you do not need to keep two systems for your data, using both table storage and blob storage. Blob storage already have a way to present data as a directory/files hierarchy.

Extension Points

Blob Storage Api Broker

On top of the above design I made an Owin Middleware that wraps in between clients and blob storage, basicly just forwarding requests from clients to blob storage. This step is off cause not needed, as you can delegate normal sas tokens and talk directly to blob storage from clients. But it makes it easy to hook into when actions are executed on files. Each tenant will get its own endpoint files/teantid//

Using such an API would also enable you to hook into whatever token authentication system you may be useing already to validate the authenticate and authorize the incoming requests and then sign the requests in this API.

Blob Storage Metadata

Using the above api broker extension, combined with metadata one can actually take it a step further and modify incoming requests to always include metadata and add in filters on the xml returned to blob storage before sending it to clients to filter out containers or blobs. One example would be when users delete a blob, then set a x-ms-meta-status:deleted and filter them out when returning blobs/containers. This way you can add different procedures for deleting data behind the scenes.

One should be careful here, since you don't want to put to much logic in here since it adds a penalty on all requests, but doing it smart can make this work very nice.

This extensions would also allow you to allow your users to create "empty" subfolders inside a container, but placing a zero byte file with a status:hidden that also will be filtered out. (remember that blob storage only can show virtual folders if there is something in them). This could also be achieved using table storage.

Azure Search

Another great extension point is that for each blob you could keep it in Azure Search to be able to find content, and this is most likely my favorite. I dont see any good solution using just blob storage or table storage that could give you a good search functionality or to some extend even a good filtering experience. With Azure Search this would give users a really rich experience for finding their content again.

Snapshots

Another extension is that snapshots could be created for every time a file is modified automatically. This becomes even easier with the broker api, otherwise monitoring logs is an options.

These ideas comes from a project that I started that I wanted to share, but since I am busy the coming months at work I don't see myself releasing my project before the summer holidays give me time to finish. The motivation of the project is to provide a nuget package that enables other developers to quickly set up this broker api that i mentioned above and configure a multi tenant blob storage solution.

I kindly ask you to vote up this answer if you read this and believe such a project could have saved you time in your current development process. This way I can see if I could use more time on the project or not.

I think that gaurav Mantris answer is more spot on for the question above, but just wanted to share my ideas on the topic.