What's the point of using Amazon SimpleDB?

Question

I thought that I could use SimpleDB to take care of the most challenging area of my application (as far as scaling goes) - twitter-like comments, but with location on top - till

Answer 1

I do not buy all the hype around SimpleDB and based on the following limitations can not see a reason why it should be used (I understand that now you can build almost anything with almost any technology, but this is not the reason to select one).

So the limitations I have seen:

• can be run only on amazon AWS, you should also pay for a whole bunch of staff
• maximum size of domain (table) is 10 GB
• attribute value length (size of field) is 1024 bytes
• maximum items in select response - 2500
• maximum response size for Select (the maximum amount of data that can return you) - 1Mb, actually you can check all the limitations here
• has drivers only for a few languages (java, php, python, ruby, .net)
• does not allow case insensitive search. You have to introduce additional lowercase field/application logic.
• sorting can be done only on one field
• because of 5s timelimit count in can behave strange. If 5 seconds passed and the query has not finished, you end up with a partial number and a token which allows you to continue query. Application logic is responsible for collecting all this data an summing up.
• everything is a UTF-8 string, which makes it a pain in the ass to work with non string values (like numbers, dates).
• sorting behaves strange for numbers (due to the fact that everything is a string). So now you have to do a shamanic dance with padding
• both do not have transactions and joins
• no compound, geostatic, multiple column indices, no foreign keys

If this is not enough, then you also have to forget about the basic things like group by, sum average, distinct as well as data manipulation. In whole the query language is pretty rudimentary and reminds a small subset of what SQL can do.

So the functionality is not really way richer than Redis/Memcached, but I highly doubt that it performs as good as these two dbs for their use cases.

SimpleDB position itself as a schema-less document-base nosql database but the query syntax of MongoDB/CounchDB is way more expressive and their limitations are way more reasonable.

And at last - do not forget about vendor locking. If in a couple of years Azure (or something else that would appear) will provide a cloud hosting 5 times cheaper than AWS, it would be really hard to switch.

Answer 2

I use SDB on a couple of large-ish applications. The 10 GB limit per domain does worry me, but we are gambling on Amazon allowing this to be extended if we need it. They have a request form on their site if you want more space.

As far as cross domain joins, don't think of SDB as a traditional database. During the migration of my data to SDB, I had to denormalize some of it so I could manually do the cross domain joins.

The 1000 byte per attribute limitation was tough to work around also. One of the applications I have is a blog service which stores posts and comments in the database. While porting it over to SDB, I ran into this limitation. I ended up storing the posts and comments as files in S3, and read that in my code. Since this server is on EC2, the traffic to S3 isn't costing anything extra.

Perhaps one of the other problems to watch out for is the eventual consistency model on SDB. You can't write data and then read it back with any guarantee that the newly written data will be returned to you. Eventually the data will be updated.

All of this said, I still love SDB. I don't regret switching to it. I moved from a SQL 2005 server. I think I had a lot more control with SQL, but once giving up that control, I have more flexibility. Not needing to pre-define the schema is awesome. With a strong and robust caching layer in your code, it's easy to make SDB more flexible.

Answer 3

The point of SimpleDB .. is to be used, not as a database for all your data, but as an index for data in other non traditional "DB" data stores, such as S3. This is how I use SimpleDB for things like ETL processes. The data in my S3 data lake must be indexed but S3 has no suitable index, and this one of the best use cases for SimpleDB (IMHO)

Google this: "simpledb s3 index"

Note, it does not have to be for S3 specifically. You may have data in EFS / EBS, or even SES that you want indexed. SimpleDB is a great solution for providing a simple fast index for just about anything. I find DynamoDB overkill and unnecessarily over complicated in terms of provisioning for your needed throughput and how that relates to cost, and have heard horror stories related to that. With SimpleDB the performance is consistently good and the costs are predictable.

READ THIS: https://segment.com/blog/the-million-dollar-eng-problem/

Given that fact and that for anything complicated, there are other solutions for data storage/indexing such as Sphinx, Postgres, Mongo ..etc, my question has always been what is the point of the DynamoDB cost trap when other solutions are just as fast but, DONT NEED THROUGHPUT CONFIGURATION, and have predictable costs. DynamoDB is an AWS money grab (IMHO). They can't phase out SimpleDB because there are too many existing customers in-the-know that rely on it. AWS relies on it themselves as well. If it truly was supplanted by DynamoDB as they claim then everyone would have moved over to Dynamo and this would not be a discussion.