Can I use AllocateIDs as “string” ? datastore

Question

I need to assign a unique identifier to each entity in datastore. I've found AllocateIDs but the issue is that it generates integers and I use strings for the keys. Is it safe to convert the integer to a string or there is a risk of collision (i.e. datastore to return me the same integer that I'm currently using as a string key). ?

Answer 1

Let's clear some things first:

The identifier part of an entity's key can either be

• a key name string
• or an integer numeric ID

But not both. So when you save an entity, its key either has a string id called name OR (exclusive OR) an int64 id called intID.

The 2 optional identifier fields are distinct: if you have an entity with name="1234" it is different from that of with intID=1234.

When you save a new entity without explicitly specifying a name or intID, the datastore will assign a new unique intID identifier to it. The datastore will never assign a string name by itself. You can only have entities with string name identifiers if you specify a string name yourself.

The datastore knows about the automatically generated intIDs it generates itself, and will never generate the same intID twice (well behavior). However when saving a new entity if you specify an intID yourself, you have to take care about it being unique. This would require to first check if the intID you whish to use is not yet in use (e.g. by querying it first to see if no entity has that intID yet) but even this would not be 100% guarantee that by the time you end up actually saving an entity with this will still be unused. The AllocateIDs() function can be used to obtain a continuous range of intIDs which the datastore will not use to generate intIDs by itself later on, so you are free to use the allocated range of intIDs safely. This also means that if there are concurrent requests also trying to save new entities (either in the same instance or in other instances), they will also never end up using these intIDs if identifier generation is left to the datastore.

Back to you question

Do you really need manual identifier assignment? In most of the cases this is only used/required if you already have a unique property of the entity which cannot be the same for 2 different entities (for example entity is Person which has a property IdentityCardId which is already unique to each person).

If you have such a unique property, you may use that which itself by nature ensures uniqueness. If you have no such property, then you should not use manual identifier assignment in the first place, you can just use/rely on automatic intID assignment of the datastore.

Note that you can have mixed identifiers of different entities of the same kind (e.g. you can have a Person with intID and another Person with name).

As noted above, intIDs and names are distinct. So AllocateIDs() does not take entities with name identifiers into account just because the name contains a valid number. From the datastore you don't get help to "allocate" a name identifier (analog to the AllocateIDs() to allocate intIDs), so it must be application logic to ensure the assigned name is unique else you will end up "overwriting"/replacing an existing entity.

Answer 2

You can avoid any possibility of a collision by prefixing an id with any string that you know you never use for your keys. For example:

String key = "id" + entity.getKey().getId();

And all of this is necessary only if you ever use a sequence of integers for your string keys.

Answer 3

The way to get the datastore to generate string keys for you is to use NewIncompleteKey.

ikey := datastore.NewIncompleteKey(ctx, "Thing", nil)
key, _ := datastore.Put(ctx, ikey, nil) // TODO: check the error
fmt.Println(key.Encode()) // randomly-generated string key

Then to get it back from the string key...

ks := req.FormValue("key") // e.g., from an HTTP request parameter
key, _ := datastore.DecodeKey(ks) // TODO: check the error
var t Thing
_ = datastore.Get(ctx, key, &t) // TODO: check the error