Explanation of NHibernate HiLo

Question

I\'m struggling to get my head round how the HiLo generator works in NHibernate. I\'ve read the explanation here which made things a little clearer.

My understanding

Answer 1

I believe your understanding is more or less correct. The max_lo parameter is simply used to determine the number of Ids available for any given Hi value.

My best guess is that NHibernate's default max_lo value is 32768. Thus a Hi value of 1 would start your Ids at 32768 and run you right up to 65535. A Hi value of 2 would start at 65536 and run up another max_lo Ids.

Basically you use the max_lo value to control Id fragmentation. 32768 is likely not the optimal value for every situation.

It is important to note however that this only works within the scope of a SessionFactory. If you are stopping/starting your application and reinitializing the SessionFactory a whole bunch, it's going to increment the Hi value upon startup anyway and you're going to see your Ids jump pretty quickly.

Answer 2

For those wondering how to choose a good max_lo value, the trade-off is essentially between:

• Frequency with which you need to query a new hi value from the db.
• Maximum amount of unique numbers you actually can generate.

A lower max_lo will make sure there is no "waste" of id's, which in turn governs the moment at which you will hit the implicit limit of your datatype (which will likely be int). The price you pay is that each client needs to query and increase the hi value more frequently.

A higher max_lo is useful to reduce the frequency of queries that get and increment hi, but result in more waste.

The metrics you need to take into account to determine the optimal value are:

• Frequency at which new entities are created and need an ID
• Frequency at which the application restarts / gets recycled (anything that results in a new NHibernate SessionFactory)

Let's consider a web application that is hosted in IIS and is recycled every 24 hours. The entities are Customer and Order.

Now lets assume:

• 10000 new Orders per 24 hours
• 10 new customers per 24 hours

Then the perfect max_lo is 10000 for Orders and 10 for Customers. Of course, in the real world you can never determine it so precicely and clearly, but you should get the idea here!

Now let's consider different scenario where we pick totally wrong (ridiculous) max_lo's:

• Suppose a 10 customers are making orders simultaneously every second, with a max_lo of only 10 on orders, every second there is a superfluous database call to increment hi.
• Suppose your app is a desktop app and is installed on 50 clients (support staff?), that each start it about twice a day. Together they create about 100 helpdesk tickets a day. Now let's say we stick with the max_lo default of 32767. Hi is incremented 100 times a day (50 clients * 2), which means you will hit the maximum value of int in less than 2 years, should you have forgotten the important fact that hi gets incremented so frequently. A good max_lo here would be (100 tickets / 50 clients) = only 2.

Hopes this helps with conceptualizing the HiLo algorithm and its implications in general, while also giving you the maths to actually stick a number on max_lo.

Answer 3

NHibernate 3.1.1 does this to generate ID using HiLo

if (lo > maxLo)
{
    long hival = <GetNextHiFromDB>
    lo = hival == 0 ? 1 : 0;
    hi = hival * (this.maxLo + 1L);
}
long result = hi + lo;
lo++;
return result;

Inside NHibernate configuration you specify maxLo. If maxLo is set to 100 you will get 101 ids for each hi value.

Answer 4

Looking at the keys generated by my Nhibernate 3 HiLo objects, the algorithm looks like: (Hi * Lo) + Hi

So with my Hivalue in the DB as 390 and with my configuration as follows:

<id name="TimeclockId" column="TimeclockId" type="Int64" unsaved-value="0">
      <generator class="hilo">
        <param name="where">TableId = 1</param>
        <param name="table">HiValue</param>
        <param name="column">NextValue</param>
        <param name="max_lo">10</param>
      </generator>
    </id>

I restart my app pool and get (390 * 10) + 390 = 4290, the range being 4290 - 4300.

This is the reason why you get seemingly strange gaps in your primary keys because the next generated key from a hi value of 391 is 4301, and the range is 4301 - 4311.