How to get the next number in a sequence

Question

I have a table like this:

+----+-----------+------+-------+--+
| id | Part      | Seq  | Model |  |
+----+-----------+------+-------+--+
| 1  | Head      | 0         


        

Answer 1

If you do not maintain a counter table, there are two options. Within a transaction, first select the MAX(seq_id) with one of the following table hints:

1. WITH(TABLOCKX, HOLDLOCK)
2. WITH(ROWLOCK, XLOCK, HOLDLOCK)

TABLOCKX + HOLDLOCK is a bit overkill. It blocks regular select statements, which can be considered heavy even though the transaction is small.

A ROWLOCK, XLOCK, HOLDLOCK table hint is probably a better idea (but: read the alternative with a counter table further on). The advantage is that it does not block regular select statements, ie when the select statements don't appear in a SERIALIZABLE transaction, or when the select statements don't provide the same table hints. Using ROWLOCK, XLOCK, HOLDLOCK will still block insert statements.

Of course you need to be sure that no other parts of your program select the MAX(seq_id) without these table hints (or outside a SERIALIZABLE transaction) and then use this value to insert rows.

Note that depending on the number of rows that are locked this way, it is possible that SQL Server will escalate the lock to a table lock. Read more about lock escalation here.

The insert procedure using WITH(ROWLOCK, XLOCK, HOLDLOCK) would look as follows:

DECLARE @target_model INT=3;
DECLARE @part VARCHAR(128)='Spine';
BEGIN TRY
    BEGIN TRANSACTION;
    DECLARE @max_seq INT=(SELECT MAX(seq) FROM dbo.table_seq WITH(ROWLOCK,XLOCK,HOLDLOCK) WHERE model=@target_model);
    IF @max_seq IS NULL SET @max_seq=0;
    INSERT INTO dbo.table_seq(part,seq,model)VALUES(@part,@max_seq+1,@target_model);
    COMMIT TRANSACTION;
END TRY
BEGIN CATCH
    ROLLBACK TRANSACTION;
END CATCH

---

An alternative and probably a better idea is to have a counter table, and provide these table hints on the counter table. This table would look like the following:

CREATE TABLE dbo.counter_seq(model INT PRIMARY KEY, seq_id INT);

You would then change the insert procedure as follows:

DECLARE @target_model INT=3;
DECLARE @part VARCHAR(128)='Spine';
BEGIN TRY
    BEGIN TRANSACTION;
    DECLARE @new_seq INT=(SELECT seq FROM dbo.counter_seq WITH(ROWLOCK,XLOCK,HOLDLOCK) WHERE model=@target_model);
    IF @new_seq IS NULL 
        BEGIN SET @new_seq=1; INSERT INTO dbo.counter_seq(model,seq)VALUES(@target_model,@new_seq); END
    ELSE
        BEGIN SET @new_seq+=1; UPDATE dbo.counter_seq SET seq=@new_seq WHERE model=@target_model; END
    INSERT INTO dbo.table_seq(part,seq,model)VALUES(@part,@new_seq,@target_model);
    COMMIT TRANSACTION;
END TRY
BEGIN CATCH
    ROLLBACK TRANSACTION;
END CATCH

The advantage is that fewer row locks are used (ie one per model in dbo.counter_seq), and lock escalation cannot lock the whole dbo.table_seq table thus blocking select statements.

You can test all this and see the effects yourself, by placing a WAITFOR DELAY '00:01:00' after selecting the sequence from counter_seq, and fiddling with the table(s) in a second SSMS tab.

---

PS1: Using ROW_NUMBER() OVER (PARTITION BY model ORDER BY ID) is not a good way. If rows are deleted/added, or ID's changed the sequence would change (consider invoice id's that should never change). Also in terms of performance having to determine the row numbers of all previous rows when retrieving a single row is a bad idea.

PS2: I would never use outside resources to provide locking, when SQL Server already provides locking through isolation levels or fine-grained table hints.

Answer 2

Let's first list the challenges:

1. We cannot use a normal constraint as there are existing null values and we also need to cater for duplicates as well as gaps - if we look at the existing data. This is fine, we will figure it out ;-> in step 3

2. We require safety for concurrent operations (thus some form or mix of transactions, isolation levels and possibly a "kinda SQL mutex".) Gut feel here is a stored proc for a couple of reasons:

   2.1 It protects more easily from sql injection

   2.2 We can control the isolation levels (table locking) more easily and recover from some issues which come with this kind of requirement

   2.3 We can use application level db locks to control the concurrency

3. We must store or find the next value on every insert. The word concurrency tells us already that there will be contention and probably high throughput (else please stick to single threads). So we must already be thinking: do not read from the same table you want to write to in an already complicated world.

So with that short prequel, let's attempt a solution:

As a start, we are creating your original table and then also a table to hold the sequence (BodyPartsCounter) which we are setting to the last used sequence + 1:

    CREATE TABLE BodyParts
        ([id] int identity, [Part] varchar(9), [Seq] varchar(4), [Model] int)
    ;

    INSERT INTO BodyParts
        ([Part], [Seq], [Model])
    VALUES
        ('Head', NULL, 3),
        ('Neck', '1', 3),
        ('Shoulders', '2', 29),
        ('Shoulders', '2', 3),
        ('Stomach', '5', 3)
    ;

    CREATE TABLE BodyPartsCounter
        ([id] int
        , [counter] int)
    ;

    INSERT INTO BodyPartsCounter
        ([id], [counter])
    SELECT 1, MAX(id) + 1 AS id FROM BodyParts
    ;

Then we need to create the stored procedure which will do the magic. In short, it acts as a mutex, basically guaranteeing you concurrency (if you do not do inserts or updates into the same tables elsewhere). It then get's the next seq, updates it and inserts the new row. After this has all happened it will commit the transaction and release the stored proc for the next waiting calling thread.

SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author:      Charlla
-- Create date: 2016-02-15
-- Description: Inserts a new row in a concurrently safe way
-- =============================================
CREATE PROCEDURE InsertNewBodyPart 
@bodypart varchar(50), 
@Model int = 3
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;

    BEGIN TRANSACTION;

    -- Get an application lock in your threaded calls
    -- Note: this is blocking for the duration of the transaction
    DECLARE @lockResult int;
    EXEC @lockResult = sp_getapplock @Resource = 'BodyPartMutex', 
                   @LockMode = 'Exclusive';
    IF @lockResult = -3 --deadlock victim
    BEGIN
        ROLLBACK TRANSACTION;
    END
    ELSE
    BEGIN
        DECLARE @newId int;
        --Get the next sequence and update - part of the transaction, so if the insert fails this will roll back
        SELECT @newId = [counter] FROM BodyPartsCounter WHERE [id] = 1;
        UPDATE BodyPartsCounter SET [counter] = @newId + 1 WHERE id = 1;

        -- INSERT THE NEW ROW
        INSERT INTO dbo.BodyParts(
            Part
            , Seq
            , Model
            )
            VALUES(
                @bodypart
                , @newId
                , @Model
            )
        -- END INSERT THE NEW ROW
        EXEC @lockResult = sp_releaseapplock @Resource = 'BodyPartMutex';
        COMMIT TRANSACTION;
    END;

END
GO

Now run the test with this:

EXEC    @return_value = [dbo].[InsertNewBodyPart]
    @bodypart = N'Stomach',
    @Model = 4

SELECT  'Return Value' = @return_value

SELECT * FROM BodyParts;
SELECT * FROM BodyPartsCounter

This all works - but be careful - there's a lot to consider with any kind of multithreaded app.

Hope this helps!

Answer 3

I would not try to store the Seq value in the table in the first place.

As you said in the comments, your ID is IDENTITY, which increases automatically in a very efficient and concurrent-safe way by the server. Use it for determining the order in which rows were inserted and the order in which the Seq values should be generated.

Then use ROW_NUMBER to generate values of Seq partitioned by Model (the sequence restarts from 1 for each value of Model) as needed in the query.

SELECT
    ID
    ,Part
    ,Model
    ,ROW_NUMBER() OVER(PARTITION BY Model ORDER BY ID) AS Seq
FROM YourTable