Recursive CTE Concept Confusion

Question

I am trying to understand the concepts of using CTE in my SQL code. I have gone through a number of online posts explaining the concept but I cannot grasp how it iterates to pre

Answer 1

Well, a short introduction to recursive CTEs:

A recursive CTE is rather something iterativ, than really recursive. The anchor query is taken to get some initial result set. With this set we can dive deeper. Try these simple cases:

Just a counter, not even a JOIN needed...

The 1 of the anchor will lead to a 2 in the UNION ALL. This 2 is passed into the UNION ALL again and will be returned as a 3 and so on...

WITH recCTE AS
(
    SELECT 1 AS Mycounter 

    UNION ALL

    SELECT recCTE.MyCounter+1
    FROM recCTE 
    WHERE recCTE.MyCounter<10
)
SELECT * FROM recCTE;

A counter of 2 columns

This is exactly the same as above. But we have two columns and deal with them separately.

WITH recCTE AS
(
    SELECT 1 AS Mycounter1, 10 AS MyCounter2 

    UNION ALL

    SELECT recCTE.MyCounter1+1,recCTE.MyCounter2+1
    FROM recCTE 
    WHERE recCTE.MyCounter1<10
)
SELECT * FROM recCTE;

Now we have two rows in the initial query

Running alone, the initial query will return two rows. Both with the counter==1 and two different values for the Nmbr-column

WITH recCTE AS
(
    SELECT MyCounter=1, Nmbr FROM(VALUES(1),(10)) A(Nmbr)

    UNION ALL

    SELECT recCTE.MyCounter+1, recCTE.Nmbr+1
    FROM recCTE 
    WHERE recCTE.MyCounter<10
)
SELECT * FROM recCTE ORDER BY MyCounter,Nmbr;

Now we get 20 rows back, not 10 as in the examples before. This is, because both rows of the anchor are used independently.

We can use the recursive CTE in a JOIN

In this example we will create a derived set first, then we will join this to the recursive CTE. Guess why the first row carries "X" instead of "A"?

WITH SomeSet AS (SELECT * FROM (VALUES(1,'A'),(2,'B'),(3,'C'),(4,'D'),(5,'E'),(6,'F'),(7,'G'),(8,'H'),(9,'I'),(10,'J')) A(id,Letter))
,recCTE AS
(
    SELECT MyCounter=1, Nmbr,'X' AS Letter FROM(VALUES(1),(10)) A(Nmbr)

    UNION ALL

    SELECT recCTE.MyCounter+1, recCTE.Nmbr+1, SomeSet.Letter
    FROM SomeSet 
    INNER JOIN recCTE ON SomeSet.id=recCTE.MyCounter+1
    WHERE recCTE.MyCounter<10
)
SELECT * FROM recCTE ORDER BY MyCounter,Nmbr;

This will use a self-referring join to simulate your hierarchy, but with one gap-less chain

WITH SomeSet AS (SELECT * FROM (VALUES(1,'A',NULL),(2,'B',1),(3,'C',2),(4,'D',3),(5,'E',4),(6,'F',5),(7,'G',6),(8,'H',7),(9,'I',8),(10,'J',9)) A(id,Letter,Previous))
,recCTE AS
(
    SELECT id,Letter,Previous,' ' PreviousLetter FROM SomeSet WHERE Previous IS NULL

    UNION ALL

    SELECT SomeSet.id,SomeSet.Letter,SomeSet.Previous,recCTE.Letter
    FROM SomeSet 
    INNER JOIN recCTE ON SomeSet.Previous=recCTE.id
)
SELECT * FROM recCTE:

And now almost the same as before, but with several elements with the same "previous".

This is - in principles - your hierarchy

WITH SomeSet AS (SELECT * FROM (VALUES(1,'A',NULL),(2,'B',1),(3,'C',2),(4,'D',2),(5,'E',2),(6,'F',3),(7,'G',3),(8,'H',4),(9,'I',1),(10,'J',9)) A(id,Letter,Previous))
,recCTE AS
(
    SELECT id,Letter,Previous,' ' PreviousLetter FROM SomeSet WHERE Previous IS NULL

    UNION ALL

    SELECT SomeSet.id,SomeSet.Letter,SomeSet.Previous,recCTE.Letter
    FROM SomeSet 
    INNER JOIN recCTE ON SomeSet.Previous=recCTE.id
)
SELECT * FROM recCTE

Conclusio

The key points

• The anchor query must return at least one row, but may return many
• The second part must match the column list (as any UNION ALL query)
• The second part must refer to the cte in its FROM-clause
  • either directly, or
  • through a JOIN
• The second part will be called over and over using the result of the call before
• Each row is handled separately (a hidden RBAR)
• You can start with a Manager (top-most-node) and walk down by querying for employees with this manager id, or
• You can start with the lowest in hierarchy (the ones, where no other row exists, using their id as manager id) and move up the list
• As it is a hidden RBAR you can use this for row-by-row actions like string cummulation.

An example for the last statement

See how the column LetterPath is built.

WITH SomeSet AS (SELECT * FROM (VALUES(1,'A',NULL),(2,'B',1),(3,'C',2),(4,'D',2),(5,'E',2),(6,'F',3),(7,'G',3),(8,'H',4),(9,'I',1),(10,'J',9)) A(id,Letter,Previous))
,recCTE AS
(
    SELECT id,Letter,Previous,' ' PreviousLetter,CAST(Letter AS VARCHAR(MAX)) AS LetterPath FROM SomeSet WHERE Previous IS NULL

    UNION ALL

    SELECT SomeSet.id,SomeSet.Letter,SomeSet.Previous,recCTE.Letter,recCTE.LetterPath + SomeSet.Letter 
    FROM SomeSet 
    INNER JOIN recCTE ON SomeSet.Previous=recCTE.id
)
SELECT * FROM recCTE