CTE slow performance on Left join

Question

I need to provide a report that shows all users on a table and their scores. Not all users on said table will have a score, so in my solution I calculate the score first using a

Answer 1

UPDATE

I accepted Alan's answer, i ended up doing the following. Posting examples hoping the formatting helps someone, it slowed me down a bit...or maybe I am just slow heh heh.

1. Changed my Scalar UDF to InlineTVF

SCALAR Function 1-

    ALTER FUNCTION [dbo].[fn_WorkDaysAge]
(
    -- Add the parameters for the function here
    @first_date DATETIME,
    @second_date DATETIME
)
RETURNS int
AS
BEGIN
    -- Declare the return variable here
    DECLARE @WorkDays int

    -- Add the T-SQL statements to compute the return value here
SELECT @WorkDays = COUNT(*)
FROM DateDimension
WHERE Date BETWEEN @first_date AND @second_date
AND workingday = '1' 

    -- Return the result of the function
    RETURN @WorkDays

END

iTVF function 1-

    ALTER FUNCTION [dbo].[fn_iTVF_WorkDaysAge] 
(   
    -- Add the parameters for the function here
 @FirstDate as Date, 
 @SecondDate as Date
)
RETURNS TABLE  AS RETURN 

SELECT WorkDays = COUNT(*)
FROM DateDimension
WHERE Date BETWEEN @FirstDate AND @SecondDate
AND workingday = '1' 

I then updated my next function the same way. I added the CROSS APPLY (something ive personally not used, im still a newbie) as indicated below and replaced the UDFs with the field names in my case statement.

Old Code

INNER JOIN tblTauClassList AS T
  ON T.SaRacf = racf
WHERE
--FILTERS BY A ROLLING 15 BUSINESS DAYS UNLESS THE DAYS BETWEEN CURRENT DATE AND TAU START DATE ARE <15
agent_stats.DateTime >=
    CASE WHEN dbo.fn_WorkDaysAge(TauStart, GETDATE()) <15 THEN TauStart ELSE
        dbo.fn_WorkDate15(TauStart) 
    END

New Code

INNER JOIN tblTauClassList AS T
  ON T.SaRacf = racf
--iTVFs
CROSS APPLY dbo.fn_iTVF_WorkDaysAge(TauStart, GETDATE()) as age
CROSS APPLY dbo.fn_iTVF_WorkDate_15(TauStart) as roll
WHERE
--FILTERS BY A ROLLING 15 BUSINESS DAYS UNLESS THE DAYS BETWEEN CURRENT DATE AND TAU START DATE ARE <15
agent_stats.DateTime >=
    CASE WHEN age.WorkDays <15 THEN TauStart ELSE
        roll.Date 
    END

New code runs in 3-4 seconds. I will go back and index the appropriate tables per your recommendation and probably gain more efficiency there.

Cannot thank you enough!