Not sure if this would be faster or slower than DavidG's suggestion, but in order to get the same matches with only one check, you can do:
DECLARE @categoryId INT
SET @categoryId = 3
SELECT *
FROM myTable
WHERE CHARINDEX(',' + CAST(@categoryId AS VARCHAR(MAX)) + ',', ',' + categoryIds + ',') > 0