SSIS LookUp is not dealing with NULLs like the docs say it should

Question

I have an SSIS data flow that uses a lookup. Sometimes the value to be looked up (in my stream, not in the lookup table) is null.

The MSDN Docs say:

Answer 1

You can use a Conditional Split to break the data into two sets, one where the given value is null and the rest. Only perform the lookup on the output that has non-null values and then combine the results back with the data set that contains the null values using a Union All. You can still trap for unmatched values on the lookup and not worry about having to add a null entry to your lookup table

Answer 2

I know this is a late answer, but for anyone searching on this like I was, I found this to be the simplest answer:

In the lookup connection, use a SQL query to retrieve your data and add UNION SELECT NULL, NULL to the bottom.

For example:

SELECT CarId, CarName FROM Cars 
UNION SELECT NULL, NULL

Preview will show an additional row of CarId = Null and CarName = Null that will be available in the lookpup.

Answer 3

To work around this issue within SSIS, there is an alternative approach to a previous SO answer which can be applied.

Within the Lookup Transformation, you can redirect rows on error and then pass them to another destination which can simply be the same intended destination table within your database.

Therefore your destination table within the database will still receive all the rows (477 in screen shot below).

This approach therefore avoids the need to put dummy NULL values into your lookup table within your database, with the trade offs being:

• An extra step within your SSIS package.
• Error rows (non-NULL non-matches in this case) will always be loaded into the destination table. To help identify these rouge records, you can export the destination table into a txt file and then diff with the input source file to see any differences.

Answer 4

I had never noticed that line in BOL about the full cache mode. The answer to your problem is what you've already stated about having NULL in the reference set of data. A quick proof would be these two sample data flows.

In this example, I generate 4 rows of data: 1, 2, 3, and NULL (Column_isNull = true) and hit an in memory table that has all the values and perform a lookup between Column in the data flow and c1 defined in the in-memory table. It blows up as you described

I then added one more value to the lookup table, NULL and voila, the full-cache lookup is able to match NULL to NULL.

Takeaway

To make a NULL input value match in a lookup component, the reference data set must have a corresponding NULL value available as well as have the cache mode set to FULL.