Tricky TSQL: NOT IN

We’ve done this before , but we can go one better this time.

Let’s take this step by step.

NULL means “I don’t know”.It stand for an unknown value.

Nothing can be equal to NULL.We simply can’t say that 1 = NULL, or ‘ABBA’ = NULL, because we don’t know what value NULL might possibly be in the real world.

Nothing can be NOT equal to NULL.We can’t say that 1 <>NULL, or ‘ABBA’ <>NULL. NULL may well turn out to be 1, or ABBA, or whatever valuewe’re trying to check. We simply don’t know.

A simple example

So let’s take a very simple example, to demonstrate how this works. This particular query will return the value‘True’, because 1 is demonstrably equal to 1:

IF 1 = 1
SELECT 'True!';

Yep, that makes sense. I can tell: 1 = 1.

This query will not return‘True’; it will return nothing, because 1 cannot be proven to be equal to NULL:

IF 1 = NULL
SELECT 'True!';

Wehave an unknown, and nothing can possibly be equal to the unknown.

Another way to put this, is to use the keyword IN:

IF 1 IN ( SELECT NULL AS a )
SELECT 'True';

That parenthetical SELECT NULL simply returns NULL, so the query ends up asking: “is the value 1 in the set of values (NULL) ?” We have no idea, so the querydoes not return ‘True’.

Almost the same thing, but trickier

Here’s yet another query that will not return‘True’; 1 cannot be proven to be NOT equal to NULL:

IF 1 <> NULL
SELECT 'True!';

NULL isunknown, and nothing can possibly be NOT equal to the unknown.

NULL will mess up your NOT IN queries

And the absolute trickiest bit the one that willtrip you up, if you’re not thinking? NOT IN .

This query also returns nothing!

IF 1 NOT IN ( SELECT 2 AS a
UNION
SELECT NULL AS a )
SELECT 'True';

The IF statement asks: “is 1 in the collection of values (2, NULL)?” And we must say, “ah, no idea. That can’t be proven. Therefore we can’t return true.”

The value 1 cannot be determined to not be in the set of (2, NULL).

Play around withthese, let it sink in. And watch out for NULLs in your NOT IN result sets!

OneFix So, always use IS NULL or IS NOT NULL for your NULL comparison needs. And if you have a [value] NOT IN situation, read on.

Here’s what I did* to fix this exact situation in a recent query:

SELECT ID
FROM Table1
WHERE ID NOT IN ( SELECT ID
FROM Table2
WHERE ID > 0 );

While we could have used ID IS NOT NULL instead of ID > 0 , it would make any relevant index on Table2 useless to us. In this particular case, ID is restricted to onlypositive above 0, and NULL.

To read up on even moresolutions, see Mladen Pradjic’s article JOIN vs IN EXISTS .

Happy days,

Jen

www.MidnightDBA.com/Jen