I am considering adding a column to a database table, and that column will be null possibly for most rows however I do want to be able to query on that column for particular
I would suggest a filtered index, such as WHERE column IS NOT NULL;
- this will allow you to create an index that only bothers to index the non-NULL values, and ignores all of the rows with no value. You'll probably want to make sure the index covers the queries you want to run with this type of predicate, so that you don't have to go back into the whole table to lookup the other columns the query needs to output (or use in a join, or otherwise filter, etc).
More details here.