How to detect and remove a column that contains only null values?

Question

In my table table1 there are 6 columns Locations,a,b,c,d,e.

Locations [a]   [b]   [c]  [d]   [e]

[1]       10.00 Null  Null 20.00 Null

[2]         


        

Answer 1

How to detect whether a given column has only the NULL value:

SELECT 1  -- no GROUP BY therefore use a literal
  FROM Locations
HAVING COUNT(a) = 0 
       AND COUNT(*) > 0;

The resultset will either consist of zero rows (column a has a non-NULL value) or one row (column a has only the NULL value). FWIW this code is Standard SQL-92.

Answer 2

SELECT * FROM table1 WHERE c IS NOT NULL  -- or also SELECT COUNT(*)

To detect if indeed this column has no values at all.

ALTER TABLE table1 DROP COLUMN c

is the query to remove the column if it is deemed desirable.

Answer 3

Here is a fast (and ugly) stored proc that takes the name of the table and print (or drop if you want it to) the fields that are full of nulls.

ALTER procedure mysp_DropEmptyColumns 
  @tableName nvarchar(max)
as begin
  declare @FieldName nvarchar(max)
  declare @SQL nvarchar(max)
  declare @CountDef nvarchar(max)
  declare @FieldCount int

  declare fieldNames cursor  local fast_forward for
    select c.name
      from syscolumns c 
        inner join sysobjects o on c.id=o.id
      where o.xtype='U'
        and o.Name=@tableName

  open fieldNames 
  fetch next from fieldNames into @FieldName
  while (@@fetch_status=0)
  begin
    set @SQL=N'select @Count=count(*) from "'+@TableName+'" where "'+@FieldName+'" is not null'
    SET @CountDef = N'@Count int output';
    exec sp_executeSQL @SQL, @CountDef, @Count = @FieldCount output
    if (@FieldCount=0)
    begin
      set @SQL = 'alter table '+@TableName+' drop column '+@FieldName
      /* exec sp_executeSQL @SQL */
      print @SQL
    end
    fetch next from fieldNames into @FieldName
  end

  close fieldNames
end

This uses a cursor, and is a bit slow and convoluted, but I suspect that this is a kind of procedure that you'll be running often

Answer 4

Try this stored procedure with your table name as input.

alter proc USP_DropEmptyColumns 

@TableName varchar(255)
as
begin

Declare @col varchar(255), @cmd varchar(max)

DECLARE getinfo cursor for
SELECT c.name FROM sys.tables t JOIN sys.columns c ON t.Object_ID = c.Object_ID
WHERE t.Name = @TableName

OPEN getinfo

FETCH NEXT FROM getinfo into @col

WHILE @@FETCH_STATUS = 0
BEGIN
    SELECT @cmd = 'IF NOT EXISTS (SELECT top 1 * FROM [' + @TableName + '] WHERE [' + @col + '] IS NOT NULL) 

                        BEGIN 
                        ALTER TABLE [' + @TableName + ']  DROP Column [' + @col + ']
                        end'
    EXEC(@cmd)

    FETCH NEXT FROM getinfo into @col
END

CLOSE getinfo
DEALLOCATE getinfo

end

Answer 5

PROC PRINT DATA=TABLE1;RUN;

PROC TRANSPOSE DATA=TABLE1 OUT=TRANS1;VAR A B C D E;RUN;

DATA TRANS2;SET TRANS1;IF COL1 = . AND COL2 = . THEN DELETE;RUN;

PROC TRANSPOSE DATA=TRANS2 OUT=TABLE2 (DROP=_NAME_);VAR COL1-COL2;RUN;

PROC PRINT DATA=TABLE2;RUN;

Answer 6

SQL is more about working on rows rather than columns.

If you're talking about deleting rows where c is null, use:

delete from table1 where c is null

If you're talking about dropping a column when all rows have null for that column, I would just find a time where you could lock out the DB from users and execute one of:

select c from table1 group by c
select distinct c from table1
select count(c) from table1 where c is not null

Then, if you only get back just NULL (or 0 for that last one), weave your magic (the SQL Server command may be different):

alter table table1 drop column c

Do this for whatever columns you want.

You really need to be careful if you're deleting columns. Even though they may be full of nulls, there may be SQL queries out there that use that column. Dropping the column will break those queries.