Filling in missing values with forward-backward method with lag in SAS

£可爱£侵袭症+ 提交于 2019-12-13 12:30:31

问题


Assume that you have a table with user name, counter and score for each counter.

data have;
input user $  counter  score;
cards;
A 1 .
A 2 .
A 3 40
A 4 .
A 5 20
A 6 .
B 1 30
B 2 .
C 1 .
C 2 .
C 3 .
;
run;

Some scores are missing beween some counters, and you want to put the same score as previous counter. So the result will look like below:

A 1 40
A 2 40
A 3 40
A 4 40
A 5 20
A 6 20
B 1 30
B 2 30
C 1 .
C 2 .
C 3 .

I managed to fill the missing score values forward by using the lag function like below:

data result1a;
  set have(keep=user);
  by user;

  *Look ahead;
    merge have have(firstobs=2 keep=score rename=(score=_NextScore));

    if first.user then do;
        if score= . then score=_NextScore;
        end;
    else do;
        _PrevScore = lag(score);
        if score= . then score=_PrevScore;
    end;
    output;
run;

Then I sorted the table backward by using descending funtion on counter like below:

proc sort data = result1a out= result1b; 
by user descending counter ;
run;

Then finally I would fill the missing values forward in raaranged table (going backward according to the initial table) by using the lag function again like below.

I used the lag function in do-loop, because I wanted to update the previous value in each step (For example, the value 40 would be carried from the first score to the last score in the group all the way).

However, I get strange result. All missing values don't geta real value. Any idea about fixing the last data-step?

data result1c;
set result1b;
by user;

   if first.user then do;
        if score= . then score=_NextScore;
        else score = score;

        end;
   else do;
        _PrevScore = lag(score);
        if score= . then 
        score=_PrevScore;
        else score = score;
   end;
   output;
run;

回答1:


Don't need to use lag, use retain (or equivalent). Here's a double DoW loop solution that does it in one datastep (and, effectively, one read - it buffers the read so this is as efficient as a single read).

First we loop through the dataset to get the first score found, so we can grab that for the initial prev_score value. Then setting that, and re-looping through the rows for that user and outputting. There's no actual retain here since I am doing the looping myself, but it's similar to if there were a retain prev_score; and this was a normal data step loop. I don't actually retain it since I want it to go missing when a new user is met.

data want;
  do _n_ = 1 by 1 until (last.user);
    set have;
    by user;
    if missing(first_score) and not missing(score) then 
      first_score = score;

  end;
  prev_score = first_score;
  do _n_ = 1 by 1 until (last.user);
    set have;
    by user;
    if missing(score) then
      score = prev_score;
    prev_score = score;
    output;
  end;
run;



回答2:


lag() is a commonly misunderstood function. The name implies that when you call it SAS looks back at the previous row and grabs the value, but this is not at all the case.

In fact, lag<n>() is a function that creates a "queue" with n values. When you call lag<n>(x), it pushes the current value of x into that queue and reads a previous value from it (of course the pushing only occurs once per row). So if you have lag<n>() within a condition, the pushing only occurs when that condition is satisfied.

To fix your problem, you need the lag() function to run for every row, and to run after score has been corrected:

data result1c;
set result1b;
by user;
if first.user then do;
    if score= . then score=_NextScore;
    else score = score;
end;
else do;
    if score= . then 
    score=_PrevScore;
    else score = score;
end;
_PrevScore = lag(score);
output;
run;

EDIT: I got hung up on the misuse of lag and didn't present a working alternative. Because you're modifying score, it's a bad idea to use lag at all. Retain will work here:

data result1c;
set result1b;
by user;
retain _PrevScore;
if first.user then do;
    if score= . then score=_NextScore;
    else score = score;
end;
else do;
    if score= . then 
    score=_PrevScore;
    else score = score;
end;
_PrevScore = score;
output;
run;


来源:https://stackoverflow.com/questions/42961591/filling-in-missing-values-with-forward-backward-method-with-lag-in-sas

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!