missing-data

I follow the method in this post to replace missing values with the group mode, but encounter the "IndexError: index out of bounds". df['SIC'] = df.groupby('CIK').SIC.apply(lambda x: x.fillna(x.mode()[0])) I guess this is probably because some groups have all missing values and do not have a mode. Is there a way to get around this? Thank you! mode is quite difficult, given that there really isn't any agreed upon way to deal with ties. Plus it's typically very slow. Here's one way that will be "fast". We'll define a function that calculates the mode for each group, then we can fill the missing

I'm conducting a factor analysis of several variables in R using factanal() (but am open to using other packages). I want to determine each case's factor score, but I want the factor scores to be unstandardized and on the original metric of the input variables. When I run the factor analysis and obtain the factor scores, they are standardized with a normal distribution of mean=0, SD=1, and are not on the original metric of the input variables. How can I obtain unstandardized factor scores that have the same metric as the input variables? Ideally, this would mean a similar mean, sd, range, and

I need to forward fill values in a column of a dataframe within groups. I should note that the first value in a group is never missing by construction. I have the following solutions at the moment. df = pd.DataFrame({'a': [1,1,2,2,2], 'b': [1, np.nan, 2, np.nan, np.nan]}) # desired output a b 1 1 1 1 2 2 2 2 2 2 Here are the three solutions that I've tried so far. # really slow solutions df['b'] = df.groupby('a')['b'].transform(lambda x: x.fillna(method='ffill')) df['b'] = df.groupby('a')['b'].fillna(method='ffill') # much faster solution, but more memory intensive and ugly all around tmp = df

I have the following data.frame, "subset" Time A B C 2016-10-07 06:16:46 NA NA 41 2016-10-07 06:26:27 40 39 42 2016-10-07 06:38:23 NA 40 NA 2016-10-07 06:41:06 42 42 44 2016-10-07 06:41:06 NA 42 44 2016-10-07 06:41:06 NA 42 44 2016-10-07 06:41:07 44 43 48 2016-10-07 06:41:41 NA 43 48 2016-10-07 06:42:44 45 42 48 2016-10-07 06:48:40 46 45 48 I would like to have a plot where "Time" is the x-axis, "A" is a line and "B" and "C" are points. However, when i plot this, the only line that appears for "A" is the one connecting the last 2 dots (45 and 46), because these are the only 2 consecutive

A <- data.frame(Item_A = c("00EF", "00EF", "00EF", "00EF", "00EF", "00FR", "00FR"), Item_B = c(NA, NA, NA, NA, "JAMES RIVER", NA, NA)) B <- data.frame(Item_A = c("00EF", "00EF", "00EF", "00FR", "00FR"), Item_B = c("JAMES RIVER", NA, "JAMES RIVER", "RICE MIDSTREAM", "RICE MIDSTREAM")) Expected: A <- data.frame(Item_A = c("00EF", "00EF", "00EF", "00EF", "00EF", "00FR", "00FR"), Item_B = c("JAMES RIVER", "JAMES RIVER", "JAMES RIVER", "JAMES RIVER", "JAMES RIVER", "RICE MIDSTREAM", "RICE MIDSTREAM")) B <- data.frame(Item_A = c("00EF", "00EF", "00EF", "00FR", "00FR"), Item_B = c("JAMES RIVER",

The Julia 1.0.0 documentation says this about missing values in Julia and R: In Julia, missing values are represented by the missing object rather than by NA. Use ismissing(x) instead of isna(x). The skipmissing function is generally used instead of na.rm=TRUE (though in some particular cases functions take a skipmissing argument). Here is example code in R that I would like to duplicate in Julia: > v = c(1, 2, NA, 4) > is.na(v) [1] FALSE FALSE TRUE FALSE (First note that is.na is the R function's correct spelling, not isna as shown in the quote above, but that is not my point.) If I follow

I have seen many questions that touch on this topic but haven't yet found an answer. If I have missed a question that does answer this question, please do mark this and point us to the question. Scenario: We have a benchmark dataset, we have imputation methods, we systematically delete values from the benchmark and use two different imputation methods. Thus we have a benchmark, imputedData1 and imputedData2. Question: Is there a function that can produce a number that represents the difference between the benchmark and imputedData1 or/and the difference between the benchmark and imputedData2.