missing-data

问题 This is what my data.table/dataframe looks lke library(data.table) dt <- fread(' STATE ZIP PA 19333 PA 19327 PA 19333 PA NA PA 19355 PA 19333 PA NA PA 19355 PA NA ') I have three missing values in the ZIP column. I want to fill the missing values with nonmissing sample values of ZIPs according to their probability of occuring in the dataset. So for example ZIP 19333 occurs three times in the dataset and ZIP 19355 occurs twice in the dataset and 19327 occurs once. So ZIP 19333 has 50%

问题 This question already has answers here : Remove rows with all or some NAs (missing values) in data.frame (16 answers) Closed 4 years ago . I have a data frame like this col_1 col_2 col_3 col_4 12344 53445 34335 AAA 12545 56565 12123 AAB NA 54556 32323 ABB NA NA NA NA 43434 65654 NA ABA I want to get rows with at least non-NA value, or put another way, rows with all NAs (row 5 in this case) should be removed. Can you give me some advice? 回答1: if your data frame is named dta: dta[rowSums(!is.na

问题 I'm having an issue where my Hadoop job on AWS's EMR is not being saved to S3. When I run the job on a smaller sample, the job stores the output just fine. When I run the same command but on my full dataset, the job completes again, but there is nothing existing on S3 where I specified my output to go. Apparently there was a bug with AWS EMR in 2009, but it was "fixed". Anyone else ever have this problem? I still have my cluster online, hoping that the data is buried on the servers somewhere.

问题 I have a problem with NA in a factor variable since ggplot includes them in the plot as if they are another category/level. I would like to drop the missing data. I am sorry I don't have code handy at the moment, I tried to remove factor levels from dataset that I found at data() and it did not work. Had someone the same problem? I tried the solution suggested here Remove unused factor levels from a ggplot bar plot but I get an error Error: unexpected symbol in: mycode Can someone suggest

问题 I have a data set in which the variable GENDER containing two levels Male(M) and Female(F) has lot of Missing values . How do i deal with missing value? What are the different methods to handle these missing values. Any help would be appreciated. 回答1: There are several techniques in order to estimate a missing value. I've been writing a paper for a project at Uni regarding such methods. I will briefly explain 5 commonly used missing data imputation techniques. Hereinafter we will consider a

问题 I have a data set in which the variable GENDER containing two levels Male(M) and Female(F) has lot of Missing values . How do i deal with missing value? What are the different methods to handle these missing values. Any help would be appreciated. 回答1: There are several techniques in order to estimate a missing value. I've been writing a paper for a project at Uni regarding such methods. I will briefly explain 5 commonly used missing data imputation techniques. Hereinafter we will consider a

问题 please take a look at this: http://jsfiddle.net/2rNzr/ var chart = new Highcharts.Chart({ chart: { renderTo: 'container' }, xAxis: { categories: ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'] }, series: [{ data: [29.9, '', 106.4, 129.2, 144.0, 176.0, 135.6, 148.5, 216.4, 194.1, 95.6, 54.4] }] }); you'll notice that the data has a blank value in it (the second value), which causes the line graph to display incorrectly. Is this a bug? What is the correct

问题 please take a look at this: http://jsfiddle.net/2rNzr/ var chart = new Highcharts.Chart({ chart: { renderTo: 'container' }, xAxis: { categories: ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'] }, series: [{ data: [29.9, '', 106.4, 129.2, 144.0, 176.0, 135.6, 148.5, 216.4, 194.1, 95.6, 54.4] }] }); you'll notice that the data has a blank value in it (the second value), which causes the line graph to display incorrectly. Is this a bug? What is the correct

问题 How to handle missing values in datasets before applying machine learning algorithm??. I noticed that it is not a smart thing to drop missing NAN values. I usually do interpolate (compute mean) using pandas and fill it up the data which is kind of works and improves the classification accuracy but may not be the best thing to do. Here is a very important question. What is the best way to handle missing values in data set? For example if you see this dataset, only 30% has original data.

问题 I'm looking for something similar to na.locf() in the zoo package, but instead of always using the previous non- NA value I'd like to use the nearest non- NA value. Some example data: dat <- c(1, 3, NA, NA, 5, 7) Replacing NA with na.locf (3 is carried forward): library(zoo) na.locf(dat) # 1 3 3 3 5 7 and na.locf with fromLast set to TRUE (5 is carried backwards): na.locf(dat, fromLast = TRUE) # 1 3 5 5 5 7 But I wish the nearest non- NA value to be used. In my example this means that the 3