data-cleaning

I am dealing with pandas DataFrames like this: id x 0 1 10 1 1 20 2 2 100 3 2 200 4 1 NaN 5 2 NaN 6 1 300 7 1 NaN I would like to replace each NAN 'x' with the previous non-NAN 'x' from a row with the same 'id' value: id x 0 1 10 1 1 20 2 2 100 3 2 200 4 1 20 5 2 200 6 1 300 7 1 300 Is there some slick way to do this without manually looping over rows? You could perform a groupby/forward-fill operation on each group: import numpy as np import pandas as pd df = pd.DataFrame({'id': [1,1,2,2,1,2,1,1], 'x':[10,20,100,200,np.nan,np.nan,300,np.nan]}) df['x'] = df.groupby(['id'])['x'].ffill() print

When I don't use a pipe, I can change the original daframe using this command df<-slice(df,-c(1:3))%>% # delete top 3 rows df<-select(df,-c(Col1,Col50,Col51)) # delete specific columns How would one do this with a pipe? I tried this but the slice and select functions don't change the original dataframe. df%>% slice(-c(1:3))%>% select(-c(Col1,Col50,Col51)) I'd like to change the original df. You can definitely do the assignment by using an idiom such as df <- df %>% ... or df %>% ... -> df . But you could also avoid redundancy (i.e., stating df twice) by using the magrittr compound assignment

For eg - Say I have data in the following format - Current Format I would need the data to be formatted in the following format for ease of use - Required Format Of course the data contains a lot more records - I'm looking for an easy way to transpose data in this way for large sets of data. Any help will be appreciated :) This is very easy with PowerQuery. It is inbuilt for Excel 2016 and a freely available add in for Version from 2010 to 2013. You would set your data up as a table excluding the first row which contains the text Number of Cases (Ctrl + T whilst bring up window to create table