Convert data from long format to wide format with multiple measure columns

Question

I am having trouble figuring out the most elegant and flexible way to switch data from long format to wide format when I have more than one measure variable I want to bring

Answer 1

Note -Sept 2019: within tidyr, the gather()+spread() approach (described in this answer) has more or less been replaced by the pivot_wider() approach (described in `this newer tidyr answer). For current info about the transition, see the pivoting vignette.

---

Here's a solution with the tidyr package, which has essentially replaced reshape and reshape2. As with those two packages, the strategy it to make the dataset longer first, and then wider.

library(magrittr); requireNamespace("tidyr"); requireNamespace("dplyr")
my.df %>%
  tidyr::gather(key=variable, value=value, c(X, Y)) %>%   # Make it even longer.
  dplyr::mutate(                                          # Create the spread key.
    time_by_variable   = paste0(variable, "_", TIME)
  ) %>%
  dplyr::select(ID, time_by_variable, value) %>%          # Retain these three.
  tidyr::spread(key=time_by_variable, value=value)        # Spread/widen.

After the tidyr::gather() call, the intermediate dataset is:

ID TIME variable value
1   A    1        X     1
2   B    1        X     2
3   C    1        X     3
...
28  A    5        Y    28
29  B    5        Y    29
30  C    5        Y    30

The eventual result is:

  ID X_1 X_2 X_3 X_4 X_5 Y_1 Y_2 Y_3 Y_4 Y_5
1  A   1   4   7  10  13  16  19  22  25  28
2  B   2   5   8  11  14  17  20  23  26  29
3  C   3   6   9  12  15  18  21  24  27  30

tidyr::unite() is an alternative, suggested by @JWilliman. This is functionally equivalent to the dplyr::mutate() and dplyr::select() combination above, when the remove parameter is true (which is the default).

If you're not accustomed to this type of manipulation, the tidyr::unite() may be a small obstacle because it's one more function you have to learn & remember. However, it's benefits include (a) more concise code (ie, four lines are replaced by one) and (b) fewer places to repeat variable names (ie, you don't have to repeat/modify variables in the dplyr::select() clause).

my.df %>%
  tidyr::gather(key=variable, value=value, c(X, Y)) %>%           # Make it even longer.
  tidyr::unite("time_by_variable", variable, TIME, remove=T) %>%  # Create the spread key `time_by_variable` while simultaneously dropping `variable` and `TIME`.
  tidyr::spread(key=time_by_variable, value=value)                # Spread/widen.

Answer 2

The pivot_wider() function is tidyr's 2nd generation approach (released in tidyr 1.0.0).

library(magrittr); requireNamespace("tidyr");

my.df %>%
  tidyr::pivot_wider(
    names_from  = c(TIME), # Can accommodate more variables, if needed.
    values_from = c(X, Y)
  )

Result:

# A tibble: 3 x 11
  ID      X_1   X_2   X_3   X_4   X_5   Y_1   Y_2   Y_3   Y_4   Y_5
  <fct> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int>
1 A         1     4     7    10    13    16    19    22    25    28
2 B         2     5     8    11    14    17    20    23    26    29
3 C         3     6     9    12    15    18    21    24    27    30

This is probably preferable to the previous tidyr approach (that uses a combination of gather() and spread()).

More capabilities are described in the pivoting vignette. This example is particularly concise because your desired specifications match the defaults of the id_cols and names_sep.

Answer 3

In order to handle multiple variables like you want, you need to melt the data you have before casting it.

library("reshape2")

dcast(melt(my.df, id.vars=c("ID", "TIME")), ID~variable+TIME)

which gives

  ID X_1 X_2 X_3 X_4 X_5 Y_1 Y_2 Y_3 Y_4 Y_5
1  A   1   4   7  10  13  16  19  22  25  28
2  B   2   5   8  11  14  17  20  23  26  29
3  C   3   6   9  12  15  18  21  24  27  30

---

EDIT based on comment:

The data frame

num.id = 10 
num.time=10 
my.df <- data.frame(ID=rep(LETTERS[1:num.id], num.time), 
                    TIME=rep(1:num.time, each=num.id), 
                    X=1:(num.id*num.time), 
                    Y=(num.id*num.time)+1:(2*length(1:(num.id*num.time))))

gives a different result (all entries are 2) because the ID/TIME combination does not indicate a unique row. In fact, there are two rows with each ID/TIME combinations. reshape2 assumes a single value for each possible combination of the variables and will apply a summary function to create a single variable is there are multiple entries. That is why there is the warning

Aggregation function missing: defaulting to length

You can get something that works if you add another variable which breaks that redundancy.

my.df$cycle <- rep(1:2, each=num.id*num.time)
dcast(melt(my.df, id.vars=c("cycle", "ID", "TIME")), cycle+ID~variable+TIME)

This works because cycle/ID/time now uniquely defines a row in my.df.

Answer 4

   reshape(my.df,
           idvar = "ID",
           timevar = "TIME",
           direction = "wide")

gives

  ID X.1 Y.1 X.2 Y.2 X.3 Y.3 X.4 Y.4 X.5 Y.5
1  A   1  16   4  19   7  22  10  25  13  28
2  B   2  17   5  20   8  23  11  26  14  29
3  C   3  18   6  21   9  24  12  27  15  30

Answer 5

Using the data.table_1.9.5, this can be done without the melt as it can handle multiple value.var columns. You can install it from here

 library(data.table)
 dcast(setDT(my.df), ID~TIME, value.var=c('X', 'Y'))
 #   ID 1_X 2_X 3_X 4_X 5_X 1_Y 2_Y 3_Y 4_Y 5_Y
 #1:  A   1   4   7  10  13  16  19  22  25  28
 #2:  B   2   5   8  11  14  17  20  23  26  29
 #3:  C   3   6   9  12  15  18  21  24  27  30