How to Extract keywords from a Data Frame in R

问题

I am new to text-mining in R. I want to remove stopwords (i.e. extract keywords) from my data frame's column and put those keywords into a new column.

I tried to make a corpus, but it didn't help me.

df$C3 is what I currently have. I would like to add column df$C4, but I can't get it to work.

df <- structure(list(C3 = structure(c(3L, 4L, 1L, 7L, 6L, 9L, 5L, 8L, 
       10L, 2L), .Label = c("Are doing good", "For the help", "hello everyone", 
       "hope you all", "I Hope", "I need help", "In life", "It would work", 
       "On Text-Mining", "Thanks"), class = "factor"), C4 = structure(c(2L, 
       4L, 1L, 6L, 3L, 7L, 5L, 9L, 8L, 3L), .Label = c("doing good", 
       "everyone", "help", "hope", "Hope", "life", "Text-Mining", "Thanks", 
       "work"), class = "factor")), .Names = c("C3", "C4"), row.names = c(NA, 
       -10L), class = "data.frame")

head(df)
#               C3          C4
# 1 hello everyone    everyone
# 2   hope you all        hope
# 3 Are doing good  doing good
# 4        In life        life
# 5    I need help        help
# 6 On Text-Mining Text-Mining

回答1:

This solution uses packages dplyr and tidytext.

library(dplyr)
library(tidytext)

# subset of your dataset
dt = data.frame(C1 = c(108,20, 999, 52, 400),
                C2 = c(1,3,7, 6, 9),
                C3 = c("hello everyone","hope you all","Are doing good","in life","I need help"), stringsAsFactors = F)

# function to combine words (by pasting one next to the other)
f = function(x) { paste(x, collapse = " ") }

dt %>%
  unnest_tokens(word, C3) %>%      # split phrases into words
  filter(!word %in% stop_words$word) %>%   # keep appropriate words
  group_by(C1, C2) %>%             # for each combination of C1 and C2
  summarise(word = f(word)) %>%    # combine multiple words (if there are multiple)
  ungroup()                        # forget the grouping

# # A tibble: 2 x 3
#        C1    C2  word
#      <dbl> <dbl> <chr>
#   1    20     3  hope
#   2    52     6  life

The problem here is that the "stop words" built in that package filter out some of the words you want to keep. Therefore, you have to add a manual step where you specify words you need to include. You can do something like this:

dt %>%
  unnest_tokens(word, C3) %>%      # split phrases into words
  filter(!word %in% stop_words$word | word %in% c("everyone","doing","good")) %>%   # keep appropriate words
  group_by(C1, C2) %>%             # for each combination of C1 and C2
  summarise(word = f(word)) %>%    # combine multiple words (if there are multiple)
  ungroup()                        # forget the grouping

# # A tibble: 4 x 3
#        C1    C2       word
#      <dbl> <dbl>      <chr>
#   1    20     3       hope
#   2    52     6       life
#   3   108     1   everyone
#   4   999     7 doing good

回答2:

This is one of the first things I did in R, it may not be the best but something like:

library(stringi)

 df2 <- do.call(rbind, lapply(stop$stop, function(x){
    t <- data.frame(c1= df[,1], c2 = df[,2], words = stri_extract(df[,3], coll=x))
    t<-na.omit(t)}))

Example data:

 df = data.frame(c1 = c(108,20,99), c2 = c(1,3,7), c3 = c("hello everyone", "hope you all", "are doing well"))

 stop = data.frame(stop = c("you", "all"))

Then after you can reshapedf2 using:

df2 = data.frame(c1 = unique(u$c1), c2 = unique(u$c2), words = paste(u$words, collapse= ','))

Then cbind df and df2

回答3:

I would use the tm-package. It has a little dictionary with english stopwords. You can replace these stopwords with a white space using gsub():

library(tm)
prep      <- tolower(paste(" ", df$C3, " "))
regex_pat <- paste(stopwords("en"), collapse = " | ")
df$C4     <- gsub(regex_pat, " ", prep)
df$C4     <- gsub(regex_pat, " ", df$C4)
#    C3               C4
# 1  hello everyone   hello everyone  
# 2    hope you all             hope  
# 3  Are doing good             good  
# 4         In life             life  
# 5     I need help        need help

You can easily add new words like c("hello", "othernewword", stopwords("en")).

来源：https://stackoverflow.com/questions/45371139/how-to-extract-keywords-from-a-data-frame-in-r

标签

corpus