Remove punctuation but keep hyphenated phrases in R text cleaning

问题

Is there any effective way to remove punctuation in text but keeping hyphenated expressions, such as "accident-prone"?

I used the following function to clean my text

clean.text = function(x)
{
  # remove rt
  x = gsub("rt ", "", x)
  # remove at
  x = gsub("@\\w+", "", x)
  x = gsub("[[:punct:]]", "", x)
  x = gsub("[[:digit:]]", "", x)
  # remove http
  x = gsub("http\\w+", "", x)
  x = gsub("[ |\t]{2,}", "", x)
  x = gsub("^ ", "", x)
  x = gsub(" $", "", x)
  x = str_replace_all(x, "[^[:alnum:][:space:]'-]", " ")
  #return(x)
}

and apply it on hyphenated expressions that returned

my_text <- "accident-prone"
new_text <- clean.text(text)
new_text
[1] "accidentprone"

while my desired output is

"accident-prone"

I have referenced this thread but didn't find it worked on my situation. There must be some regex things that I haven't figured out. It will be really appreciated if someone could enlighten me on this.

回答1:

Putting my two cents in, you could use (*SKIP)(*FAIL) with perl = TRUE and remove any non-word characters:

data <- c("my-test of #$%^&*", "accident-prone")
(gsub("(?<![^\\w])[- ](?=\\w)(*SKIP)(*FAIL)|\\W+", "", data, perl = TRUE))

Resulting in

[1] "my-test of"     "accident-prone"

See a demo on regex101.com.

Here the idea is to match what you want to keep

(?<![^\\w])[- ](?=\\w)
# a whitespace or a dash between two word characters
# or at the very beginning of the string

let these fail with (*SKIP)(*FAIL) and put what you want to be removed on the right side of the alternation, in this case

\W+

effectively removing any non-word-characters not between word characters.
You'd need to provide more examples for testing though.

回答2:

The :punct: set of characters includes the dash and you are removing them. You could make an alternate character class that omits the dash. You do need to pay special attention to the square-brackets placements and escape the double quote and the backslash:

 (test <- gsub("[]!\"#$%&'()*+,./:;<=>?@[\\^_`{|}~]", "", "my-test of #$%^&*") )
[1] "my-test of "

The ?regex (help page) advises against using ranges. I investigated whether there might be any simplification using my local ASCII sequence of punctuation, but it quickly became obvious that was not the way to go for other reasons. There were 5 separate ranges, and the "]" was in the middle of one of them so there would have been 7 ranges to handle in addition to the "]" which needs to come first.

来源：https://stackoverflow.com/questions/49115193/remove-punctuation-but-keep-hyphenated-phrases-in-r-text-cleaning

标签

regex

stemming

punctuation

hyphenation