How to do named entity recognition (NER) using quanteda?

余生长醉 提交于 2020-03-20 20:54:06

问题


Having a dataframe with text

df = data.frame(id=c(1,2), text = c("My best friend John works and Google", "However he would like to work at Amazon as he likes to use python and stay at Canada")

Without any preprocessing

How is it possible to extract name entity recognition like this

Example results words

dfresults = data.frame(id=c(1,2), ner_words = c("John, Google", "Amazon, python, Canada")

回答1:


You can do this without quanteda, using the spacyr package -- a wrapper around the spaCy library mentioned in your linked article.

Here, I have slightly edited your input data.frame.

df <- data.frame(id = c(1, 2), 
                 text = c("My best friend John works at Google.", 
                          "However he would like to work at Amazon as he likes to use Python and stay in Canada."),
                 stringsAsFactors = FALSE)

Then:

library("spacyr")
library("dplyr")

# -- need to do these before the next function will work:
# spacy_install()
# spacy_download_langmodel(model = "en_core_web_lg")

spacy_initialize(model = "en_core_web_lg")
#> Found 'spacy_condaenv'. spacyr will use this environment
#> successfully initialized (spaCy Version: 2.0.10, language model: en_core_web_lg)
#> (python options: type = "condaenv", value = "spacy_condaenv")

txt <- df$text
names(txt) <- df$id

spacy_parse(txt, lemma = FALSE, entity = TRUE) %>%
    entity_extract() %>%
    group_by(doc_id) %>%
    summarize(ner_words = paste(entity, collapse = ", "))
#> # A tibble: 2 x 2
#>   doc_id ner_words             
#>   <chr>  <chr>                 
#> 1 1      John, Google          
#> 2 2      Amazon, Python, Canada


来源:https://stackoverflow.com/questions/57289663/how-to-do-named-entity-recognition-ner-using-quanteda

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!