phrases | 易学教程

Python: Chunking others than noun phrases (e.g. prepositional) using Spacy, etc

阅读更多关于 Python: Chunking others than noun phrases (e.g. prepositional) using Spacy, etc

问题 Since I was told Spacy was such a powerful Python module for natural speech processing, I am now desperately looking for a way to group words together to more than noun phrases, most importantly, prepositional phrases. I doubt there is a Spacy function for this but that would be the easiest way I guess (SpacySpaCy import is already implemented in my project). Nevertheless, I'm open for any possibility of phrase recognition/ chunking. 回答1: Here's a solution to get PPs. In general you can get

Algorithm for generating a 'top list' using word frequency

阅读更多关于 Algorithm for generating a 'top list' using word frequency

问题 I have a big collection of human generated content. I want to find the words or phrases that occur most often. What is an efficient way to do this? 回答1: Don't reinvent the wheel. Use a full text search engine such as Lucene. 回答2: The simple/naive way is to use a hashtable. Walk through the words and increment the count as you go. At the end of the process sort the key/value pairs by count. 回答3: the basic idea is simple -- in executable pseudocode, from collections import defaultdict def

Statistical sentence suggestion model like spell checking

阅读更多关于 Statistical sentence suggestion model like spell checking

问题 There are already spell checking models available which help us to find the suggested correct spellings based on a corpus of trained correct spellings. Can the granularity be increased to "word" from alphabet so that we can have even phrase suggestions , such that if an incorrect phrase is entered then it should suggest the nearest correct phrase from the corpus of correct phrases, of course it is trained from a list of valid phrases. Are there any python libraries which achieve this

PHP Replacing swear words with phrases

阅读更多关于 PHP Replacing swear words with phrases

So I get how to replace certain words with other ones. What I'm trying to figure out is how to take a word and replace it with a phrase and eliminate all other input. For example: bad word is 'dog' user inputs -> 'You smell like a dog.' instead of it replacing 'dog' with 'rainbow' or something, I want it to echo something like: 'You are a potty mouth'. Here's what I have for code: <?php $find = array('dog', 'cat', 'bird'); $replace = 'You are a potty mouth.'; if (isset ($_POST['user_input'])&&!empty($_POST['user_input'])) { $user_input = $_POST['user_input']; $user_input_new = str_ireplace(

PHP Replacing swear words with phrases

阅读更多关于 PHP Replacing swear words with phrases

问题 So I get how to replace certain words with other ones. What I'm trying to figure out is how to take a word and replace it with a phrase and eliminate all other input. For example: bad word is 'dog' user inputs -> 'You smell like a dog.' instead of it replacing 'dog' with 'rainbow' or something, I want it to echo something like: 'You are a potty mouth'. Here's what I have for code: <?php $find = array('dog', 'cat', 'bird'); $replace = 'You are a potty mouth.'; if (isset ($_POST['user_input'])&

Statistical sentence suggestion model like spell checking

阅读更多关于 Statistical sentence suggestion model like spell checking

There are already spell checking models available which help us to find the suggested correct spellings based on a corpus of trained correct spellings. Can the granularity be increased to "word" from alphabet so that we can have even phrase suggestions , such that if an incorrect phrase is entered then it should suggest the nearest correct phrase from the corpus of correct phrases, of course it is trained from a list of valid phrases. Are there any python libraries which achieve this functionality already or how to proceed for this for an existing large gold standard phrase corpus to get

How to get frequently occurring phrases with Lucene

阅读更多关于 How to get frequently occurring phrases with Lucene

问题 I would like to get some frequently occurring phrases with Lucene. I am getting some information from TXT files, and I am losing a lot of context for not having information for phrases e.g. "information retrieval" is indexed as two separate words. What is the way to get the phrases like this? I can not find anything useful on internet, all the advices, links, hints especially examples are appreciated! EDIT: I store my documents just by title and content: Document doc = new Document(); doc.add

How to get frequently occurring phrases with Lucene

阅读更多关于 How to get frequently occurring phrases with Lucene

I would like to get some frequently occurring phrases with Lucene. I am getting some information from TXT files, and I am losing a lot of context for not having information for phrases e.g. "information retrieval" is indexed as two separate words. What is the way to get the phrases like this? I can not find anything useful on internet, all the advices, links, hints especially examples are appreciated! EDIT: I store my documents just by title and content: Document doc = new Document(); doc.add(new Field("name", f.getName(), Field.Store.YES, Field.Index.NOT_ANALYZED)); doc.add(new Field("text",

Extract Arabic phrases from a given text in java

阅读更多关于 Extract Arabic phrases from a given text in java

问题 Can you help me in finding a regex that take list of phrases and check if one of these phrases exist in the given text, please? Example: If I have in the hashSet the following words: كيف الحال إلى أين أين يوجد هل من أحد هنا And the given text is: كيف الحال أتمنى أن تكون بخير I want to get after performing regex: كيف الحال My initial code: HashSet<String> QWWords = new HashSet<String>(); QWWords.add("كيف الحال"); QWWords.add("إلى أين"); QWWords.add("أين يوجد"); QWWords.add("هل من أحد هنا");