word-embedding | 易学教程

Sentence embedding in keras

阅读更多关于 Sentence embedding in keras

I am trying a simple document classification using sentence embeddings in keras. I know how to feed word vectors to a network, but I have problems using sentence embeddings. In my case, I have a simple representation of sentences (adding the word vectors along the axis, for example np.sum(sequences, axis=0) ). My question is, what should I replace the Embedding layer with in the code below to feed sentence embeddings instead? model = Sequential() model.add(Embedding(len(embedding_weights), len(embedding_weights[0]), weights=[embedding_weights], mask_zero=True, input_length=MAX_SEQUENCE_LENGTH,

How do I create a Keras Embedding layer from a pre-trained word embedding dataset?

阅读更多关于 How do I create a Keras Embedding layer from a pre-trained word embedding dataset?

How do I load a pre-trained word-embedding into a Keras Embedding layer? I downloaded the glove.6B.50d.txt (glove.6B.zip file from https://nlp.stanford.edu/projects/glove/ ) and I'm not sure how to add it to a Keras Embedding layer. See: https://keras.io/layers/embeddings/ You will need to pass an embeddingMatrix to the Embedding layer as follows: Embedding(vocabLen, embDim, weights=[embeddingMatrix], trainable=isTrainable) vocabLen : number of tokens in your vocabulary embDim : embedding vectors dimension (50 in your example) embeddingMatrix : embedding matrix built from glove.6B.50d.txt

keras understanding Word Embedding Layer

阅读更多关于 keras understanding Word Embedding Layer

From the page I got the below code: from numpy import array from keras.preprocessing.text import one_hot from keras.preprocessing.sequence import pad_sequences from keras.models import Sequential from keras.layers import Dense from keras.layers import Flatten from keras.layers.embeddings import Embedding # define documents docs = ['Well done!', 'Good work', 'Great effort', 'nice work', 'Excellent!', 'Weak', 'Poor effort!', 'not good', 'poor work', 'Could have done better.'] # define class labels labels = array([1,1,1,1,1,0,0,0,0,0]) # integer encode the documents vocab_size = 50 encoded_docs =

Value Error problem with multicell Dimensions must be equal, but are 20 and 13

阅读更多关于 Value Error problem with multicell Dimensions must be equal, but are 20 and 13

问题 I am working with python 3.6.5 and tensorflow 1.8.0 Nr of neurons are 10 at the moment, input in this example is 3 I have already build a recurrent neuronal network and now wanted to improve it. I need some help! Here is a little excerpt of the code to reproduce my error: You can also replace BasicRNN by LSTM or GRU to get the other messages. import numpy as np import tensorflow as tf batch_size = 10 nr_inputs = 3 nr_outputs = 4 nr_steps = 4 nr_layers = 2 def mini_batch ( Xdata, ydata, batch

Gensim Word2Vec select minor set of word vectors from pretrained model

阅读更多关于 Gensim Word2Vec select minor set of word vectors from pretrained model

问题 I have a large pretrained Word2Vec model in gensim from which I want to use the pretrained word vectors for an embedding layer in my Keras model. The problem is that the embedding size is enormous and I don't need most of the word vectors (because I know which words can occure as Input). So I want to get rid of them to reduce the size of my embedding layer. Is there a way to just keep desired wordvectors (including the coresponding indices!), based on a whitelist of words? 回答1: Thanks to this

Using pretrained gensim Word2vec embedding in keras

阅读更多关于 Using pretrained gensim Word2vec embedding in keras

问题 I have trained word2vec in gensim. In Keras, I want to use it to make matrix of sentence using that word embedding. As storing the matrix of all the sentences is very space and memory inefficient. So, I want to make embedding layer in Keras to achieve this so that It can be used in further layers(LSTM). Can you tell me in detail how to do this? PS: It is different from other questions because I am using gensim for word2vec training instead of keras. 回答1: Let's say you have following data that

Explain with example: how embedding layers in keras works

阅读更多关于 Explain with example: how embedding layers in keras works

问题 I don't understand the Embedding layer of Keras. Although there are lots of articles explaining it, I am still confused. For example, the code below isfrom imdb sentiment analysis: top_words = 5000 max_review_length = 500 embedding_vecor_length = 32 model = Sequential() model.add(Embedding(top_words, embedding_vecor_length, input_length=max_review_length)) model.add(LSTM(100)) model.add(Dense(1, activation='sigmoid')) model.compile(loss='binary_crossentropy', optimizer='adam', metrics=[

Use LSTM tutorial code to predict next word in a sentence?

阅读更多关于 Use LSTM tutorial code to predict next word in a sentence?

问题 I've been trying to understand the sample code with https://www.tensorflow.org/tutorials/recurrent which you can find at https://github.com/tensorflow/models/blob/master/tutorials/rnn/ptb/ptb_word_lm.py (Using tensorflow 1.3.0.) I've summarized (what I think are) the key parts, for my question, below: size = 200 vocab_size = 10000 layers = 2 # input_.input_data is a 2D tensor [batch_size, num_steps] of # word ids, from 1 to 10000 cell = tf.contrib.rnn.MultiRNNCell( [tf.contrib.rnn

CBOW v.s. skip-gram: why invert context and target words?

阅读更多关于 CBOW v.s. skip-gram: why invert context and target words?

问题 In this page, it is said that: [...] skip-gram inverts contexts and targets, and tries to predict each context word from its target word [...] However, looking at the training dataset it produces, the content of the X and Y pair seems to be interexchangeable, as those two pairs of (X, Y): (quick, brown), (brown, quick) So, why distinguish that much between context and targets if it is the same thing in the end? Also, doing Udacity's Deep Learning course exercise on word2vec, I wonder why they

What does a weighted word embedding mean?

阅读更多关于 What does a weighted word embedding mean?

In the paper that I am trying to implement, it says, In this work, tweets were modeled using three types of text representation. The first one is a bag-of-words model weighted by tf-idf (term frequency - inverse document frequency) (Section 2.1.1). The second represents a sentence by averaging the word embeddings of all words (in the sentence) and the third represents a sentence by averaging the weighted word embeddings of all words, the weight of a word is given by tf-idf (Section 2.1.2). I am not sure about the third representation which is mentioned as the weighted word embeddings which is