word-embedding

I don't understand the Embedding layer of Keras. Although there are lots of articles explaining it, I am still confused. For example, the code below isfrom imdb sentiment analysis: top_words = 5000 max_review_length = 500 embedding_vecor_length = 32 model = Sequential() model.add(Embedding(top_words, embedding_vecor_length, input_length=max_review_length)) model.add(LSTM(100)) model.add(Dense(1, activation='sigmoid')) model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy']) print(model.summary()) model.fit(X_train, y_train, nb_epoch=3, batch_size=64) In this code, what

I've been trying to understand the sample code with https://www.tensorflow.org/tutorials/recurrent which you can find at https://github.com/tensorflow/models/blob/master/tutorials/rnn/ptb/ptb_word_lm.py (Using tensorflow 1.3.0.) I've summarized (what I think are) the key parts, for my question, below: size = 200 vocab_size = 10000 layers = 2 # input_.input_data is a 2D tensor [batch_size, num_steps] of # word ids, from 1 to 10000 cell = tf.contrib.rnn.MultiRNNCell( [tf.contrib.rnn.BasicLSTMCell(size) for _ in range(2)] ) embedding = tf.get_variable( "embedding", [vocab_size, size], dtype=tf

In this page, it is said that: [...] skip-gram inverts contexts and targets, and tries to predict each context word from its target word [...] However, looking at the training dataset it produces, the content of the X and Y pair seems to be interexchangeable, as those two pairs of (X, Y): (quick, brown), (brown, quick) So, why distinguish that much between context and targets if it is the same thing in the end? Also, doing Udacity's Deep Learning course exercise on word2vec , I wonder why they seem to do the difference between those two approaches that much in this problem: An alternative to

I am working with python 3.6.5 and tensorflow 1.8.0 Nr of neurons are 10 at the moment, input in this example is 3 I have already build a recurrent neuronal network and now wanted to improve it. I need some help! Here is a little excerpt of the code to reproduce my error: You can also replace BasicRNN by LSTM or GRU to get the other messages. import numpy as np import tensorflow as tf batch_size = 10 nr_inputs = 3 nr_outputs = 4 nr_steps = 4 nr_layers = 2 def mini_batch ( Xdata, ydata, batch_size ) : global global_counter result = None Xbatch = np.zeros( shape=[batch_size, nr_steps, nr_inputs]

I have a large pretrained Word2Vec model in gensim from which I want to use the pretrained word vectors for an embedding layer in my Keras model. The problem is that the embedding size is enormous and I don't need most of the word vectors (because I know which words can occure as Input). So I want to get rid of them to reduce the size of my embedding layer. Is there a way to just keep desired wordvectors (including the coresponding indices!), based on a whitelist of words? Thanks to this answer (I've changed the code a little bit to make it better). you can use this code for solving your

I have sentence embedding output X of a sentence pair of dimension 2*1*300 . I want to split this output into two vectors of shape 1*300 to calculate its absolute difference and product. x = MaxPooling2D(pool_size=(1,MAX_SEQUENCE_LENGTH),strides=(1,1))(x) x_A = Reshape((1,EMBEDDING_DIM))(x[:,0]) x_B = Reshape((1,EMBEDDING_DIM))(x[:,1]) diff = keras.layers.Subtract()([x_A, x_B]) prod = keras.layers.Multiply()([x_A, x_B]) nn = keras.layers.Concatenate()([diff, prod]) Currently, when I do x[:,0] it throws an error saying AttributeError: 'Tensor' object has no attribute '_keras_shape' . I assume

I have a seq2seq model which is working fine. I want to add an embedding layer in this network which I faced with an error. this is my architecture using pretrained word embedding which is working fine(Actually the code is almost the same code available here , but I want to include the Embedding layer in the model rather than using the pretrained embedding vectors): LATENT_SIZE = 20 inputs = Input(shape=(SEQUENCE_LEN, EMBED_SIZE), name="input") encoded = Bidirectional(LSTM(LATENT_SIZE), merge_mode="sum", name="encoder_lstm")(inputs) encoded = Lambda(rev_ent)(encoded) decoded = RepeatVector

In LDA model generates different topics everytime i train on the same corpus , by setting the np.random.seed(0) , the LDA model will always be initialized and trained in exactly the same way. Is it the same for the Word2Vec models from gensim ? By setting the random seed to a constant, would the different run on the same dataset produce the same model? But strangely, it's already giving me the same vector at different instances. >>> from nltk.corpus import brown >>> from gensim.models import Word2Vec >>> sentences = brown.sents()[:100] >>> model = Word2Vec(sentences, size=10, window=5, min