deep-learning

How can I calculate FLOPs and Params without 0 weights neurons affected?

北城以北 提交于 2021-02-10 16:17:00
问题 My Prune code is shown below, after running this, I will get a file named 'pruned_model.pth'. import torch from torch import nn import torch.nn.utils.prune as prune import torch.nn.functional as F from cnn import net ori_model = '/content/drive/My Drive/ECG_weight_prune/checkpoint_dir/model.pth' save_path = '/content/drive/My Drive/ECG_weight_prune/checkpoint_dir/pruned_model.pth' device = torch.device("cuda" if torch.cuda.is_available() else "cpu") model = net().to(device) model.load_state

how to use bert for long sentences? [duplicate]

别来无恙 提交于 2021-02-10 15:50:21
问题 This question already has answers here : How to use Bert for long text classification? (6 answers) Closed 5 months ago . I am trying to classify given text into news, clickbait or others. The texts which I have for training are long.distribution of lengths is shown here. Now, the question is should I trim the text at the middle and make it 512 tokens long? But, I have even documents with circa 10,000 words so won't I loose the gist by truncation? Or, should I split my text into sub texts of

how to use bert for long sentences? [duplicate]

最后都变了- 提交于 2021-02-10 15:47:33
问题 This question already has answers here : How to use Bert for long text classification? (6 answers) Closed 5 months ago . I am trying to classify given text into news, clickbait or others. The texts which I have for training are long.distribution of lengths is shown here. Now, the question is should I trim the text at the middle and make it 512 tokens long? But, I have even documents with circa 10,000 words so won't I loose the gist by truncation? Or, should I split my text into sub texts of

how to use bert for long sentences? [duplicate]

懵懂的女人 提交于 2021-02-10 15:47:03
问题 This question already has answers here : How to use Bert for long text classification? (6 answers) Closed 5 months ago . I am trying to classify given text into news, clickbait or others. The texts which I have for training are long.distribution of lengths is shown here. Now, the question is should I trim the text at the middle and make it 512 tokens long? But, I have even documents with circa 10,000 words so won't I loose the gist by truncation? Or, should I split my text into sub texts of

Why is my CPU doing matrix operations faster than GPU instead?

江枫思渺然 提交于 2021-02-10 15:38:32
问题 When I tried to verify that the GPU does matrix operations over the CPU, I got unexpected results.CPU performs better than GPU according to my experience result, it makes me confused. I used cpu and gpu to do matrix multiplication respectively.Programming environment is MXNet and cuda-10.1. with gpu: import mxnet as mx from mxnet import nd x = nd.random.normal(shape=(100000,100000),ctx=mx.gpu()) y = nd.random.normal(shape=(100000,100000),ctx=mx.gpu()) %timeit nd.dot(x,y) 50.8 µs ± 1.76 µs per

Binary classification with softmax activation always outputs 1

核能气质少年 提交于 2021-02-10 15:16:58
问题 Sorry for the quality of the question but a beginner here , I was just trying my luck with titanic dataset, but it always predicts that the passenger died. I try to explain code below: import matplotlib.pyplot as plt import numpy as np import pandas as pd import seaborn as sns import tensorflow as tf from tensorflow import keras from tensorflow.keras import layers from tensorflow.keras import losses from tensorflow.keras.layers.experimental import preprocessing import os Load dataset dataset

Binary classification with softmax activation always outputs 1

回眸只為那壹抹淺笑 提交于 2021-02-10 15:14:50
问题 Sorry for the quality of the question but a beginner here , I was just trying my luck with titanic dataset, but it always predicts that the passenger died. I try to explain code below: import matplotlib.pyplot as plt import numpy as np import pandas as pd import seaborn as sns import tensorflow as tf from tensorflow import keras from tensorflow.keras import layers from tensorflow.keras import losses from tensorflow.keras.layers.experimental import preprocessing import os Load dataset dataset

Add blocks of values to a tensor at specific locations in PyTorch

心不动则不痛 提交于 2021-02-10 14:42:03
问题 I have a list of indices: indx = torch.LongTensor([ [ 0, 2, 0], [ 0, 2, 4], [ 0, 4, 0], [ 0, 10, 14], [ 1, 4, 0], [ 1, 8, 2], [ 1, 12, 0] ]) And I have a tensor of 2x2 blocks: blocks = torch.FloatTensor([ [[1.5818, 2.3108], [2.6742, 3.0024]], [[2.0472, 1.6651], [3.2807, 2.7413]], [[1.5587, 2.1905], [1.9231, 3.5083]], [[1.6007, 2.1426], [2.4802, 3.0610]], [[1.9087, 2.1021], [2.7781, 3.2282]], [[1.5127, 2.6322], [2.4233, 3.6836]], [[1.9645, 2.3831], [2.8675, 3.3770]] ]) What I want to do is to

How to implement SegNet with preserving max-indexes in Keras

戏子无情 提交于 2021-02-10 14:42:00
问题 I'm trying to implement SegNet in Keras (tf backend) to do semantic segmentation. The most impressived trick of SgeNet is to pass max-pooling indices to the upsampling layers. However, there are many implementations of SegNet in Keras(e.g.) I find on github just using simple UpSampling (called SegNet-Basic). I notice that it can be achieved in Tensorflow with " tf.nn.max_pool_with_argmax ". So I want to know is there any similar method to get the max-pooling indices and put them back in

Add blocks of values to a tensor at specific locations in PyTorch

依然范特西╮ 提交于 2021-02-10 14:38:43
问题 I have a list of indices: indx = torch.LongTensor([ [ 0, 2, 0], [ 0, 2, 4], [ 0, 4, 0], [ 0, 10, 14], [ 1, 4, 0], [ 1, 8, 2], [ 1, 12, 0] ]) And I have a tensor of 2x2 blocks: blocks = torch.FloatTensor([ [[1.5818, 2.3108], [2.6742, 3.0024]], [[2.0472, 1.6651], [3.2807, 2.7413]], [[1.5587, 2.1905], [1.9231, 3.5083]], [[1.6007, 2.1426], [2.4802, 3.0610]], [[1.9087, 2.1021], [2.7781, 3.2282]], [[1.5127, 2.6322], [2.4233, 3.6836]], [[1.9645, 2.3831], [2.8675, 3.3770]] ]) What I want to do is to