ngram representation and distance matrix in R

问题

Assume that we have this data:

a <- c("ham","bamm","comb")

for 1-gram, this is the matrix representation of the above list.

#  h a m b c o
#  1 1 1 0 0 0
#  0 1 2 1 0 0 
#  0 0 1 1 1 1

I know that table(strsplit(a,split = "")[i]) for i in 1:length(a) will give the separated count for each of them. But I don't know how use rbind to make them as a whole since the lengths and column names are different.

After that, I want to use either Euclidean or Manhattan distance to find the similarity matrix for each of them as:

#     ham  bamm comb  
# ham  0    3    5
# bamm 3    0    4
# comb 5    4    0 

回答1:

You could also use the stringdist package.

library(stringdist)
a <- c("ham","bamm","comb")

# stringdistmatrix with qgram calculations
stringdistmatrix(a, a, method = 'qgram')

     [,1] [,2] [,3]
[1,]    0    3    5
[2,]    3    0    4
[3,]    5    4    0

recreating the 1-gram with stringdist

# creates the total count of the 1-gram
qgrams(a, q = 1L)
   h m o a b c
V1 1 4 1 2 2 1

# create a named vector if you want a nice table
names(a) <- a
qgrams(a, .list = a, q = 1L)

#V1 is the total line
     h m o a b c
V1   1 4 1 2 2 1
ham  1 1 0 1 0 0
bamm 0 2 0 1 1 0
comb 0 1 1 0 1 1

回答2:

You can do in this way :

s <- stack(setNames(strsplit(a,split=""),a))
m <- t(table(s))

> m
      values
ind    a b c h m o
  ham  1 0 0 1 1 0
  bamm 1 1 0 0 2 0
  comb 0 1 1 0 1 1

Then using dist :

> as.matrix(dist(m,method='manhattan'))
     ham bamm comb
ham    0    3    5
bamm   3    0    4
comb   5    4    0

来源：https://stackoverflow.com/questions/49252396/ngram-representation-and-distance-matrix-in-r