I am trying to reproduce with R an algorithm described in Sutton and Barto (2018), but I was not able to produce a matrix with arrows as the one described by the authors on
