As @Ben Bolker has suggested, you can simply define your identity matrix using diag
:
my.matrix <- diag(0.25, 4)
my.matrix
## [,1] [,2] [,3] [,4]
## [1,] 0.25 0.00 0.00 0.00
## [2,] 0.00 0.25 0.00 0.00
## [3,] 0.00 0.00 0.25 0.00
## [4,] 0.00 0.00 0.00 0.25