While construction with sized vector as default value, given by @leemes, is quite an answer, there is an alternative without using an additional vector and copy ctor:
assert(COL >= 0);
assert(ROW >= 0);
vector> matrix;
for (size_t i{0}; i != COL; ++i)
{
matrix.emplace_back(ROW);
}