So if I want to create a big array
like above, what should I do?
Avoid using the stack for these cases (in other words, avoid creating arrays like these which aren't heap-allocated when working inside a function). Just to give you an idea, my thread-local stack is only 16 kilobytes large. 4501 * 4501 * 4 (assuming 4 bytes per int) = ~81 megabytes.
Consider something like this instead:
typedef vector<int> Row;
typedef vector<Row> Matrix;
Matrix dp(4501, Row(4501) );
If you want to create a 10x50 matrix:
Matrix dp(10, Row(50) );
You can use this just like your normal dp array had it not overflowed the stack. This one will be allocated and automatically deallocated to/from the heap so that you don't have to worry about stack overflow when using it.
dp[5][10] = 123;
Good luck!
[Edit] There are also matrix solutions in boost worth looking into but suggesting boost might be a bit premature given the nature of the topic.