Is there a max array length limit in C++?

Question

Is there a max length for an array in C++?

Is it a C++ limit or does it depend on my machine? Is it tweakable? Does it depend on the type the array is made of?

Answer 1

As many excellent answers noted, there are a lot of limits that depend on your version of C++ compiler, operating system and computer characteristics. However, I suggest the following script on Python that checks the limit on your machine.

It uses binary search and on each iteration checks if the middle size is possible by creating a code that attempts to create an array of the size. The script tries to compile it (sorry, this part works only on Linux) and adjust binary search depending on the success. Check it out:

import os

cpp_source = 'int a[{}]; int main() {{ return 0; }}'

def check_if_array_size_compiles(size):
        #  Write to file 1.cpp
        f = open(name='1.cpp', mode='w')
        f.write(cpp_source.format(m))
        f.close()
        #  Attempt to compile
        os.system('g++ 1.cpp 2> errors')
        #  Read the errors files
        errors = open('errors', 'r').read()
        #  Return if there is no errors
        return len(errors) == 0

#  Make a binary search. Try to create array with size m and
#  adjust the r and l border depending on wheather we succeeded
#  or not
l = 0
r = 10 ** 50
while r - l > 1:
        m = (r + l) // 2
        if check_if_array_size_compiles(m):
                l = m
        else:
                r = m

answer = l + check_if_array_size_compiles(r)
print '{} is the maximum avaliable length'.format(answer)

You can save it to your machine and launch it, and it will print the maximum size you can create. For my machine it is 2305843009213693951.

Answer 2

I would agree with the above, that if you're intializing your array with

 int myArray[SIZE] 

then SIZE is limited by the size of an integer. But you can always malloc a chunk of memory and have a pointer to it, as big as you want so long as malloc doesnt return NULL.

Answer 3

If you have to deal with data that large you'll need to split it up into manageable chunks. It won't all fit into memory on any small computer. You can probably load a portion of the data from disk (whatever reasonably fits), perform your calculations and changes to it, store it to disk, then repeat until complete.

Answer 4

I'm surprised the max_size() member function of std::vector has not been mentioned here.

"Returns the maximum number of elements the container is able to hold due to system or library implementation limitations, i.e. std::distance(begin(), end()) for the largest container."

We know that std::vector is implemented as a dynamic array underneath the hood, so max_size() should give a very close approximation of the maximum length of a dynamic array on your machine.

The following program builds a table of approximate maximum array length for various data types.

#include <iostream>
#include <vector>
#include <string>
#include <limits>

template <typename T>
std::string mx(T e) {
    std::vector<T> v;
    return std::to_string(v.max_size());
}

std::size_t maxColWidth(std::vector<std::string> v) {
    std::size_t maxWidth = 0;

    for (const auto &s: v)
        if (s.length() > maxWidth)
            maxWidth = s.length();

    // Add 2 for space on each side
    return maxWidth + 2;
}

constexpr long double maxStdSize_t = std::numeric_limits<std::size_t>::max();

// cs stands for compared to std::size_t
template <typename T>
std::string cs(T e) {
    std::vector<T> v;
    long double maxSize = v.max_size();
    long double quotient = maxStdSize_t / maxSize;
    return std::to_string(quotient);
}

int main() {
    bool v0 = 0;
    char v1 = 0;

    int8_t v2 = 0;
    int16_t v3 = 0;
    int32_t v4 = 0;
    int64_t v5 = 0;

    uint8_t v6 = 0;
    uint16_t v7 = 0;
    uint32_t v8 = 0;
    uint64_t v9 = 0;

    std::size_t v10 = 0;
    double v11 = 0;
    long double v12 = 0;

    std::vector<std::string> types = {"data types", "bool", "char", "int8_t", "int16_t",
                                      "int32_t", "int64_t", "uint8_t", "uint16_t",
                                      "uint32_t", "uint64_t", "size_t", "double",
                                      "long double"};

    std::vector<std::string> sizes = {"approx max array length", mx(v0), mx(v1), mx(v2),
                                      mx(v3), mx(v4), mx(v5), mx(v6), mx(v7), mx(v8),
                                      mx(v9), mx(v10), mx(v11), mx(v12)};

    std::vector<std::string> quotients = {"max std::size_t / max array size", cs(v0),
                                          cs(v1), cs(v2), cs(v3), cs(v4), cs(v5), cs(v6),
                                          cs(v7), cs(v8), cs(v9), cs(v10), cs(v11), cs(v12)};

    std::size_t max1 = maxColWidth(types);
    std::size_t max2 = maxColWidth(sizes);
    std::size_t max3 = maxColWidth(quotients);

    for (std::size_t i = 0; i < types.size(); ++i) {
        while (types[i].length() < (max1 - 1)) {
            types[i] = " " + types[i];
        }

        types[i] += " ";

        for  (int j = 0; sizes[i].length() < max2; ++j)
            sizes[i] = (j % 2 == 0) ? " " + sizes[i] : sizes[i] + " ";

        for  (int j = 0; quotients[i].length() < max3; ++j)
            quotients[i] = (j % 2 == 0) ? " " + quotients[i] : quotients[i] + " ";

        std::cout << "|" << types[i] << "|" << sizes[i] << "|" << quotients[i] << "|\n";
    }

    std::cout << std::endl;

    std::cout << "N.B. max std::size_t is: " <<
        std::numeric_limits<std::size_t>::max() << std::endl;

    return 0;
}

On my macOS (clang version 5.0.1), I get the following:

|  data types | approx max array length | max std::size_t / max array size |
|        bool |   9223372036854775807   |             2.000000             |
|        char |   9223372036854775807   |             2.000000             |
|      int8_t |   9223372036854775807   |             2.000000             |
|     int16_t |   9223372036854775807   |             2.000000             |
|     int32_t |   4611686018427387903   |             4.000000             |
|     int64_t |   2305843009213693951   |             8.000000             |
|     uint8_t |   9223372036854775807   |             2.000000             |
|    uint16_t |   9223372036854775807   |             2.000000             |
|    uint32_t |   4611686018427387903   |             4.000000             |
|    uint64_t |   2305843009213693951   |             8.000000             |
|      size_t |   2305843009213693951   |             8.000000             |
|      double |   2305843009213693951   |             8.000000             |
| long double |   1152921504606846975   |             16.000000            |

N.B. max std::size_t is: 18446744073709551615

On ideone gcc 8.3 I get:

|  data types | approx max array length | max std::size_t / max array size |
|        bool |   9223372036854775744   |             2.000000             |
|        char |   18446744073709551615  |             1.000000             |
|      int8_t |   18446744073709551615  |             1.000000             |
|     int16_t |   9223372036854775807   |             2.000000             |
|     int32_t |   4611686018427387903   |             4.000000             |
|     int64_t |   2305843009213693951   |             8.000000             |
|     uint8_t |   18446744073709551615  |             1.000000             |
|    uint16_t |   9223372036854775807   |             2.000000             |
|    uint32_t |   4611686018427387903   |             4.000000             |
|    uint64_t |   2305843009213693951   |             8.000000             |
|      size_t |   2305843009213693951   |             8.000000             |
|      double |   2305843009213693951   |             8.000000             |
| long double |   1152921504606846975   |             16.000000            |

N.B. max std::size_t is: 18446744073709551615

It should be noted that this is a theoretical limit and that on most computers, you will run out of memory far before you reach this limit. For example, we see that for type char on gcc, the maximum number of elements is equal to the max of std::size_t. Trying this, we get the error:

prog.cpp: In function ‘int main()’:
prog.cpp:5:61: error: size of array is too large
  char* a1 = new char[std::numeric_limits<std::size_t>::max()];

Lastly, as @MartinYork points out, for static arrays the maximum size is limited by the size of your stack.

Answer 5

One thing I don't think has been mentioned in the previous answers.

I'm always sensing a "bad smell" in the refactoring sense when people are using such things in their design.

That's a huge array and possibly not the best way to represent your data both from an efficiency point of view and a performance point of view.

cheers,

Rob

Answer 6

To summarize the responses, extend them, and to answer your question directly:

No, C++ does not impose any limits for the dimensions of an array.

But as the array has to be stored somewhere in memory, so memory-related limits imposed by other parts of the computer system apply. Note that these limits do not directly relate to the dimensions (=number of elements) of the array, but rather to its size (=amount of memory taken). Dimensions (D) and in-memory size (S) of an array is not the same, as they are related by memory taken by a single element (E): S=D * E.

Now E depends on:

• the type of the array elements (elements can be smaller or bigger)
• memory alignment (to increase performance, elements are placed at addresses which are multiplies of some value, which introduces
  ‘wasted space’ (padding) between elements
• size of static parts of objects (in object-oriented programming static components of objects of the same type are only stored once, independent from the number of such same-type objects)

Also note that you generally get different memory-related limitations by allocating the array data on stack (as an automatic variable: int t[N]), or on heap (dynamic alocation with malloc()/new or using STL mechanisms), or in the static part of process memory (as a static variable: static int t[N]). Even when allocating on heap, you still need some tiny amount of memory on stack to store references to the heap-allocated blocks of memory (but this is negligible, usually).

The size of size_t type has no influence on the programmer (I assume programmer uses size_t type for indexing, as it is designed for it), as compiler provider has to typedef it to an integer type big enough to address maximal amount of memory possible for the given platform architecture.

The sources of the memory-size limitations stem from

• amount of memory available to the process (which is limited to 2^32 bytes for 32-bit applications, even on 64-bits OS kernels),
• the division of process memory (e.g. amount of the process memory designed for stack or heap),
• the fragmentation of physical memory (many scattered small free memory fragments are not applicable to storing one monolithic structure),
• amount of physical memory,
• and the amount of virtual memory.

They can not be ‘tweaked’ at the application level, but you are free to use a different compiler (to change stack size limits), or port your application to 64-bits, or port it to another OS, or change the physical/virtual memory configuration of the (virtual? physical?) machine.

It is not uncommon (and even advisable) to treat all the above factors as external disturbances and thus as possible sources of runtime errors, and to carefully check&react to memory-allocation related errors in your program code.

So finally: while C++ does not impose any limits, you still have to check for adverse memory-related conditions when running your code... :-)