Does an array object explicitly contain the indexes?

Question

Since day one of learning Java I\'ve been told by various websites and many teachers that arrays are consecutive memory locations which can store the specified number of data al

Answer 1

On your first picture arr[0] to arr[4] are not references to the array elements. They are just illustrative labels for the location.

Answer 2

The "consecutive memory locations" is an implementation detail and may be wrong. For example, Objective-C mutable arrays do not use consecutive memory locations.

To you, it mostly doesn't matter. All you need to know is that you can access an array element by supplying the array and an index, and some mechanism unknown to you uses the array and the index to produce the array element.

There is obviously no need for the array to store indexes, since for example every array in the world with five array elements has the indexes 0, 1, 2, 3, and 4. We know these are the indexes, no need to store them.

Answer 3

The critical piece to understand is that memory allocated for an array is contiguous. So given the address of the initial element of an array, i.e., arr[0], this contiguous memory allocation scheme helps the runtime to determine the address of array element given its index.

Say we have declared int[] arr = new int[5], and its initial array element, arr[0], is at address 100. To reach the third element in the array all that the runtime needs to perform is following the math 100 + ((3-1)*32) = 164 (assuming 32 is the size of an integer). So all that the runtime needs is the address of the initial element of that array. It can derive all other addresses of array elements based on the index and the size of the datatype the array stores.

Just an off-topic note: Although the array occupies a contiguous memory location, the addresses are contiguous only in the virtual address space and not in the physical address space. A huge array could span multiple physical pages that may not be contiguous, but the virtual address used by the array will be contiguous. And mapping of a virtual address to a physical address is done by OS page tables.

Answer 4

Does an array object explicitly contain the indexes?

Short answer: No.

Longer answer: Typically not, but it theoretically could do.

Full answer:

Neither the Java Language Specification nor the Java Virtual Machine Specification makes any guarantees about how arrays are implemented internally. All it requires is that array elements are accessed by an int index number having a value from 0 to length-1. How an implementation actually fetches or stores the values of those indexed elements is a detail private to the implementation.

A perfectly conformant JVM could use a hash table to implement arrays. In that case, the elements would be non-consecutive, scattered around memory, and it would need to record the indexes of elements, to know what they are. Or it could send messages to a man on the moon who writes the array values down on labeled pieces of paper and stores them in lots of little filing cabinets. I can't see why a JVM would want to do these things, but it could.

What will happen in practice? A typical JVM will allocate the storage for array elements as a flat, contiguous chunk of memory. Locating a particular element is trivial: multiply the fixed memory size of each element by the index of the wanted element and add that to the memory address of the start of the array: (index * elementSize) + startOfArray. This means that the array storage consists of nothing but raw element values, consecutively, ordered by index. There is no purpose to also storing the index value with each element, because the element's address in memory implies its index, and vice-versa. However, I don't think the diagram you show was trying to say that it explicitly stored the indexes. The diagram is simply labeling the elements on the diagram so you know what they are.

The technique of using contiguous storage and calculating the address of an element by formula is simple and extremely quick. It also has very little memory overhead, assuming programs allocate their arrays only as big as they really need. Programs depend on and expect the particular performance characteristics of arrays, so a JVM that did something weird with array storage would probably perform poorly and be unpopular. So practical JVMs will be constrained to implement contiguous storage, or something that performs similarly.

I can think of only a couple of variations on that scheme that would ever be useful:

1. Stack-allocated or register-allocated arrays: During optimization, a JVM might determine through escape analysis that an array is only used within one method, and if the array is also a smallish fixed size, it would then be an ideal candidate object for being allocated directly on the stack, calculating the address of elements relative to the stack pointer. If the array is extremely small (fixed size of maybe up to 4 elements), a JVM could go even further and store the elements directly in CPU registers, with all element accesses unrolled & hardcoded.

2. Packed boolean arrays: The smallest directly addressable unit of memory on a computer is typically an 8-bit byte. That means if a JVM uses a byte for each boolean element, then boolean arrays waste 7 out of every 8 bits. It would use only 1 bit per element if booleans were packed together in memory. This packing isn't done typically because extracting individual bits of bytes is slower, and it needs special consideration to be safe with multithreading. However, packed boolean arrays might make perfect sense in some memory-constrained embedded devices.

Still, neither of those variations requires every element to store its own index.

I want to address a few other details you mentioned:

arrays store the specified number of data all of the same type

Correct.

The fact that all an array's elements are the same type is important because it means all the elements are the same size in memory. That's what allows for elements to be located by simply multiplying by their common size.

This is still technically true if the array element type is a reference type. Although in that case, the value of each element is not the object itself (which could be of varying size) but only an address which refers to an object. Also, in that case, the actual runtime type of objects referred to by each element of the array could be any subclass of the element type. E.g.,

Object[] a = new Object[4]; // array whose element type is Object
// element 0 is a reference to a String (which is a subclass of Object)
a[0] = "foo";

// element 1 is a reference to a Double (which is a subclass of Object)
a[1] = 123.45;

// element 2 is the value null (no object! although null is still assignable to Object type)
a[2] = null;

// element 3 is a reference to another array (all arrays classes are subclasses of Object)
a[3] = new int[] { 2, 3, 5, 7, 11 };

arrays are consecutive memory locations

As discussed above, this doesn't have to be true, although it is almost surely true in practice.

To go further, note that although the JVM might allocate a contiguous chunk of memory from the operating system, that doesn't mean it ends up being contiguous in physical RAM. The OS can give programs a virtual address space that behaves as if contiguous, but with individual pages of memory scattered in various places, including physical RAM, swap files on disk, or regenerated as needed if their contents are known to be currently blank. Even to the extent that pages of the virtual memory space are resident in physical RAM, they could be arranged in physical RAM in an arbitrary order, with complex page tables that define the mapping from virtual to physical addresses. And even if the OS thinks it is dealing with "physical RAM", it still could be running in an emulator. There can be layers upon layers upon layers, is my point, and getting to the bottom of them all to find out what's really going on takes a while!

Part of the purpose of programming language specifications is to separate the apparent behavior from the implementation details. When programming you can often program to the specification alone, free from worrying about how it happens internally. The implementation details become relevant however, when you need to deal with the the real-world constraints of limited speed and memory.

Since an array is an object and object references are stored on the stack, and actual objects live in the heap, object references point to actual objects

This is correct, except what you said about the stack. Object references can be stored on the stack (as local variables), but they can also be stored as static fields or instance fields, or as array elements as seen in the example above.

Also, as I mentioned earlier, clever implementations can sometimes allocate objects directly on the stack or in CPU registers as an optimization, although this has zero effect on your program's apparent behavior, only its performance.

The compiler just knows where to go by looking at the provided array index number during runtime.

In Java, it's not the compiler that does this, but the virtual machine. Arrays are a feature of the JVM itself, so the compiler can translate your source code that uses arrays simply to bytecode that uses arrays. Then it's the JVM's job to decide how to implement arrays, and the compiler neither knows nor cares how they work.

Answer 5

The array, as you say, will only store objects of the same type. Each type will have a corresponding size, in bytes. For example in an int[] each element will occupy 4 bytes, each byte in a byte[] will occupy 1 byte, each Object in an Object[] will occupy 1 word (because it's really a pointer to the heap), etc.

The important thing is that each type has a size and every array has a type.

Then, we get to the problem of mapping an index to a memory position at runtime. It's actually very easy because you know where the array starts and, given the array's type, you know the size of each element.

If your array starts at some memory position N you can use the the given index I and element size S to compute that the memory you're looking for will be at memory address N + (S * I).

This is how Java finds memory positions for indexes at runtime without storing them.

Answer 6

Your two diagrams are, apart from labels that are strictly for human consumption, equivalent and identical.

That is to say that in the first diagram, the labels arr[0], arr[1], etc., are not part of the array. They are simply there for illustrative purposes, indicating how the array elements are laid out in memory.

What you were told, namely that arrays are stored in contiguous locations in memory (at least insofar as virtual addresses are concerned; on modern hardware architectures, these need not map into contiguous physical addresses) and array elements are located based on their size and index, is correct. (At least in... well, it is definitely correct in C/C++. It is almost certainly correct in most, if not all, Java implementations. But it is likely incorrect in languages that allow for sparse arrays or arrays that can grow or shrink dynamically.)

The fact that the array reference is created in the stack whereas the array data are placed on the heap is an implementation-specific detail. Compilers that compile Java directly to machine code may implement array storage differently, taking into account the specific characteristics of the target hardware platform. In fact, a clever compiler may place, e.g., small arrays in the stack in their entirety, and use the heap only for larger arrays, to minimize the need for garbage collection, which can impact performance.