What is a loop invariant?

Question

I\'m reading \"Introduction to Algorithm\" by CLRS. In chapter 2, the authors mention \"loop invariants\". What is a loop invariant?

Answer 1

Previous answers have defined a loop invariant in a very good way.

Following is how authors of CLRS used loop invariant to prove correctness of Insertion Sort.

Insertion Sort algorithm(as given in Book):

INSERTION-SORT(A)
    for j ← 2 to length[A]
        do key ← A[j]
        // Insert A[j] into the sorted sequence A[1..j-1].
        i ← j - 1
        while i > 0 and A[i] > key
            do A[i + 1] ← A[i]
            i ← i - 1
        A[i + 1] ← key

Loop Invariant in this case: Sub-array[1 to j-1] is always sorted.

Now let us check this and prove that algorithm is correct.

Initialization: Before the first iteration j=2. So sub-array [1:1] is the array to be tested. As it has only one element so it is sorted. Thus invariant is satisfied.

Maintenance: This can be easily verified by checking the invariant after each iteration. In this case it is satisfied.

Termination: This is the step where we will prove the correctness of the algorithm.

When the loop terminates then value of j=n+1. Again loop invariant is satisfied. This means that Sub-array[1 to n] should be sorted.

This is what we want to do with our algorithm. Thus our algorithm is correct.

Answer 2

Invariant in this case means a condition that must be true at a certain point in every loop iteration.

In contract programming, an invariant is a condition that must be true (by contract) before and after any public method is called.

Answer 3

I like this very simple definition: (source)

A loop invariant is a condition [among program variables] that is necessarily true immediately before and immediately after each iteration of a loop. (Note that this says nothing about its truth or falsity part way through an iteration.)

By itself, a loop invariant doesn't do much. However, given an appropriate invariant, it can be used to help prove the correctness of an algorithm. The simple example in CLRS probably has to do with sorting. For example, let your loop invariant be something like, at the start of the loop, the first i entries of this array are sorted. If you can prove that this is indeed a loop invariant (i.e. that it holds before and after every loop iteration), you can use this to prove the correctness of a sorting algorithm: at the termination of the loop, the loop invariant is still satisfied, and the counter i is the length of the array. Therefore, the first i entries are sorted means the entire array is sorted.

An even simpler example: Loops Invariants, Correctness, and Program Derivation.

The way I understand a loop invariant is as a systematic, formal tool to reason about programs. We make a single statement that we focus on proving true, and we call it the loop invariant. This organizes our logic. While we can just as well argue informally about the correctness of some algorithm, using a loop invariant forces us to think very carefully and ensures our reasoning is airtight.

Answer 4

Loop invariant is a mathematical formula such as (x=y+1). In that example, x and y represent two variables in a loop. Considering the changing behavior of those variables throughout the execution of the code, it is almost impossible to test all possible to x and y values and see if they produce any bug. Lets say x is an integer. Integer can hold 32 bit space in the memory. If that number exceeds, buffer overflow occurs. So we need to be sure that throughout the execution of the code, it never exceeds that space. for that, we need to understand a general formula that shows the relationship between variables. After all, we just try to understand the behavior of the program.

Answer 5

It is hard to keep track of what is happening with loops. Loops which don't terminate or terminate without achieving their goal behavior is a common problem in computer programming. Loop invariants help. A loop invariant is a formal statement about the relationship between variables in your program which holds true just before the loop is ever run (establishing the invariant) and is true again at the bottom of the loop, each time through the loop (maintaining the invariant). Here is the general pattern of the use of Loop Invariants in your code:

... // the Loop Invariant must be true here
while ( TEST CONDITION ) {
// top of the loop
...
// bottom of the loop
// the Loop Invariant must be true here
}
// Termination + Loop Invariant = Goal
...
Between the top and bottom of the loop, headway is presumably being made towards reaching the loop's goal. This might disturb (make false) the invariant. The point of Loop Invariants is the promise that the invariant will be restored before repeating the loop body each time. There are two advantages to this:

Work is not carried forward to the next pass in complicated, data dependent ways. Each pass through the loop in independent of all others, with the invariant serving to bind the passes together into a working whole. Reasoning that your loop works is reduced to reasoning that the loop invariant is restored with each pass through the loop. This breaks the complicated overall behavior of the loop into small simple steps, each which can be considered separately. The test condition of the loop is not part of the invariant. It is what makes the loop terminate. You consider separately two things: why the loop should ever terminate, and why the loop achieves its goal when it terminates. The loop will terminate if each time through the loop you move closer to satisfying the termination condition. It is often easy to assure this: e.g. stepping a counter variable by one until it reaches a fixed upper limit. Sometimes the reasoning behind termination is more difficult.

The loop invariant should be created so that when the condition of termination is attained, and the invariant is true, then the goal is reached:

invariant + termination => goal
It takes practice to create invariants which are simple and relate which capture all of goal attainment except for termination. It is best to use mathematical symbols to express loop invariants, but when this leads to over-complicated situations, we rely on clear prose and common-sense.

Answer 6

Sorry I don't have comment permission.

@Tomas Petricek as you mentioned

A weaker invariant that is also true is that i >= 0 && i < 10 (because this is the continuation condition!)"

How it's a loop invariant?

I hope I am not wrong, as far as I understand^[1], Loop invariant will be true at the beginning of the loop (Initialization), it will be true before and after each iteration (Maintenance) and it will also be true after the termination of the loop (Termination). But after the last iteration i becomes 10. So, the condition i >= 0 && i < 10 becomes false and terminates the loop. It violates the third property (Termination) of loop invariant.

[1] http://www.win.tue.nl/~kbuchin/teaching/JBP030/notebooks/loop-invariants.html