Loop “Forgets” to Remove Some Items [duplicate]

Answer 1

Quoting from the docs:

Note: There is a subtlety when the sequence is being modified by the loop (this can only occur for mutable sequences, i.e. lists). An internal counter is used to keep track of which item is used next, and this is incremented on each iteration. When this counter has reached the length of the sequence the loop terminates. This means that if the suite deletes the current (or a previous) item from the sequence, the next item will be skipped (since it gets the index of the current item which has already been treated). Likewise, if the suite inserts an item in the sequence before the current item, the current item will be treated again the next time through the loop. This can lead to nasty bugs that can be avoided by making a temporary copy using a slice of the whole sequence, e.g.,

for x in a[:]:
    if x < 0: a.remove(x)

---

Iterate over a shallow copy of the list using [:]. You're modifying a list while iterating over it, this will result in some letters being missed.

The for loop keeps track of index, so when you remove an item at index i, the next item at i+1th position shifts to the current index(i) and hence in the next iteration you'll actually pick the i+2th item.

Lets take an easy example:

>>> text = "whoops"
>>> textlist = list(text)
>>> textlist
['w', 'h', 'o', 'o', 'p', 's']
for char in textlist:
    if char.lower() in 'aeiou':
        textlist.remove(char)

Iteration 1 : Index = 0.

char = 'W' as it is at index 0. As it doesn't satisfies that condition you'll do noting.

Iteration 2 : Index = 1.

char = 'h' as it is at index 1. Nothing more to do here.

Iteration 3 : Index = 2.

char = 'o' as it is at index 2. As this item satisfies the condition so it'll be removed from the list and all the items to it's right will shift one place to the left to fill the gap.

now textlist becomes :

   0    1    2    3    4
`['w', 'h', 'o', 'p', 's']`

As you can see the other 'o' moved to index 2, i.e the current index so it'll be skipped in the next iteration. So, this is the reason some items are bring skipped in your iteration. Whenever you remove an item the next item is skipped from the iteration.

Iteration 4 : Index = 3.

char = 'p' as it is at index 3.

....

---

Fix:

Iterate over a shallow copy of the list to fix this issue:

for char in textlist[:]:        #note the [:]
    if char.lower() in 'aeiou':
        textlist.remove(char)

---

Other alternatives:

List comprehension:

A one-liner using str.join and a list comprehension:

vowels = 'aeiou'
text = "Hey look Words!"
return "".join([char for char in text if char.lower() not in vowels])

regex:

>>> import re
>>> text = "Hey look Words!"
>>> re.sub('[aeiou]', '', text, flags=re.I)
'Hy lk Wrds!'

Answer 2

text = "Hey look Words!"

print filter(lambda x: x not in "AaEeIiOoUu", text)

Output

Hy lk Wrds!

Answer 3

You're iterating over a list and deleting elements from it at the same time.

First, I need to make sure you clearly understand the role of char in for char in textlist: .... Take the situation where we have reached the letter 'l'. The situation is not like this:

['H', 'e', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
                      ^
                    char

There is no link between char and the position of the letter 'l' in the list. If you modify char, the list will not be modified. The situation is more like this:

['H', 'e', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
                      ^
char = 'l'

Notice that I've kept the ^ symbol. This is the hidden pointer that the code managing the for char in textlist: ... loop uses to keep track of its position in the loop. Every time you enter the body of the loop, the pointer is advanced, and the letter referenced by the pointer is copied into char.

Your problem occurs when you have two vowels in succession. I'll show you what happens from the point where you reach 'l'. Notice that I've also changed the word "look" to "leap", to make it clearer what's going on:

advance pointer to next character ('l') and copy to char

['H', 'e', 'y', ' ', 'l', 'e', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
                   -> ^
char = 'l'

char ('l') is not a vowel, so do nothing

advance pointer to next character ('e') and copy to char

['H', 'e', 'y', ' ', 'l', 'e', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
                        -> ^
char = 'e'

char ('e') is a vowel, so delete the first occurrence of char ('e')

['H', 'e', 'y', ' ', 'l', 'e', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
                           ^

['H', 'e', 'y', ' ', 'l',      'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
                           ^

['H', 'e', 'y', ' ', 'l',   <- 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
                           ^

['H', 'e', 'y', ' ', 'l', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
                           ^

advance pointer to next character ('p') and copy to char

['H', 'e', 'y', ' ', 'l', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
                             -> ^
char = 'p'

When you removed the 'e' all the characters after the 'e' moved one place to the left, so it was as if remove had advanced the pointer. The result is that you skipped past the 'a'.

In general, you should avoid modifying lists while iterating over them. It's better to construct a new list from scratch, and Python's list comprehensions are the perfect tool for doing this. E.g.

print ''.join([char for char in "Hey look Words" if char.lower() not in "aeiou"])

But if you haven't learnt about comprehensions yet, the best way is probably:

text = "Hey look Words!"

def anti_vowel(text):

  textlist = list(text)
  new_textlist = []

  for char in textlist:
    if char.lower() not in 'aeiou':
      new_textlist.append(char)

    return "".join(new_textlist)

print anti_vowel(text)

Answer 4

You're modifying the data you're iterating over. Don't do that.

''.join(x for x in textlist in x not in VOWELS)

Answer 5

Others have already explained the issue with your code. For your task, a generator expression is easier and less error prone.

>>> text = "Hey look Words!"
>>> ''.join(c for c in text if c.lower() not in 'aeiou')
'Hy lk Wrds!'

or

>>> ''.join(c for c in text if c not in 'AaEeIiOoUu')
'Hy lk Wrds!'

however, str.translate is the best way to go.

Answer 6

Other answers tell you why for skips items as you alter the list. This answer tells you how you should remove characters in a string without an explicit loop, instead.

Use str.translate():

vowels = 'aeiou'
vowels += vowels.upper()
text.translate(None, vowels)

This deletes all characters listed in the second argument.

Demo:

>>> text = "Hey look Words!"
>>> vowels = 'aeiou'
>>> vowels += vowels.upper()
>>> text.translate(None, vowels)
'Hy lk Wrds!'
>>> text = 'The Quick Brown Fox Jumps Over The Lazy Fox'
>>> text.translate(None, vowels)
'Th Qck Brwn Fx Jmps vr Th Lzy Fx'

In Python 3, the str.translate() method (Python 2: unicode.translate()) differs in that it doesn't take a deletechars parameter; the first argument is a dictionary mapping Unicode ordinals (integer values) to new values instead. Use None for any character that needs to be deleted:

# Python 3 code
vowels = 'aeiou'
vowels += vowels.upper()
vowels_table = dict.fromkeys(map(ord, vowels))
text.translate(vowels_table)

You can also use the str.maketrans() static method to produce that mapping:

vowels = 'aeiou'
vowels += vowels.upper()
text.translate(text.maketrans('', '', vowels))