How do I remove duplicate words from a list in python without using sets?

Question

I have the following python code which almost works for me (I\'m SO close!). I have text file from one Shakespeare\'s plays that I\'m opening: Original text file:

\"Bu

Answer 1

You did have a couple logic error with your code. I fixed them, hope it helps.

fname = "stuff.txt"
fhand = open(fname)
AllWords = list()      #create new list
ResultList = list()    #create new results list I want to append words to

for line in fhand:
    line.rstrip()   #strip white space
    words = line.split()    #split lines of words and make list
    AllWords.extend(words)   #make the list from 4 lists to 1 list

AllWords.sort()  #sort list

for word in AllWords:   #for each word in line.split()
    if word not in ResultList:    #if a word isn't in line.split            
        ResultList.append(word)   #append it.


print(ResultList)

Tested on Python 3.4, no importing.

Answer 2

Below function might help.

   def remove_duplicate_from_list(temp_list):
        if temp_list:
            my_list_temp = []
            for word in temp_list:
                if word not in my_list_temp:
                    my_list_temp.append(word)
            return my_list_temp
        else: return []

Answer 3

mylist = ['Arise', 'But', 'It', 'Juliet', 'Who', 'already', 'and', 'and', 'and', 'breaks', 'east', 'envious', 'fair', 'grief', 'is', 'is', 'is', 'kill', 'light', 'moon', 'pale', 'sick', 'soft', 'sun', 'sun', 'the', 'the', 'the', 'through', 'what', 'window', 'with', 'yonder']
newlist = sorted(set(mylist), key=lambda x:mylist.index(x))
print(newlist)
['Arise', 'But', 'It', 'Juliet', 'Who', 'already', 'and', 'breaks', 'east', 'envious', 'fair', 'grief', 'is', 'kill', 'light', 'moon', 'pale', 'sick', 'soft', 'sun', 'the', 'through', 'what', 'window', 'with', 'yonder']

newlist contains a list of the set of unique values from mylist, sorted by each item's index in mylist.

Answer 4

This should do the job:

fname = input("Enter file name: ")
fh = open(fname)
lst = list()
for line in fh:
    line = line.rstrip()
    words = line.split()
    for word in words:
        if word not in lst:
            lst.append(word)
lst.sort()
print(lst)

Answer 5

This should work, it walks the list and adds elements to a new list if they are not the same as the last element added to the new list.

def unique(lst):
    """ Assumes lst is already sorted """
    unique_list = []
    for el in lst:
        if el != unique_list[-1]:
            unique_list.append(el)
    return unique_list

You could also use collections.groupby which works similarly

from collections import groupby

# lst must already be sorted 
unique_list = [key for key, _ in groupby(lst)]

Answer 6

There is a problem with your code. I think you mean:

for word in line.split():   #for each word in line.split()
    if words not in ResultList:    #if a word isn't in ResultList