Python Pandas Replace Special Character

Question

For some reason, I cannot get this simple statement to work on the ñ. It seems to work on anything else but doesn't like that character. Any ideas?

DF['NAME']=DF['NAME'].str.replace("ñ","n")

Thanks

Answer 1

I'm assuming you're using Python 2.x here and this is likely a Unicode problem. Don't worry, you're not alone--unicode is really tough in general and especially in Python 2, which is why it's been made standard in Python 3.

If all you're concerned about is the ñ, you should decode in UTF-8, and then just replace the one character.

That would look something like the following:

DF['name'] = DF['name'].str.decode('utf-8').replace(u'\xf1', 'n')

As an example:

>>> "sureño".decode("utf-8").replace(u"\xf1", "n")
u'sureno'

If your string is already Unicode, then you can (and actually have to) skip the decode step:

>>> u"sureño".replace(u"\xf1", "n")
u'sureno'

Note here that u'\xf1' uses the hex escape for the character in question.

Update

I was informed in the comments that <>.str.replace is a pandas series method, which I hadn't realized. The answer to this possibly might be something like the following:

DF['name'] = map(lambda x: x.decode('utf-8').replace(u'\xf1', 'n'), DF['name'].str)

or something along those lines, if that pandas object is iterable.

Another update

It actually just occurred to me that your issue may be as simple as the following:

DF['NAME']=DF['NAME'].str.replace(u"ñ","n")

Note how I've added the u in front of the string to make it unicode.

Answer 2

You can use replace function with special character to be replaced with a different value of your choice in the following way.

if your dataframe is df and you have to do it in all the columns that are string. in case of mine I am doing it for "\n"

df= df.applymap(lambda x: x.replace("\n"," "))