<0xEF> character showing up in files. How to remove them?

Question

I am doing compressing of JavaScript files and the compressor is complaining that my files have  character in them.

How can I search for these cha

Answer 1

On Unix/Linux:

sed 's/\xEF\xBB\xBF//' < inputfile > outputfile

On MacOSX

sed $'s/\xEF\xBB\xBF//' < inputfile > outputfile

Notice the $ after sed for mac.

On Windows

There is Super Sed an enhanced version of sed. For Windows this is a standalone .exe, intended for running from the command line.

Answer 2

Thanks for the previous answers, here's a sed(1) variant just in case:

sed '1s/^\xEF\xBB\xBF//'

Answer 3

You can easily remove them using vim, here are the steps:

1) In your terminal, open the file using vim:

vim file_name

2) Remove all BOM characters:

:set nobomb

3) Save the file:

:wq

Answer 4

Save the file without code signature.

Answer 5

In Sublime Text you can install the Highlighter package and then customize the regular expression in your user settings.

Here I added \uFEFF to the end of the highlighter_regex property.

{
    "highlighter_enabled": true,
    "highlighter_regex": "(\t+ +)|( +\t+)|[\u2026\u2018\u2019\u201c\u201d\u2013\u2014\uFEFF]|[\t ]+$",
    "highlighter_scope_name": "invalid",
    "highlighter_max_file_size": 1048576,
    "highlighter_delay": 3000
}

To overwrite the default package settings place the file here:

~/.config/sublime-text-3/Packages/User/highlighter.sublime-settings

Answer 6

Another method to remove those characters - using Vim:

vim -b fileName

Now those "hidden" characters are visible (<feff>) and can be removed.