\377\376 Appended to file (Windows -> Unix)

不想你离开。 提交于 2020-01-11 09:58:06

问题


Morning All -

I have a problem and the solution has alluded me for a few days now.

I have an SSIS package that performs the following. 1. Run SQL script 2. Export results to flat file (UTF-8 Encoded, ; delimitated, and \n for new lines) 3. FTP results to solaris machine ( binary format )

The problem is, that when the file is shows up on my solaris box it has the following at the start of the file.

\377\376

I have tried dos2unix and still has not corrected the issue. In fact it changes the \377\376 to \227\226, not very helpful.

My question, any way to remove these characters from my file? When they are there they mess with grep and other unix tools like head.

Any help is appreciated.

Thanks


回答1:


This was an easy fix that eluded me for a few days. Thought I would answer my own question incase anyone is searching for an answer.

By default any SSIS or windows encoded file is UCS-2-LITTLE-ENDIAN encoded. The easiest way is to encode the file on your unix server with the following commands.

  1. Switch over to UTF-8 (or whatever encoding you need) with iconv

    iconv -f UCS-2-LITTLE-EDIAN -t UTF-8 input > output

  2. Remove the carriage returns that ms adds to the end of liens.

    unix2dos -ascii utf-8-file outputfile

And that will solve your problem.




回答2:


Dos2unix version 6.0 and higher can convert Windows Unicode UTF-16 files to Unix UTF-8. It will also remove the Byte Order Mark (BOM). Get the latest dos2unix here

There is a Windows version available.




回答3:


As the previews answers stated, using dos2unix made the job, in my case I used:

dos2unix.exe -r -v -f -D utf8 <FileName>

in which:

-r, --remove-bom remove Byte Order Mark (default)

-v, --verbose verbose operation

-f, --force force conversion of binary files

-D, --display-enc set encoding of displayed text messages encoding ansi, unicode, utf8, default to ansi

And the BOM char was removed



来源:https://stackoverflow.com/questions/12462340/377-376-appended-to-file-windows-unix

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!