chcp 65001 and a .bat file

Question

I have a problem with chcp 65001 command in Windows shell.

I need to generate a list of files in a folder. So I ran cmd.exe, typed

cd folder
dir /B /O:N > list_of_files.txt

It worked, but I had a problem with special, non-ASCII characters which are in some file names. So I added chcp 65001

Everything worked, but when I put these commands into a .bat file, the script doesn't work.

So

cd folder
chcp 65001
dir /B /O:N > list_of_files.txt

doesn't generate the list.

and

cd folder
chcp 65001 && dir /B /O:N > list_of_files.txt

as well as

cd folder
chcp 65001 > nul && dir /B /O:N > list_of_files.txt

generates the list, but with the default encoding :/.

Everything works in cmd.exe, but not in .bat files.

I've read the topic: stackoverflow.com/questions/2182568/batch-script-is-not-executed-if-chcp-was-called, but it didn't help.

EDIT: I partially solved my problem, changing chcp 65001 to chcp 1250 because all characters were in this encoding. But actually this doesn't answer the question.

Answer 1

Use cmd /U. See http://ss64.com/nt/cmd.html:

Most common text files are ANSI, use these switches when you need to convert the character set. These options will affect piping or redirecting to a file:

• /A Output ANSI characters
• /U Output UNICODE characters (UCS-2 Little Endian)

Here's my attempt (launch it under cmd /A, of course):

@ECHO OFF >NUL
SETLOCAL EnableExtensions

:: create a UNICODE file with Byte Order Mark using `wmic` 
chcp 852 >NUL
>list_of_files.txt wmic os get localdatetime

:: store a line with BOM to a variable
:: although FINDSTR does not support UTF-16 files
:: it will read first three bytes at least
for /F "delims=" %%G in ('
    findstr "^" list_of_files.txt
  ') do set "UTF8BOM=%%G"

:: write BOM only* to a file (* echo writes hexadecimal value FFFE0D0A)
:: the `<NUL set /p =text` trick does not work: chokes down leading `FF`  
>list_of_files.txt echo(%UTF8BOM:~0,2%

chcp 65001 >NUL
:: add CRLF in  Unicode (hexadecimal 0D000A00)
>>list_of_files.txt cmd /U /C echo(

:: add result of `dir /B /O:N` in Unicode 
>>list_of_files.txt cmd /U /C dir /B /O:N

:: check the result: still invalid first line, see output
type list_of_files.txt
chcp 852 >NUL

Output. Still invalid first line (that hexadecimal 0D0A), sorry; use another method to get pure Utf-8 byte order mark:

==>cmd /A /C D:\bat\SO\UTF8BOM32182619.bat
਍
cpANSI_OoCcSsUu.txt
cpANSI_ÖöÇ窺Üü.txt
escrzyaie.txt
ěščřžýáíé.txt
list_of_files.txt

==>

Answer 2

"chcp 65001" does not work before Windows 7. It will cause the batch to terminate immediately. There is no work-around.

I have verified this by directly testing 2003, XP, Vista, 2008, 7, 8, and 10.

Answer 3

Tested on Windows 7 only, may not work on Windows Vista.

Apparently chcp doesn't affect dir directly.

Parse the output of dir and print it via echo:

chcp 65001
>list_of_files.txt (for /f "delims=" %%a in ('dir /B /O:N') do echo %%a)

Note: the output file won't have UTF-8 Byte Order Mark.

Answer 4

it looks like a problem I recently met

cd folder
dir /B /O:N > list_of_files.tmp
cmd /U /C type list_of_files.tmp>list_of_files.txt
del list_of_files.tmp

Answer 5

On Windows 2003 worked this:

chcp 65001 && cmd /C dir C:\WINDOWS\* && chcp 866

C:\windows\* - only sample

&& chcp 866 - default code page and this allow to continue batch