Character with encoding UTF8 has no equivalent in WIN1252

Question

I am getting the following exception:

Caused by: org.postgresql.util.PSQLException: ERROR: character 0xefbfbd of encoding \"UTF8\" has no equivalent in \"WIN         


        

Answer 1

That looks like the byte sequence 0xBD, 0xBF, 0xEF as a little-endian integer. This is the UTF8-encoded form of the Unicode byte-order-mark (BOM) character 0xFEFF.

I'm not sure what Postgre's normal behaviour is, but the BOM is normally used only for encoding detection at the beginning of an input stream, and is usually not returned as part of the result.

In any case, your exception is due to this code point not having a mapping in the Win1252 code page. This will occur with most other non-Latin characters too, such as those used in Asian scripts.

Can you change the database encoding to be UTF8 instead of 1252? This will allow your columns to contain almost any character.

Answer 2

I have face this issue when my Windows 10 using Mandarin China as default language. This problem has occurred because I did try to import a database with UTF-8. Checking via psql and do "\l", it shows collate and cytpe is Mandarin China.

The solution, reset OS language back to US and re-install PostgreSQL. As the collate back to UTF-8, you can reset back your OS language again.

I write the full context and solution here https://www.yodiw.com/fix-utf8-encoding-win1252-cputf8-postgresql-windows-10/

Answer 3

This problem appeared for us around 19/11/2016 with our old Access 97 app accessing a postgresql 9.1 DB.

This was solved by changing the driver to UNICODE instead of ANSI (see plang comment).

Answer 4

I had a similar issue, and I solved by setting the encoding to UTF8 with \encoding UTF8 in the client before attempting an INSERT INTO foo (SELECT * from bar WHERE x=y);. My client was using WIN1252 encoding but the database was in UTF8, hence the error.

More info is available on the PostgreSQL wiki under Character Set Support (devel docs).

Answer 5

You can change encoding

Example

String tmp // String that will be saved in postgre database
String utfString = new String(tmp.getBytes(Charset.forName("utf-8")));

I use java.nio.Charset to set charset;

Answer 6

Here's what worked for me : 1 enable ad-hoc queries in sp_configure. 2 add ODBC DSN for your linked PostgreSQL server. 3 make sure you have both ANSI and Unicode (x64) drivers (try with both). 4 run query like this below - change UID, server ip, db name and password. 5 just keep the query in last line in postgreSQL format.

EXEC sp_configure 'show advanced options', 1
RECONFIGURE
GO
EXEC sp_configure 'ad hoc distributed queries', 1
RECONFIGURE
GO

SELECT * FROM OPENROWSET('MSDASQL', 
'Driver=PostgreSQL Unicode(x64); 
uid=loginid;
Server=1.2.3.41;
port=5432;
database=dbname;
pwd=password',

'select * FROM table_name limit 10;')