How is this website fixing the encoding?

Question

I am trying to turn this text:

×וויר. העתיד של רשתות חברתיות והתקשורת ×©×œ× ×•

Into this text:

Answer 1

If you look closely at the gibberish, you can tell that each Hebrew character is encoded as 2 characters - it appears that של is encoded as של.

This suggests that you are looking at UTF8 or UTF16 as ASCII. Converting to UTF8 will not help because it is already ASCII and will keep that encoding.

You can read each pair of bytes and reconstruct the original UTF8 from them.

Here is some C# I came up with - this is very simplistic (doesn't fully work - too many assumptions), but I could see some of the characters converted properly:

private string ToProperHebrew(string gibberish)
{
   byte[] orig = Encoding.Unicode.GetBytes(gibberish);
   byte[] heb = new byte[orig.Length / 2];

   for (int i = 0; i < orig.Length / 2; i++)
   {
     heb[i] = orig[i * 2];
   }

   return Encoding.UTF8.GetString(heb);
}

If appears that each byte was re-encoded as two bytes - not sure what encoding was used for this, but discarding one byte seemed to be the right thing for most doubled up characters.