I have an application which sends a POST request to the VB forum software and logs someone in (without setting cookies or anything).
Once the user is logged in I cre
I have written a C# method that url-encodes ALL symbols:
/// <summary>
/// !#$345Hf} → %21%23%24%33%34%35%48%66%7D
/// </summary>
public static string UrlEncodeExtended( string value )
{
char[] chars = value.ToCharArray();
StringBuilder encodedValue = new StringBuilder();
foreach (char c in chars)
{
encodedValue.Append( "%" + ( (int)c ).ToString( "X2" ) );
}
return encodedValue.ToString();
}
Url Encoding is easy in .NET. Use:
System.Web.HttpUtility.UrlEncode(string url)
If that'll be decoded to get the folder name, you'll still need to exclude characters that can't be used in folder names (*, ?, /, etc.)
I've been experimenting with the various methods .NET provide for URL encoding. Perhaps the following table will be useful (as output from a test app I wrote):
Unencoded UrlEncoded UrlEncodedUnicode UrlPathEncoded EscapedDataString EscapedUriString HtmlEncoded HtmlAttributeEncoded HexEscaped
A A A A A A A A %41
B B B B B B B B %42
a a a a a a a a %61
b b b b b b b b %62
0 0 0 0 0 0 0 0 %30
1 1 1 1 1 1 1 1 %31
[space] + + %20 %20 %20 [space] [space] %20
! ! ! ! ! ! ! ! %21
" %22 %22 " %22 %22 " " %22
# %23 %23 # %23 # # # %23
$ %24 %24 $ %24 $ $ $ %24
% %25 %25 % %25 %25 % % %25
& %26 %26 & %26 & & & %26
' %27 %27 ' ' ' ' ' %27
( ( ( ( ( ( ( ( %28
) ) ) ) ) ) ) ) %29
* * * * %2A * * * %2A
+ %2b %2b + %2B + + + %2B
, %2c %2c , %2C , , , %2C
- - - - - - - - %2D
. . . . . . . . %2E
/ %2f %2f / %2F / / / %2F
: %3a %3a : %3A : : : %3A
; %3b %3b ; %3B ; ; ; %3B
< %3c %3c < %3C %3C < < %3C
= %3d %3d = %3D = = = %3D
> %3e %3e > %3E %3E > > %3E
? %3f %3f ? %3F ? ? ? %3F
@ %40 %40 @ %40 @ @ @ %40
[ %5b %5b [ %5B %5B [ [ %5B
\ %5c %5c \ %5C %5C \ \ %5C
] %5d %5d ] %5D %5D ] ] %5D
^ %5e %5e ^ %5E %5E ^ ^ %5E
_ _ _ _ _ _ _ _ %5F
` %60 %60 ` %60 %60 ` ` %60
{ %7b %7b { %7B %7B { { %7B
| %7c %7c | %7C %7C | | %7C
} %7d %7d } %7D %7D } } %7D
~ %7e %7e ~ ~ ~ ~ ~ %7E
Ā %c4%80 %u0100 %c4%80 %C4%80 %C4%80 Ā Ā [OoR]
ā %c4%81 %u0101 %c4%81 %C4%81 %C4%81 ā ā [OoR]
Ē %c4%92 %u0112 %c4%92 %C4%92 %C4%92 Ē Ē [OoR]
ē %c4%93 %u0113 %c4%93 %C4%93 %C4%93 ē ē [OoR]
Ī %c4%aa %u012a %c4%aa %C4%AA %C4%AA Ī Ī [OoR]
ī %c4%ab %u012b %c4%ab %C4%AB %C4%AB ī ī [OoR]
Ō %c5%8c %u014c %c5%8c %C5%8C %C5%8C Ō Ō [OoR]
ō %c5%8d %u014d %c5%8d %C5%8D %C5%8D ō ō [OoR]
Ū %c5%aa %u016a %c5%aa %C5%AA %C5%AA Ū Ū [OoR]
ū %c5%ab %u016b %c5%ab %C5%AB %C5%AB ū ū [OoR]
The columns represent encodings as follows:
UrlEncoded: HttpUtility.UrlEncode
UrlEncodedUnicode: HttpUtility.UrlEncodeUnicode
UrlPathEncoded: HttpUtility.UrlPathEncode
EscapedDataString: Uri.EscapeDataString
EscapedUriString: Uri.EscapeUriString
HtmlEncoded: HttpUtility.HtmlEncode
HtmlAttributeEncoded: HttpUtility.HtmlAttributeEncode
HexEscaped: Uri.HexEscape
NOTES:
HexEscape
can only handle the first 255 characters. Therefore it throws an ArgumentOutOfRange
exception for the Latin A-Extended characters (eg Ā).
This table was generated in .NET 4.0 (see Levi Botelho's comment below that says the encoding in .NET 4.5 is slightly different).
EDIT:
I've added a second table with the encodings for .NET 4.5. See this answer: https://stackoverflow.com/a/21771206/216440
EDIT 2:
Since people seem to appreciate these tables, I thought you might like the source code that generates the table, so you can play around yourselves. It's a simple C# console application, which can target either .NET 4.0 or 4.5:
using System;
using System.Collections.Generic;
using System.Text;
// Need to add a Reference to the System.Web assembly.
using System.Web;
namespace UriEncodingDEMO2
{
class Program
{
static void Main(string[] args)
{
EncodeStrings();
Console.WriteLine();
Console.WriteLine("Press any key to continue...");
Console.Read();
}
public static void EncodeStrings()
{
string stringToEncode = "ABCD" + "abcd"
+ "0123" + " !\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~" + "ĀāĒēĪīŌōŪū";
// Need to set the console encoding to display non-ASCII characters correctly (eg the
// Latin A-Extended characters such as ĀāĒē...).
Console.OutputEncoding = Encoding.UTF8;
// Will also need to set the console font (in the console Properties dialog) to a font
// that displays the extended character set correctly.
// The following fonts all display the extended characters correctly:
// Consolas
// DejaVu Sana Mono
// Lucida Console
// Also, in the console Properties, set the Screen Buffer Size and the Window Size
// Width properties to at least 140 characters, to display the full width of the
// table that is generated.
Dictionary<string, Func<string, string>> columnDetails =
new Dictionary<string, Func<string, string>>();
columnDetails.Add("Unencoded", (unencodedString => unencodedString));
columnDetails.Add("UrlEncoded",
(unencodedString => HttpUtility.UrlEncode(unencodedString)));
columnDetails.Add("UrlEncodedUnicode",
(unencodedString => HttpUtility.UrlEncodeUnicode(unencodedString)));
columnDetails.Add("UrlPathEncoded",
(unencodedString => HttpUtility.UrlPathEncode(unencodedString)));
columnDetails.Add("EscapedDataString",
(unencodedString => Uri.EscapeDataString(unencodedString)));
columnDetails.Add("EscapedUriString",
(unencodedString => Uri.EscapeUriString(unencodedString)));
columnDetails.Add("HtmlEncoded",
(unencodedString => HttpUtility.HtmlEncode(unencodedString)));
columnDetails.Add("HtmlAttributeEncoded",
(unencodedString => HttpUtility.HtmlAttributeEncode(unencodedString)));
columnDetails.Add("HexEscaped",
(unencodedString
=>
{
// Uri.HexEscape can only handle the first 255 characters so for the
// Latin A-Extended characters, such as A, it will throw an
// ArgumentOutOfRange exception.
try
{
return Uri.HexEscape(unencodedString.ToCharArray()[0]);
}
catch
{
return "[OoR]";
}
}));
char[] charactersToEncode = stringToEncode.ToCharArray();
string[] stringCharactersToEncode = Array.ConvertAll<char, string>(charactersToEncode,
(character => character.ToString()));
DisplayCharacterTable<string>(stringCharactersToEncode, columnDetails);
}
private static void DisplayCharacterTable<TUnencoded>(TUnencoded[] unencodedArray,
Dictionary<string, Func<TUnencoded, string>> mappings)
{
foreach (string key in mappings.Keys)
{
Console.Write(key.Replace(" ", "[space]") + " ");
}
Console.WriteLine();
foreach (TUnencoded unencodedObject in unencodedArray)
{
string stringCharToEncode = unencodedObject.ToString();
foreach (string columnHeader in mappings.Keys)
{
int columnWidth = columnHeader.Length + 1;
Func<TUnencoded, string> encoder = mappings[columnHeader];
string encodedString = encoder(unencodedObject);
// ASSUMPTION: Column header will always be wider than encoded string.
Console.Write(encodedString.Replace(" ", "[space]").PadRight(columnWidth));
}
Console.WriteLine();
}
}
}
}
Levi Botelho commented that the table of encodings that was previously generated is no longer accurate for .NET 4.5, since the encodings changed slightly between .NET 4.0 and 4.5. So I've regenerated the table for .NET 4.5:
Unencoded UrlEncoded UrlEncodedUnicode UrlPathEncoded WebUtilityUrlEncoded EscapedDataString EscapedUriString HtmlEncoded HtmlAttributeEncoded WebUtilityHtmlEncoded HexEscaped
A A A A A A A A A A %41
B B B B B B B B B B %42
a a a a a a a a a a %61
b b b b b b b b b b %62
0 0 0 0 0 0 0 0 0 0 %30
1 1 1 1 1 1 1 1 1 1 %31
[space] + + %20 + %20 %20 [space] [space] [space] %20
! ! ! ! ! %21 ! ! ! ! %21
" %22 %22 " %22 %22 %22 " " " %22
# %23 %23 # %23 %23 # # # # %23
$ %24 %24 $ %24 %24 $ $ $ $ %24
% %25 %25 % %25 %25 %25 % % % %25
& %26 %26 & %26 %26 & & & & %26
' %27 %27 ' %27 %27 ' ' ' ' %27
( ( ( ( ( %28 ( ( ( ( %28
) ) ) ) ) %29 ) ) ) ) %29
* * * * * %2A * * * * %2A
+ %2b %2b + %2B %2B + + + + %2B
, %2c %2c , %2C %2C , , , , %2C
- - - - - - - - - - %2D
. . . . . . . . . . %2E
/ %2f %2f / %2F %2F / / / / %2F
: %3a %3a : %3A %3A : : : : %3A
; %3b %3b ; %3B %3B ; ; ; ; %3B
< %3c %3c < %3C %3C %3C < < < %3C
= %3d %3d = %3D %3D = = = = %3D
> %3e %3e > %3E %3E %3E > > > %3E
? %3f %3f ? %3F %3F ? ? ? ? %3F
@ %40 %40 @ %40 %40 @ @ @ @ %40
[ %5b %5b [ %5B %5B [ [ [ [ %5B
\ %5c %5c \ %5C %5C %5C \ \ \ %5C
] %5d %5d ] %5D %5D ] ] ] ] %5D
^ %5e %5e ^ %5E %5E %5E ^ ^ ^ %5E
_ _ _ _ _ _ _ _ _ _ %5F
` %60 %60 ` %60 %60 %60 ` ` ` %60
{ %7b %7b { %7B %7B %7B { { { %7B
| %7c %7c | %7C %7C %7C | | | %7C
} %7d %7d } %7D %7D %7D } } } %7D
~ %7e %7e ~ %7E ~ ~ ~ ~ ~ %7E
Ā %c4%80 %u0100 %c4%80 %C4%80 %C4%80 %C4%80 Ā Ā Ā [OoR]
ā %c4%81 %u0101 %c4%81 %C4%81 %C4%81 %C4%81 ā ā ā [OoR]
Ē %c4%92 %u0112 %c4%92 %C4%92 %C4%92 %C4%92 Ē Ē Ē [OoR]
ē %c4%93 %u0113 %c4%93 %C4%93 %C4%93 %C4%93 ē ē ē [OoR]
Ī %c4%aa %u012a %c4%aa %C4%AA %C4%AA %C4%AA Ī Ī Ī [OoR]
ī %c4%ab %u012b %c4%ab %C4%AB %C4%AB %C4%AB ī ī ī [OoR]
Ō %c5%8c %u014c %c5%8c %C5%8C %C5%8C %C5%8C Ō Ō Ō [OoR]
ō %c5%8d %u014d %c5%8d %C5%8D %C5%8D %C5%8D ō ō ō [OoR]
Ū %c5%aa %u016a %c5%aa %C5%AA %C5%AA %C5%AA Ū Ū Ū [OoR]
ū %c5%ab %u016b %c5%ab %C5%AB %C5%AB %C5%AB ū ū ū [OoR]
The columns represent encodings as follows:
HttpUtility.UrlEncode
HttpUtility.UrlEncodeUnicode
HttpUtility.UrlPathEncode
WebUtility.UrlEncode
Uri.EscapeDataString
Uri.EscapeUriString
HttpUtility.HtmlEncode
HttpUtility.HtmlAttributeEncode
WebUtility.HtmlEncode
Uri.HexEscape
NOTES:
HexEscape can only handle the first 255 characters. Therefore it throws an ArgumentOutOfRange exception for the Latin A-Extended characters (eg Ā).
This table was generated in .NET 4.5 (see answer https://stackoverflow.com/a/11236038/216440 for the encodings relevant to .NET 4.0 and below).
EDIT:
Ideally these would go in a class called "FileNaming" or maybe just rename Encode to "FileNameEncode". Note: these are not designed to handle Full Paths, just the folder and/or file names. Ideally you would Split("/") your full path first and then check the pieces. And obviously instead of a union, you could just add the "%" character to the list of chars not allowed in Windows, but I think it's more helpful/readable/factual this way. Decode() is exactly the same but switches the Replace(Uri.HexEscape(s[0]), s) "escaped" with the character.
public static List<string> urlEncodedCharacters = new List<string>
{
"/", "\\", "<", ">", ":", "\"", "|", "?", "%" //and others, but not *
};
//Since this is a superset of urlEncodedCharacters, we won't be able to only use UrlEncode() - instead we'll use HexEncode
public static List<string> specialCharactersNotAllowedInWindows = new List<string>
{
"/", "\\", "<", ">", ":", "\"", "|", "?", "*" //windows dissallowed character set
};
public static string Encode(string fileName)
{
//CheckForFullPath(fileName); // optional: make sure it's not a path?
List<string> charactersToChange = new List<string>(specialCharactersNotAllowedInWindows);
charactersToChange.AddRange(urlEncodedCharacters.
Where(x => !urlEncodedCharacters.Union(specialCharactersNotAllowedInWindows).Contains(x))); // add any non duplicates (%)
charactersToChange.ForEach(s => fileName = fileName.Replace(s, Uri.HexEscape(s[0]))); // "?" => "%3f"
return fileName;
}
Thanks @simon-tewsi for the very usefull table above!
Since .NET Framework 4.5 and .NET Standard 1.0 you should use WebUtility.UrlEncode. Advantages over alternatives:
It is part of .NET Framework 4.5+, .NET Core 1.0+, .NET Standard 1.0+, UWP 10.0+ and all Xamarin platforms as well. HttpUtility, while being available in .NET Framework earlier (.NET Framework 1.1+), becomes available on other platforms much later (.NET Core 2.0+, .NET Standard 2.0+) and it still unavailable in UWP (see related question).
In .NET Framework, it resides in System.dll
, so it does not require any additional references, unlike HttpUtility
.
It properly escapes characters for URLs, unlike Uri.EscapeUriString
(see comments to drweb86's answer).
It does not have any limits on the length of the string, unlike Uri.EscapeDataString
(see related question), so it can be used for POST requests, for example.