This answer to what looks like the same question:
Convert integer to hex and hex to integer
..does not work for me.
I am not able to go to a HEX string
Thanks for giving some more explicit examples. As far as I can tell from the documentation and Googling, this is not possible in MSSQL 2005 without a UDF or other procedural code. In MSSQL 2008 the CONVERT() function's style parameter now supoprts binary data, so you can do it directly like this:
select convert(int, convert(varbinary, '0x89', 1))
In previous versions, your choices are:
If converting the data is only for display purposes, the application might be the easiest solution: data formatting usually belongs there anyway. If you must do it in a query, then a UDF is easiest but the performance may not be great (I know you said you preferred not to use a UDF but it's not clear why). I'm guessing that upgrading to MSSQL 2008 just for this probably isn't realistic.
Finally, FYI the version number you included is the version of Management Studio, not the version number of your server. To get that, query the server itself with select @@version
or select serverproperty('ProductVersion')
.