I\'m having a hell of a time replacing commas only within parentheses for this string:
select distinct \\n(\'\'+rtrim(ent.custno)+\' - \'+rtrim(ent.com
Assuming your parentheses are paired and can be nested (balanced), and you need to replace all commas inside those balanced parentheses, you can use a regex to match those substrings and replace the commas with a match evaluator:
cols = Regex.Replace(cols, @"\((?>[^()]|(?)\(|(?<-c>)\))*(?(c)(?!))\)", m => m.Value.Replace(",", ";"));
See IDEONE demo