Unfortunately, the best answer is "use them each as appropriate where necessary".
Parenthesis () in JavaScript are used for function calls, to surround conditional statements, or for grouping to enforce Order of Operations.
function myFunc() {
if (condition1) {
}
if ( (1 + 2) * 3) {
// very different from (1 + 2 * 3)
}
}
Braces {} are used during the declaration of Object Literals, or to enclose blocks of code (function definitions, conditional blocks, loops, etc).
var objLit = {
a: "I am an Object Literal."
};
Brackets [] are typically mostly used for accessing the properties of an Object (or the elements of an Array), so mylist[3] fetches the fourth element in the Array.
var mylist = [1,2,3,4];
alert(mylist[2]);
It doesn't help that you're trying to start with jQuery, which also uses its own Selector Grammar within Strings that are passed to function calls (which can make it look much more complicated than it actually is). This: $('a[rel="nofollow self"]') is only a single function call, and the inner brackets are handled by jQuery.
