I've mentioned this before, but the tool I find most useful is an application of Reap and Sow which mimics/extends the behavior of GatherBy:
SelectEquivalents[x_List,f_:Identity, g_:Identity, h_:(#2&)]:=
Reap[Sow[g[#],{f[#]}]&/@x, _, h][[2]];
This allows me to group lists by any criteria and transform them in the process. The way it works is that a criteria function (f) tags each item in the list, each item is then transformed by a second supplied function (g), and the specific output is controlled by a third function (h). The function h accepts two arguments: a tag and a list of the collected items that have that tag. The items retain their original order, so if you set h = #1& then you get an unsorted Union, like in the examples for Reap. But, it can be used for secondary processing.
As an example of its utility, I've been working with Wannier90 which outputs the spatially dependent Hamiltonian into a file where each line is a different element in the matrix, as follows
rx ry rz i j Re[Hij] Im[Hij]
To turn that list into a set of matrices, I gathered up all sublists that contain the same coordinate, turned the element information into a rule (i.e. {i,j}-> Re[Hij]+I Im[Hij]), and then turned the collected rules into a SparseArray all with the one liner:
SelectEquivalents[hamlst,
#[[;; 3]] &,
#[[{4, 5}]] -> (Complex @@ #[[6 ;;]]) &,
{#1, SparseArray[#2]} &]
Honestly, this is my Swiss Army Knife, and it makes complex things very simple. Most of my other tools are somewhat domain specific, so I'll probably not post them. However, most, if not all, of them reference SelectEquivalents.
Edit: it doesn't completely mimic GatherBy in that it cannot group multiple levels of the expression as simply as GatherBy can. However, Map works just fine for most of what I need.
Example: @Yaroslav Bulatov has asked for a self-contained example. Here's one from my research that has been greatly simplified. So, let's say we have a set of points in a plane
In[1] := pts = {{-1, -1, 0}, {-1, 0, 0}, {-1, 1, 0}, {0, -1, 0}, {0, 0, 0},
{0, 1, 0}, {1, -1, 0}, {1, 0, 0}, {1, 1, 0}}
and we'd like to reduce the number of points by a set of symmetry operations. (For the curious, we are generating the little group of each point.) For this example, let's use a four fold rotation axis about the z-axis
In[2] := rots = RotationTransform[#, {0, 0, 1}] & /@ (Pi/2 Range[0, 3]);
Using SelectEquivalents we can group the points that produce the same set of images under these operations, i.e. they're equivalent, using the following
In[3] := SelectEquivalents[ pts, Union[Through[rots[#] ] ]& ] (*<-- Note Union*)
Out[3]:= {{{-1, -1, 0}, {-1, 1, 0}, {1, -1, 0}, {1, 1, 0}},
{{-1, 0, 0}, {0, -1, 0}, {0, 1, 0}, {1, 0, 0}},
{{0,0,0}}}
which produces 3 sublists containing the equivalent points. (Note, Union is absolutely vital here as it ensures that the same image is produced by each point. Originally, I used Sort, but if a point lies on a symmetry axis, it is invariant under the rotation about that axis giving an extra image of itself. So, Union eliminates these extra images. Also, GatherBy would produce the same result.) In this case, the points are already in a form that I will use, but I only need a representative point from each grouping and I'd like a count of the equivalent points. Since, I don't need to transform each point, I use the Identity function in the second position. For the third function, we need to be careful. The first argument passed to it will be the images of the points under the rotations which for the point {0,0,0} is a list of four identical elements, and using it would throw off the count. However, the second argument is just a list of all the elements that have that tag, so it will only contain {0,0,0}. In code,
In[4] := SelectEquivalents[pts,
Union[Through[rots[#]]]&, #&, {#2[[1]], Length[#2]}& ]
Out[4]:= {{{-1, -1, 0}, 4}, {{-1, 0, 0}, 4}, {{0, 0, 0}, 1}}
Note, this last step can just as easily be accomplished by
In[5] := {#[[1]], Length[#]}& /@ Out[3]
But, it is easy with this and the less complete example above to see how very complex transformations are possible with a minimum of code.
