What is in your Mathematica tool bag? [closed]

Answer 1

One of the nice things about the Mathematica notebook interface is that it can evaluate expressions in any language, not just Mathematica. As a simple example, consider creating a new Shell input cell type that passes the contained expression to the operating system shell for evaluation.

First, define a function that delegates evaluation of a textual command to the external shell:

shellEvaluate[cmd_, _] := Import["!"~~cmd, "Text"]

The second argument is needed and ignored for reasons that will become apparent later. Next, we want to create a new style called Shell:

1. Open a new notebook.
2. Select the menu item Format/Edit Stylesheet...
3. In the dialog, beside Enter a style name: type Shell.
4. Select the cell bracket beside the new style.
5. Select the menu item Cell/Show Expression
6. Overwrite the cell expression with the Step 6 Text given below.
7. Once again, select the menu item Cell/Show Expression
8. Close the dialog.

Use the following cell expression as the Step 6 Text:

Cell[StyleData["Shell"],
 CellFrame->{{0, 0}, {0.5, 0.5}},
 CellMargins->{{66, 4}, {0, 8}},
 Evaluatable->True,
 StripStyleOnPaste->True,
 CellEvaluationFunction->shellEvaluate,
 CellFrameLabels->{{None, "Shell"}, {None, None}},
 Hyphenation->False,
 AutoQuoteCharacters->{},
 PasteAutoQuoteCharacters->{},
 LanguageCategory->"Formula",
 ScriptLevel->1,
 MenuSortingValue->1800,
 FontFamily->"Courier"]

Most of this expression was copied directly form the built-in Program style. The key changes are these lines:

 Evaluatable->True,
 CellEvaluationFunction->shellEvaluate,
 CellFrameLabels->{{None, "Shell"}, {None, None}},

Evaluatable enables the SHIFT+ENTER functionality for the cell. Evaluation will call the CellEvaluationFunction passing the cell content and content type as arguments (shellEvaluate ignores the latter argument). CellFrameLabels is just a nicety that let's the user identify that this cell is unusual.

With all of this in place, we can now enter and evaluate a shell expression:

1. In the notebook created in the steps above, create an empty cell and select the cell bracket.
2. Select the menu item Format/Style/Shell.
3. Type a valid operating system shell command into the cell (e.g. 'ls' on Unix or 'dir' on Windows).
4. Press SHIFT+ENTER.

It is best to keep this defined style in a centrally located stylesheet. Furthermore, evaluation functions like shellEvaluate are best defined as stubs using DeclarePackage in init.m. The details of both of these activities are beyond the scope of this response.

With this functionality, one can create notebooks that contain input expressions in any syntax of interest. The evaluation function can be written in pure Mathematica, or delegate any or all parts of the evaluation to an external agency. Be aware that there are other hooks that relate to cell evaluation, like CellEpilog, CellProlog and CellDynamicExpression.

A common pattern involves writing the input expression text to a temporary file, compiling the file in some language, running the program and capturing the output for ultimate display in the output cell. There are plenty of details to address when implementing a full solution of this kind (like capturing error messages properly), but one must appreciate the fact that it is not only possible to do things like this, but practical.

On a personal note, it is features like this that makes the notebook interface the center of my programming universe.

Update

The following helper function is useful for creating such cells:

evaluatableCell[label_String, evaluationFunction_] :=
  ( CellPrint[
      TextCell[
        ""
      , "Program"
      , Evaluatable -> True
      , CellEvaluationFunction -> (evaluationFunction[#]&)
      , CellFrameLabels -> {{None, label}, {None, None}}
      , CellGroupingRules -> "InputGrouping"
      ]
    ]
  ; SelectionMove[EvaluationNotebook[], All, EvaluationCell]
  ; NotebookDelete[]
  ; SelectionMove[EvaluationNotebook[], Next, CellContents]
  )

It is used thus:

shellCell[] := evaluatableCell["shell", Import["!"~~#, "Text"] &]

Now, if shellCell[] is evaluated, the input cell will be deleted and replaced with a new input cell that evaluates its contents as a shell command.

Answer 2

Start without a blank notebook open

I was bothered by having Mathematica start with a blank notebook open. I could close this notebook with a script, but it would still flash open briefly. My hack is to create a file Invisible.nb containing:

Notebook[{},Visible->False]

And add this to my Kernel\init.m :

If[Length[Notebooks["Invisible*"]] > 0, 
  NotebookClose[Notebooks["Invisible*"][[1]]]
]

SetOptions[$FrontEnd,
  Options[$FrontEnd, NotebooksMenu] /. 
    HoldPattern["Invisible.nb" -> {__}] :> Sequence[]
]

I now start Mathematica by opening Invisible.nb

There may be a better way, but this has served me well.

---

Customized Fold and FoldList

Fold[f, x] is made equivalent to Fold[f, First@x, Rest@x]

Incidentally, I believe this may find its way into a future version of Mathematica.

Surprise! This has been implemented, though it is presently undocumented. I am informed that it was implemented in 2011 by Oliver Ruebenkoenig, apparently not long after I posted this. Thank you Oliver Ruebenkoenig!

Unprotect[Fold, FoldList]

Fold[f_, h_[a_, b__]] := Fold[f, Unevaluated @ a, h @ b]
FoldList[f_, h_[a_, b__]] := FoldList[f, Unevaluated @ a, h @ b]

(* Faysal's recommendation to modify SyntaxInformation *)
SyntaxInformation[Fold]     = {"ArgumentsPattern" -> {_, _, _.}};
SyntaxInformation[FoldList] = {"ArgumentsPattern" -> {_, _., {__}}};

Protect[Fold, FoldList]

Updated to allow this:

SetAttributes[f, HoldAll]
Fold[f, Hold[1 + 1, 2/2, 3^3]]

f[f[1 + 1, 2/2], 3^3]

---

"Dynamic Partition"

See Mathematica.SE post #7512 for a new version of this function.

Frequently I want to partition a list according to a sequence of lengths.

pseudo-code example:

partition[{1,2,3,4,5,6}, {2,3,1}]

Output: {{1,2}, {3,4,5}, {6}}

I came up with this:

dynP[l_, p_] := 
 MapThread[l[[# ;; #2]] &, {{0} ~Join~ Most@# + 1, #} &@Accumulate@p]

Which I then completed with this, including argument testing:

dynamicPartition[l_List, p : {_Integer?NonNegative ..}] :=
  dynP[l, p] /; Length@l >= Tr@p

dynamicPartition[l_List, p : {_Integer?NonNegative ..}, All] :=
  dynP[l, p] ~Append~ Drop[l, Tr@p] /; Length@l >= Tr@p

dynamicPartition[l_List, p : {_Integer?NonNegative ..}, n__ | {n__}] :=
  dynP[l, p] ~Join~ Partition[l ~Drop~ Tr@p, n] /; Length@l >= Tr@p

The third argument controls what happens to elements beyond the split specification.

---

Szabolcs's Mathematica tricks

The one I use most frequently is the Paste Tabular Data Palette

CreatePalette@
 Column@{Button["TSV", 
    Module[{data, strip}, 
     data = NotebookGet[ClipboardNotebook[]][[1, 1, 1]];
     strip[s_String] := 
      StringReplace[s, RegularExpression["^\\s*(.*?)\\s*$"] -> "$1"];
     strip[e_] := e;
     If[Head[data] === String, 
      NotebookWrite[InputNotebook[], 
       ToBoxes@Map[strip, ImportString[data, "TSV"], {2}]]]]], 
   Button["CSV", 
    Module[{data, strip}, 
     data = NotebookGet[ClipboardNotebook[]][[1, 1, 1]];
     strip[s_String] := 
      StringReplace[s, RegularExpression["^\\s*(.*?)\\s*$"] -> "$1"];
     strip[e_] := e;
     If[Head[data] === String, 
      NotebookWrite[InputNotebook[], 
       ToBoxes@Map[strip, ImportString[data, "CSV"], {2}]]]]], 
   Button["Table", 
    Module[{data}, data = NotebookGet[ClipboardNotebook[]][[1, 1, 1]];
     If[Head[data] === String, 
      NotebookWrite[InputNotebook[], 
       ToBoxes@ImportString[data, "Table"]]]]]}

---

Modify external data from within Compile

Recently Daniel Lichtblau showed this method I had never seen before. In my opinion it significantly extends the utility of Compile

ll = {2., 3., 4.};
c = Compile[{{x}, {y}}, ll[[1]] = x; y];

c[4.5, 5.6]

ll

(* Out[1] = 5.6  *)

(* Out[2] = {4.5, 3., 4.}  *)

Answer 3

One trick I've used, which allows you to emulate the way most built-in functions work with bad arguments (by sending a message and then returning the whole form unevaluated) exploits a quirk of the way Condition works when used in a defintion. If foo should only work with one argument:

foo[x_] := x + 1;
expr : foo[___] /; (Message[foo::argx, foo, Length@Unevaluated[expr], 1]; 
                    False) := Null; (* never reached *)

If you have more complex needs, it's easy to factor out the argument validation and message generation as an independent function. You can do more elaborate things by using side effects in Condition beyond just generating messages, but in my opinion most of them fall into the "sleazy hack" category and should be avoided if possible.

Also, in the "metaprogramming" category, if you have a Mathematica package (.m) file, you can use the "HeldExpressions" element to get all the expressions in the file wrapped in HoldComplete. This makes tracking things down much easier than using text-based searches. Unfortunately, there's no easy way to do the same thing with a notebook, but you can get all the input expressions using something like the following:

inputExpressionsFromNotebookFile[nb_String] :=
 Cases[Get[nb],
  Cell[BoxData[boxes_], "Input", ___] :>
   MakeExpression[StripBoxes[boxes], StandardForm],
  Infinity]

Lastly, you can use the fact that Module emulates lexical closures to create the equivalent of reference types. Here's a simple stack (which uses a variation the Condition trick for error handling as a bonus):

ClearAll[MakeStack, StackInstance, EmptyQ, Pop, Push, Peek]
 With[{emptyStack = Unique["empty"]},
  Attributes[StackInstance] = HoldFirst;
  MakeStack[] :=
   Module[{backing = emptyStack},
    StackInstance[backing]];

  StackInstance::empty = "stack is empty";

  EmptyQ[StackInstance[backing_]] := (backing === emptyStack);

  HoldPattern[
    Pop[instance : StackInstance[backing_]]] /;
    ! EmptyQ[instance] || (Message[StackInstance::empty]; False) :=
   (backing = Last@backing; instance);

  HoldPattern[Push[instance : StackInstance[backing_], new_]] :=
   (backing = {new, backing}; instance);

  HoldPattern[Peek[instance : StackInstance[backing_]]] /;
    ! EmptyQ[instance] || (Message[StackInstance::empty]; False) :=
   First@backing]

Now you can print the elements of a list in reverse order in a needlessly convoluted way!

With[{stack = MakeStack[], list},
 Do[Push[stack, elt], {elt, list}];

 While[!EmptyQ[stack],
  Print[Peek@stack];
  Pop@stack]]

Answer 4

I'm sure a lot of people have encountered the situation where they run some stuff, realizing it not only stuck the program, but they also haven't saved for the last 10 minutes!

EDIT

After suffering from this for some time, I one day found out that one can create auto-save from within the Mathematica code. I think that using such auto-save have helped me a lot in the past, and I always felt that the possibility itself was something that not a lot of people are aware that they can do.

The original code I used is at the bottom. Thanks to the comments I've found out that it is problematic, and that it is much better to do it in an alternative way, using ScheduledTask (which will work only in Mathematica 8).

Code for this can be found in this answer from Sjoerd C. de Vries (Since I'm not sure if it's OK to copy it to here, I'm leaving it as a link only.)

---

The solution below is using Dynamic. It will save the notebook every 60 seconds, but apparently only if its cell is visible. I'm leaving it here only for completion reasons. (and for users of Mathematica 6 and 7)

/EDIT

To solve it I use this code in the beginning of a notebook:

Dynamic[Refresh[NotebookSave[]; DateString[], UpdateInterval -> 60]]

This will save your work every 60 seconds.
I prefer it to NotebookAutoSave[] because it saves before the input is processed, and because some files are more text than input.

I originally found it here: http://en.wikipedia.org/wiki/Talk:Mathematica#Criticisms

Note that once running this line, saving will happen even if you close and re-open your file (as long as dynamic updating is enabled).

Also, since there is no undo in Mathematica, be careful not to delete all your content, since saving will make it irreversible (as a precaution move, I remove this code from every finished notebook)

Answer 5

PutAppend with PageWidth -> Infinity

In Mathematica using of the PutAppend command is the most straightforward way to maintain a running log file with results of intermediate computations. But it uses by default PageWith->78 setting when exporting expressions to a file and so there is no guarantee that every intermediate output will take only one line in the log.

PutAppend does not have any options itself but tracing its evaluations reveals that it is based on the OpenAppend function which has the PageWith option and allows changing its default value by the SetOptions command:

In[2]:= Trace[x>>>"log.txt",TraceInternal->True]
Out[2]= {x>>>log.txt,{OpenAppend[log.txt,CharacterEncoding->PrintableASCII],OutputStream[log.txt,15]},Null}

So we can get PutAppend to append only one line at a time by setting:

SetOptions[OpenAppend, PageWidth -> Infinity]

---

UPDATE

There is a bug introduced in version 10 (fixed in version 11.3): SetOptions no longer affects the behavior of OpenWrite and OpenAppend.

A workaround is to implement your own version of PutAppend with explicit PageWidth -> Infinity option:

Clear[myPutAppend]
myPutAppend[expr_, pathtofile_String] :=
 (Write[#, expr]; Close[#];) &[OpenAppend[pathtofile, PageWidth -> Infinity]]

Note that we also may implement it via WriteString as shown in this answer, but in this case it will be necessary to preliminarily convert the expression into the corresponding InputForm via ToString[expr, InputForm].

Answer 6

I find it really useful when developing packages to add this keyboard shortcut to my SystemFiles/FrontEnd/TextResources/Windows/KeyEventTranslations.tr file.

(* Evaluate Initialization Cells: Real useful for reloading library changes. *)

Item[KeyEvent["i", Modifiers -> {Control, Command}],
    FrontEndExecute[
        FrontEndToken[
            SelectedNotebook[],
            "EvaluateInitialization"]]],

Next for every Packagename.m I make a PackagenameTest.nb notebook for testing and the first 2 cells of the test notebook are set as initialization cells. In the first cell I put

Needs["PackageManipulations`"]

to load the very useful PackageManipulations library which was written by Leonid. The second cell contains

PackageRemove["Packagename`Private`"]
PackageRemove["Packagename`"]
PackageReload["Packagename`"]

which all do the actual package reloading. Note the first two lines are there only to Remove all symbols as I like to keep the contexts as clean as possible.

Then the workflow for writing and testing a package becomes something like this.

1. Save changes to Packagename.m.
2. Go to PackagenameTest.nb and do CTRL + ALT + i.

This causes the initialization cells to reload the package, which makes testing real simple.