How do I convert a IPython Notebook into a Python file via commandline?

Question

I\'m looking at using the *.ipynb files as the source of truth and programmatically \'compiling\' them into .py files for scheduled jobs/tasks.

The

Answer 1

For converting all *.ipynb format files in current directory to python scripts recursively:

for i in *.ipynb **/*.ipynb; do 
    echo "$i"
    jupyter nbconvert  "$i" "$i"
done

Answer 2

There's a very nice package called nb_dev which is designed for authoring Python packages in Jupyter Notebooks. Like nbconvert, it can turn a notebook into a .py file, but it is more flexible and powerful because it has a lot of nice additional authoring features to help you develop tests, documentation, and register packages on PyPI. It was developed by the fast.ai folks.

It has a bit of a learning curve, but the documentation is good and it is not difficult overall.

Answer 3

Here is a quick and dirty way to extract the code from V3 or V4 ipynb without using ipython. It does not check cell types, etc.

import sys,json

f = open(sys.argv[1], 'r') #input.ipynb
j = json.load(f)
of = open(sys.argv[2], 'w') #output.py
if j["nbformat"] >=4:
        for i,cell in enumerate(j["cells"]):
                of.write("#cell "+str(i)+"\n")
                for line in cell["source"]:
                        of.write(line)
                of.write('\n\n')
else:
        for i,cell in enumerate(j["worksheets"][0]["cells"]):
                of.write("#cell "+str(i)+"\n")
                for line in cell["input"]:
                        of.write(line)
                of.write('\n\n')

of.close()

Answer 4

I had this problem and tried to find the solution online. Though I found some solutions, they still have some problems, e.g., the annoying Untitled.txt auto-creation when you start a new notebook from the dashboard.

So eventually I wrote my own solution:

import io
import os
import re
from nbconvert.exporters.script import ScriptExporter
from notebook.utils import to_api_path


def script_post_save(model, os_path, contents_manager, **kwargs):
    """Save a copy of notebook to the corresponding language source script.

    For example, when you save a `foo.ipynb` file, a corresponding `foo.py`
    python script will also be saved in the same directory.

    However, existing config files I found online (including the one written in
    the official documentation), will also create an `Untitile.txt` file when
    you create a new notebook, even if you have not pressed the "save" button.
    This is annoying because we usually will rename the notebook with a more
    meaningful name later, and now we have to rename the generated script file,
    too!

    Therefore we make a change here to filter out the newly created notebooks
    by checking their names. For a notebook which has not been given a name,
    i.e., its name is `Untitled.*`, the corresponding source script will not be
    saved. Note that the behavior also applies even if you manually save an
    "Untitled" notebook. The rationale is that we usually do not want to save
    scripts with the useless "Untitled" names.
    """
    # only process for notebooks
    if model["type"] != "notebook":
        return

    script_exporter = ScriptExporter(parent=contents_manager)
    base, __ = os.path.splitext(os_path)

    # do nothing if the notebook name ends with `Untitled[0-9]*`
    regex = re.compile(r"Untitled[0-9]*$")
    if regex.search(base):
        return

    script, resources = script_exporter.from_filename(os_path)
    script_fname = base + resources.get('output_extension', '.txt')

    log = contents_manager.log
    log.info("Saving script at /%s",
             to_api_path(script_fname, contents_manager.root_dir))

    with io.open(script_fname, "w", encoding="utf-8") as f:
        f.write(script)

c.FileContentsManager.post_save_hook = script_post_save

To use this script, you can add it to ~/.jupyter/jupyter_notebook_config.py :)

Note that you may need to restart the jupyter notebook / lab for it to work.