I setup a Hadoop cluster with Spark 2.4, on my client I installed Jupyter notebook and the SparkMagic module.
Everything seems to work fine within the same notebook.