The problem is that you are calling handleError, then passing the result to expect. If handleError throws, then expect never even gets called.
You need to defer calling handleError until expect is called, so that expect can see what happens when the function is called. Fortunately, this is what expect wants:
expect(function () { handleError(true); }).to.not.throw();
expect(function () { handleError("anything else") }).to.throw("stop js execution");
If you read the documentation for throw, you'll see that the associated expect should be passed a function.
