PyQt: Exit QSystemTrayIcon program after QMessageBox

I have a simple script primarily based on QSystemTrayIcon. Everything works find, and there's an option there on right-click on the taskbar icon that exits the program. I would like to add a QMessageBox, and on choosing yes, exit the program; otherwise, do nothing.

I'm familiar with all that, but it doesn't work as it should, and hence the question. I created a minimal example to demonstrate the problem:

import sys
from PyQt5 import QtCore, QtGui, QtWidgets


class SystemTrayIcon(QtWidgets.QSystemTrayIcon):
    def __init__(self, icon, parent=None):
        QtWidgets.QSystemTrayIcon.__init__(self, icon, parent)
        self.menu = QtWidgets.QMenu(parent)
        self.exit_action = self.menu.addAction("Exit")
        self.setContextMenu(self.menu)
        self.exit_action.triggered.connect(self.slot_exit)

        self.msg_parent = QtWidgets.QWidget()

    def slot_exit(self):
        reply = QtWidgets.QMessageBox.question(self.msg_parent, "Confirm exit",
                                               "Are you sure you want to exit Persistent Launcher?",
                                               QtWidgets.QMessageBox.Yes, QtWidgets.QMessageBox.No)
        # if reply == QtWidgets.QMessageBox.Yes:
        #     QtCore.QCoreApplication.exit(0)


def main():
    app = QtWidgets.QApplication(sys.argv)

    tray_icon = SystemTrayIcon(QtGui.QIcon("TheIcon.png"))

    tray_icon.show()
    sys.exit(app.exec_())

if __name__ == '__main__':
    main()

Now you see, at the slot_exit() function, whether I choose yes or no, the program exits (with code 0, no errors). The commented part is what I expect to be used to determine the action based on the choice. Could you please help me figure out why this behavior is happening and what's the right way to exit only on "yes"?

I'm using Windows 10, 64-bit with Python Anaconda 3.5.2 32-bit, and PyQt 5.7.

The problem is that Qt is made to exit when all Windows are closed. Simply disable that with: