Prevent Android app from being associated with every file type, when sharing

Question

In my app I have a custom file extension for saved data. I have my app showing up when a user selects this particular type of file and does "Open With".

However