I am making use of a contenteditable div in combination with the rangy Javascript library to insert HTML at the cursor position.
End of the day the contents of the div c
I made a jsfiddle with a quick sample of how to look at the range of the selection and use the previousSibling property of the startContainer to find the button: jsfiddle
Put the cursor in aaa and it will show the button is prev sibling. put it in ccc and bbb will show.
So with this you could handle the keydown event of the div, check if key is backspace + previousSibling is your button, and remove it with jQuery.
previousSibling is null if it is a textnode though, just FYI.