ID attributes cannot start with a number and they should be unique. In any case, you can use :eq() to select a specific row using a 0-based integer:
// Remove the third row
$("#test tr:eq(2)").remove();
Alternatively, rewrite your HTML so that it's valid:
<table id="test">
<tr id=test1><td>bla</td></tr>
<tr id=test2><td>bla</td></tr>
<tr id=test3><td>bla</td></tr>
<tr id=test4><td>bla</td></tr>
</table>
And remove it referencing just the id:
$("#test3").remove();