The solid corners are a limitation of Firefox (and other Gecko-based browsers). MDN says:
Dotted and dashed rounded border corners are rendered as solid in Gecko; see bug 382721.
Why not something like
<form action="checklogin.php" method="post" style="display:block;border:1px dotted #0000ff;-moz-border-radius:7px;-webkit-border-radius:7px;border-radius:10px; width:200px; padding:10px; margin-left:auto; margin-right:auto;" >
<h2 style="text-align:center; font-weight:bold">Admin Login</h2>
<labe for="username">Username : </label>
<input name="username" id="username" type="text"/>
<labe for="password">Password : </label>
<input name="password" id="password" type="password" />
<input name="submit" id="submit" type="submit" value="Log in" style="display:block" />
</form>
You will avoid the table and for users it is easier to process labels on top of the inputs not on left. You can check it here jsfiddle