Got it.
Now this is my code:
$p = get_adjacent_post(1, '', 1);
if(!empty($p)) echo ' ';
$n = get_adjacent_post(1, '', 0);
if(!empty($n)) echo ' ';
The function returns an object of the prev/next post which I can use for generating my links. The first parameter is for restricting the post on the same cat.
I searched in wordpress codex a few times yesterday but didn't come across this function, now stumled upon it by accident.
If someone has a better/simpler/faster method please post to get an accepted answer.