So I nearly implemented a free-flight camera using vectors and something like gluLookAt.
The movement in all 4 directions and rotation around the Y-axis work fine.
The problem you're facing is the exact same that I had trouble with the first time I tried to implement the camera movement. The problem occurs because if you first turn so that you are looking straight down the X axis and then try to "tilt" the camera by rotating around the X axis, you will effectively actually spin around the direction you are looking.
I find that the best way to handle camera movement is to accumulate the angles in separate variables and every time rotate completely from origin. If you do this you can first "tilt" by rotating around the X-axis then turn by rotating around the Y-axis. By doing it in this order you make sure that the tilting will always be around the correct axis relative to the camera. Something like this:
public void pan(float turnSpeed)
{
totalPan += turnSpeed;
updateOrientation();
}
public void tilt(float turnSpeed)
{
totalTilt += turnSpeed;
updateOrientation();
}
private void updateOrientation()
{
float afterTiltX = 0.0f; // Not used. Only to make things clearer
float afterTiltY = (float) Math.sin(totalTilt));
float afterTiltZ = (float) Math.cos(totalTilt));
float vecX = (float)Math.sin(totalPan) * afterTiltZ;
float vecY = afterTiltY;
float vecZ = (float)Math.cos(totalPan) * afterTiltZ;
center = eye + vecmath.vector(vecX, vecY, vecZ);
}
I don't know if the syntax is completely correct. Haven't programmed in java in a while.