The picture is always taken in the orientation the camera is built into the device. To get your image rotated correctly you'll have to read the orientation information that is stored into the picture (EXIF meta data). There it is stored how the device was oriented, when the image was taken.
Here is some code that reads the EXIF data and rotates the image accordingly:
file
is the name of the image file.
BitmapFactory.Options bounds = new BitmapFactory.Options();
bounds.inJustDecodeBounds = true;
BitmapFactory.decodeFile(file, bounds);
BitmapFactory.Options opts = new BitmapFactory.Options();
Bitmap bm = BitmapFactory.decodeFile(file, opts);
ExifInterface exif = new ExifInterface(file);
String orientString = exif.getAttribute(ExifInterface.TAG_ORIENTATION);
int orientation = orientString != null ? Integer.parseInt(orientString) : ExifInterface.ORIENTATION_NORMAL;
int rotationAngle = 0;
if (orientation == ExifInterface.ORIENTATION_ROTATE_90) rotationAngle = 90;
if (orientation == ExifInterface.ORIENTATION_ROTATE_180) rotationAngle = 180;
if (orientation == ExifInterface.ORIENTATION_ROTATE_270) rotationAngle = 270;
Matrix matrix = new Matrix();
matrix.setRotate(rotationAngle, (float) bm.getWidth() / 2, (float) bm.getHeight() / 2);
Bitmap rotatedBitmap = Bitmap.createBitmap(bm, 0, 0, bounds.outWidth, bounds.outHeight, matrix, true);
UPDATE 2017-01-16
With the release of the 25.1.0 Support Library, an ExifInterface Support Library was introduced, which should perhaps make the access to the Exif attributes easier. See the Android Developer's Blog for an article about it.