Scaling AR pictures based on the distance from the camera

Question

I’m developing an augmented reality iPhone app.

What it should basically do is display pictures assigned to geographical locations when you look at

Answer 1

I think I managed to solve my problem. Let me explain how I did it for if might be of use to others. If you find this approach wrong, I will be grateful for your feedback or any further clues.

I decided to assign physical dimensions in metres to my virtual billboards. This discussion helped me find out the parameters of the iPhone 4 camera: focal length and the dimensions of the CCD sesnor. What's more, these values also helped me calculate a proper FOV for my AR app (see Calculating a camera's angle of view).

This website helped me calculate the size in millimeters of a physical object image produced on a CCD sensor. So if my billboards have width and height in metres and their distance from the camera is known as well as the focal length of the camera, I can calculate their size on the sensor.

(Focal Length * Object Dimension) / Lens-to-object distance = Image Size (on the sensor)

double CalculatePhysicalObjectImageDimensionOnCCD(double cameraFocalLength_mm, double physicalObjectDimension_m, double distanceFromPhysicalObject_m)
{
    double physicalObjectDimension_mm = physicalObjectDimension_m * 1000;
    double distanceFromPhysicalObject_mm = distanceFromPhysicalObject_m * 1000;
    return (cameraFocalLength_mm * physicalObjectDimension_mm) / distanceFromPhysicalObject_mm;
}

I have little knowledge on that matter so I’m not sure if the approach I took then is OK, but I just decided to calculate how much larger the iPhone screen is compared to the dimensions of the CCD sensor. So by a simple mathematical operation I get a sensor-to-screen size ratio. Because the width-to-height ratio of the sensor and the screen seem to be different, I calculated the ratio in a kind of cranky way:

double GetCCDToScreenSizeRatio(double sensorWidth, double sensorHeight, double screenWidth, double screenHeight)
{
    return sqrt(screenWidth * screenHeight) / sqrt(sensorWidth * sensorHeight);
}

Then the ratio I get can be treated as a multiplier. First I calculate a dimension of my virtual billboard on the sensor and then multiply it by the ratio. This way I get the actual size of the billboard in pixels. That’s it. So when I call the function below just by providing the width of my billboard and the distance from it, it returns the width in pixels of the billboard as viewed on the screen. Same for the height of the billboard to get both dimensions.

const double CCD_DIM_LONGER_IPHONE4 = 4.592; //mm
const double CCD_DIM_SHORTER_IPHONE4 = 3.450; //mm
const double FOCAL_LENGTH_IPHONE4 = 4.28; //mm

double CalculatePhysicalObjectImageDimensionOnScreen_iPhone4(double physicalObjectDimension_m, double distanceFromPhysicalObject_m)
{
    double screenWidth = [UIScreen mainScreen].bounds.size.width;
    double screenHeight = [UIScreen mainScreen].bounds.size.height;

    return CalculatePhysicalObjectImageDimensionOnScreen(FOCAL_LENGTH_IPHONE4, physicalObjectDimension_m, distanceFromPhysicalObject_m, CCD_DIM_LONGER_IPHONE4, CCD_DIM_SHORTER_IPHONE4, screenWidth, screenHeight);
}

double CalculatePhysicalObjectImageDimensionOnScreen(double cameraFocalLength_mm, double physicalObjectDimension_m, double distanceFromPhysicalObject_m, double ccdSensorWidth, double ccdSensorHeight, double screenWidth, double screenHeight)
{
    double ccdToScreenSizeRatio = GetCCDToScreenSizeRatio(ccdSensorWidth, ccdSensorHeight, screenWidth, screenHeight);
    double dimensionOnCcd = CalculatePhysicalObjectImageDimensionOnCCD(cameraFocalLength_mm, physicalObjectDimension_m, distanceFromPhysicalObject_m);

    return dimensionOnCcd * ccdToScreenSizeRatio;
}

It seems that it works perfect compared ty my previous, stupid approach of linear scaling. I also noticed, by the way, that it is really important to know the FOV for your camera when registering virtual objects on an AR view. Here’s how to calculate the FOV based on CCD sensor dimensions and the focal length.

It’s so difficult to find these values anywhere! I wonder why they are not accessible programmatically (at least my research showed me that they are not). It seems that it is necessary to prepare hard-coded values and then check the model of the device the app is running on to decide which of the values to choose when doing all the calculations above :-/.