Determine all camera parameters by a simultanous minimization process.

::camera_calibration performs the calibration of a video camera. Thus, known 3D model points (with coordinates NX, NY, NZ) are projected in the image and the sum of the squared distance between these projections and the corresponding image points (with coordinates NRow, NCol) is minimized.

In case of converging, the exact internal (CamParam) and external (NFinalPose) camera parameters are determined by this minimization algorithm. The parameters StartCamParam and NStartPose are used as initial values for the minimization process. Since this algorithm simultaneously handles correspondences between image and model points from different images, it is also called multi image calibration.

In general camera calibration means the exact determination of the parameters, which model the (optical) projection of any 3D world point P_W into a (sub-)pixel [r,c] on the CCD sensor of the camera. This is of importance, if the original 3D pose of an object has to be computed using an image (e.g., measuring of industrial pieces).

The underlying camera model is a pinhole camera with radial distortions if the focal length passed in StartCamParam is greater than 0. It describes the transform of a 3D world point P_W into a (sub-)pixel [r,c] of the video image by the following equations:

  P_C = (x,y,z) = R*P_W + T

    u = Focus * x / z
    v = Focus * y / z

   u' = (2*u) / (1+sqrt(1-4*Kappa*(u^2+v^2)))
   v' = (2*v) / (1+sqrt(1-4*Kappa*(u^2+v^2)))

    c = u' / Sx + Cx
    r = v' / Sy + Cy

  P_C = (x,y,z) = R*P_W + T

    u = x
    v = y

   u' = (2*u) / (1+sqrt(1-4*Kappa*(u^2+v^2)))
   v' = (2*v) / (1+sqrt(1-4*Kappa*(u^2+v^2)))

    c = u' / Sx + Cx
    r = v' / Sy + Cy

The total of 15 camera parameters can be divided into the internal and external camera parameters:

Internal camera parameters:

  These parameters describe the characteristics of the used video
  camera, especially the dimension of the CCD sensor itself and the
  projection properties of the used combination of lens, camera, and
  framegrabber.  The camera model (as described above) contains the
  following 8 parameters:

  Focus:       Focal length of the lens.  0 for telecentric lenses.
  Kappa:       Distortion coefficient to model the pillow- or
               barrel-shaped distortions caused by the lens.
  Sx:          Scale factor, corresponds to the horizontal distance
               between two neighboring cells on the CCD sensor.
               Attention: This value increases, if the image is
               subsampled!
  Sy:          Scale factor, corresponds to the vertical distance
               between two neighboring cells on the CCD sensor.
               Since in most cases the image signal is sampled
               line-synchronously, this value is determined by the
               dimension of the CCD sensor and needn't be estimated
               for pinhole cameras by the calibration process.
       Attention: This value increases, if the image is
       subsampled!
  Cx:          Column coordinate of the image center point (center
               of the radial distortion).
  Cy:          Row coordinate of the image center point (center of
               the radial distortion).
  ImageWidth:  Width of the sampled video image.  Attention: This
               value decreases, if the image is subsampled!
  ImageHeight: Height of the sampled video image.  Attention: This
               value decreases, if the image is subsampled!

External camera parameters:

  These 7 parameters describe the position and orientation of the
  camera and are also called as camera pose.  The relative position
  of the camera with regard of a world or object coordinate system
  is defined by a 3D translational vector (parameters 1, 2, and
  3).  The orientation of the camera is described by the three
  rotation angles around the three coordinate axes (parameters 4, 5,
  and 6).

  The last value within the camera pose encodes the representation
  type of the described 3D transform: E.g., the value '0' is the
  code of the representation type 1 and says, that the transform of
  a point is described, hereby the 3D rotation is performed before
  the 3D translation, that the three rotations are specified as
  angles (in degrees), and that the rotation order is
  Gamma-Beta-Alpha, i.e., the first rotation is around the Z-axis,
  the second one around the new Y-axis, and the third one around the
  new X-axis, which is generated after the second rotation.
  ::camera_calibration operates with all types of 3D transform
  for NStartPose.  The meaning of the other representation
  types are explained at ::create_pose.

  Note that in contrast to the external camera parameters the
  internal ones are the same for all positions and orientations of
  the camera.

The use of ::camera_calibration leads to some questions, which are dealed with in the following sections:

How to generate a appropriate calibration table?

The simplest method to determine the internal parameter of a CCD camera is the use of the planar calibration table generated by the operator ::create_caltab. In case of small distances between object and lens it may be sufficient to print the calibration pattern by a laser printer and to mount it on a cardboard. Otherwise -- especially by using a wide-angle lens -- it is possible to print the PostScript file on a large inkjet printer and to mount it on a aluminum table. It is very important, that the mark coordinates in the calibration table description file correspond to the real ones on the calibration table with high accuracy. Thus, the calibration table description file has to be modified in accordance with the measurement of the calibration table!

How to take a set of suitable images?

If you use the planar calibration table, you can proceed in the following way: With the combination of lens (fixed distance!), camera, and framegrabber to be calibrated a set of images of the calibration table has to be taken, see ::open_framegrabber and ::grab_image. The following items have to be considered:

   - At least a total of 10 to 20 images should be taken into account.
   - The calibration table has to be completely visible (incl. border!).
   - Reflections etc. on the calibration table should be avoided.
   - Within the set of images the calibration table should appear in
     different positions and orientations: Once left in the image,
     once right, once (left and right) at the bottom, once (left or
     right) at the top, from different distances etc.  At this, the
     calibration table should be rotated a little around its X-
     and/or Y-axis, so the perspective distortions of the
     calibration pattern are clearly visible.  Thus, the external
     camera parameters (camera pose with regard of the calibration
     table) should be set to a large variety of different values!
   - The calibration table should fill at least a quarter of the
     whole image to ensure the robust detection of the marks.

How to extract the calibration marks in the images?

If a planar calibration table is used, for each image the operators ::find_caltab and ::find_marks_and_pose can be used to determine the coordinates of the calibration marks and to compute a rough estimate for the external camera parameters. The concatenation of these values can directly be used as initial values for the external camera parameters (NStartPose) in ::camera_calibration.

Obviously, images, on which the segmentation of the calibration table (::find_caltab) has failed or the calibration marks haven't been determined successfully by ::find_marks_and_pose, should not be used.

How to find suitable initial values for the internal camera parameters?

The operators ::find_marks_and_pose (determination of initial values for the external camera parameters) and ::camera_calibration require initial values for the internal camera parameters. These parameters can be provided by a appropriate text file (see ::read_cam_par), which can be generated by ::write_cam_par or can be edited manually.

The following should be considered for the initial values of the single parameters:

  - Focus:       The initial value is the nominal focal length of the
                 the used lens, e.g., 0.008 m.
  - Kappa:       Use 0.0 as initial value. The calibratied value normally 
                 lies between -1000.0 and -50000.0 1/(m*m) depending on
                 the used lens.
  - Sx:          The initial value for the horizontal distance between
                 two neighboring CCD cells depends on the dimension of
                 the used CCD chip of the camera (see technical 
                 specifications of the camera). Generally, common CCD chips 
                 are either 1/3''-Chips (e.g., SONY XC-73, SONY XC-777), 
                 1/2''-Chips (e.g., SONY XC-999, Panasonic WV-CD50), or 
                 2/3''-Chips (e.g., SONY DXC-151, SONY XC-77). Notice: 
                 The value of Sx increases, if the image is subsampled! 
                 Appropriate initial values are:

                             Full image (768*576)   Subsampling (384*288)
                 1/3"-Chip      0.0000055 m            0.0000110 m
                 1/2"-Chip      0.0000086 m            0.0000172 m
                 2/3"-Chip      0.0000110 m            0.0000220 m

                 The value for Sx is calibrated, since the video signal
                 of a CCD camera normally isn't sampled pixel-synchronously.
  - Sy:          Since most off-the-shelf cameras have quadratic pixels,
                 the same values for Sy are valid as for Sx.  In contrast
                 to Sx the value for Sy will NOT be calibrated for pinhole
                 cameras, because the video signal of a CCD camera normally
                 is sampled line-synchronously. Thus, the initial value
                 is equal to the final value. Appropriate initial values are:

                             Full image (768*576)   Subsampling (384*288)
                 1/3"-Chip      0.0000055 m            0.0000110 m
                 1/2"-Chip      0.0000086 m            0.0000172 m
                 2/3"-Chip      0.0000110 m            0.0000220 m

  - Cx and Cy:   Initial values for the coordinates of the image center
                 is the half image width and half image height. 
                 Notice: The values of Cx and Cy decrease, if the image 
                 is subsampled! Appropriate initial values are:

                             Full image (768*576)   Subsampling (384*288)
                 Cx             384.0                  192.0
                 Cy             288.0                  144.0

  - ImageWidth and ImageHeight: These two parameters are set by the
                 the used framegrabber and therefore are not calibrated.
                 Appropriate initial values are:
    
                             Full image (768*576)   Subsampling (384*288)
                 ImageWidth     768                    384
                 ImageHeight    576                    288

Which camera parameters have to be estimated?

The input parameter EstimateParams is used to select which camera parameters to estimate. Usually this parameter is set to 'all', i.e., all 6 external camera parameters (translation and rotation) and all internal camera parameters have to be determined. Otherwise, EstimateParams contains a tuple of strings indicating the combination of parameters to estimate.

What is the order within the single parameters?

The length of the tuple NStartPose corresponds to the number of calibration images, e.g., using 15 images leads to a length of the tuple NStartPose equal to 15*7=105 (15 times the 7 external camera parameters). The first 7 values correspond to the camera pose of the first image, the next 7 values to the pose of the second one, etc.

This fixed number of calibration images has to be considered within the tuples with the coordinates of the 3D model marks and the extracted 2D marks. If 15 images are used, the length of the tuples NRow and NCol is 15 times the length of the tuples with the coordinates of the 3D model marks (NX, NY, and NZ). If every image consists 49 marks, the length of the tuples NRow and NCol is 15*49=735, while the length of the tuples NX, NY, and NZ is 49. The order of the values in NRow and NCol is ``image after image'', i.e., using 49 marks the first 3D model point corresponds to the 1st, 50th, 99th, 148th, 197th, 246th, etc. extracted 2D mark.

The 3D model points can be read from a calibration table description file using the operator ::caltab_points. Initial values for the camera pose can be determined by applying ::find_marks_and_pose for each image. The tuple NStartPose is set by the concatenation of all these camera poses.

What is the meaning of the output parameters?

If the camera calibration process is finished successfully, i.e., the minimization process is converged, the output parameters CamParam and NFinalPose contain the computed exact values for the internal and external camera parameters. The length of the tuple NFinalPose corresponds to the length of the tuple NStartPose.

The representation types of NFinalPose correspond to the representation type of the first tuple of NStartPose. You can convert the representation type by ::convert_pose_type. The sense of all kind of representation types for the 3D transform are explained at ::create_pose.

The computed average errors (Errors) give an impression of the accuracy of the calibration. The error values (deviations in x- and y-coordinates) are measured in pixels.

Must I use a planar calibration table?

No. The operator ::camera_calibration is designed in a way, that the input tuples NX, NY, NZ, NRow, and NCol can contain any 3D/2D correspondences, see the above paragraph explaining the order of the single parameters.

Thus, it makes no difference, how the required 3D model marks and the corresponding extracted 2D marks are determined. On the one hand it is possible to use a 3D calibration pattern, on the other hand you also can use any characteristic points (natural landmarks) with known position in the world. By setting EstimateParams to 6, it is possible to compute the world position of the camera! Hereby at least three 3D/2D-correspondences are necessary as input. Homogeneous transformation matrices are useful to generate NStartPose, see program example in ::hom_mat3d_to_pose and/or program example in ::create_pose for generating NStartPose directly.

The minimization process of the calibration depends on the initial values of the internal (StartCamParam) and external (NStartPose) camera parameters. The computed average errors Errors give an impression of the accuracy of the calibration. The errors (deviations in x- and y-coordinates) are measured in pixels.

HTuple StartCamPar,NX,NY,NZ;
HTuple RCoord1,CCoord1,StartPose1;
HTuple RCoord2,CCoord2,StartPose2;
HTuple RCoord3,CCoord3,StartPose3;
HTuple StartPoses, RCoords, CCoords;
HTuple CamParam,NFinalPose,Errors;
// read calibration images
HImage Image1("calib-01.tiff");
HImage Image2("calib-02.tiff");
HImage Image3("calib-03.tiff");
// find calibration pattern
HRegion Caltab1 = Image1.FindCaltab("caltab.descr",3,112,5);
HRegion Caltab2 = Image2.FindCaltab("caltab.descr",3,112,5);
HRegion Caltab3 = Image3.FindCaltab("caltab.descr",3,112,5);
// find calibration marks and start poses
StartCamPar[7] = 576;          // ImageHeight
StartCamPar[6] = 768;          // ImageWidth
StartCamPar[5] = 288;          // Cy
StartCamPar[4] = 384;          // Cx
StartCamPar[3] = 0.000011;     // Sy
StartCamPar[2] = 0.000011;     // Sx
StartCamPar[1] = 0.0;          // Kappa
StartCamPar[0] = 0.008;        // Focus
RCoord1 = Image1.FindMarksAndPose(Caltab1,"caltab.descr",StartCamPar,
                                  128,10,&CCoord1,&StartPose1);
RCoord2 = Image2.FindMarksAndPose(Caltab2,"caltab.descr",StartCamPar,
                                  128,10,&CCoord2,&StartPose2);
RCoord3 = Image3.FindMarksAndPose(Caltab3,"caltab.descr",StartCamPar,
                                  128,10,&CCoord3,&StartPose3);
// read 3D positions of calibration marks
::caltab_points("caltab.descr",&NX,&NY,&NZ);
// camera calibration
StartPoses = (StartPose1.Append(StartPose2)).Append(StartPose3);
RCoords = (RCoord1.Append(RCoord2)).Append(RCoord3);
CCoords = (CCoord1.Append(CCoord2)).Append(CCoord3);
::camera_calibration(NX,NY,NZ,RCoords,CCoords,StartCamPar,StartPoses,
                     11,&CamParam,&NFinalPose,&Errors);
// write internal camera parameters to file
::write_cam_par(CamParam,"campar.dat");

::camera_calibration returns H_MSG_TRUE if all parameter values are correct and the desired camera parameters have been determined by the minimization algorithm. If necessary, an exception handling is raised.

::find_marks_and_pose, ::caltab_points, ::read_cam_par

::write_pose, ::pose_to_hom_mat3d, ::disp_caltab, ::sim_caltab

::find_caltab, ::find_marks_and_pose, ::disp_caltab, ::sim_caltab, ::write_cam_par, ::read_cam_par, ::create_pose, ::convert_pose_type, ::write_pose, ::read_pose, ::pose_to_hom_mat3d, ::hom_mat3d_to_pose, ::caltab_points, ::create_caltab

Camera calibration