What is the meaning of the parameters? 
This document explains what the parameters of the evaluation protocol are and how to set them. The example in the text shows how to select the parameters for evaluation of similarity measures for rigid registration of simulated BrainWeb images. Why normalization? In the case of rigid registration of 3D images the parametrical space is six dimensional (three translations and three rotations). The parametrical space should be normalized because all dimensions of the parametrical space are not measured in the same units. Also similarity measure is differently affected if the values of different parameters are changed for the same value (if translation changes for 1 mm (1 unit in the nonnormalized parametrical space) the effect is different than if the rotation changes for 1 rad (1 unit in the nonnormalized parametrical space)). How is normalization taken into account? The radius R is the radius of the hypersphere in the muldimensional normalized parametrical space (6 D for rigid registration, 12 D for affine and 21 D for projective transformation). It can be set to initial misalignment of two images or according to the capture range that a similarity measure is expected to have. Points, at which the similarity measure should be evaluated, are computed inside the hypersphere in the normalized parametrical space. The hypersphere is centered in the normalized goldstandard point, which is obtained by dividing the gold standard translations and rotations by corresponding normalization parameters. To obtain the points in the nonnormalized parametrical space (translations and rotations), the computed points in the normalized parametrical space are multiplied by corresponding normalization parameters. How to select the normalization parameters? The dimensions of the images are 181 x 217 x 181 voxels of size (1 x 1 x 1) mm. The smallest dimension of the images is 181 mm. The normalization parameters for the three translations are set to 10 % of the minimal dimension of the floating image, which is in this case 18.1 mm. The normalization parameters for rotation are set to the rotation around one corner of the image, which causes the mean shift of voxels for 18.1 mm. In radians this is approximatelly 0.108. How to select N and M? How dense the parametrical space will be evaluated depends on the number of lines in the hypersphere N and on the number of evenly spaced points M along each line. We have experimentally established that the properties stabilize if N is larger than 40. So we have decided to set it to 50. Larger values can be used for N. We can say that where omega is a constant and D is the number of dimensions. From our experiments it can be concluded that omega is around 1.9. There are many ways to select the number of points M on a line. M and R define the step size (2*R/M). M can be set to a value for which a step size will be smaller than the size of a voxel. For a given R, the number of points M may be selected with respect to the accuracy that a similarity measurebased registration is supposed to accomplish. In the sampling example for 2D parametrical space x1 and x2 are the two normalized dimensions of the parametrical space and X0 is the normalized gold standard position. If the parameters are set according to instructions above, the first step of the protocol produces two files of the same points written in different formats. The first file (0.7 MB) is organized in the same format that the file with the results should be. The second file (0.6 MB) is organized in the following way:
The link to both files is sent to the specified email address so that the reasearchers can decide which file format is more suitable for them. Additional file formats can be added if needed by the users. Step 2: This step depends on the specific implementation of a similarity measure. The reasearchers interested in evaluation should compute the values of their similarity measure at the points determined in the previous step and format the results like they are formatted in the example file. The file should begin with the normalization parameters. The points in the paramaterical space in which the user computed the values of the similarity measure must be written in the lines, which begin with the string "Point: ". The coordinates of the points should be written in the same order as the normalization parameters. For example, the first coordinate must correspond to the first normalization parameter, etc. The values of the similarity measures should be written in the next line, which begins with the string "CF: ". Also, the lines must be numbered. When the values of the similarity measure are computed and the results correctly formatted, they should be submitted to the last step of the evaluation process where the properties are computed. What are the properties of the similarity measures? Properties of the similarity measures are computed in the following way:
Properties are explained in more detail in the paper. How to select the parameters of the third step? In the first step of the evaluation protocol researcher can set the number of lines N to a large number and then decide to compute the properties only for a certain number of them or compute the properties for different number of lines and then compare the results. Parameters number 2 and 3 are the indexes of the lines at which the evaluation should start and end. What is parameter number 4? When the computation of the parameters starts all points in the input file get normalized with the normalization factors written at the beginning of the file. The properties are then computed in the normalized parametrical space but they need to be presented in the nonnormalized parametrical space. For this reason one dimension is chosen as the reference dimension and the properties are multiplied with the chosen normalization parameter.
