The stereoscopic triangulation of a three dimensional coordinate from two two dimensional images from two different viewpoints is achieved using an estimation technique for the near intersection of two rays. In order to achieve this, certain intrinsic properties of the viewpoints are needed:

• Horizontal Field of View
• Vertical Field of View
• Limits of x coordinates
• Limits of y coordinates

As well as information about the layout of the Wii Remotes and the detected infra-red sources:

• The relative positions and orientations of the two cameras
• The two dimensional coordinates of a point from each viewpoint

Using the fields of view, a vector from the camera position to the detected point can be obtained. These vectors can be thought of as rays from the infra-red source to the camera. Any detected point on a ray will report the same two dimensional coordinates.

Using a bit of linear algebra, the information from the two cameras can be used to make an estimation of the intersection of the lines. This is achieved by finding the point that minimizes the distance between that point and the two rays:

Minimization of Errors by Minimization of the Distance

The points that are reported by the two Wii Remotes are not necessarily sorted in the same order:

Any two points can be used to estimate a three dimensional point. Therefore, it is necessary to find pairs of points which would most likely yield an intersection. This is achieved by minimizing the distances. A matrix of prospective three dimensional points is generated by taking a pairwise permutation of the two dimensional points and using stereoscopic triangulation. The most likely points are chosen as the final points.

Minimization of Errors by Predictive Interpolation

The predictive abilities of the tracking algorithm depend on two important factors:

1. The instantaneous velocity - the vector difference between the current point and the previous point
2. The average velocity - the average of the instantaneous velocities over a period of time

A prediction is made by taking a weighted average of the two velocities and adding this vector to the last point. This prediction is then compared to the observed values and the most likely points are chosen as the final points.

Polling and Asynchronous Events

A slightly modified version of the asynchronous model described Asynchronous Methods was used in the point tracker. At a set interval, the point tracker asynchronously examines the state of the two Wii Remotes and solves for up to four three dimensional points, which are subsequently added to the point buffer. The point buffer is then examined by the gesture recognizer for any recognizable patterns.

Point Tracking Ability

The point tracker implemented, although not perfect, was sufficient for the gesture recognition system. The tracking methods used would fail in some cases, for example, when a user had two infra-red sources merge and upon reaching the edge of one of the Wii Remote's viewports, only one leaves and one stays. In this case, the system thinks that there are still two of the pointers when there should only be one. Any problems that the tracker encountered were easily dealt with by removing all of the points from both of the viewports (i.e. hiding the infra-red sources), and move the points back into the volume of gesture recognition.

<< Back to Project