This is the test that hawkeye prediction patch did using a ball machine, that's the accuracy that is said when talk about, for heavens sake who knows how inaccurate it is in real conditions.
A detailed analysis of HawkEYE technology, cricket is in the middle of the report have copied a few excerpts here, but highly recommend checking it out to get the whole picture.
http://www2.geog.ucl.ac.uk/~mdisney/teaching/1006/papers/collins_hawkeye.pdf
You cannot be serious! Public understanding of technology with special reference to “Hawk-Eye”
There is a danger that Hawk-Eye as used could inadvertently cause naïve
viewers to overestimate the ability of technological devices to resolve disagreement among humans because measurement errors are not made salient.
For example, virtual reconstructions can easily be taken to show “exactly what
really happened.” Suggestions are made for how confidence levels might be
measured and represented and “health warnings” attached to reconstructions. A
general principle for the use of sports decision aids is put forward. A set of open
questions about Hawk-Eye is presented which, if answered, could help inform
discussions of its use and accuracy.
For a number of years after the introduction of Hawk-Eye, cricket commentators would simply remark on
what Hawk-Eye showed on the screen, giving the impression, perhaps inadvertently, that
the virtual reality represented exactly what would actually have happened had the pad not
been struck. This is where our analysis of Hawk-Eye begins.
A cricket ball is not uniformly spherical. Around its “equator” it has a raised seam and
the two “hemispheres” become more asymmetrical as the game goes on. The trajectory of the
ball after it hits the ground can vary enormously. The bounce depends on the speed, the hardness and texture of the ball—which changes during the game, the state of the ground at the
exact point of the bounce, the spin on the ball and the position of the seam. The “swing”—
which is the aerodynamically induced curve in the flight of the ball, which can be in any
plane—depends on the ball’s speed, its spin, its state, its orientation, the orientation of the
seam and the state of the atmosphere. As a result, what happens to the ball after it bounces is
not going to be fully predictable from its pre-bounce trajectory so that, as far as we can see,
Hawk-Eye has to estimate the post-bounce trajectory largely or entirely from the post-bounce
behavior of the ball {caveat 3} for which it can gather data between the bounce and impact
on the pad. This certainly seems to be the implication of the claim made by Paul Hawkins,
the Director of Hawk-Eye Innovations, in response to a criticism of Dennis Lillee, the
Australian fast bowler:
“… Hawk-Eye simply observes and then calculates the actual trajectory of the ball.
Whether the cause of this trajectory was due to atmospheric conditions, the wicket, or the
ball hitting the seam is irrelevant from a Hawk-Eye perspective. Hawk-Eye just tracks
what happened—it does not try to predict nor to answer why it happened.”
So, if the ball rears up unexpectedly after hitting the seam or a crack on the pitch, HawkEye will track the trajectory off the pitch to predict the future course of the ball.
Similarly, the tracking system will come into play if the ball shoots along the ground after
hitting a dry spot on the pitch.
Our concern in analysing what Hawk-Eye can do is to understand more fully what it means to
“track” and “predict” the path of the ball. Predictions are extrapolations and the accuracy of
these extrapolations is limited by, among other things, the quality of the data. No measurement
is ever exact. Heisenberg established this as a deep principle of physics with the “uncertainty
principle,” but here we are talking of macroscopic measuring processes such as are discussed
by, say Thomas Kuhn in his 1961 paper on measurement and, of course, by physicists and most
other scientists as a matter of ordinary fact in their day-to-day work. As a result, it is normal
in science to associate a measurement with an estimate of its potential error.
A decision is not a measurement. A decision is binary like the “guilty/not guilty” decision of an English jury; in cricket the batsman is either “OUT” or “NOT OUT.” The process
of what we will call “digitization” is used to turn inexact measurements into discrete decisions. In most sports, the referees or umpires are the people who do the digitization and
what we are discussing here could be described as technical aids to digitization. The bails
in cricket are one such aid to digitization. As discussed above, it can sometimes be difficult
to tell whether the ball has touched the wicket or not and the falling of a bail converts this
uncertainty into one of two discrete possibilities which have merely to be “read off” by the
umpire.
by paul hawkins
… Hawk-Eye has shown that balls pitched on roughly the same area on the wicket have
passed the stumps at widely varying heights. And in tests conducted, thousands of deliveries were bowled from a bowling machine and filmed by Hawk-Eye. The camera feeds
were cut about two metres from the stumps, approximately the point where the batsman
would normally intercept the ball. When the ball hit the wicket, Hawk-Eye was able to
determine, to within about 5 mm, the point of impact.
… “Hawk-Eye requires between 1 to 2 feet of travel after the ball has pitched to be able
to accurately track the ball out of the bounce (this is significantly less than an umpire
requires). In instances when this does not happen, a Hawk-Eye replay is not offered to
TV.” (18 December 2003)
19
… in most cases Hawk-Eye’s output is accurate to within five millimetres in predicting
the path of the ball. The accuracy levels are highest when the ball has traveled a fair distance after pitching, but even when the point of contact is very close to the pitch of the
ball, the accuracy levels are still within 20mm
Sport has changed as a result of television replays whether one likes it or not. For
example, at least some television viewers find that soccer is being spoiled for them by the
number of blatantly incorrect refereeing decisions visible on television replays. Sports decision aids, including television replays (as in rugby union), have a valuable role to play in
undoing some of this damage. But the exact way all these things are used depends on a prior
302 Public Understanding of Science 17(3)
understanding of the relationship between what these devices can do and the way normal
human judgment works. Automated sports decision aids, if their capabilities were presented
in a transparent way, could add still more to the enjoyment of sport and, in addition, to a better
public understanding of the limits and possibilities of technology. If the Automated Decision
Principle were followed, and with the increasing speed of computation potentially making
automated decision devices more accurate and more capable of analysing and presenting, in
real time, the magnitude of uncertainty associated with any reconstruction, the future for the
technology as an aid to human judgment, seems bright.
