The rise and fall of the Gans experiment

by

Brendan McKay
and friends

Summary

We summarize the efforts to independently replicate the "cities" experiment of Harold Gans that appeared to provide independent proof of the reality of the "Torah codes". As we will see, all attempts have failed to observe the same phenomenon, including one conducted by a joint team of codes supporters and skeptics.

The original experiment

In the famous Great Rabbis experiments of Witztum, Rips and Rosenberg (WRR), names and appellations of 66 famous rabbis were matched to the dates of their birth and death. Soon afterwards, a similar experiment was conducted by Mr Harold Gans, at that time an employee of the US National Security Agency. In the Gans experiment, the same names and appellations (with a minor caveat) were used, but the names of the communities of birth and death replaced the dates. Gans obtained a very strong positive result in favour of the codes.

The data for the Gans experiment (other than the names and appellations) is attributed to a chemist Mr. Zvi Inbal, at that time a lecturer on the codes for the organization Arachim and a friend of Doron Witztum. After the experiment had been run, either Witztum or Inbal provided Gans with some very complicated rules by which the data had allegedly been extracted from two encyclopedias.

Doubts about the quality of the data, especially as to whether it was produced in an objective fashion, followed soon after similar doubts had been raised concerning the names and appellations. Gans has strenuously defended the data against such attack. More recently he claims to have rechecked it all and found an even stronger result after a few errors were corrected. However, as of the last communication we had with him, he was still refusing to show us his new data.

Such arguments are best settled by replication, that is by independent repetition of the experiment.

First attempt at replication

The first attempt at replicating the experiment was designed by Barry Simon, Chair of Mathematics at Caltech. A description of that experiment is here. Simon's basic idea was to collect the data that actually appears in the encyclopedias, exactly as it appears without modification of any sort. The data was collected by some Israeli assistants who were paid for their services. The result was totally negative, without even a slight hint of codes.

A few years later, Doron Witztum found quite a few mistakes in the data of Simon's experiment, mostly due to the data collectors misunderstanding some of their instructions. Witztum of course claimed victory but omitted to mention that correcting the errors didn't make the result any better! (In fact, many of the "errors" claimed by Witztum are not errors at all. It doesn't matter because there is no sign of codes using either all of Witztum's "corrections" or just those which are valid.)

Second and third attempts at replication

The most serious attempts at replicating the Gans experiment were made by a committee at the Hebrew University of Jerusalem. The committee was formed in 1997 and comprised:

The committee designed two replications and engaged separate experts to compile the data for each. One of the experiments included many of the design decisions made in the Gans experiment, such as the choice of prefixes for the community names, while the other left such decisions to the discretion of the expert. The experts were chosen by Professor Furstenberg, who kept their identities secret even from the other committee members. Funding to pay the experts was provided by Mr. Alec Gindis and the Aish HaTorah organization (known as a leading opponent and leading supporters of the codes, respectively).

The experiments were finally completed in 2002. No sign of codes was detected in either case. One experiment obtained a result of 0.463 and the other 0.617. A positive result would have required a very small quantity, less than 0.001 and even that would have been about 1000 times weaker than Gans originally obtained.

The official report of the committee is here.

Some tentative analysis

It has already been suggested that the result of the committee experiments was due to inaccuracies in the data, although no specific examples have been brought to our attention. It should be obvious that values like 0.463 and 0.617 cannot be explained in that manner, but we did a brief experimental test of this claim. We took the original data of Gans and corrupted some fraction of it by permuting the letters of the locality names at random. We did this 10 times for each of a range of corruption percentages. The results are shown in the figure below, where the two purple lines indicate the results of the committee's two experiments when run in the Gans fashion. It is seen that an impossibly high error rate in the data is required, much greater than the amounts by which the committee's data differs from the Gans data.

Another experiment we did directly concerns the issue of how Gans' data got to be the way it is. If the positive result was due to some process that somehow biased the data towards those locality names that performed well in the experiment, then we ought to find that the data uniquely used by Gans performs very well and the data not used by Gans performs very poorly. The former is already expected from the results above, but the latter is not and is not a consequence of any codes hypothesis that we know of. To define the data for these tests, we took all the names of places of birth or death given to us by Professor Furstenberg. This included the two expert compilations and some extra names generated by Professor Furstenberg himself (spelling the names in one list in the manner of the other list). Then we ran the following in the Gans manner:

Gans data without Furstenberg data:   4 / million
Furstenberg data without Gans data:   996092 / million

Although the last result is not so high as to be impossible by chance, it is high enough to provide some support for the hypothesis that names performing worse than random were specifically rejected.

Conclusion

One can speculate about the reason for Gans's initial positive result, and some limited speculations are made above. However, the main lesson from this story is clear: Gans's alleged evidence for codes has failed the standard scientific requirement that it be replicable. In accordance with the rules by which science is conducted, its conclusions must be set aside.


Back to the Torah Codes page
Back to the Mathematical Miracles page