The Newcomb-Benford Law

Simon Newcomb (12 Mar 1835 – 11 July 1909), a self-taught Canadian-American mathematical astronomer, was a man of many diverse accomplishments (though a rather unpleasant individual in some ways) but nowadays principally remembered for noticing (at some point in 1881) that the later pages of his university library copy of the logarithm tables were progressively less well-thumbed than the earlier ones.

The tabulation was evidently to a good many significant figures, requiring a good many pages, of course.

The pages at the beginning of the book, for those numbers starting with a 1, were the most worn, while the pages at the end of the book, for those numbers starting with a 9, were the least worn. The pages in between, for intermediate numbers starting with digits between 2 and 8, were of steadily decreasing dilapidation.

What could account for this, he wondered?

In academic robe, 1905

We have to be rather cautious at this point...

Logarithms are typically displayed for the numbers (to so many significant figures) between zero and unity and are purely mathematical in origin. They are not at issue, and aren’t subject to the Benford effect. If you don’t believe me, please see the Frank Castle four-figure log table.

Though it was logarithms that had caught his eye, it was the table of antilogarithms, which represented the real-world data that the logarithms had processed, that were indeed subject to the effect. Please see the corresponding Frank Castle four-figure antilog table for visual confirmation and (further down) a tabulation of my own tally.

Please click here for the log table, here for the antilog table,
and here for the nitty-gritty of using them (no gain without pain).

Indeed, he devised an inspired formula to quantify the relative frequencies of the initial digits of the numbers in the antilogarithm table. It gave the probability of a real-world number beginning with the digit d as being log (d + 1) – log (d).

This can be graphically displayed as per the ‘borrowed’ barchart below

As also by the next such, in which Frank Benford’s hijacking of the idea in 1938 is clearly evident.

Stigler’s law of eponomy is validated once again!

Perhaps of interest is my own tally of the digits as per the Frank Castle antilog table:

(I've repeated this exercise using the Frank Castle 5 figure antilog table and the only difference is that 301->302 and 177->176)

Newcomb was a busy man, and this was just an interesting oddity, so after dashing off a note to The American Journal of Mathematics, he probably forgot all about it.

Frank Benford

But waiting round the corner with a sockful of sand (as P G Wodehouse would have put it) was the Efficient Baxter as personified by Frank Benford, whose tireless researches into countless numerical tabulations of almost every aspect of humanly recorded data have corroborated Newcomb’s formula (subject to certain T & C’s, of course).

Benford seems to have quoted Newcomb’s formula as his own (calling it the “The Law of Anomalous Numbers” in his publication Proc Am Phil Soc vol 78 (1938) p551, and saying that the law “evidently goes deeper among the roots of primal causes than our number system unaided can explain”) but made no effort to justify it theoretically. And neither shall I, as statistics are my Achilles heel.

But, nevertheless, I’d like to put it under the microscope briefly. Another way to express the formula is simply log (d + 1) – log (d) = log (1 + 1/d), and as tabulated below.

Dropping down to one place of decimals it's also neat to note that

It is said that the law (or perhaps one should say tendency) is invariant to change of logarithmic base (whether binary or decimal or duodecimal, for example) and is also independent of the units of measurement (whether, for example, gallons per mile or litres per kilometre). In fact, it’s an inexhaustible topic for mathematical mining for generations to come.

It has also been very effectively weaponised in the investigation of bogus financial data, as best described (along with much else relating to its fundamentals) by Alex Bellos in chapter 2 of “Alex through the looking glass”, Bloomsbury 2015.