INTRODUCTORY NOTE TO
THE LIMITING INFORMATION CAPACITY OF A NEURONAL LINK
Donald M. MacKay
##
I first met Warren McCulloch in 1949 at King's College, London, where I set up a seminar at which McCulloch tried out his Logical Calculus of the Ideas Immanent in Nervous Activity on a fascinated gathering of physicists, communication and control engineers and physiologists. I had just drafted an exploratory paper on The Combination of Digital and Analogical Techniques in the Design of Analytical Engines, in which I had ventured to argue that purely digital encoding of information could not permit the most general possible transformation of information, for which I suggested that variable threshold logic elements would be desirable. McCulloch expressed generous enthusiasm for this heterodox effort, a copy of which he took back to Chicago to duplicate for distribution (I never knew to whom!)
In 1951 I came to the United States as a Rockefeller Fellow, and was made welcome in McCulloch's basement laboratory at Illinois Neuropsychiatric Institute in Chicago. The fashion for comparing the nervous system with a digital computer was still in the ascendant, and people were inclined to justify it by alleging that digital encoding of information was necessary if a synapse was to transmit with maximum informational efficiency. McCulloch and I fell to arguing amicably about this - he at that time hopeful of binary logic as a model of neural interaction, and I excited by the possibilities of combining analog and digital principles in order to increase the flexibility of the representation and processing of information in the brain.
It was agreed that he should track down some relevant data for a typical synaptic link and I should compute the maximum rate (in bits per second) at which an engineer could in principle use a synapse to transmit information in various modes. I wrote the paper and McCulloch characteristically wanted me to take sole credit for it; but I persuaded him to join as coauthor on the grounds that he had both provoked the question and provided the physiological data to answer it, and that in any case the point might be better taken by digital modellers if it came with his explicit blessing.
As we were careful to emphasize, the exercise was meant simply to give a feeling for the relevant orders of magnitude, and not at all to advance specific models of the processes actually used in the central nervous system. It showed clearly that informational efficiency did not demand binary digital encoding of synaptic signals; but we left open the question whether, and if so in what ways, the time domain was actually used in cerebral information processing. This is still an open question, likely to become increasingly important as multi-electrode recording brings relevant data within reach.
* * *
The paper contains one misstatement, which was my fault. It is only in an interval modulation system using a constant clock rate (pulse-position modulation) that all intervals should ideally be used with equal frequency. With the most general form of pulse-interval modulation it would be advantageous to use shorter intervals somewhat more frequently than longer. The effect of this on our Figure 3, however, would be relatively small. As calculated by A.M. Andrew (Communication Theory, edited by Willis Jackson, Butterworths, London. 1952, p. 100), corresponding values of CT/CV and TR/ΔT (Figure 3) for the optimal pulse interval distribution are:
These slightly higher limiting values of CT/CB only reinforce our general conclusions.
Postscript to The Limiting Information Capacity of a Neuronal Link
The line of thought sketched here is developed a little further in:
MacKay, D.M.: Self-Organization in the Time Domain, Self-Organizing Systems 1962. Yovits, Jacobi and Goldstein, eds., Spartan Books, 37-48,1962.
For further research:
Wordcloud: Analog, Argue, Binary, Capacity, Chicago, Combining, Communication, Compute, Data, Digital, Distribution, Domain, Efficiency, Effort, Encoding, Engines, Enthusiasm, Exercise, Figure, General, Information, Interval, Limiting, Link, Logic, London, Mackay, Maximum, McCulloch, Model, Modulation, Nervous, Neuronal, Open, Order, Paper, Possible, Principle, Processing, Question, Rate, Relevant, Self-Organizing, Synapse, Synaptic, System, Transmit, Used, Value
Keywords: Link, Paper, Digital, Logic, Mackay, Information, Activity, Effort, College
Google Books: http://asclinks.live/e6bd
Google Scholar: http://asclinks.live/f9fz
Jstor: http://asclinks.live/5txi