Techniques of measurement and computation prior to geometry are often called mathematics (e.g. Egyptian or Babylonian mathematics), but doing so misses the element of systematic proof characteristic of mathematics. From proof comes mathematics' distinctive property of certainty.

The ancient Greeks called Thales their first philosopher as well as mathematician. And as he and his followers were concerned with natural philosophy--indeed they created the concept of nature--he was also the first physicist. Ancient Greek tradition therefore suggests that mathematics, philosophy and physics had a common origin, and even originator.

How much historical truth is represented by this ancient tradition can never be determined with certainty, but in any case, it is not our primary concern. Like many such traditions, it represents another, deeper, truth: these three intellectual disciplines each attempt, through the systematic use of logic, to reduce areas of knowledge to axioms, and thereby, to certainty. The materials with which each has to work, and therefore the quality of success of each, differ greatly. As their differences are discussed, their underlying similarities should be remembered.

Even religion, on a strictly intellectual level, shares basic similarities in the following sense. The portion of the Cosmos that is, or that we hope is derivable from/explainable in terms of basic laws/truths/axioms, is what we call nature. To the extent that humans are part of that portion, they are part of nature; to the extent that they are not, they are supernatural. But since, by definition, the axioms upon which nature is based are unexplainable, they are supernatural. If you believe that they must come from "somewhere" it must be outside nature. That somewhere is the Hebrew definition of God that is a cornerstone of Western Civilization.

A simple illustration of the structure of an algebraic proof shown in Figure 1.

In general, as in this example, anyone can check the correctness of the lines of inference leading to each statement. Anyone can do this because even a simple electronic circuit can and any reasonably functioning brain can perform as well as a circuit.

The ability to record and then check the smallest detail of each of many logical steps is necessary to the existence of mathematics. It guarantees that each theorem is as certain as the axioms upon which it is based, and it enables the growth of vast chains of logic, which suffer no decrease in certainty along them. This in turn requires that the total number of axioms defining a branch of mathematics be not too large.

If these things are true, the following steps can be taken in a finite amount of time: First, every axiom can be identified, examined, clarified and agreed upon as being true by a community of rational people. Then the set of axioms can be checked for self-consistency. This ensures that it is impossible that one proof be found proving a statement true and another be found proving it false.

A theorem is not proved until every logical step in it is verified. Once this is done, it is, by definition of rationality, impossible for a rational person to say that the theorem is less certainly true than any axiom supporting it. Thus, a rational person who doubts a theorem, must doubt the truth of an axiom upon which it is based. This is important to emphasize because of a widespread tendency to reject a scientific deduction, which though solidly based on solid axioms, may violate common sense or prejudice. It is a tendency especially found in people who take pride in a sense of rationality that they have confused with common sense.

Much of this confusion is traceable to lack of experience. Everyone makes plausible inferences, but few carry through rigorous mathematical proofs. As a consequence, they may have a verbal understanding of the difference between plausibility and proof, but not an experiential one. This is an invaluable lesson once was (but apparently no longer is) imparted, in a general liberal education, by the study of Euclidean geometry.

The discovery of mathematics took a long time; stone tables of numbers giving instances of the Pythagorean theorem, a²+b²=c² were compiled several centuries before the theorem was proven. People saw the same relation between the sides of right triangles being repeated in instance after instance. They must have known each was an instance of a general result, something always true, but could not figure out how to prove it.

Why was it important to have a proof? Did the repetition not make the general truth sufficiently obvious? If your belief in the theorem were based on large tables of instances, could you be certain that the theorem was exact?

If your belief was based on the theorem, you could be certain, and this has enormous consequences. When the theorem was applied to the case a=1, b=1, it gave a result for c which, the Pythagoreans proved, did not exist–at least within the concept of number as was then understood (that concept being confined to the rational numbers). They proved that there existed no (rational) number the square of which was 2. And yet, clearly, there appeared to be lengths that corresponded to the square root of 2!

A conceptual framework for understanding this did not appear until the late nineteenth century; it took more than two millennia. And even today a residual related question remains unsolved: the consistency between the concept of the continuity, as it applies to lengths and to the real numbers.

At a less technical but more philosophical level, another question can be asked. Does something exist if it cannot be produced, even in principle? For example, the square root of two is not rational, and we can only write out rationals; therefore, it is literally impossible to produce it–to write it down. Neither can this be done, even in principle; we only have an algorithm (for finding square roots) that allows us to approach it, ever more closely. But are we approaching something that is?

The answer is not as trivial as it may first appear. It is easy to say, for example, we can define it to exist as a limit. But can we define what exists? Can I define God to exist as the limiting ‘First Cause’? Obviously, there is disagreement concerning the powers of definition, and more generally, the meaning of ‘to exist’.

We shall see that mathematics is the but simplest context in which the deep implications of reduction manifest themselves.

Another characteristic of mathematics is that a few axioms lead to an endless number of theorems; equivalently, mathematics reduces a endless number of truths to a relatively few basic truths. Only the physical sciences is similar in this respect, and that is because the truths of physics are also mathematical in form.

The source of this infinite reducing power is the endless number of mathematical structures that can be created from the basic elements of any branch of mathematics: each structure is related to others, and the statement of a relation is a theorem.

The structures of plane Geometry are the figures that can be composed by ruler and compass on a plane: figures made up of straight lines and circular arcs.

The structures of the branch of mathematics called the Theory of Numbers are subsets of the natural integers. For example, there are the even, the odd, the prime, the perfect,…numbers, also particular numbers, particular pairs of numbers, and so on. Properties of these subsets as well as relations between them all give rise to theorems.

The natural numbers form the simplest infinitely large system, and yet, it is infinitely rich in complexity and mystery. Historically, its puzzles have intrigued the greatest mathematicians. The Pythagoreans believed that the structures within it were key to the structure of nature, a belief long derided as mere mysticism. But amongst mystics are found those whose intuitions are so far ahead of their times and abilities that they can do no more than sound the way they do. And in fact there is a current turn in physics towards integers--towards basing theory on the concept of bits of information--that may eventually lead us back to the Pythagorean idea.

Planar structures can be interconnected by Geometry, subsets of the integers, by the Theory of Numbers, only because in each case the objects are composed of homogeneous elements simply related to one another. All planar structures of Geometry are composed of points, straight line segments, and circular arcs. All subset of integers are composed of integers. This is true of every branch of mathematics.

Relations between structures are possible because they are made up of similar elements. It is therefore not surprising that such elements are often call 'atoms' and the structures composed of them, 'molecules', and even the Pythagoreans used a similar analogy in the context of ancient Greek atomism.

Figure 2 is a stylized picture of uniform pulses, each with time width t, in a digital channel. Such a train of pulses can be interpreted as representing a number in the binary, or base-2, system. Each pulse represents a value 1, whereas each gap in the pulse train represents a value 0.

The channel shown communicates one binary digit, or bit, either a 1 or a 0, in time t. Its capacity is defined to be 1/t bits. A neuron cannot fire its action potential faster than about 1000 times a second. Taking t=.001sec, its capacity, measured in bps=bits per second, is approximately

C =1 bit/(0.001sec) = 1000 bps.

Information in any form can be converted into the form of bits through a variety of techniques, of which one is always coding. Figure 3 illustrates a simple example of the coding of a digitized signal (digitization will be discussed separately).

In Figure 3Figure digitization produces 4 rather than the 2 amplitude levels of the simple train of pulses of Figure 2. Labeling the levels 0,1,2,3,–the digits of the base-4 system–the signal shown takes on the values 110122333322210. Suppose this signal needs to be sent somewhere using a two-level channel like that in Figure 2. One simple way is by first coding the 4 levels in binary notation:

The table shows how 4 symbols can be represented by pairs of bits. Doing this, the original message

1,1,0,1,2,2,3,3,3,3,2,2,2,1,0

(commas are put in to help the eye) becomes a string of bits

01,01,00,01,10,10,11,11,11,11,10,10,10,01,00.

In this form, the message in Figure 3 can be sent as a simple train of binary digits. We see thatbase-2 and base-4 messages can be re-created from the other; they are equivalent representations of the same thing.

In a time T, a channel with capacity C can transmit the equivalent of CT bits; the time it takes do deliver some number of digits varies inversely with capacity. Channel capacity applies to and can be used to rate the delivery power of any communications device.

Figure 3 also illustrates digitization, the transformation of continuous into discrete forms of data. Pulses of 4 discrete heights (0,1,2,3) are used to approximate and represent a continuous curve. What allows this to be done? Clearly, it is only possible it provides an approximation to the original curve that is sufficient for whatever purpose is at hand: the approximation must retain relevant, and should discard irrelevant data.

One way to decide if the digitization of a signal provides a sufficiently accurate approximation, is to have a definite encoding and decoding algorithm in hand. The encoder produces the digitized signal from the original; the decoder produces a continuous curve from the digitized signal. There are any number of mathematical algorithms that will ensure that original and final curves do not differ from one another in relevant details in all circumstances to be encountered.

It is possible to look at ever finer details of a continuous curve. The more details that are relevant, the more refined a digitization scheme is needed. If all details of continuous curves were relevant to a task, digitization would require an infinite number of levels and would consequently become impossible.

It will soon become apparent (if not already so) that a fundamental assumption of perception must be that this is never the case. To perceive anything is, first of all, to assume that all but a finite number of details about it are relevant, and to discard, in the process of perception, as many of the irrelevant details as possible. Thus, in effect, digitization performs reduction by discarding almost all of the infinite amount of irrelevant information inherent in continuous data.

The human visual system reduces the data input to it by a factor of about ten million.

A rough diagram of the human visual system which is the neuronal circuitry connecting retinal receptors to the higher cortex is indicated in Fig.4. It shows its order of magnitude of data reduction. Information at the rate of about one billion bits per second leaves the rods and cones at the surface of the retina, but the higher cortex can only handle about 100 bits per second. Thus the information flow into our visual system is about ten million times greater than the higher cortex can handle.

A template, a standardized form, is usually used to reduce the labor of production: to produce a straight line we use a straight edge, for business letters, word processing software supply templates. Far less well known, and the source of great insight into human behavior, is the biological and psychological use of templates to reduce the labor of information processing throughout during its input as well as output.

In fact, labor reduction is an understatement; without the use of templates, no sensory perception at all would be possible. Digitization forces all input information into an either/or form, thereby reducing an effectively infinite to a finite number of possibilities. Thus, in Figure 3, the all the values of a continuous curve are forced into either this or that value.

As we shall see, templates used in digitization, composed of their various sets of discrete levels, are but the simplest and most fundamental templates used to process sensory input. There are, for example, templates that force images to have (or at least appear to have) continuous existence, boundaries, motion, surfaces, and so on. Further up the processing chain, we use learned templates to identify images as mother, friends, and so on. And further yet, as a device to reduce the labor of thought we force ideas into conceptual templates that correspond to words, and complexes of words that extend to theories and whole world-views.

Before discussing reduction in biology and psychology, a simpler example taken from modern data communication technology will be discussed. Figure 5 illustrates how a simple black and white image (no grey) on computer and television monitors might be painted, one spot, or pixel, at a time, by electron beams on their fluorescent screens. As an electron beam sweeps across the inner face of a screen, it is turned on or off at fixed intervals. When on, the electrons hit the screen causing the fluorescent material painted on it to glow thereby producing a bright spot. A dark spot appears when it is off. Shown in figure 5 is the top row of the upper pattern in Figure 6. The electron gun scans horizontally along each row and consecutively from the top to bottom row.

The world is such that we are seldom interested in transmitting random patterns. More common are images composed of linear segments such as that shown in the lower half of Figure 6. These are not random; correlations between bit values are apparent. Expressed in the language of correlations between neighboring bits: 1 has a tendency to be preceded and succeeded by a 1; 0, by a 0. Thus, the bit string corresponding to the lower image starts off as 0011111111110001111111111000....,. Its runs of consecutive 0’s and 1’s reflect the order apparent in the original image.

Pictures of this type can be coded in fewer bits than 1 per pixel. Figure 8 shows how a commonly used type of code does this. It specifies, in binary notation, the number of bits in alternating runs of 0’s and 1’s using a fixed 4 bit code. This code uses 96 bits to specify the lower image instead of the 144 bits required by giving each bit individually– about a 30% reduction in code length.

Obviously, the string of coded bits is meaningless without the proper circuitry receiving it: the circuitry necessary to read each 4 bit sequence, keep track of whether it refers to on or off bits, and transform that information into a sequence of 0’s or 1’s. This circuitry is another form of template: it produces bits in a standardized form, here, long sequences of bits of one or another type from (generally) far fewer input bits. It can be compared to a straight edge which produces long lines from two pieces of information that correspond (in graphs) to the intercept and slope of the line.

Such a template, existing in a receiver’s circuitry, is analogous to knowledge existing in a brain. In either case, knowledge is not information but that which allows information to be properly interpreted and used. In the case of image processing, this means the reproduction of the original image. In the case of the brain, knowledge, interpretation, and meaning are much deeper issues, but at least one philosophical view–pragmatism--would maintain that the proof of knowledge is similar to that of the receiver: the use that is made of information as manifest in resultant actions (mental but ultimately physical).

The string of coded bits is also produced by a circuit which analyzes the original pixel pattern, counting up similar consecutive and translating the number into four bits of binary notation. As we shall see, this is the use of templates most readily apparent in the first stages of the sensory system where input signals are classified according to which template pattern they best fit. These templates represent knowledge of what to expect in terms of input. Do we expect a world of linear images? Such expectations in humans, at the most basic level, are inborn; they are carried by genetic codes which predispose nerve growth in the sensory system.

Think of a pen on a plotting machine instead of an electron beam on a monitor’s screen. The machine is told a point to move its pen, horizontally on a piece of paper, by giving the point’s coordinates. It is constructed to move only along a straight line between where it is and the next point it is given. It is also told whether the pen is up (not touching the paper) or down (touching and therefore drawing) while moving, and is told how wide to draw the line.

In Figure 9, the paper has 12 horizontal points, labelled 0,…,11 starting from the left, and an equal number of vertical points starting from the top. These coordinates require at least 4 bits to be expressed in binary. The coordinates (0,0) (decimal) at the upper left are written in binary as (0000,0000), and the lower right coordinates of the line, at (11,11) (decimal) are (1011,1011) (binary).

To code line width, 3 bits might be used as follows. Line width zero, means no line at all; the pen is up–not touching the paper. This is coded as 000 (binary). The thinnest line is coded as 001 (binary), and the thickest as 111(binary). Supposing that line width means pixel width, and noting that the line in the figure is about 2 pixels wide measured perpendicularly to its direction (except where cut off by the page), it would be coded as 010 (binary).

We can now construct the set of information bits to be sent to the plotter. It is a sequence of instructions of the form: (line width, destination point). To draw the line shown in the figure requires two instructions:

(1) (width 0, go to (0,0)) ® (000,0000,0000).

(2) (width 2, go to (11,11))® (010,1011,1011).

The pen will first go, making no line, from wherever it is, to the upper left point on the page. It will then go, while drawing a 2 pixel width line, straight to the lower right corner. The commas and brackets in the instruction are just aids to the eye. The machine simply gets the 2´ 11=22 bits 0000000000001010111011 which it is hard wired to convert into its actions.

Using such a code, the 144 original bits needed to specify the lower line pixel by pixel as been reduced to 22. This is (22/144)=0.15 bits per pixel.

A real screen might have about 50,000 pixels. On account of the increased screen size, the same line would require only a few more than 22 bits to specify. Suppose that it took 25 bits and that a picture had an average of 100 lines. This means that each image would average 2,500 bits to encode, 5% of 50,000 bits. For line drawing of this general type, this suggests that each pixel provides .05 bits of information.

Differential templates are essential for natural data.

There are no natural images containing uniform straight lines like that in Figure 9. The line, straight or otherwise, of zero or uniform width, is an ideal, artificial construct biologically hard-wired into us via templates in the visual system. Realizing this, first consciously and then in code, led to higher efficiency in artificial communication systems..

The image in Figure 9 is more natural than previous ones in that it exhibits effects of nature’s characteristic continuity. A finely enough grained image of nature appears continuous at its boundaries: darkness shades into light, one color into another, perfect straightness shades into irregularities.

To capture shades of brightness, more than 2 brightness levels are needed. Figure 9 four brightness levels, labelled 0,1,2,3. Those of the first row are all zero; those of the second row are, 001110000000; and so on. A blow-up of pixels and their brightness levels in one region illustrate how the difference between the j^th and j-1^th pixel brightness is then coded in the position of the j^th pixel. No difference is greater than one unit in magnitude. 3f differences were greater, = scheme would have had to be created 8 bright more than 4 brightness levels.

0,1,2,3(decimal) requires 2 bits to be expressed in binary as 00,01,10,11 (binary) respectively. Without a difference code, each pixel requires two bits if to transmit 4 levels. It is useful to note that a formula for this number of bits per pixel is log₂4=2. If 2³=8 brightness levels had been necessary, then log₂8=3 bits could be used to express them, and so on.

Using the differential code only +1, 0, -1 need be sent. Three values can be coded by 2 binary numbers; we could code —1 as 0 and then, in binary, similarly 0 coded as 1 and then in binary, as 01, and finally changing +1 to 2 which in binary is 10. But in this scheme, the bit combination, 11, is wasted and there is no advantage in using differences for 4 brightness levels, although there is, for 8 and higher.

An much better procedure can be used with a difference code for pixels having any number of brightness levels. By using an important technique called block coding, the average number of bits needed to transmit information turns out to be log₂3=1.58 bits per pixel.

Differential strategies are regularly used in all modern systems of image storage and transmission. They are of great economic importance. When a phone company halves its code length it doubles the number of messages it can send without doubling its investment in construction and maintenance of lines. When a computer owner gets image compression software it may save the cost of another hard disk, or thousands of them for large companies. These are applications already of importance. Their potential importance of world-wide image communications is first emerging.

Figure 4 show the structure of the retina where the first stages of data reduction take place. It shows a network of cells that act as a biologically hard wired electro-chemical circuits. They are implementations of templates, that are derived, presumably, from evolutionary experience. These templates express biologically incarnate knowledge–in-built knowledge of which we are unconscious.

One piece of this knowledge is that the world is generally made up of discrete objects the light from which changes rapidly mainly at their edges. These determine where things are, and this is important to know–more important than knowing about the relatively slight gradations in the light that typically come from the objects surfaces. The visual system is therefore set up to devote its channel capacity to edge information. The biological solution to this problem is a form of difference coding.

Retinal images rarely cause large differences in the light hitting neighboring photoreceptors. Large differences occur at edges. Visual systems utilize this by the way they combine the output signals of large numbers of photoreceptors within roughly circular areas on the retina. The signal from each area goes to a single (retinal ganglion) cell. That area is called that cell's receptive field. These retinal ganglion cells constitute the optic nerve that leads from the retina to the portions of the visual system where further signal processing takes place.

An on-center ganglion’s firing rate increases with the area of illumination of the central on region and decreases with the area of illumination of the surrounding off region.

The response for four extreme examples is illustrated in Figure 12. At the top, the on-center region is illuminated and the inhibitory region is not. This produces the most rapid ganglion firing rate during the on-off time interval.

Next, below the top, both fields are illuminated and the response is reduced because signals from neurons in the inhibitory surround field reduce the firing rate that would result from the on-center neurons alone.

Third from the top, no light reaches any of the receptors within the total receptive field of the ganglion.

When only the inhibitory surround field is illuminated, as shown at the bottom of the figure, the response is cut off entirely.

Firing rates for a completely illuminated and for a completely dark receptive field (the two middle states of illumination in the figure) do not differ by much on the scale set by the maximum firing rate. That is, receptive fields are relatively insensitive to the average light hitting them. This helps explain the eye’s ability to easily adjust to different average light levels. Because of the visual system’s strategy, the eye can detect as little as a 2% difference in brightness between a small point of light and a uniform background projected on a screen; and it can do the same thing even when the background brightness ranges in brightness by a factor of five.

If the eye had not followed this strategy, but had instead detected average levels of light, its sensitivity to relative light levels would have necessarily been less. For each ganglion has a fixed channel capacity. To the extent it is used to transmit information about average light levels, it is unavailable to transmit other information, e.g. about relative light levels. The greater the precision with which anything is described, the more bits of information needed. For example, to report an intensity level of 0.193 (in some units) requires 4 digits. Suppose we discard the last digit as being irrelevant, and report the intensity as 0.19. Less digits are used and the precision is less.

Retinal circuitry is configured to discard data about precise average light levels in favor of data about relative light levels. In the context of evolutionary survival the discarded data was deemed to be relatively irrelevant. Picking out shapes defined by boundaries at which light changed rapidly was more relevant to survival than knowing precise light levels.

This first stage of data reduction by the visual system illustrates the two characteristic methods of reduction, found in both unconscious biological and in conscious scientific data processing. One is the elimination of irrelevant data an example of which has been unnecessary precision in information about average light intensity.

The second method is the elimination of redundant information. Data is made redundant primarily by knowledge. This knowledge allows the system to supply by itself–to predict–some data in terms of other data. Conscious knowledge can be in a number of forms, ranging from physical theories to simple connections between ideas created from direct experience. Unconscious knowledge is in the form of templates that are physically expressed in neuronal circuitry.

The visual system assumes that the world is generally made up of discrete objects that can be identified most effectively by their shapes. Its first job is to locate and trace these boundaries. Then it guesses that the color between boundaries are extensions of the color sensed at boundaries. It fills in the color by itself; the brain perceives the filled in color.

It also spot checks these assumptions by occasionally focusing the eye at a few points between boundaries, although it can never gather information from all points and never attempts to do so. It neither can nor does gather such information at all times; it simply assumes that the color is slowly changing in time. The information it does not gather is deemed to be redundant on the basis of the visual system’s ‘theory’ that time and motion are continuous.

The output of oriented linear arrays of on-center or off-center fields are each combined and input to certain other neurons in the visual cortex. The latter fire when a line segment is imaged on the array feeding them. Together, the cells in an array and the neuron it feeds, constitute a circuit that therefore ‘detects’ a line segment imaged at some point and orientation on the retina.

Each such segment is subjectively perceived as being smooth and continuous, but of course not only is this continuity a mental fabrication, it is physically impossible. Material lines all have irregularities of at least micro-crystalline size. As long as the image of an irregularity is smaller than some length determined by construction of the array detecting it, it is ignored. In effect, reality is forced into the template defined by the array.

Insofar as what happens in the brain, the perception of a line segment indicates only that a particular kind of neuron has fired, and ultimately something similar seems to be true of all perceptions. There is an ancient, but naïve, seeing something, means that a corresponding image is created somewhere in the brain. Not only does this not describe what exists in the brain, but we must also wonder about the sense in which what is perceived ‘really exists’ outside the brain. Continuity is perceived, but is it ever out there? And if not, what about all of sensual reality?