Computationally Inequivalent Summations and Their Parenthetic Forms
aa r X i v : . [ c s . D M ] M a y Computationally Inequivalent Summationsand Their Parenthetic Forms
Laura Monroe and Vanessa JobUltrascale Systems Research CenterLos Alamos National LaboratoryLos Alamos, NM 87501USA [email protected]@lanl.gov
Abstract
Floating-point addition on a finite-precision machine is not associative, so not allmathematically equivalent summations are computationally equivalent. Making thisassumption can lead to numerical error in computations. Proper ordering and paren-thesizing is a low-overhead way of mitigating such error in a floating point summation.Ordered and parenthesized summations fall into equivalence classes. We describethese classes, and the parenthetic forms summations in these classes take. We providesummation-related interpretations for sequences known in other contexts, and give newrecursive and closed formulas for sequences not previously related to summation.We also introduce a data structure that facilitates understanding of these objects,and use it to consider certain forms of summation used by default in widely usedcomputer languages. Finally, we relate this data structure to other mathematicalconstructs from the fields of mathematical analysis and algorithmic analysis.
Two summations are mathematically equivalent if they are comprised of the same summands,under any parenthesization and any ordering, since addition is associative and commutative.This is not true for floating-point addition on computers of finite precision [1, 2].The assumption that mathematically equivalent summations are also computationallyequivalent can lead to calculation error, and affect the accuracy of the result to the detrimentof the overall calculation, since error in one step can propagate and influence the entire run.To complicate matters, there can easily be a very large number of non-equivalent summations,so it is not feasible to simply examine all possible cases.In this paper, we look at the structure of computationally equivalent forms of floating-point summations on n term. We enumerate the non-equivalent groupings and orderings,1nd discuss in detail some cases that are of interest because of their high degree of accuracyor because they are default in commonly used programming languages.These structures and this kind of analysis appear in a range of disparate fields. Forexample, the form and ordering of trees with similar structure is important in mathemat-ical phylogenetics [3]. There is also a connection between the summation-based structuresdiscussed here, the Karatsuba recursion tree [4, 5, 6], and the Tagaki function [7, 8]. Two summations are computationally equivalent if they differ only by some series of pairwisetranspositions. Two computationally equivalent summations will always have the same valueon any system following IEEE 754 [9] because the commutativity of addition is guaranteedunder this standard. However, associativity is not guaranteed and should not be assumedon a finite-precision machine.The equivalence relation in use throughout this paper is accordingly that imposed byIEEE 754: pairwise commutativity. All isomorphisms we discuss in this paper are in termsof this relation. When we call two summations equivalent in this paper, we mean compu-tationally equivalent on a IEEE-754-compliant finite-precision machine. We do not meanmathematically equivalent, which of course does assume associativity.Adding the same summands with different groupings and orderings that are computa-tionally inequivalent can produce different results in practice, and can induce rounding error[1, 2]. This is discussed in detail in many references going back decades.As an example of computationally inequivalent summations, consider a summation prob-lem with one very large number and many very small numbers. Two groupings can lead todifferent results: • When adding a large floating-point number to a small one, the sum may exceed theprecision of the computer, and the significant digits of the small number are lost. • On the other hand, if many very small floats are added together first before addingthe large number, the resulting sum of small elements may add to the end sum, eventhough some of the significant digits of the sum of small elements are still lost.
In this paper, we use the terms parenthesization and grouping interchangeably.Two summations that are computationally equivalent must have equivalent parentheticform. However, having equivalent parenthetic form is not sufficient for computational equiv-alence. 2his reason for this is the importance of ordering. For any parenthetic form, one canalways select a particular set of floating-point summands and impose an ordering in such away that floating-point error takes place. On the other hand, optimal grouping combinedwith the best ordering for that parenthesization can lead to much-reduced rounding error[10]. For this reason, when we enumerate computationally inequivalent summations, we mustconsider both grouping and ordering.A simple example of two computationally inequivalent summations that have equivalentparenthetic form is (( a + b ) + c ) and ( a + ( b + c )). The sum ( a + ( b + c )) is computationallyequivalent to (( b + c )+ a ), by pairwise commutativity, and (( b + c )+ a ) has the same parentheticform as (( a + b ) + c ). However, the ordering of additions means that that (( b + c ) + a ) maynot obtain the same results as (( a + b ) + c , and so they are not computationally equivalent. In some sense, selecting a grouping is answering the question: “In what manner should thesummation take place?”, whereas selecting an ordering answers: “How shall we instantiatethe chosen grouping?”Parenthesization is relevant to such things as parallelization, where a balanced tree ispreferred, for good load-balancing. However, in a parallel implementation, there may not becontrol over the order in which the parallel sub-sums are added, and this must be taken intoaccount when designing a parallelization scheme. Also, in a perfectly load-balanced scheme,one may find oneself obliged to group very large summands with very small, thus greatlyenhancing the likelihood of computational error.Parenthesization also comes into play when designing a default summation form for agiven programming language. The specifications for the language may permit the program-mer to sum values without defining the parenthesization. In that case, the compiler willaddress the parenthesization in some default manner that seems optimal.On the other hand, ordering influences the amount of rounding error one might see. Thereare parenthesizations that for certain distributions of summand values are unacceptably likelyto give results prone to rounding error, as in the load-balanced scheme discussed above.One approach might be to start with a rough ordering, perhaps on subsets of the sum-mands, select groupings that work for each subset, and then add the sums of the subset sumsin some appropriate grouping.One might have computational constraints on the ordering of a summation rather thanthe grouping, for example on a streaming problem, but we do not address that question inthis paper.
When a sequence in this paper associated with a class of summations turns out to existin the Online Encyclopedia of Integer Sequences (OEIS) [11], we name the OEIS sequence3nd cite the listing. This paper links several disparate OEIS sequences into a family ofsummation-related sequences.The OEIS resource can be very useful in the investigation of mathematical objects as-sociated with sequences. It lists other mathematical objects having the same sequences,and thus can give insight into the mathematics of the problem itself and its connection toother problems. It is well worth looking at any such references given, in the hope of findingcommonalities with seemingly unrelated problems.
A convenient and clarifying way of representing a summation is as a leaf-labeled rooted fullbinary tree. A full binary tree is one in which every node has either zero or two children[12]. In a summation tree, a node with two children represents the parenthesized sum of itschildren. The leaf nodes are labeled with the variables, and the internal nodes with +.In this paper, we use the terms summation and leaf-labeled summation tree interchange-ably. We also use the terms parenthetic form and unlabeled summation tree interchangeably.
We consider in this paper both abstract parenthetic forms and actual summations on theseforms. Parenthetic forms are abstract, correspond to unlabeled summation trees, describeonly the parenthesization, and may represent a class of summation trees, or a family ofsummations. Actual summations correspond to leaf-labeled trees, and describe both theparenthesization and the ordering of the variables. A summation is a concrete instantiationof some abstract parenthetic form.Two summations that are computationally equivalent for any choice of summands musthave isomorphic summation trees. However, having isomorphic summation trees is not suf-ficient for computational equivalence. Figure 1 is an example of two summations that haveisomorphic unlabeled summation trees, yet have non-isomorphic leaf-labeled summation treesand are computationally inequivalent.
Two unlabeled summation trees are isomorphic if one can be obtained from the other bya sequence of reversing the children of some set of nodes. This is in accordance with thepairwise commutativity of addition we assume. Parenthetic forms that are equivalent up topairwise commutativity of summands map to isomorphic summation trees.This refers only to isomorphic equivalence of parenthetic form; it is not the same ascomputational equivalence on instantiated summations, which depends on ordering as wellas grouping. 4 + ++ c + fa b d e ++ ++ c f +e d b a
Figure 1: Two isomorphic summation trees, or parenthetic forms, that represent computa-tionally inequivalent summations. These trees represent ((( a + b ) + c ) + (( d + e ) + f )) and((( e + d ) + c ) + ( f + ( b + a ))), respectively. Although the summands are the same and thetrees are isomorphic in form, the summations themselves are not computationally equivalent,because of the ordering of the summands: the summation on the right cannot be derivedfrom the summation on the left by a sequence of pairwise transpositions. Two leaf-labeled summation trees are isomorphic if one can be obtained from the other bya sequence of reversing the children of some set of nodes. Again, this is in accordance withthe pairwise commutativity of addition. Computationally equivalent summations map toisomorphic leaf-labeled trees.
A summation tree can be considered as a type of PQ tree. A PQ tree is a labeled treerepresenting a set of permutations on n elements, where the n leaf nodes are labeled withthe elements and the internal nodes are labeled as P or Q , where the P nodes have at leasttwo children with all permutations of the children equivalent, and the Q nodes have at leastthree children, with reversals of the orderings of the children equivalent [13]. A summationtree is a binary PQ-tree, and since it is binary, all of the internal nodes are labeled with P .To aid in the enumeration and understanding of summation trees, we introduce a typeof PQ summation tree called an SD-tree. We use this structure to establish a formula anda general method for enumerating members of a class of summations. An SD-tree is a binary tree in which all internal nodes have two children, and where aninternal node is labeled S if its two children have the { S } ame number of descendant leafnodes, and D if its two children have { D } ifferent numbers of descendant leaf nodes. Onemight also consider a leaf-labeled SD-tree, which is an instantiation of the parenthetic formrepresented by the SD-tree in question, and represents an actual summation. An SD-tree is5 variant of a PQ-tree. An example of a leaf-labeled SD-tree is shown in figure 2. SD DS c S fa b d e
Figure 2: An example of an SD-tree. This tree represents ((( a + b )+ c )+(( d + e )+ f )). This isthe same summation as in the first tree in Figure 1. The nodes labeled S have children withthe same number of descendant leaves. The nodes labeled D have children with differentnumbers of descendant leaves.Two summation SD-trees are considered isomorphic if one can be obtained from the otherby a sequence of reversing the children of some set of nodes (this is the same definition asin the non-SD summation tree case). Lemma 1.
Two isomorphic SD-trees with n leaf nodes have the same number of S -nodesand the same number of D -nodes.Proof. Follows from pairwise commutativity of addition.
Remark . The converse is not true. Two SD-trees with n leaf nodes and with the samenumber of S -nodes and D -nodes need not be isomorphic, as in Figure 3. If the number of S-nodes in the elements of an equivalence class of summation SD-trees isknown or can be quantified, then there is a simple formula to enumerate the computationallyinequivalent summations in that class.
Lemma 3.
Let T be SD-tree with n leaf nodes. The number of computationally inequivalentleaf-labeled SD-trees that are isomorphic to T is n !2 ε , where ε is the number of internal nodesof T labeled S, i.e., with left and right subtrees having the same number of leaf nodes.Proof. There are n ! ways to order the n leaf nodes of the tree T . Without loss of generality,it suffices to consider SD-trees where the number of leaf descendants of the left child of anode is always less than or equal to the number of leaf descendants of the right child. (Thisis because of pairwise commutativity.)Call the set of left descendant leaf nodes of a node d ℓ , and the set of right descendantleaf nodes d r . | d ℓ | = | d r | if and only if an ordering transposing d ℓ and d r gives a tree that6 D SDS DDD SS
Figure 3: Not all trees with the same number of leaves and the same number of S -nodesare isomorphic. Here are two non-isomorphic SD-trees with 6 leaves, 2 S -nodes and 3 D -nodes. This example is smallest in terms of the number of leaves of the SD-tree. (There isalso another example of non-isomorphic SD-trees with 6 leaves, this having 3 S -nodes and 2 D -nodes.)is the same as the original tree, up to transposition of the two subtrees. So such subtreesare combinatorially counted twice. The number of trees that are the same as another givenordering is 2 ε , so the total number of non-equivalent trees isomorphic to T is n !2 ε . Corollary 4.
Let T be an SD-tree with n leaf nodes. The number of non-equivalent sum-mations corresponding (up to tree isomorphism) to T is σ ( n ) = n !2 ε , where ε is the number ofinternal nodes of T σ ( n ) labeled S, i.e., with left and right subtrees having the same number ofleaf nodes. Corollary 4 implies that it suffices to (1) show that all summation trees representing classmembers are isomorphic to each other as SD-trees, (2) show that all summations with treesisomorphic to that class are in the class, and (3) provide a formula for the number of S nodesin the representative summation tree. The enumeration then follows immediately. We willfollow this method throughout this paper to enumerate summations of certain interestingclasses. Certain classes of summation problems have constraints placed on the grouping that mayinduce isomorphic parenthetic forms. When such constraints exist, we can discuss the prob-lem in terms of the summation trees that have forms conforming to the constraints. In otherwords, we fix the parenthesization in accordance with the constraint, then set the ordering.7n the next two sections, we first discuss summations on n variables, grouped and or-dered without any constraints on the grouping. We then turn to summations with groupingconstraints that induce specific parenthetic forms, and discuss in depth two important cases. We assume in this section that all summations are completely and explicitly parenthesized,and that pairwise commutativity holds. We make no other assumptions; in other words,there are no constraints imposed on the parenthesizations.The following formula is known to apply to leaf-labeled binary trees, and is shown byStanley [14], Callan [15] and Dale [16]. Walters makes this observation on commutative,non-associative multiplication, which has the same properties needed for this purpose asaddition [17].We observe that the number of computationally inequivalent summations on n variablesis the same as the number of leaf-labeled rooted binary trees with n leaf nodes. We give asketch of a tree-based proof here, and also provide a new combinatorial proof. Proposition 5. [14, 15, 16] The number of computationally inequivalent summations on n terms is (2 n − Proof 1, Tree-based Proof (sketch).
We proceed via induction, and note that all n -variablesums are obtained by taking the appropriate ( n − n th variableto one of its sub-sums. There are ( n −
2) + signs in an ( n − n − − n − −
1) + 1 = (2 n −
3) possible sub-sums. Since there is no overlap, there isno duplication, and this is the exact number. So by the induction hypothesis, the numberof computationally inequivalent summations on n variables is (2 n − n − − n − n − n − Proof 2, Combinatorial Proof.
We can order the n variables in n ! ways to make n ! unparen-thesized sums. For each of these sums, there are C n − = n (cid:0) n − n − (cid:1) distinct ways to paren-thesize them, where C n − is the n th Catalan number [14, 18].Each sum contains n − n − n ! · n · (cid:0) n − n − (cid:1) n − = ( n − n − n − n − · n − = (2 n − n − · n − = (2 n − n − · (2 n − · · · ·
2= (2 n − n − n − · · · · · · · n − · (2 n − · · · ·
2= (2 n − Remark . This sequence in Proposition 5 is OEIS A001147 [17]. There are many differentinterpretations of this sequence.
We discuss pairwise and ladder summation in depth in this section, two important casesthat are used by default in certain widely used programming languages. Both of these havegrouping constraints and both are examples of classes that can be described in terms ofisomorphic SD-trees.
DD dS ca b
Figure 4: An example of a ladder, or serial, SD-tree, representing ((( a + b ) + c ) + d ). A ladder summation is one in which the summation proceeds pairwise in the order in whichthe terms appear. For example, the ladder summation on four summands is ((( a + b )+ c )+ d ),and its tree is shown in Figure 4. Ladder summation corresponds to the C language defaultof left-to-right associativity on summations with ungrouped summands [19].9 emark . By pairwise commutativity, the same result is guaranteed in C on an IEEE-754-compliant system upon transposition of the first two elements of an ungrouped summation,but is not guaranteed after any other transposition.
Lemma 8.
There is a unique ladder SD-tree with n leaf nodes, and it has exactly one S -node.Proof. The lowest internal level of the a ladder-summation tree consists of one node withtwo leaf children. Every other node in the tree has one child that is a leaf; the other is theroot of another ladder summation. So only the unique lowest internal node is labeled S.
Lemma 9.
All SD-trees having exactly one S -node are isomorphic to some ladder SD-tree.Proof. Any subtree of an SD-tree must have at least one S -node, except when it consists of asingle leaf node. This means one of the top branches of an SD-tree with exactly one S -nodeis a single leaf node. The conclusion follows by induction on the number of leaf nodes. Corollary 10.
Ladder SD-trees are exactly those with one S -node, up to isomorphism. Proposition 11.
The number of computationally inequivalent ladder summations on n vari-ables is n !2 (2) Proof.
Follows immediately from Corollary 4 and Corollary 10.
Remark . This is OEIS A001710 [20], the number of even permutations on n letters. Pairwise summation is a recursive summation method that proceeds by dividing the set of n summands in half (or almost in half, if n is odd), summing on the two subsets, then addingthe two sums. Pairwise summation is known to be fairly accurate, and in some cases is nearlyas accurate as such gold-standard techniques as Kahan summation [2]. Pairwise summationis the default on ungrouped summands in NumPy [21] and in Julia [22].We discuss here three different forms for the sequence σ ( n ), the number of pairwisesummations on n variables. Two are recursive and one is a closed form. One of the recursiveformulations is previously known, and is shown by David in [23]. Proposition 13. [23] The number of computationally inequivalent pairwise summations on n variables is σ ( n ) = ( (cid:0) mm (cid:1) σ ( m ) , if n = 2 m ; (cid:0) m +1 m (cid:1) σ ( m ) σ ( m + 1) , if n = 2 m + 1 . (3) Remark . Enumerating the pairwise summations on n elements is equivalent to enumerat-ing the tournaments on n teams. This is the classical formulation of this problem, and givesrise to the sequence OEIS A096351 [24]. 10 S Sa ec g (a) Pairwise summation SD-tree forthe set of 4 summands { a, c, e, g } .This tree has 3 S -nodes. DD SS ec ga b (b) Pairwise summation SD-tree for the setof 5 summands { a, b, c, e, g } . This tree has 2 S -nodes. SD DS Sc ga b e f (c) Pairwise summation SD-tree for the set of6 summands { a, b, c, e, f, g } . This tree has 3 S -nodes. DS DS S S ga b c d e f (d) Pairwise summation SD-tree for the setof 7 summands { a, b, c, d, e, f, g } . This treehas 4 S -nodes. SS SS S S Sa b c d e f g h (e) Pairwise summation SD-tree for the set of 8summands { a, b, c, d, e, f, g, h } . This tree has 7 S -nodes. Figure 5: The sub-figures show the changing shape of pairwise SD-trees as summands areadded, and show how the internal nodes change (or not) from S to D and vice versa. Newnodes are added in alternating fashion, so that leaf descendants of all internal nodes areevenly distributed. The S -nodes are grey, and children of S -nodes have the same number ofleaf descendants. We show here a new formula for this sequence, based on the tree representation of a pairwisesummation, and the method presented using the SD-tree-based approach from Corollary 4.Pairwise summation gives rise to a binary tree in which every node that is not a leaf11as exactly two children, and every level is completely filled except for the lowest. This isa full complete binary tree , except in the spacing of its lowest level [12]. The lowest levelis not filled from left to right, but instead the number of descendants of a node differs byat most one from the number of descendants of its siblings, so at every level of the tree,the children of a node are evenly or almost evenly divided between the left and the rightbranches. Examples are shown in Figure 5.
Lemma 15.
All pairwise-summation SD-trees on n elements are isomorphic, and thereforehave the same number of S -nodes.Proof. A pairwise-summation SD-tree is constructed by extending each node by two children.If a node has k leaf descendants, then its children must have (cid:4) k (cid:5) and (cid:6) k (cid:7) leaf descendants.These can be transposed, by pairwise commutativity of addition. That they have the samenumber of S -nodes follows by Lemma 1. Lemma 16.
Any summation with a summation tree isomorphic to a pairwise-summationSD-tree is itself a pairwise summation.Proof.
Follows from the definition of pairwise summation. S -nodes in a pairwise summation. We now know that all pairwisesummation SD-trees have the same number of S nodes, and all we need do is count them toobtain the exponent in the denominator of the enumeration. Define ǫ ( n ) be the number of S nodes in an SD-tree. We show here two different forms for the sequence ǫ ( n ). Proposition 17.
The number of S -nodes in a pairwise summation SD-tree with n leaf nodesis ε ( n ) = ε ( m ) + 1 , if n = 2 m ; ε ( m ) + ε ( m + 1) , if n = 2 m + 1;0 if n = 1 . (4) Proof.
All subtrees of a pairwise summation SD-tree are themselves pairwise summationSD-trees. We proceed via induction.If n = 2 m , then the pairwise summation divides the 2 m leaf nodes evenly between thetwo subtrees, and the root node is an S -node. So the number of S -nodes in the SD-tree isequal to 1 + 2 ε ( m ), by the induction hypothesis.If n = 2 m + 1, then the pairwise summation divides the 2 m + 1 leaf nodes into twosubtrees having m and m + 1 nodes, and the root node is not an S -node. So the number of S -nodes in the SD-tree is equal to ε ( m ) + ε ( m + 1), by the induction hypothesis. Proposition 18.
The number of S -nodes in a pairwise summation SD-tree with n leaf nodesis ε ( n ) = ⌊ log ( n ) ⌋ X i =0 h(cid:16)(cid:16)j n i k + 1 (cid:17) mod 2 (cid:17) × i + ( − ⌊ n i ⌋ +1 ) mod 2 ) × ( n mod 2 i ) i (5)12 emark . Equation (5) is simpler than it looks. ⌊ log ( n ) ⌋ is one less than the number ofbits in n . The expression ( (cid:4) n i (cid:5) + 1) mod 2) is just the negation of the i th bit of n and enablesthe functions seen in Equation 7 and 8 from { , } to { , } and {− , } , respectively. Proof.
Because every SD-tree is a full complete binary tree, there are 2 i nodes at level i except for the bottom level. At each level, the descendant leaf nodes are almost evenlydivided between the nodes: at the i th level, each of the 2 i nodes has at least (cid:4) n i (cid:5) leafdescendants, and ( n mod 2 i ) of these nodes have an additional leaf descendant.Consider the binary representation of the number n where the bits are ordered from leastsignificant to most significant. If the i th bit of n is 0, then (cid:4) n i (cid:5) is even and 2 i − ( n mod 2 i )nodes have an even number of descendants. If the i th bit of n is 1, then (cid:4) n i (cid:5) is odd, and( n mod 2 i ) nodes have an even number of descendants. A node in a pairwise summation treeis an S -node if it has an even number of leaf descendants. So the number of S -nodes at level i is ( i − ( n mod 2 i ) , if the i th bit of n is 0;( n mod 2 i ) , if the i th bit of n is 1 . (6)We observe the following two equations: (cid:16)j n i k + 1 (cid:17) mod 2 = ( , if the i th bit of n is 0;0 , if the i th bit of n is 1 . (7)( − ⌊ n i ⌋ +1 ) mod 2 ) = ( -1 , if the i th bit of n is 0;1 , if the i th bit of n is 1 . (8)Putting Equations (6), (7), and (8) together, we see that the number of S -nodes at level i is (cid:16)(cid:16)j n i k + 1 (cid:17) mod 2 (cid:17) × i + ( − ⌊ n i ⌋ +1 ) mod 2 ) × ( n mod 2 i )Summing across the levels proves the proposition. Remark . Equation (5) is easily calculated as bitwise operations. Here is one C encoding: int bits_n = (int) floor(log2(n))+1;int exp = 0;for (int i=0; i Remark . The exponent ε ( n ) in Equations (4) and (5) is an offset variant of the sequenceOEIS A268289 [25], the cumulative deficient binary digit sum. In the context of S -nodes, ε ( n ) is undefined at n = 0 and starts with ε (1) = 0. The SD-tree equation in Proposition 17is the same formula referenced by Sloane for OEIS A268289 [25]. The closed form shown inEquation (5) is new for that sequence. 13 emark . Hwang provides an extensive analysis of solutions to divide-and-conquer recur-rences in [26]. Equation (5) could also be obtained from Equation (4) using the methods inthat paper. Instead, we prove Equation (5) directly by analyzing S -nodes in Proposition 18. The exponent in the denomi-nator of the enumeration of the equivalence class of pairwise summations has been derived,and the enumeration itself immediately follows. Corollary 23. The number of computationally inequivalent pairwise summations on n vari-ables is σ ( n ) = n !2 ε ( n ) , where ε ( n ) = ε ( m ) + 1 , if n = 2 m ; ε ( m ) + ε ( m + 1) , if n = 2 m + 1 . , if n = 1 . (9) Proof. Follows directly from Corollary 4 and Proposition 17. Corollary 24. The number of computationally inequivalent pairwise summations on n vari-ables is σ ( n ) = n !2 ε ( n ) , where ε ( n ) = ⌊ log ( n ) ⌋ X i =0 (cid:16)(cid:16)j n i k + 1 (cid:17) mod 2 (cid:17) · i +( − ⌊ n i ⌋ +1 ) mod 2 ) · ( n mod 2 i )(10) Proof. Follows directly from Corollary 4 and Proposition 18. Remark . Equation (9) is a new recursive formula for OEIS A096351 [24] and Equation(10) is a new closed formula for the same. There are interesting connections between S -nodes in a pairwise summation, the Tagakifunction [7] and Karatsuba’s classical addition algorithm [4]. OEIS A268289, i.e., the numberof S -nodes in a pairwise summation, is related to the Tagaki function; there are a numberof identities linking the two [6, 8]. Baruchel discusses the recursion tree of Karatsuba’smultiplication algorithm in terms of two types of nodes in the tree, direct and indirect. Heconjectured in [5] and proved in [6] that the sequence ε ( n + 1) in Equation (11), the set ofelements in the indirect path that contribute to a term of degree n in the Karatsuba addition,is A268289 n +1 .We provide a direct proof of Baruchel’s identity on ε ( n ) (A268289) here, but in terms ofthe number of S -nodes of pairwise summations (or full complete binary trees with the lowestlevels spaced as in a pairwise summation), rather than the Karatsuba recursion tree.To prove this, we consider an n -leafed pairwise-summation tree T , and label the ( n − T breadth-first with the integers from 1 to ( n − T has two14ubtrees: the left T L and the right T R . The internal nodes of these could be labeled in thesame manner as those of T . Without loss of generality, if n = 2 m + 1, T L has the greaternumber ( m + 1) of leaf nodes and T R has m leaf nodes. If n = 2 m , then both T L and T R have m leaf nodes. This ordering of subtrees is key to the mapping from T L and T R to T that we define for the proof.We define f , mapping the internal nodes of T L and T R into the internal nodes of T : f ( x ) = ( x, if x ∈ T L ;2 x + 1 , if x ∈ T R . The function f is a bijection between the internal nodes of T L ∪ T R and the non-root internalnodes of T . This mapping is illustrated in Figure 6. (a) Two pairwise summation SD-trees, theleft, T L , with 4 leaves and the right, T R , with3. Internal nodes are numbered breadth-first, and the S -nodes are grey. 12 34 5 6 (b) A pairwise summation SD-tree T with 7leaves. The left subtree is isomorphic to T L , andthe right subtree is isomorphic to T R . Internalnodes are numbered breadth-first, and S -nodesare grey. Figure 6: The mapping f is illustrated here. Internal nodes x from T L are sent to (2 x ), so 1,2, and 3 from T L (circles) are mapped to internal nodes 2, 4, and 6 of tree T . Internal nodes x from T R are sent to (2 x + 1), so internal node 2 of T R (square) is mapped to internal node5 of T , and internal node 1 of T R is mapped to node 3 of T . Proposition 26. Let ε ( n ) be the number of S -nodes in a pairwise summation SD-tree. Then ε ( n ) = |{ k | ≤ k < n, ( n − k − 1) mod 2 ⌊ log k ⌋ +1 < ⌊ log k ⌋ }| (11) Proof. It suffices to show that the S -nodes of T are exactly those whose labels are in the setdescribed by Equation (11). The proof proceeds by induction on tree height. We considerthe two cases: n = 2 m + 1 ( n odd) and n = 2 m ( n even).First, the tree root satisfies condition 11 if and only if n is even: the root is labeled 1, so( n − − 1) mod 2 ⌊ log ⌋ +1 < ⌊ log ⌋ if and only if ( n − − 1) = 0 mod 2 if and only if n iseven. 15 ase 1: n odd, n = 2 m + 1. By the inductive hypothesis, x is an S -node of T L if andonly if (( m + 1) − x − 1) mod 2 ⌊ log ( x ) ⌋ +1 < ⌊ log ( x ) ⌋ , which is true if and only if(2( m + 1) − x − 2) mod 2 ⌊ log ( x ) ⌋ +2 < ⌊ log ( x ) ⌋ +1 , which is the same as( n − x − 1) mod 2 ⌊ log (2 x ) ⌋ +1 < ⌊ log (2 x ) ⌋ , which is true if and only if( n − k − 1) mod 2 ⌊ log ( k ) ⌋ +1 < ⌊ log ( k ) ⌋ , where k is an even node of T . (12)Again, by the induction hypothesis, x is an S -node of T R if and only if( m − x − 1) mod 2 ⌊ log ( x ) ⌋ +1 < ⌊ log ( x ) ⌋ , which is true if and only if(2 m − x − 2) mod 2 ⌊ log (2 x +1) ⌋ +1 < ⌊ log ( x +1) ⌋ , which is the same as(( n − − (2 x + 1) − 1) mod 2 ⌊ log (2 x +1) ⌋ +1 < ⌊ log ( x +1) ⌋ , which is true if and only if( n − k − 1) mod 2 ⌊ log ( k ) ⌋ +1 < ⌊ log ( k ) ⌋ , where k is an odd node of T , and k ≥ . (13)Statement (12) shows there is a bijection between even nodes of T that satisfy the conditionin the proposition and the S -nodes of T L , and statement (13) shows a bijection betweenodd non-root nodes of T that satisfy the condition and the S -nodes of T R . Since n is odd,the root node does not satisfy the condition. So the number of nodes in T that satisfy thecondition in Equation (11) is ε ( m + 1) + ε ( m ), and this is ε ( n ), by Proposition 17. Case 2: n even, n = 2 m . The proof in this case proceeds in the same manner as theabove. By the induction hypothesis, x is an S -node of T L if and only if( m − x − 1) mod 2 ⌊ log ( x ) ⌋ +1 < ⌊ log ( x ) ⌋ , which is true if and only if(2 m − x − 2) mod 2 ⌊ log ( x ) ⌋ +2 < ⌊ log ( x ) ⌋ +1 , which is the same as(( n − − x − 1) mod 2 ⌊ log (2 x +1) ⌋ +1 < ⌊ log (2 x +1 ⌋ ) , which is true if and only if( n − k − 1) mod 2 ⌊ log ( k ) ⌋ +1 < ⌊ log ( k ) ⌋ , where k is an even node of T . (14)Again, by the induction hypothesis, x is an S -node of T R if and only if( m − x − 1) mod 2 ⌊ log ( x ) ⌋ +1 < ⌊ log ( x ) ⌋ , which is true if and only if(2 m − x − 2) mod 2 ⌊ log ( x ) ⌋ +2 < ⌊ log ( x ) ⌋ +1 , which is true if and only if( n − (2 x + 1) − 1) mod 2 ⌊ log (2 x +1) ⌋ +1 < ⌊ log (2 x +1) ⌋ , which is true if and only if( n − k − 1) mod 2 ⌊ log ( k ) ⌋ +1 < ⌊ log ( k ) ⌋ , where k is an odd node of T , and k ≥ . (15)Statement (14) shows there is a bijection between even nodes of T that satisfy the conditionin the proposition and the S -nodes of T L , and (15) shows a bijection between odd non-rootnodes of T that satisfy the condition and the S -nodes of T R . The root node does satisfythe condition when n is even. So the number of nodes in T that satisfy the condition inEquation (11) is 2 ε ( m ) + 1, and this is ε ( n ), by Proposition 17.16 .2.3 A remark on orderings for pairwise summations vs. tournaments The parenthetic form of grouping for a pairwise summation is the same as the form for atournament. However, the optimal ordering for a summation may well be different thanthat of a tournament. This contrast illustrates the problem-specific nature of selecting thepreferred or best ordering, even after a grouping method has been selected.As a heuristic, the most accurate pairwise summation groups pairs of similar magnitudeat each level of the summation tree. This tends to minimize rounding error that results fromthe use of finite precision on digital computers [10].The best tournament might also match teams of equal strength at each level of thetournament, leading to interesting games, at least in early games. This is equivalent to thebest ordering for a pairwise summation. On the other hand, one might prefer better teamsto be matched with worse teams early in the competition, in order to allow the best teamsto persevere into the final rounds of the tournament. This would be a very poor choice fora pairwise summation, and is more likely to lead to rounding errors and inaccuracy. Thechoice of ordering here then would depend on the goals of the tournament. Proposition 27. The upper bound for the number of computationally inequivalent summa-tions on n summands on a class of isomorphic SD-trees is n !2 . This is the number of laddersummations on n summands.Proof. This can be seen by considering that all nodes in a summation tree must have zero ortwo children, so leaf nodes at the lowest level must come in pairs, and any tree must have atleast one of these pairs. This means an upper bound is less than or equal to n !2 , by Corollary4. But this bound is met by the class of ladder summations on n terms. Remark . This upper bound is OEIS A001710 [20], the number of even permutations on n letters. See Proposition 11. Proposition 29. The lower bound for the number of summations on n summands on a classof isomorphic SD-trees is n !2 β ( n ) , where β ( n ) is the highest power of that divides n ! .Proof. Follows from Corollary 4. Remark . The sequence β ( n ) is OEIS A011371 [27], the highest power of 2 dividing n !.The sequence n !2 β ( n ) is OEIS A049606 [28], the largest odd divisor of n !. Lemma 31. The largest power of that divides k ! is (2 k − . roof. Let 2 β (2 k ) be the largest power of 2 that divides 2 k !. For 1 ≤ i ≤ k there are k i = 2 k − i numbers in { , , . . . , k } divisible by 2 i . Each of these adds one to β (2 k ). So β (2 k ) = k P i =1 k − i = k − P i =0 i = 2 k − Lemma 32. Let n = 2 k + r , where < r < k . The largest power of that divides n !(2 k )! = n ( n − · · · (2 k + 1) is the same as the largest power of that divides ( n − k )! = r ! .Proof. k does not divide any of { k + 1 , k + 2 , . . . , k + r = n } , so largest power of of 2that divides n !(2 k )! = n ( n − · · · (2 k + 1) is the same as the largest power of 2 that divides( n − k )!. Lemma 33. Let n = 2 k + r , where < r < k . The largest power of that divides n ! is thelargest power of that divides k ! plus the largest power of that divides ( n − k )! = r ! Proof. Follows from Lemmas 31 and 32. Proposition 34. Let n = 2 k + r , where ≤ r < k . The bound in Proposition 29 onsummations on n summands is met by a parenthetic form µ ( n ) that has the recursive form µ ( n ) = ( µ (2 k ) + µ ( r )) where µ (2 i ) is the pairwise summation on i elements.Case 1, r = 0 . A pairwise summation on 2 k elements is represented by a perfect binary tree,one in which all interior nodes have two children, and all leaf nodes are at the same level k [29]. A perfect binary tree has 2 k leaf nodes and 2 k − S -node. So by Corollary 4 and Lemma 31, the bound is met. Case 2, r > . k is not equal to n − k , so the top node is not an S -node and the problemreduces to finding the largest powers of 2 that divide 2 k and r . The conclusion follows byLemma 33. Remark . The parenthetic form shown in Proposition 34 is not necessarily the only one thatmeets the bound. For example, Table 1 shows 3 non-equivalent parenthetic forms meetingthe bound β ( n ) from Proposition 27 for each of n = 7 , , , 14, and 15 non-equivalentparenthetic forms meeting the bound for n = 15. Proposition 36. The number of non-isomorphic summation trees (or parenthetic forms)with n unlabeled leaf nodes (or summands) is α ( n ) = ⌊ n ⌋ X i =1 α ( i ) α ( n − i ) (16)18 roof. Partition n as i and n − i . A summation tree is formed by attaching an i -leaf tree asthe left subtree and an ( n − i )-leaf tree as the right. Because of commutativity, it suffices toconsider only the case where i ≤ n − i . Summing across such i gives the proposition. Remark . This sequence is OEIS A000992, the n th half-Catalan number. An equivalentobservation is made by Callan [30]. In Table 1, this sequence is in the rightmost column. Proposition 38. The number of non-isomorphic summation trees (or parenthetic forms)with n unlabeled leaf nodes (or summands) and s S -nodes is τ ( n, s ) = ⌊ n − ⌋ P j =1 s P i =0 τ ( j, i ) τ ( n − j, s − i ) , if n is odd ; ⌊ n − ⌋ P j =1 s P i =0 τ ( j, i ) τ ( n − j, s − i ) + s − P i =0 τ ( n , i ) τ ( n , s − − i ) , if n is even . (17) where τ ( n, s ) = ( , if n = 0 and s = 0 (the unique empty tree) ;1 , if n = 1 and s = 0 (the unique single-node tree) . Proof. At every stage, the first term n in the calculation is reduced by half, so this recursionterminates, according to the rules for termination of τ ( n, s ) above.If n is odd, the children of the top node cannot both have the same number of leafdescendants, so the top node cannot be an S -node. The formula follows by counting thesub-trees on the 2-partitions of n , where the left sub-tree has i S -nodes and the right has s − i S -nodes, for 0 ≤ i ≤ s , and the two counts are multiplied together.If n is even, the formula is as above, but we must also include the case where the topnode is an S-node. In that case, both children have n leaf nodes, and the summation ison sub-trees where there are n leaf nodes in each sub-tree. In this case, the root is oneof the S -nodes, so the left sub-tree has i S -nodes and the right has s − − i S -nodes, for0 ≤ i ≤ s − Remark . The number of parenthetic forms increases much more slowly than the totalnumber of summations. Table 1 shows the enumeration of parenthetic forms on n summandsand s S -nodes for n up to 16 summands. Remark . The formula in Proposition 38 is recursive and the proof is constructive, so onecan more or less laboriously calculate all the parenthetic forms on N summands. Proposition 41. The number of parenthetic forms on n ≥ with exactly S -nodes is τ ( n, 2) = ( ( m − , if n = 2 m + 1;( m − m − , if n = 2 m. (18)19able 1: The number τ ( n, s ) of summation trees with n leaf nodes and s S -nodes, calculatedup to n = 15. The rows represent n , the number of leaf nodes, and the columns represent s ,the number of S -nodes. α ( n ) is the sum of all the τ ( n, s ) in the n th row. n s 1 2 3 4 5 6 7 8 9 10 11 12 13 14 α ( n )1 02 Proof. We proceed by induction, and observe that the proposition is true by inspection for n = 1 , , , 4. We also observe that no tree having exactly 2 S -nodes can have an S -noderoot. This means that either (a) one of the subtrees below the root has two S -nodes and theother has none, or else (b) both subtrees have exactly one S -node.In case (a), the subtree with no S -nodes consists of only one leaf node. The other subtreehas n − S -nodes. So the number of trees with such subtrees is τ ( n − , S -node, so by Corollary 10, it is the unique ladder tree ofits size. Each subtree with ℓ leaf nodes has a sibling with n − ℓ leaf nodes, and there is oneladder tree at each of these sizes, so the number of such double ladder trees is (cid:4) n (cid:5) − n isodd, and n − n is even (excluding when the siblings have an equal number of leaf nodes).Putting (a) and (b) together, τ ( n, 2) = τ ( n − , 2) + (cid:4) n (cid:5) − 1. Using the induction hypothesis, τ ( n, 2) = ( ( m − m − 2) + ( m − 1) = ( m − , if n = 2 m + 1;( m − + ( m − 2) = ( m − m − , if n = 2 m. Remark . τ ( n, 1) counts the (unique) ladder summations, as discussed in Proposition 11. τ ( n, 2) is sequence OEIS A002620 [31], the quarter-squares, with an offset. The bold-facecolumn α ( n ) in Table 1 is the half-Catalan sequence OEIS A000992 [30] seen in Proposition36. None of the other sequences in Table 1 seem to be listed in the OEIS.20 Conclusion We have classified a number of summation types arising in real calculations. Table 2 in theAppendix presents a summary of the main combinatorial results derived or referenced in thispaper, with other interpretations and OEIS numbers where applicable. We thank Terry Grov´e for asking the questions that inspired this work, and for many ad-ditional conversations. We also thank Andy DuBois, Shane Fogerty, Brett Neuman, ChrisMauney, and Bob Robey for many interesting and helpful discussions on the subject.This work was performed at Los Alamos National Laboratory, managed by Triad Na-tional Security, LLC, under U.S. Government contract 89233218CNA000001, and at LANL’sUltrascale Systems Research Center (USRC) at the New Mexico Consortium, supported bythe U.S. Department of Energy contract DE-FC02-06ER25750.This publication is unclassified, and has been assigned LANL identifier LA-UR-20-23426.In memory of John Conway and Vera Pless. References [1] D. Goldberg, “What every computer scientist should know about floating-point arith-metic,” ACM Comput. Surv. , vol. 23, p. 548, Mar. 1991.[2] N. J. Higham, “The accuracy of floating point summation,” SIAM J. Sci. Comput ,vol. 14, pp. 783–799, 1993.[3] N. A. Rosenberg, “Enumeration of lonely pairs of gene trees and species trees by meansof antipodal cherries,” Advances in Applied Mathematics , vol. 102, pp. 1 – 17, 2019.[4] A. Karatsuba and Y. Ofman, “Multiplication of multidigit numbers on automata,” Soviet Physics Doklady , vol. 7, p. 595, 12 1962.[5] T. Baruchel, “Flattening Karatsubas recursion tree into a single summation,” SN Com-puter Science , vol. 1, Nov 2019.[6] T. Baruchel, “Properties of the cumulated deficient binary digit sum.” https://arxiv.org/abs/1908.02250 , 2019.[7] T. Takagi, “A simple example of the continuous function without derivative,” TokyoSugaku-Butsurigakkwai Hokoku , vol. 1, pp. 176–177, 1901.[8] J. Lagarias, “The Takagi function and its properties,” Functions in Number Theory andTheir Probabilistic Aspects , 12 2011. 219] IEEE, “IEEE Standard for Floating-Point Arithmetic,” IEEE Std. 754-2019 (Revisionof IEEE 754-2008) , pp. 1–84, 2019.[10] V. Job, T. Grov´e, S. Fogerty, C. Mauney, L. Monroe, and R. Robey, “Order matters:A case study on reducing floating point error in sums through ordering and grouping.”Manuscript submitted to the International Conference for High-Performance Comput-ing, Networking, Storage and Analysis, 2020.[11] OEIS Foundation Inc., “The On-Line Encyclopedia of Integer Sequences.” http://oeis.org , 2020.[12] D. Knuth, The Art of Computer Programming: Volume 1: Fundamental Algorithms .Pearson Education, 1997.[13] K. S. Booth and G. S. Lueker, “Testing for the consecutive ones property, intervalgraphs, and graph planarity using PQ-tree algorithms,” Journal of Computer and Sys-tem Sciences , vol. 13, no. 3, pp. 335 – 379, 1976.[14] R. Stanley and S. Fomin, Enumerative Combinatorics: Volume 2 . Cambridge Studiesin Advanced Mathematics, Cambridge University Press, 1997.[15] D. Callan, “A combinatorial survey of identities for the double factorial.” https://arxiv.org/abs/0906.1317 , 2009.[16] M. Dale and J. Moon, “The permuted analogues of three Catalan sets,” Journal ofStatistical Planning and Inference , vol. 34, no. 1, pp. 75 – 87, 1993.[17] OEIS Foundation Inc., “The On-Line Encyclopedia of Integer Sequences: SequenceA001147.” https://oeis.org/A001147 , 2020.[18] OEIS Foundation Inc., “The On-Line Encyclopedia of Integer Sequences: SequenceA000108.” https://oeis.org/A000108 , 2020.[19] ISO/IEC, “Iso International Standard 9899:2018 Programming Language C.” , 2018.[20] OEIS Foundation Inc., “The On-Line Encyclopedia of Integer Sequences: SequenceA001710.” https://oeis.org/A001710 , 2020.[21] SciPy.org. https://docs.scipy.org/doc/numpy/reference/generated/numpy.sum.html ,2020.[22] JuliaLang.org. https://github.com/JuliaLang/julia/pull/4039 , 2013.[23] H. David, The method of paired comparisons . Griffin’s statistical monographs andcourses, C. Griffin, 1988. 2224] OEIS Foundation Inc., “The On-Line Encyclopedia of Integer Sequences: SequenceA096351.” http://oeis.org/A096351 , 2020.[25] OEIS Foundation Inc., “The On-Line Encyclopedia of Integer Sequences: SequenceA268289.” http://oeis.org/A268289 , 2020.[26] H.-K. Hwang, S. Janson, and T.-H. Tsai, “Exact and asymptotic solutions of a divide-and-conquer recurrence dividing at half: Theory and applications,” ACM Trans. Algo-rithms , vol. 13, Oct. 2017.[27] OEIS Foundation Inc., “The On-Line Encyclopedia of Integer Sequences: SequenceA011371.” https://oeis.org/A011371 , 2020.[28] OEIS Foundation Inc., “The On-Line Encyclopedia of Integer Sequences: SequenceA049606.” http://oeis.org/A049606 , 2020.[29] Y. Zou and P. E. Black, “Dictionary of Algorithms and Data Structures.” https://xlinux.nist.gov/dads/HTML/perfectBinaryTree.html , 2019.[30] OEIS Foundation Inc., “The On-Line Encyclopedia of Integer Sequences: SequenceA000992.” https://oeis.org/A000992 , 2020.[31] OEIS Foundation Inc., “The On-Line Encyclopedia of Integer Sequences: SequenceA002620.” https://oeis.org/A002620 , 2020.2010 Mathematics Subject Classification : Primary 05A10, Secondary 05C30, 65G50. Keywords: summations, numerical error, binary trees(Concerned with sequences A000992, A001147, A001710, A002620, A011371, A060818,A049606, A096351 and A268289.) 23 Appendix: Summary of results Table 2: This table is a summary of the main combinatorial results derived or referenced inthis paper, with references, alternative interpretations and OEIS numbers where applicable. Summation interpretation Refs. Other meaning OEIS Inequivalent parenthetic forms (i.e., non-isomorphic summation trees) No. of non-isomorphic summation treeswith n unlabeled leaf nodes Prop. 36 Half-Catalannumbers A000992No. of non-isomorphic summation treeswith n unlabeled leaf nodes and 1 S -node(ladder or serial summations) Cor. 10 The constant 1No. of non-isomorphic summation treeswith n unlabeled leaf nodes and 2 S -nodes Prop. 41 Quarter-squares A002620No. of non-isomorphic summation treeswith n unlabeled leaf nodes and s S -nodes Prop. 38 Computationally inequivalent summations No. of computationally inequivalentsummations on n elements having aparticular summation SD-tree Cor. 4No. of computationally inequivalentsummations on n summands Prop. 5 Double factorial ofodd numbers A001147No. of computationally inequivalent ladder,or serial, summations on n summands Prop. 11 No. of evenpermutations on n elements A001710No. of computationally inequivalentpairwise summations on n summands Prop. 13,Cor. 23, 24 No. of tournamentson n teams A096351 S -nodes and bounds Lower bound for the number of S -nodes ina summation tree on n summands Prop. 27 The constant 1Upper bound for the number of S -nodes ina summation tree on n summands Prop. 29 Largest number k where 2 k divides n ! A011371No. of S -nodes in a pairwise summationtree with n leaf nodes Prop.17, 18 Cumulative deficientbinary digit sum A268289Lower bound for the number ofcomputationally inequivalent summationson n summands on a given SD-tree Prop. 29 Largest odd divisorof n ! A049606Upper bound for the number ofcomputationally inequivalent summationson n summands on a given SD-tree Prop. 27 No. of evenpermutations on n elements A001710elements A001710