Peer-Reviewed Journal Details
Mandatory Fields
Pearlmutter, BA;Siskind, JM
2008
January
ACM Transactions on Programming Languages and Systems
Reverse-mode AD in a functional framework: Lambda the ultimate backpropagator
Published
11 ()
Optional Fields
AUTOMATIC DIFFERENTIATION
30
We show that reverse-mode AD (Automatic Differentiation)-a generalized gradient-calculation operator-can be incorporated as a first-class function in an augmented lambda calculus, and therefore into a functional-programming language. Closure is achieved, in that the new operator can be applied to any expression in the augmented language, yielding an expression in that language. This requires the resolution of two major technical issues: (a) how to transform nested lambda expressions, including those with free-variable references, and (b) how to support self application of the AD machinery. AD transformations preserve certain complexity properties, among them that the reverse phase of the reverse-mode AD transformation of a function have the same temporal complexity as the original untransformed function. First-class unrestricted AD operators increase the expressive power available to the numeric programmer, and may have significant practical implications for the construction of numeric software that is robust, modular, concise, correct, and efficient.
NEW YORK
0164-0925
10.1145/1330017.1330018
Grant Details