Abstract

AD is generally cast as transforming first-order code which consumes and produces numeric arrays. We open with a laundry list of practical weaknesses of this standard formulation and standard implementation technologies. We proceed to a discussion of attempts to generalize AD: to make it apply to a wider class of procedures, to raise its semantic level, to increase its robustness, and to make it first-class. We sketch out links to Synthetic Differential Geometry and to Programming Language Theory, and summarize a few ways in which AD can be formalized. Refactoring of AD opens the possibility of using techniques being developed for efficient AD to serve other ends, such as security monitoring or probabilistic computation. We close with some case studies of ways in which general AD, combined with extremely aggressive compilation, can allow highly succinct and modular construction of complex adaptive systems. (Joint work with Jeffrey Mark Siskind.)