Differentiating the Warm-Ups