Alex Martelli is a senior staff engineer at Google. Alex is a co-author of the just-published 3rd edition of Python in a Nutshell, and was a co-editor of the Python Cookbook. He is a PSF Fellow, and won the 2002 Activators’ Choice Award and the 2006 Frank Willison Award for contributions to the Python community
Automated testing is at the heart of modern development and operations, but it's often segregated between developer-focused unit testing and separate, often only semi-automated integration testing. The talk shows that the best kind of testing deploys in many layers, re-using test-components in various configurations for multiple testing purposes.
The role of automated testing at the heart of modern development and operations is, by now, a given. However, the traditional approach to testing, which cleaves it drastically into two—developer-focused unit testing and separate (and, alas, often only semi-automated) integration testing—is not optimal in the modern, fluid world of DevOps.
Nothing short of full automation is suitable for continuous integration; any “testing” requiring humans has a drastically different place in the continuum of development and deployment and should best be called by a completely different name, such as "quality assurance". Within the realm of fully automated testing, the best approach, just as for other kinds of software, is highly modular.
This talk highlights the proper design of components meant for testing purposes and explains how such a design lets you compose multiple, layered testing suites that span the gamut from unit tests (meant to run all the time during development) to full-fledged end-to-end tests of whole systems—and, crucially, the often-neglected intermediate layers, bridging the thoroughness of end-to-end tests with unit tests’ ability to pinpoint the location of any problems that emerge, enabling rapid fixes of most such problems.
The talk also discusses the use of modular, layered testing components to validate software refactoring (all the way to deep rearchitecting of a system) and (when deployed in a load-testing arrangement) identify and validate software (and architectural) optimizations.