Measurement is the first step that leads to control and eventually to improvement. If you can’t measure something, you can’t understand it. If you can’t understand it, you can’t control it. If you can’t control it, you can’t improve it. ~ H. James Harrington
The pithy maxim “Measure twice, cut once” is the title of a book by one of my heroes- master woodworker, Norm Abram of PBS series “The New Yankee Workshop.” Silently repeating this advice to myself as I undertake making the cuts for a new woodworking project has saved me countless hours of labor and untold dollars in materials. Besides the obvious wisdom of double checking a measurement before committing a board to the teeth of sharp saw, there are other critical aspects of measuring to be considered:
- The accuracy of the measuring instrument must precise and consistent. Poorly designed or hastily constructed measuring tools lead to work products of uncertain quality.
- The instrument must be calibrated in the right units of measurement. To make a measurement in inches and fractions, the rule must be in the right units. A metric rule or a rule incremented in tenths of an inch may be accurate for its respective purpose but is inaccurate if measuring for fractions of inches. The impulse to simply convert between units of measure with confidence in its accuracy is a fallacy. Conversion always introduces error and cannot be consistently replicated.
- Identical measuring instruments yield different results. Slight variations in the scale and condition are introduced through use, wear and tear. The same instrument should be used throughout a project for accuracy and consistency. Switching instrumentation in the middle of a project introduces unequal lengths and loose joints (error and uncertainty).
- Two craftsmen using the same instrument will measure differently. Each has his own technique and method of marking. One craftsman cannot cut with confidence on the measurement made by another unless he knows the method and assumptions underlying the mark of the other.
- The method of recording the measured mark determines its accuracy. Felt tip markers, carpenter’s pencils, fine lead pencils, marking knives and scratch awls are all common methods for recording a measurement. Each recording device has its own pluses and minuses and none are the right tool in all cases. The material being measured and the level of accuracy required determine the appropriate recording device.
- Mid-stream shortcuts always lead to mistakes. During the course of a project and the repetition of tasks, the creative (and lazy) always learn enough to find a way to make the work go faster. Improving processes and increasing productivity is a good thing unless introduced in the middle of a project. Changes in techniques or assumptions during a project add complexity and inevitably introduce errors. Lessons learned and shortcuts are best left for future projects (rather than the current one).
A lot has been made recently of election metrics and it’s about time the idea got some traction in the US elections community. Doug Chapin has written several times recently about data and metrics and has noted a movement among election administrators to begin to measure, analyze and use data. What to measure, how to measure it, how to analyze it for meaning and how to leverage it are all questions that need to be asked by everyone. Unlike the myriad of terms, definitions and meanings found in the fifty + election codes around the country, the answers to these questions must be consistent among the profession to yield meaningful data, comparisons and insights.
In a gallant first cut at collecting, aggregating and analyzing election data from around the country, the Pew released a compilation of comparative election metrics intended to score performance among the states in February 2013 along the lines of the “Democracy Index” proposed by Hether Gerken. Like good craftsmen, those working on the project had well-tuned saws and sharp chisels. What they did not have, however, is valid, reliable measures of comparable material recorded using similar assumptions and techniques in the same unit of accuracy with equal accuracy.
Even where the data were comparable, meanings of the data were not clearly defined. For example, the meaning to be assigned to the number of provisional ballots cast is ambiguous and contradictory:
“Unless provisional ballots are being given to voters for other administrative reasons, a large number may indicate problems with voter registration records. The meaning of a small number of provisional ballots, from an election administration standpoint, is more open to question. On the one hand, a small number may indicate that registration records are up to date; on the other hand, small numbers may be the result of poll workers not offering voters with registration problems the provisional ballot option when appropriate.” (Pew Center on the States, “Election Performance Index”, February 2013, p. 38) Emphasis added.
The danger of proceeding further into the data-driven world of empirical analysis without clarifying and standardizing definitions, specifying measuring instruments and units of measurement, and formulating measurement protocols (before the fact) is not merely fuzzy research. Fuzzy research will tell a story to the public and politicians that is not accurate, that is not credible, and which may not be entirely ethical. The real danger is that policies and decisions will be made on the basis of fuzzy research and careers and reputations will be made and damaged. The results will be just like an inferior quality, poorly measured woodshop project regardless of the quality of the tools and the earnestness of the craftsman.
The parallels in the art of measurement between my twin passions of woodworking and elections administration are insightful to the present situation. My point is not to criticize the Pew’s bold undertaking nor is it to put a damper on the emerging enthusiasm and commitment to gathering and using election data. By pointing to the wisdom of masters of other crafts, I hope to shape the future of data collection and analysis in our craft of election administration.
“While ‘measure twice and cut once’ is always pithy advice, it more important to measure accurately and to know that you have.” ~Aldren Watson “Hand Tools: Their Ways and Workings”