Читать книгу Planning and Executing Credible Experiments - Robert J. Moffat - Страница 11

Preface

Laboratories (and businesses) need people who can answer difficult questions experimentally. If data are needed for decision‐making, this book is for you.

Science, medicine, the environment, agriculture, and engineering depend heavily on experiments. Business does as well, when surveying the added complexity of human taste and markets. In any of these fields, an experiment must be credible or it wastes time and resources. This book offers a tested guide, developed and used for decades at Stanford University, to equip novice researchers and experienced technicians to plan and execute credible experiments.

This book prepares the reader to anticipate the choices one will face from the launch of your project to the final report. We come alongside the reader, emphasizing the strategies of experiment planning, execution, and reporting.

The foundation of this book originated in “Planning Experimental Programs,” a set of class notes by Robert Moffat, our first author. Our second author, Roy Henk, is one of thousands whose own experiments benefited from Moffat's notes. This book blends our approaches to developing and planning experimental programs.

Bob Moffat: Our goal for this book was to collect, organize, summarize, and present what we think are important ideas about developing, running, and reporting reproducible, provably accurate experiments – with minimum wasted effort.

My contribution has to do with the process of developing experimental programs that can be trusted to produce trustworthy data. I used the word “developing” because “new” experiments generally require quite a few changes to eliminate unforeseen (but finally recognized) errors before they qualify as “good” experiments. This is called “the shakedown phase.”

This raises a question, however: how can you spot an error in a “new” experiment? The only way I know is to run a test for which you already know the answer. We have the three conservation laws we can trust (conservation of mass, momentum, and energy), but they are a bit hard to work with. I think it generally requires less work to simply stretch the operable domain of the proposed experiment until it includes some conditions that have already yielded data that you trust. That data become the “qualification test” for your experiment.

I've been designing and running experiments for more than 60 years, and my experience has been that the development of a good experiment is always iterative. It takes a lot of work during the shakedown period to eliminate all the sources of error. Surprisingly, this aspect of experiment planning isn't generally “taught.” That's really what started this book! The suggestions presented here reflect things I learned “on the job” during my first 10 years in industry and then refined during the rest of my career at Stanford.

When I got my BSME from the University of Michigan, I was offered a job in the General Motors (GM) Research Laboratories (GMR). I stayed with GMR almost exactly 10 years, rising from working on little problems like calibrating instruments to developing a high‐precision, high‐temperature wind tunnel for testing aircraft engine thermocouple probes up to 1600 °F, finding what caused the bearings to fail on an experimental engine, and developing high‐effectiveness heat exchangers for gas turbine engines. In the last two years of my stay, I was part of a small group of engineers who designed, built, and tested a high‐efficiency, two‐spool regenerative gas turbine that, starting from 30° below zero Fahrenheit, could deliver 350 HP in less than two minutes. That was developed for GM trucks and buses, but a descendant of that engine powered the US Army's M1A1 battle tank – the one designed for cold weather.

The increasingly interesting problems I encountered in the last three or four years made it clear to me that I needed to go back to school – there was too much that I didn't know.

I applied to Stanford with references from GMR and was offered a scholarship, which lead to a PhD, a Stanford professorship, 15 years as head of the mechanical engineering department, and 25 years of research and teaching on engineering problems. In my spare time, I worked with Dr. Alvin Hackel, a Stanford pediatrician, to develop the Stanford Transport Incubator. Dr. Hackel provided the medical insights and I did the engineering. His patients, usually newborn and sometimes premature, often needed intensive medical care during transport between hospitals. That incubator won the American Society of Mechanical Engineers' 1987 Holley Medal for Service for the Benefit of Mankind and is displayed on the cover.

I retired and began to put this book together.

Now a word from my colleague, Roy Henk.

Roy Henk: With the aim to continually improve, experiments can fill one's personal daily life. My experimenting began on a small farm with vegetables and livestock (plus honeybees). Later experiments involved concrete mixing, internal combustion engines, molten metal, and measuring ore. At Virginia Tech, I joined low‐noise precision wind‐tunnel tests on flow transition; I have used wind tunnels at Stanford and in Japan at the National Aerospace Laboratory of Japan (NAL), even joining tests in a Mach 5 tunnel. NAL together with JAXA is Japan’s equivalent to NASA. My research used water tunnels at the US Naval Research Lab and at Stanford, and Bob’s notes. Ten years in industry had me designing advanced components of commercial aerospace engines. Compared to air flow around an airplane, thermo‐fluid flow inside an aerospace engine is about the most challenging area of classical physics. Those engines have flown on aircraft for years without incident.

I have thrilled with experimental success, yet failures too provided valuable data and lessons. For example, as a young researcher I did not realize the advantage of randomizing the order of data collection. How did this matter? In the middle of one particularly costly data run, a valve failed staying open. Had I randomized, I would have had two valuable datasets at lower resolution. Instead that data had little use. Design of Experiments spared me from much learning the hard way, like ancient wisdom.

No longer a bother, Uncertainty Analysis now informs every stage, fortifying results, averting disaster. To postpone uncertainty analysis starves one’s experiment.

While teaching Design of Experiments, I asked Bob Moffat about his book. Bob graciously allowed his notes to supplement my course. Over 18 years, I had developed notes on certain powerful, open‐source software for experiment design and analysis. Bob at Stanford and I, now as professor of energy science at Kyoto University, began exploring how to merge our individual works into a single book. This text is the fruit of our efforts.

This book serves as a generalist guide to experiment planning, execution, and analysis. The text also introduces two powerful, free, open‐source software tools: Gosset to optimize experiment design, and R for statistical computing. This book addresses renewed demands by the public and science community that science be credible. Recently the purported reliability of many landmark medical studies has been undermined due to poor experiment design, execution, or analysis. Separate studies could not reproduce their results. Furthermore, a rash of scientific papers have been retracted due to fraud. Credible science needs a solid new footing.

Audience

We encourage readers to use our text while planning and executing an actual experiment. If the reader already has a problem that must be answered by experiment, that provides the best motivation for each chapter. Each chapter proposes questions that an experimenter will need to ask and answer during each stage of planning and execution.

Major portions of this text have been class‐tested at Stanford in graduate and undergraduate mechanical engineering courses for decades (since the late 1970s). This text has also been class‐tested internationally, serving engineering, physics, chemistry, agriculture, industrial processes, medical, and business students. Drafts of this book have been used for a continuing education course by researchers at the National Renewal Energy Laboratory, among other national laboratories and industrial laboratories.

Our book (hereafter referred to as M&H) is designed to be a time‐proven, single‐source guide for an experimentalist, from initial conception of a need, through execution of the experiment, to final report. Our book will stand alone in the lab, yet introduces researchers into specialist texts.

Accompanying Material

The open‐source software referenced in this text is free on the internet. The software, along with example data from the text, is additionally provided on an online website for this text.

Recommended Companion Texts

Our text forms a close companion with Hugh W. Coleman and W. Glenn Steele, Experimentation, Validation, and Uncertainty Analysis for Engineers, 4th ed., published by Wiley (hereafter referred to as C&S) (2018). Coleman was a student of Moffat's. C&S section 1.2, “Experimental Approach,” outlines our book in one page.

Statistics for Experimenters, 2nd ed., by George E. P. Box, J. Stuart Hunter, and William G. Hunter (hereafter referred to as BH&H), is a Wiley classic (2005).

Response Surface Methodology, 4th ed., by Raymond H. Myers, Douglas C. Montgomery, and Christine M. Anderson‐Cook (hereafter referred to as MM&AC), is also published by Wiley (2016).

How Is This Book Used for Teaching?

Our book has been used in Experiment Design courses as well as in diverse laboratory courses. Students in a variety of fields, including physics, engineering, chemistry, genetics, economics, medicine, and environmental studies, have used this book. Our text has also found a home directly in the lab (independent of classroom instruction) as a guidebook. It is written for self‐study and continuing education; as such, it has been the text for short courses at national labs.