Be careful what you learn from your tests →

Pilot programs and other trials can often lead us to the wrong conclusions, Megan McArdle warns:

This is one more installment in a continuing series, brought to you by the universe, entitled “promising pilot projects often don’t scale”.  They don’t scale for corporations, and they don’t scale for government agencies.  They don’t scale even when you put super smart people with expert credentials in charge of them.  They don’t scale even when you make sure to provide ample budget resources.  Rolling something out across an existing system is substantially different from even a well run test, and often, it simply doesn’t translate.

¶