Muha v. Encore Receivable Mgmt., Inc. 2007 U.S. Dist. LEXIS 74801(E.D. Wis. Sept. 28, 2007) is a case study in how not to prepare survey evidence. In this FDCPA class action, the plaintiffs offered a consumer survey as extrinsic evidence of consumer confusion. District Judge J. P. Stadtmueller found the survey unreliable and irrelevant because, inter alia:

The design of the survey questions is neither scientific nor impartial. Plaintiffs' expert did not design the questions, rather plaintiffs' counsel did. *** Plaintiffs' counsel does not describe his experience or qualifications in designing surveys, so it comes as no surprise that he has not himself been tendered as an expert. Moreover, the plaintiffs expressly concede that their expert, Kotecki, cannot testify with respect to the design of the questions. (See Pls.' Response Br. in Opp'n to Def.'s Daubert Mot. at 4 ("Plaintiffs put Mr. Kotecki forward as an expert witness only as to survey methodology.")). If a layperson, like plaintiffs' counsel or any six-year-old for that matter, drafts a survey and an expert flawlessly conducts the survey, is the survey itself scientific or expert evidence?

Among the flaws in counsel's work: (1) the survey question did not mirror the exact language from the collection letters that allegedly confused the plaintiffs; (2) the question provided none of the surrounding text found in the defendant’s letter, which made it "perfectly obvious to even the dimmest debtor that the debt collector would very much like him to pay the amount demanded straight off, sparing the debt collector any further expense;" (3) the closed-end question drafted by counsel could be answered only by selecting one of four counsel-prepared responses, which were worded in such a way that ‛no matter how participants answered the question, the plaintiffs could claim that the participants were misled or confused. Plaintiffs' counsel drafted three false or awkwardly worded answers and then gave participants a fourth option of declaring their uncertainty.“ Held, ‛Although no survey may be perfect, the court need not and should not respond reflexively to every criticism by saying it merely 'goes to the weight' of the survey rather than to its admissibility“ (internal quotes and brackets omitted).