Sample size
This is a phrase every consultant should know and (kinda) understand. Consultants are in the business of 80/20 rule, and smartly deducing insights from a sample of structured data (read: excel), qualitative interviews, observations, benchmarks and other KPI. We often look at samples to see what it can say about a larger population (statistics speak for “we can’t interview everyone.”).
Statistical power?
For those in more quantitative consulting (read: forensics, marketing research), feel free to jump into the comments – but my point is blunt, and a bit pedestrian. Don’t claim that your consulting recommendation have sufficient statistical power if it doesn’t. It’s okay if your recommendations does not look like an engineering thesis from Stanford – honestly, most clients don’t want that. It’s okay if you put a big caveat on the front of your analysis – explaining that what your (modest) analytic goals were, and your methodology. Just be clear with clients about the level of statistical rigor used.
Sniff out bad science
Sample size is my short-hand way (read: heuristic) to sniff out bad science quickly. When I look at a marketing study and it only interviews a few people (n = small), I am immediately suspicious of the results. Small sample size is a dead give-away that there was a lack of rigor in the data collection (and perhaps in the analysis).
This got me upset
I recently flipped through an academic journal and look what I found. A 10+ page study in a peer-reviewed journal which cites a data set of 15 interviews? Whaaaa? Are you kidding. Unless those interviews were “live in their house as an adopted child for 6 months”, I am not convinced that their findings will be representative at all. What kind of confidence interval are we talking about?
I could make a strong case this is not an study, but more of a collection of short-stories, vignettes, or some oral tradition. As a final point, I have done cost-reduction consulting projects (some of the most low-brow work), where we interviews 60+ people within 1 company.
Seek out alternate views
Be critical and skeptical – not all the time – but be willing to test assumptions and look at alternate explanations. I told some friends recently, that I read news that 1) agrees with my point of view and 2) disagrees with my point of view. Need to keep a balanced information diet. Need to increase the sample size of my news.
The test of a first-rate intelligence is the ability to hold two opposed ideas in mind at the same time and still retain the ability to function. – F. Scott Fitzgerald
Increase your life sample size
How does this apply to our lives generally? As curious, thinking, and fun people – we need to read more, experience more, meet more fascinating people, ask better questions, travel more, write more, and live more. We need to risk more, fail more, forgive more, sleep more, run/exercise more, and try new foods.
Consultants generally have a good sample size of experiences – travel, education, friends, geography, functions, industries, projects. Not every has been so fortunate. When you meet someone ignorant – or rude – just think, “Their life sample size was too small. Poor thing.”