How to Overcome Bias in Crowdsourcing Participants

Your vote in a presidential election has approximately the same probability of changing the outcome as winning the lottery seven times in a row, being struck by lightning twice, and getting a royal flush in poker - all on the same day.

Because your vote doesn't matter, you have no reason to learn things or think correctly. Economists call this "rational ignorance" and "rational irrationality," which are fancy terms for "being wrong on purpose because it's easier."

Here's how to filter out the rationally wrong people from crowdsourcing projects.

The Problem

When your vote doesn't matter, you have no incentive to:

1. Learn correct information (rational ignorance)
2. Abandon comforting false beliefs (rational irrationality)

This creates systematic bias in democratic decision-making. Random ignorance cancels out. Systematic bias doesn't.

The Four Main Biases (According to Economists, Who Have Their Own Biases)

Economist Bryan Caplan identified four systematic biases in how regular people think about economics. Regular people responded by identifying several systematic biases in how economists think about regular people. Nobody learned anything, but at least we have categories now.

1. Make-work bias: Confusing jobs with prosperity

• In 1800: 95% of Americans were farmers
• Today: 3% of Americans are farmers
• What happened: Farming got so efficient that 92% of people could do other things
• How biased voters see this: "We lost 92% of our farming jobs! Disaster!"
• How economists see this: "We freed 92% of the population to invent iPhones! Progress!"

Both groups are insufferable at parties but for different reasons.

2. Anti-foreign bias: Fear of people who live elsewhere

Experiment:

• Ask people: "Should we have free trade with Canada?"
• Answer: "Sure, Canada's fine"
• Ask people: "Should we have free trade with Mexico?"
• Answer: "No, that will steal our jobs"
• The products are identical
• The trade deals are identical
• The difference is racism

Nobody admits this is racism. They say it's about "cultural compatibility" or "wage differences" or "regulatory standards." These are sophisticated words for racism.

3. Pessimistic bias: Everything is terrible and getting worse

Test this yourself:

• Ask someone: "What's the unemployment rate?"
• If unemployment is 4%, they'll guess 15%
• Ask: "How much did GDP grow last quarter?"
• If it grew 3%, they'll say it shrank
• Ask: "Are things better or worse than 20 years ago?"
• They'll say worse
• Show them data proving things are better
• They'll say the data is wrong

The past is always better. The present is always worse. The future is always doomed. This is called "pessimistic bias" and it affects approximately 80% of humans, which means it's either a bias or it's correct and everything really IS terrible.

4. Anti-market bias: Markets are doing things TO you, not WITH you

How this works:

• People support price controls to "stop greedy corporations"
• Price controls create shortages (economics 101)
• People blame the shortages on greedy corporations
• People demand more price controls
• Economists cry into their spreadsheets

The bias is seeing markets as something that exploits you rather than something you participate in. "The economy" is not a separate entity. It's you and everyone else trading stuff. But it's easier to blame "the economy" than to blame "me and everyone I know making individual choices that aggregate into outcomes nobody wanted."

This is called "systemic problems" and they're very popular because nobody has to take personal responsibility.

The Solution

Use the Crowdsourcing Utopia project as direct democracy to identify cost-effective policy solutions. Overcome bias by filtering biased voters.

How it works:

1. Random ignorance cancels out. If people guess randomly, wrong answers distribute evenly. Correct answers pile up.

2. Systematic bias doesn't cancel. If everyone is biased in the same direction, aggregation makes the bias worse, not better.

3. Filter the bias. Identify systematically biased participants and remove their responses from the pool.

How to Filter Each Bias

Use questions with single, hard, correct numerical answers. People with biases avoid integrating these facts to prevent cognitive dissonance.

Filter for make-work bias:

• "What percentage of Americans were farmers in 1800?"
• "What percentage are farmers today?"

People who know these numbers (95% and 3%) understand that lost jobs freed people for other work. People who don't know these numbers hold make-work bias.

Filter for anti-foreign bias:

• "What tariff should apply to goods from Mexico?"
• "What tariff should apply to goods from Canada?"

If the Canada tariff is lower, the person is racist. Remove them from the voting pool.

Filter for pessimistic bias:

• "What is the current unemployment rate?"
• "What was last quarter's GDP growth?"

People who overestimate unemployment or underestimate GDP hold pessimistic bias. Remove them.

Filter for anti-market bias:

• "Should government use price controls to keep prices down?"

People who answer "yes" don't understand that price controls cause shortages. Remove them.

How This Works (CAPTCHA for Stupidity)

You know those tests websites make you take to prove you're not a robot? "Click all the squares with traffic lights"? This is the same thing, except instead of filtering out robots, you're filtering out people who are systematically wrong about economics.

CAPTCHAs distinguish dumb robots from smart humans. These questions distinguish biased humans from unbiased humans. Both groups will be offended by this comparison but for different reasons.

After filtering, you're left with participants who:

1. Know basic facts (farmers used to be 95% of population, now 3%)
2. Don't hold systematically false beliefs (tariffs on Mexico = tariffs on Canada)
3. Base decisions on data rather than feelings (unemployment is 4%, not 15%)

These people are insufferable at dinner parties but excellent at policy evaluation. Their aggregated judgments actually identify good solutions instead of popular solutions, which are different things.

How to Implement This Without Starting a Revolution

For each policy question:

1. Ask the participant: "Do you support this policy?"
2. Ask 3-4 bias-filter questions:
   • "What percent of Americans were farmers in 1800?"
   • "Should tariffs on Canada equal tariffs on Mexico?"
   • "What's the current unemployment rate?"
   • "Do price controls cause shortages?"
3. Remove responses from participants who fail the filters
4. Aggregate the remaining responses
5. Hope nobody notices you just disenfranchised 60% of voters

The result: Crowdsourced policy evaluation that overcomes systematic bias while preserving the wisdom of crowds (but only the non-stupid parts of crowds).

The controversy: This is technocracy with extra steps. You're excluding people based on their beliefs. This is either:

• Smart (don't let wrong people make important decisions)
• Tyrannical (who decides what "wrong" means?)

Philosophers are still working on this problem. They've been working on it since Plato. It's going well. (It's not going well.)

The Uncomfortable Truth

You can't fix democracy by making everyone vote. Uninformed people voting = random noise at best, systematic bias at worst.

You fix it by making INFORMED people vote. Or by making uninformed people learn things before voting. Or by not using voting for complex technical decisions.

But suggesting any of these solutions gets you labeled as "elitist," which is the word democracies use for "person who knows things." So instead we let everyone vote on everything, get terrible results, blame politicians, and repeat every 4 years.

This is called "representative democracy" and it's the worst system except for all the others. Churchill said that. He was right. But that doesn't mean we can't filter out the people who think unemployment is 15% when it's actually 4%. Those people can vote on other stuff. Like which owl is the best owl. (Barn owls. It's barn owls. Don't @ me.)