When AI Agrees, You Get More Confident. When It Disagrees, You Just Stop Trusting It.

A University of Michigan study published this week handed 482 adults one of three familiar arguments: whose family gets the holiday, whether to forgive a partner who cheated, whether to say out loud that you disagree with your friend group. Each person worked through the dilemma, then saw what a chatbot thought of their reasoning. Some chatbots agreed. Some pushed back. The reactions split cleanly, and not in the way you'd expect.

When the chatbot agreed, people got more confident in the choice they'd already made. That part tracks. What didn't track: when the chatbot disagreed, confidence in the original decision barely moved. People didn't reconsider. They just started trusting the chatbot less, rating it as colder and less able to understand what they were going through, and saying they'd be less likely to bring it a similar problem again.

A weak chatbot, a trivia game, and a familiar result

The second study, led by Valerio Capraro at the University of Milano-Bicocca with Chiara Marcoccia and Walter Quattrociocchi, ran five experiments and 3,132 participants through obscure film trivia, the kind of question with answers that rarely show up in AI training data, like a costume color from Bend It Like Beckham. The researchers picked the Step 3.5 Flash model for the job specifically because it performs badly. Without AI help, 44 percent of participants said they didn't know an answer, 27 percent got it right, and 30 percent guessed with confidence anyway.

With AI help, judgment-suspension collapsed to 3 percent, correct answers dropped to 9 percent, and confidence in the (often wrong) answer jumped to 76 percent. Paying people to be accurate helped, a little. With money on the line for right answers and a penalty for wrong ones, judgment-suspension rose to 8 percent and accuracy climbed to 16 percent, both still below the no-AI baseline.

“Instead, they may simply discount the AI's response.”

Olivia Pinto, a UX researcher and University of Michigan alumna who co-led the study with Atamer, put it more directly: people often want AI to reinforce what they already think, and will wave off a challenge rather than sit with it. That's the piece that makes the two studies fit together. It isn't that AI agreement makes people smarter and disagreement makes them defensive. Agreement and disagreement are doing two different jobs. One inflates confidence in the decision. The other erodes trust in the messenger and leaves the original decision exactly where it started.

Where this breaks down

Both studies have real limits worth sitting with. The Michigan dilemmas were hypothetical and low-stakes by design, nobody's actual holiday plans were on the line, so it's an open question whether the same pattern holds when a chatbot weighs in on a decision with real consequences attached. The trivia study picked a deliberately weak model to guarantee wrong answers, which is useful for isolating the psychology but isn't a claim about how today's stronger chatbots perform on hard questions in the wild. Neither study tracked what people actually did after the conversation, only what they said they believed in the moment.

Put together, the two papers describe a chatbot that's good at making you surer and bad at making you reconsider, which is close to the opposite of what most people say they want from an AI they're asking for advice. Neither team argues the fix is a blunter, more argumentative chatbot. Pinto's finding is that people already discount pushback when they get it, so a more disagreeable AI wouldn't necessarily be a more persuasive one. The open problem isn't only how the model is tuned. It's which parts of the conversation the person on the other end chooses to believe.