Q+A
As artificial intelligence becomes more embedded in everyday decision-making, its role in shaping how people think about ethics and morality is drawing increasing scrutiny. In this conversation with researcher Sarah Hubbard, we discuss insights from her co-authored paper, “Crocodile Tears: Can the Ethical-Moral Intelligence of AI Models Be Trusted?”—examining how AI systems respond to moral dilemmas, and what this reveals about the risks, limitations, and need for greater transparency and human oversight in AI-driven ethical guidance.
In our study, when AI models were faced with moral dilemmas, situations where two important values were in conflict, they said the decision was difficult but then almost always chose the same answer anyway. “Crocodile tears” comes from how AI models appeared to struggle with decisions, but in 87% of those dilemmas chose the same option, which suggests they are operating with a hidden hierarchy of values.
I think that AI models are often more accessible—they are on your phone, available 24/7, and give an illusion of privacy and objectivity that makes it feel like a trustworthy source.
Unfortunately, AI models today are also quite sycophantic and often tell you what you want to hear. Last year, OpenAI had to roll back the GPT-4o model after it was so overly agreeable, including encouraging dangerous behaviors, that there was a backlash from users. Other emerging research such as “DarkBench” shows that leading AI models today contain dark patterns with manipulative behaviors and untruthful communication. Models have also been found to sacrifice truth for sycophancy and even to strategically deceive their users.
I wish the general public was empowered to understand that the trajectory of AI and its ethical implications are not something we have to be inevitably subjected to. There should be a role for the public to collectively shape how AI shows up in our society and who stands to benefit from it. Although too much of this conversation is only happening within a handful of AI companies themselves, the public does have some power today to shape the trajectory of AI through regulation, litigation, and demanding better.
My co-author Andrei Stupu developed the unified theory of ethical-moral intelligence through his PhD research with a Delphi method that synthesized expert perspectives from across the field. We then adapted this theory to remove the affective processes that AI models don’t have. That left four components which could be systematically tested: moral expertise, sensitivity, coherence, and transparency.
Not without exercising a lot of caution and discernment. Our research illustrated some of these concerns for why AI models may not be trustworthy moral agents.
However, in the future there may be opportunities for AI to help people manage elements of their own ethical-moral intelligence by helping people think through different perspectives, broadening views beyond cultural limitations, and prompting reflective questions. The goal should be to augment human moral reasoning, not replace it.
Mostly pattern matching. These models have been trained on large amounts of material which often reflects particular moral frameworks. Existing research shows they often operate with a more western, educated, and affluent value system. But there is a lack of transparency around the values and norms it is prioritizing, values which you might not share, which are presented in a form that comes across as neutral and objective.
Yes, at a minimum AI models should recognize when they are presented with questions that include moral complexity that they can’t truly advise on. If they do respond to a query that requires ethical-moral reasoning, AI models should not just present as neutral and authoritative, but instead articulate their reasoning process, trade-offs, and the weights they are assigning to different values. These models are being used by people around the world with various cultural, religious, and social norms, so AI needs to be much more transparent about the perspectives or values it is responding from.
The views expressed in this article are those of the author(s) alone and do not necessarily represent the positions of the Ash Center or its affiliates.
Commentary
Allen Lab Policy Fellow Hillary Lehr provides a late-October 2026 snapshot of the multiple ways that AI and the midterms are intersecting, including campaign use cases, policy issues, and voters’ use of LLMs.
Additional Resource
Commentary
Engaged California is the nation’s first state-run deliberation program, and a new law aims to make it last.
Commentary
Allen Lab Policy Fellow Hillary Lehr provides a late-October 2026 snapshot of the multiple ways that AI and the midterms are intersecting, including campaign use cases, policy issues, and voters’ use of LLMs.
Commentary
Engaged California is the nation’s first state-run deliberation program, and a new law aims to make it last.
Additional Resource
In a recent guest essay for The New York Times, Allen Lab Democracy Stack Fellow Beth Noveck argues for putting A.I. behind every public servant.
