<\/p>\n
When government officials craft policies, there is often a comment period for public feedback. Lawmakers have long had to sort through fake feedback that is mass-produced in an effort to sway policy, but they\u2019ve generally gotten used to detecting it. With, artificial intelligence, however, people can submit realistic-seeming comments, which are harder to dismiss as being fabrications with no person behind them. That\u2019s just one way AI can be used to generate text and statements, in civic conversations, for which no one can be found to be answerable.<\/p>\n
This issue, among others, has caught the attention of Deb Roy, professor of media arts and sciences at MIT and director of the MIT Center for Constructive Communication (CCC). In a new essay published in the\u00a0Bulletin of the Atomic Scientists, titled, \u201cWho stands behind the words?,\u201d Roy calls for a renewed emphasis on responsibility in communication, asserting that even when AI is used to craft text, there needs to be accountability and \u201canswerability.\u201d<\/p>\n
Building better group dialogues to produce more public trust is a core part of the work of CCC. The group, along with Cortico, the closely affiliated nonprofit Roy co-founded, has helped facilitate group discussions on important issues in many situations \u2014 including civic meetings, schools, deliberative assemblies, and private organizations \u2014 and used tech tools, including AI, to help find themes among comments, and more. But the center also designs this process so that participants can check and override the tech-based output at each key step. Ultimately, a person is responsible for every public comment.\u00a0MIT News spoke to Roy about the essay and the work of the center.\u00a0<\/p>\n
Q:<\/strong> You\u2019ve identified this problem about the lack of accountability behind words that are generated by AI. When did you really first notice this?<\/p>\n A:<\/strong> It came into focus when Chat-GPT [which is powered by a large language model, or LLM] launched. As for my own history with language models, my PhD work explored what we might now call PLMs \u2014 Puny Language Models \u2014 so I\u2019m quite familiar with language models and have tracked the field\u2019s rapid evolution. But Chat-GPT turned a corner. Two things changed at the same time: the depth of the model and its knowledge, and the way you could interact with it.\u00a0<\/p>\n I became obsessed with trying to find the limits of what it could do. And I felt that no matter how hard I tried to stretch my arms around it, I couldn\u2019t. I built my tenure case at MIT on language grounding \u2014 the idea that it\u2019s not enough for an AI system to just process lots of words and other symbols. I believed that to understand what words mean, you actually need a physical body and life experiences as part of the process. So I revisited the philosophy-of-AI literature I had once been steeped in, which asks what fundamentally distinguishes the kind of system that large language models exemplify, in extraordinarily advanced form, from the kind of processing we humans engage in. I gave several presentations to my own graduate students trying to make the case that, even though AI seems so intelligent, it\u2019s actually different in kind \u2014 and I could not convince them.\u00a0<\/p>\n And I thought, I know the difference, but I\u2019m having trouble convincing even myself of what it is. And that led me to ask, \u201cWait a minute: Is there someone or something that stands behind the words, a human responsible for what is being said?\u201d That\u2019s the thing this extraordinary new language machine doesn\u2019t have.\u00a0<\/p>\n Whether these systems genuinely understand language is actively debated. But even if you grant them all the comprehension you like, they still lack standing. The buck can\u2019t stop at a thing, a software system, with nothing to lose.<\/p>\n Q:<\/strong> The Bulletin piece has these interesting examples of civic feedback forums where it looks like there are real people saying things, and instead you have a barrage of comments from AI. How did you find some of these other examples in society where things go off the rails in this way?\u00a0<\/p>\n A:<\/strong>\u00a0After the release of ChatGPT, I started hearing a concern from people inside government who review public comments on proposed rules: a suspicion that not all of the comments were written by humans.<\/p>\n My reaction was: That\u2019s going to delegitimize this channel for public feedback. That was several years ago, but everyone now knows this is happening. Public comment is a pillar of democracy. You don\u2019t just put a new rule in place. People need to have a voice and feel like they\u2019ve been heard before a rule is imposed on them. If you skip that step, people don\u2019t accept rules as legitimate. That\u2019s why that process is baked into how America works. And that pillar just cracked, and by now, in my mind, it\u2019s fallen over. Has anyone noticed? My sense is the people on the receiving end of these comments have quietly adjusted the weight they give to public comments. It was never perfect, but boy, now much of its value is being swept away.\u00a0<\/p>\n I open my essay with this example: In 2017, the Federal Communications Commission received more than 22 million public comments on its proposal to repeal net neutrality rules, and the New York attorney general later found that nearly 18 million of them were fake. As I write in the essay, \u201cThat fraud was caught because it left fingerprints: templated text, duplicated phrasing.\u201d But large language models erase the fingerprints. Now every comment can be fluent and different, and reviewers can no longer tell which ones express views that real people formed and stand behind.<\/p>\n If you have enough of the stuff that\u2019s fake \u2014 words that no one with a stake actually stands behind \u2014 it desensitizes you to what\u2019s real. That filters back to the public, and people realize: If I take the time to express myself, they\u2019re not even listening any more.<\/p>\n Q:<\/strong> Could you say a little bit more about your point that we need people with \u201cstanding,\u201d who will take \u201caccountability\u201d for all pieces of text, even when those bits of text are generated by AI?\u00a0<\/p>\n A:<\/strong> In the legal system, standing is a formal idea: sometimes a case isn\u2019t heard because the people bringing it don\u2019t have standing. But informally we use that term all the time. I\u2019m not trying to formalize standing in everyday life. Instead I\u2019m asking:\u00a0 What is the informal sense of the term, borrowed from the legal one, that we need to pay more attention to now? Here in New England, consider the open town meeting: The person who stands up to speak is generally expected to have standing, as someone from that town with a stake in what gets decided. And if it\u2019s a stranger who drove from across the state, or from out of state, everyone\u2019s like: Wait a minute, who is that? They\u2019re not from around here. In general, there\u2019s just enough friction and it\u2019s costly enough to show up somewhere you don\u2019t actually have standing, that it doesn\u2019t happen much.<\/p>\n Now LLMs and the current digital networked environment collapse those boundaries. We always took for granted that if someone has a voice, it\u2019s because they have standing. They have some basis to be heard. We need to pay attention to this, and really consider how to design systems so that the people with standing are the ones who get heard.\u00a0<\/p>\n The other idea I tried to develop is \u201canswerability.\u201d It isn\u2019t formalized in law, but it\u2019s baked into the legal process. If you have standing to give testimony, there you are, ready for cross-examination. You have to be able to answer for what you\u2019re saying. And if you\u2019re making a specific accusation about someone, you have to be willing to back it with facts.\u00a0Standing and answerability have to travel together: The right to be heard comes with the obligation to answer for what you say.<\/p>\n That doesn\u2019t mean everyone has to attach their name to everything. I write in the Bulletin essay, \u201cPrivacy and answerability aren\u2019t opposites. \u2026 An institution can verify a person and stand behind their words without exposing them.\u201d To voice a view, you only need to show that a distinct, eligible person is speaking. An accusation asks more: Someone trusted, like a reporter, has to be able to know who is speaking. As I say: \u201cDisclosure obligations should scale with power, and identity protections with vulnerability.\u201d<\/p>\n None of this means AI can\u2019t help.<\/strong> AI can do a lot of the work inside the chain of accountability \u2014 drafting, summarizing, finding patterns. If someone with a stake in a rule uses AI to draft a comment, adopts it as their own, and is ready to answer for it, that\u2019s legitimate, in this context. What AI can\u2019t do is anchor the chain. There has to be a person at the end who stands behind the words.<\/p>\n In the essay I lay out four design principles for building that in: \u201cattributable authorship,\u201d a \u201cdecomposed and inspectable process,\u201d the use of \u201cprovenance as infrastructure,\u201d and the need to \u201cwiden standing rather than concentrating it.\u201d As I write, the \u201canimating rule is that autonomy is permitted while unaccountability is not.\u201d And the metric that matters most isn\u2019t the efficiency gained by removing humans, but the number of people AI places into roles where they can answer for what they do.<\/p>\n Q:\u00a0<\/strong>There is a phrase in the documents of the Center for Constructive Communication, that we need things to move \u201cat the speed of trust,\u201d not at the \u201cspeed of tech.\u201d And that implies really thinking about building up these systems. Could you just say a little more about what that phrase means to you?<\/p>\n A:<\/strong> I learned that phrase when I was in Appleton, Wisconsin, with Katherine Cramer [of the University of Wisconsin], a longtime collaborator, talking with community organizers who were running \u201cDialogue across Divides\u201d conversations there. They had hundreds of people involved. I thought I was developing technology tools with exactly that same purpose, and was so excited to see it. And I asked one of the people who ran that organization, \u201cHow fast did you scale this?\u201d And he paused, and then he said, \u201cAt the speed of trust.\u201d\u00a0<\/p>\n And I took that to mean \u201cslow.\u201d Then I realized he was saying: For you, slow, because you haven\u2019t yet earned our trust. And I\u2019ve since learned that when you do have trust, things can move fast. So bringing in technology is really a process of engagement. It doesn\u2019t work to say, \u201cTrust us, we\u2019re doing this because we think it\u2019s for your own good.\u201d People will turn to someone they already trust and ask, \u201cIs this good for us?\u201d And if that person says \u201cYes,\u201d things can move fast.<\/p>\n Look at what\u2019s happening at the intersection of AI and society: The speed of AI and the speed of democracy are running on different clocks. Talk to people who work in democratic practice and they say, \u201cAI must slow down, because democracy can\u2019t move that fast.\u201d Talk to people in AI and they say, \u201cAI cannot slow down, democracy needs to speed up.\u201d If we can\u2019t get these two time scales in sync, we\u2019re headed for a breakway situation,\u00a0where AI gets deployed faster than democratic processes can respond, and decisions get made without the public. That\u2019s starting to happen right now.\u00a0<\/p>\n Q:\u00a0<\/strong>We need people actively doing things to build dialogue and figure out how tech tools can interact with that. At the Center for Constructive Communication, you\u2019ve done a few iterations of the center\u2019s Real Talk initiatives to build civic dialogues, as well as the Youth Voices Initiative in school settings, and currently you\u2019re working on the New Social Forms project. Could you say a few words about them?\u00a0<\/p>\n A:<\/strong> The Real Talk effort represents the longest-standing thread in this work. A huge emphasis of our work has been developing better ways to listen to one another. We create a context that is always small \u2014 small is beautiful here \u2014 where people are willing to share authentically. And we have the consent architecture and the technology to reveal the larger patterns, and then share them. We\u2019ve done this with hundreds of communities around the country. And closer to home, we\u2019ve brought the same approach to our campus here at MIT through the realtalk@MIT program, with support from the Office of the President and the Office of the Provost, to help our community build a culture around dialogue and listening.\u00a0<\/p>\n The second thing I\u2019d emphasize is that we\u2019ve been doing more and more work on deliberative processes and citizen assemblies, and on how to make them more accessible. What if a group of high schoolers says, \u201cWe have a real problem with our cellphone policy?\u201d Suppose you want to have an assembly that brings students, teachers, and administrators together, not just to listen to people\u2019s concerns but to make policy recommendations. So we did that. The center worked with a high school in New York City, where technology helped people reveal their preferences and differences and listen to each other.<\/p>\n We\u2019re also building answerability into the technology. A prototype called VoiceTrace, developed by PhD student Cassandra Overney in our group, links every output statement \u2014 a summary of what was heard in a listening process, or a recommendation coming out of a deliberation \u2014 to the specific voices behind it, so any claim can be traced back to the conversations it came from.<\/p>\n The New Social Forms project pertains to a very interesting pattern emerging in workplaces around the world, certainly the ones we\u2019re connected to. AI is an extraordinary tool for individual empowerment, but in the words of one of our collaborators at the Google Experience Institute, people are tunneling. They go deep, using AI to develop refined ideas, but when they surface, their ideas have diverged from their teammates\u2019. So groups and teams trying to work together with AI, there\u2019s a human coordination problem.<\/p>\n We\u2019re going to expand the initial work we\u2019ve started with Google because there\u2019s a lot of interest in collective communication, trust, and interaction among people trying to work together.\u00a0<\/p>\n So is that a civic context? It\u2019s a workplace, but workplaces are civic spaces, too.\u00a0They govern people\u2019s daily lives, they\u2019re where many of us build civic skills and learn to cooperate across difference, and there\u2019s a long tradition of civic life rooted in shared work.\u00a0We\u2019re all humans trying to interact with each other. These are the same issues. Justice Louis Brandeis once used the phrase \u201cindustrial democracy\u201d to capture this. When workplaces lacked certain kinds of civic processes and civility \u2014 two interestingly related terms \u2014 labor disputes used to turn violent. Procedural justice, mediators, and people hearing one another are just as necessary in the workplace. In the end, it\u2019s all about human beings.\u00a0Hearing the humanity in one another is what democracy depends on, and it\u2019s just as necessary for communities and organizations to function.<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":" When government officials craft policies, there is often a comment period for public feedback. Lawmakers have long had to sort through fake feedback that is mass-produced in an effort to sway policy, but they\u2019ve generally gotten used to detecting it. With, artificial intelligence, however, people can submit realistic-seeming comments, which are harder to dismiss as […]<\/p>\n","protected":false},"author":1,"featured_media":4627,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[],"class_list":["post-4626","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tech-news"],"_links":{"self":[{"href":"https:\/\/smartinvestingschronicle.com\/index.php\/wp-json\/wp\/v2\/posts\/4626","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/smartinvestingschronicle.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/smartinvestingschronicle.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/smartinvestingschronicle.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/smartinvestingschronicle.com\/index.php\/wp-json\/wp\/v2\/comments?post=4626"}],"version-history":[{"count":0,"href":"https:\/\/smartinvestingschronicle.com\/index.php\/wp-json\/wp\/v2\/posts\/4626\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/smartinvestingschronicle.com\/index.php\/wp-json\/wp\/v2\/media\/4627"}],"wp:attachment":[{"href":"https:\/\/smartinvestingschronicle.com\/index.php\/wp-json\/wp\/v2\/media?parent=4626"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/smartinvestingschronicle.com\/index.php\/wp-json\/wp\/v2\/categories?post=4626"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/smartinvestingschronicle.com\/index.php\/wp-json\/wp\/v2\/tags?post=4626"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}