When AI Can Fake Realit

It’s getting harder， isn’t it， to spot real from fake， AI-generated from human- generated. With generative AI，along with other advances in deep fakery， it doesn’t takemany seconds of your voice， many images of your face， tofake you， and the realism keeps increasing.

I first started working on deepfakes in 2017， when thethreat to our trust in information was overhyped， and thebig harm， in reality， was falsified sexual images. Now thatproblem keeps growing， harming women and girls world-wide.

But also， with advances in generative AI， we’re nowalso approaching a world where it’s broadly easier to makefake reality， but also to dismiss reality as possibly faked.

Now， deceptive and malicious audiovisual AI is not theroot of our societal problems， but it’s likely to contribute tothem. Audio clones are proliferating in a range of electoralcontexts.“Is it， isn’t it”claims cloud human- rights evi-dence from war zones， sexual deepfakes target women inpublic and in private， and synthetic avatars impersonatenews anchors.

I lead WITNESS. We’re a human- rights group thathelps people use video and technology to protect and de-fend their rights. And for the last five years， we’ve coordi-nated a global effort，“Prepare， Don’t Panic，”around thesenew ways to manipulate and synthesize reality， and on howto fortify the truth of critical frontline journalists and hu-man-rights defenders.

Now， one element in that is a deepfakes rapid-responsetask force， made up of media-forensics experts and compa-nies who donate their time and skills to debunk deepfakesand claims of deepfakes. The task force recently receivedthree audio clips， from Sudan， West Africa and India. Peo-ple were claiming that the clips were deepfaked， not real.

In the Sudan case， experts used a machine-learning al-gorithm trained on over a million examples of syntheticspeech to prove， almost without a shadow of a doubt， that itwas authentic.

In the West Africa case， they couldn’t reach a defini-tive conclusion because of the challenges of analyzing au-dio from Twitter， and with background noise.

The third clip was leaked audio of a politician fromIndia. Nilesh Christopher of“Rest of World”brought thecase to the task force. The experts used almost an hour ofsamples to develop a personalized model of the politician’sauthentic voice. Despite his loud and fast claims that it wasall falsified with AI， experts concluded that it at least waspartially real， notAI.

As you can see， even experts cannot rapidly and con-clusively separate true from false， and the ease of calling“that’s deepfaked”on something real is increasing. The fu-ture is full of profound challenges， both in protecting the re-al and detecting the fake.

We’re already seeing the warning signs of this chal-lenge of discerning fact from fiction. Audio and video deep-fakes have targeted politicians， major political leaders in theEU， Turkey and Mexico， and US mayoral candidates.

Political ads are incorporating footage of events thatnever happened， and people are sharing AIWV8BdctQjjK36+Wfu4gBGg6M+lvscX9JkDarUG1Zylg=-generated imag-ery from crisis zones， claiming it to be real.

Now， again， this problem is not entirely new. The hu-man-rights defenders and journalists I work with are used tohaving their stories dismissed， and they’re used to wide-spread， deceptive， shallow fakes， videos and images takenfrom one context or time or place and claimed as if they’rein another， used to share confusion and spread disinforma-tion.

And of course， we live in a world that is full of parti-sanship and plentiful confirmation bias. Given all that， thelast thing we need is a diminishing baseline of the shared，trustworthy information upon which democracies thrive，where the specter of AI is used to plausibly believe thingsyou want to believe， and plausibly deny things you want toignore.

But I think there’s a way we can prevent that future， ifwe act now; that if we "Prepare， Don’t Panic，" we’ll kindof make our way through this somehow. Panic won’t serveus well. [It] plays into the hands of governments and corpo-rations who will abuse our fears， and into the hands of peo-ple who want a fog of confusion and will use AI as an ex-cuse.

How many people were taken in， just for a minute， bythe Pope in his dripped-out puffer jacket？ You can admit it.

More seriously， how many of you know someone who’sbeen scammed by an audio that sounds like their kid？ Andfor those of you who are thinking“I wasn’t taken in， Iknow how to spot a deepfake，”any tip you know now is al-ready outdated. Deepfakes didn’t blink， they do now. Six-fingered hands were more common in deepfake land thanreal life — not so much.

Technical advances erase those visible and audibleclues that we so desperately want to hang on to as proof wecan discern real from fake. But it also really shouldn’t beon us to make that guess without any help. Between realdeepfakes and claimed deepfakes， we need big- picture，structural solutions.

We need robust foundations that enable us to discernauthentic from simulated， tools to fortify the credibility ofcritical voices and images， and powerful detection technolo-gy that doesn’t raise more doubts than it fixes. There arethree steps we need to take to get to that future. Step one isto ensure that the detection skills and tools are in the handsof the people who need them.

I’ve talked to hundreds of journalists， community lead-ers and human-rights defenders， and they’re in the sameboat as you and me and us. They’re listening to the audio，trying to think， “Can I spot a glitch？”Looking at the image，saying， “Oh， does that look right or not？”O(jiān)r maybe they’regoing online to find a detector.

And the detector they find， they don’t know whetherthey’re getting a false positive， a false negative， or a reli-able result. Here’s an example. I used a detector， which gotthe Pope in the puffer jacket right. But then， when I put inthe Easter bunny image that I made for my kids， it said thatit was human-generated. This is because of some big chal-lenges in deepfake detection.

Detection tools often only work on one single way to make a deepfake， so you need multiple tools， and they don’twork well on low-quality social media content. Confidencescore， 0.76-0.87， how do you know whether that’s reliable，if you don’t know if the underlying technology is reliable，or whether it works on the manipulation that is being used？And tools to spot an AI manipulation don’t spot a man-ual edit. These tools also won’t be available to everyone.

There’s a trade- off between security and access， whichmeans if we make them available to anyone， they becomeuseless to everybody， because the people designing the newdeception techniques will test them on the publicly avail-able detectors and evade them.

But we do need to make sure these are available to thejournalists， the community leaders， the election officials，globally， who are our first line of defense， thought throughwith attention to real-world accessibility and use. Though atthe best circumstances， detection tools will be 85 to 95 per-cent effective， they have to be in the hands of that first lineof defense， and they’re not， right now.

So for step one， I’ve been talking about detection afterthe fact. Step two — AI is going to be everywhere in ourcommunication， creating， changing， editing. It’s not goingto be a simple binary of“yes， it’s AI”or“phew， it’s not.”AI is part of all of our communication， so we need to betterunderstand the recipe of what we’re consuming.

Some people call this content provenance and disclo-sure. Technologists have been building ways to add invisi-ble watermarking to AI- generated media. They’ve alsobeen designing ways -- and I’ve been part of these efforts --within a standard called the C2PA， to add cryptographically signed metadata to files.

This means data that provides details about the con-tent， cryptographically signed in a way that reinforces ourtrust in that information. It’s an updating record of how AIwas used to create or edit it， where humans and other tech-nologies were involved， and how it was distributed. It’s ba-sically a recipe and serving instructions for the mix of AIand human that’s in what you’re seeing and hearing.

And it’s a critical part of a new AI-infused media liter-acy. And this actually shouldn’t sound that crazy. Our com-munication is moving in this direction already. If you’relike me — you can admit it — you browse your TikTok“For You”page， and you’re used to seeing videos that havean audio source， an AI filter， a green screen， a background，a stitch with another edit.

This， in some sense， is the alpha version of this trans-parency in some of the major platforms we use today. It’sjust that it does not yet travel across the internet， it’s not re-liable， updatable， and it’s not secure. Now， there are alsobig challenges in this type of infrastructure for authenticity.

As we create these durable signs of how AI and humanwere mixed， that carry across the trajectory of how media ismade， we need to ensure they don’t compromise privacy orbackfire globally.

We have to get this right. We can’t oblige a citizenjournalist filming in a repressive context or a satirical mak-er using novel gen-AI tools to parody the powerful ... tohave to disclose their identity or personally identifiable in-formation in order to use their camera or ChatGPT.

Because it’s important they be able to retain their abili-ty to have anonymity， at the same time as the tool to createis transparent. This needs to be about the how of AI-humanmedia making， not the who.

This brings me to the final step. None of this workswithout a pipeline of responsibility that runs from the foun-dation models and the open-source projects through to theway that is deployed into systems， APIs and apps， to theplatforms where we consume media and communicate.

I’ve spent much of the last 15 years fighting， essential-ly， a rearguard action， like so many of my colleagues in thehuman rights world， against the failures of social media. Wecan’t make those mistakes again in this next generation oftechnology. What this means is that governments need to en-sure that within this pipeline of responsibility for AI， thereis transparency， accountability and liability.

Without these three steps — detection for the peoplewho need it most， provenance that is rights-respecting andthat pipeline of responsibility， we’re going to get stucklooking in vain for the six-fingered hand， or the eyes thatdon’t blink. We need to take these steps. Otherwise， we riska world where it gets easier and easier to both fake realityand dismiss reality as potentially faked.

And that is a world that the political philosopher Han-nah Arendt described in these terms：“A people that no lon-ger can believe anything cannot make up its own mind. It isdeprived not only of its capacity to act but also of its capaci-ty to think and to judge. And with such a people you canthen do what you please.”That’s a world I know none ofus want， that I think we can prevent.

Thank you.

識(shí)別真假 — —即識(shí)別 AI 生成與人類生成的內(nèi)容 — —變得愈發(fā)困難，是吧？利用生成式人工智能，加上深度偽造技術(shù)的其他進(jìn)展，只需幾秒聲音、幾張臉部圖像，就能偽造你的身份，而且逼真程度不斷提升。

我于2017年開(kāi)始研究深度偽造技術(shù)，當(dāng)時(shí)信息信任遭遇的威脅被夸大，而實(shí)際上造成巨大傷害的是虛假的情色圖片。如今這個(gè)問(wèn)題愈演愈烈，傷害著全球的婦女和女童。

不過(guò)，隨著生成式人工智能的進(jìn)步，我們的世界現(xiàn)在不僅變得更容易偽造現(xiàn)實(shí)，也更容易將現(xiàn)實(shí)視為可能是偽造的。

現(xiàn)在，欺騙性和惡意的視聽(tīng)人工智能并不是我們社會(huì)問(wèn)題的根源，但它可能會(huì)為這些問(wèn)題添油加醋。各種選舉環(huán)境中，音頻克隆激增。 “是，不是”的爭(zhēng)議模糊了戰(zhàn)區(qū)的人權(quán)證據(jù)，情色深度偽造瞄準(zhǔn)了公開(kāi)和私密場(chǎng)合的婦女，合成頭像則冒充新聞主播。

我領(lǐng)導(dǎo)著一個(gè)名為“目擊者”的組織。我們是一個(gè)人權(quán)組織，幫助人們運(yùn)用視頻和技術(shù)來(lái)保護(hù)及捍衛(wèi)自己的權(quán)利。過(guò)去五年，我們圍繞這些操縱和合成現(xiàn)實(shí)的新方法及如何鞏固重要前線記者和人權(quán)捍衛(wèi)者的真相，協(xié)調(diào)開(kāi)展了一項(xiàng)全球活動(dòng)，名為“準(zhǔn)備好，不要慌”。

其中之一是深度偽造快速應(yīng)對(duì)特別小組，由媒體取證專家和公司組成，他們貢獻(xiàn)時(shí)間和技能來(lái)揭穿深度偽造及其相關(guān)主張。小組最近收到了來(lái)自蘇丹、西非和印度的三段音頻剪輯。人們聲稱這些剪輯是深度偽造的，不是真實(shí)的。

針對(duì)蘇丹的那段音頻，專家使用了一種經(jīng)由100多萬(wàn)個(gè)合成語(yǔ)音樣本訓(xùn)練的機(jī)器學(xué)習(xí)算法，幾乎毫無(wú)疑問(wèn)地證明了其真實(shí)性。

西非音頻，由于分析來(lái)自推特的音頻困難重重，再加上背景噪音的影響，專家無(wú)法得出明確結(jié)論。

第三段剪輯是印度一個(gè)政客泄露的音頻?？萍济襟w Rest of World 的尼勒什·克里斯托弗將它提交給了特別小組。專家使用了近一個(gè)小時(shí)的樣本來(lái)開(kāi)發(fā)該政客真實(shí)聲音的個(gè)性化模型。盡管他極力聲稱這全是 AI 偽造的，但專家們最終得出結(jié)論，至少部分內(nèi)容是真實(shí)的，不是 AI 偽造的。

正如你們所見(jiàn)，即使是專家也無(wú)法快速而確切地區(qū)分真假，而且愈加容易將真實(shí)的東西說(shuō)成“那是深度偽造的”。未來(lái)充滿了巨大的挑戰(zhàn)，既要保護(hù)真實(shí)，又要識(shí)別偽造。

我們已經(jīng)看到了辨別事實(shí)和虛構(gòu)這一挑戰(zhàn)的預(yù)警信號(hào)。音視頻深度偽造已經(jīng)瞄準(zhǔn)了政客，歐盟、土耳其和墨西哥的主要政治領(lǐng)袖，以及美國(guó)的市長(zhǎng)候選人。

政治廣告在使用從未發(fā)生過(guò)的事件的素材，人們?cè)诜窒韥?lái)自危機(jī)區(qū)域的 AI 生成圖像且聲稱這樣的圖像是真實(shí)的。

再?gòu)?qiáng)調(diào)一遍，這個(gè)問(wèn)題早已有之。與我合作的人權(quán)捍衛(wèi)者和記者已經(jīng)習(xí)慣了他們的報(bào)道被打回，習(xí)慣了廣泛存在的欺騙性的膚淺假新聞 — —從一個(gè)背景、時(shí)間或地點(diǎn)獲取的視頻和圖像被聲稱屬于另一個(gè)背景、時(shí)間或地點(diǎn)，用來(lái)制造混淆和傳播虛假信息。

當(dāng)然，我們生活在一個(gè)充滿黨派偏見(jiàn)和大量確認(rèn)偏見(jiàn)的世界。鑒于這一切，我們最不希望看到的就是共享可靠信息的基準(zhǔn)線不斷降低，而那樣的信息是民主制度蓬勃發(fā)展所需的 — —在此，AI 這個(gè)幽靈被用來(lái)讓你理直氣壯地相信你想相信的事情，理直氣壯地否認(rèn)你想忽視的事情。

但我認(rèn)為，如果我們立即采取行動(dòng)，我們有辦法防止這樣的未來(lái)成真；如果可以“準(zhǔn)備好，不要慌”，我們將會(huì)以某種方式渡過(guò)這個(gè)難關(guān)?？只艑?duì)我們毫無(wú)裨益。（它）正是會(huì)濫用我們恐懼的政府和企業(yè)需要的，是那些想制造混亂并會(huì)把 AI 作為借口的人需要的。

有多少人只是一時(shí)被教皇穿著時(shí)髦羽絨服所迷惑了？你可以承認(rèn)。更嚴(yán)重的是，有多少人知道有人被聽(tīng)起來(lái)像自己孩子的音頻所欺騙了？對(duì)那些認(rèn)為“我沒(méi)有被騙，我知道如何辨別深度偽造”的人來(lái)說(shuō)，現(xiàn)在知道的任何技巧都已經(jīng)過(guò)時(shí)了。深度偽造以前不會(huì)眨眼，現(xiàn)在會(huì)了。在深度偽造的世界中，六指手比現(xiàn)實(shí)生活中更常見(jiàn) — —其實(shí)并非如此。

技術(shù)進(jìn)步抹去了那些我們渴望用來(lái)證明我們能夠區(qū)分真假的可見(jiàn)和可聽(tīng)的線索。但我們也不該沒(méi)有任何幫助就做出那種猜測(cè)。在真實(shí)的深度偽造和所謂的深度偽造之間，我們需要全局性、結(jié)構(gòu)性的解決方案。

我們需要堅(jiān)實(shí)的基礎(chǔ)使我們能夠辨別真實(shí)與模擬，需要工具來(lái)加強(qiáng)關(guān)鍵聲音和圖像的可信度，需要強(qiáng)大的檢測(cè)技術(shù)且不會(huì)引起更多懷疑。我們需要采取三個(gè)步驟來(lái)邁向這樣的未來(lái)。第一步是確保那些檢測(cè)技術(shù)和工具掌握在需要它們的人手中。

我與數(shù)百名記者、社區(qū)領(lǐng)袖和人權(quán)捍衛(wèi)者進(jìn)行過(guò)交流，他們和你、我及我們的處境一樣。他們聽(tīng)音頻時(shí)會(huì)想“我能發(fā)現(xiàn)一個(gè)錯(cuò)誤嗎？”，看圖像時(shí)會(huì)說(shuō)“哦，這看起來(lái)對(duì)勁兒?jiǎn)幔俊??；蛟S，他們會(huì)上網(wǎng)找個(gè)檢測(cè)器。

他們能找到檢測(cè)器，但他們不知道自己得到的檢測(cè)結(jié)果是錯(cuò)誤肯定、錯(cuò)誤否定還是確實(shí)可靠的。

舉個(gè)例子，我用一個(gè)檢測(cè)器成功識(shí)別了教皇穿著羽絨服的圖像。但是，同一個(gè)檢測(cè)器，當(dāng)我輸入自己為孩子制作的復(fù)活節(jié)兔子的圖像時(shí)，它卻說(shuō)是人工生成的。這是由于深度偽造檢測(cè)中存在的一些重大挑戰(zhàn)。

檢測(cè)工具通常一種工具只針對(duì)一種深度偽造方法，因此需要多種工具來(lái)進(jìn)行檢測(cè)，并且它們?cè)诘唾|(zhì)量社交媒體內(nèi)容上效果不佳。置信度得分為0.76-0.87，如果不知道底層技術(shù)是否可靠，或者它是否適用于正在使用的操作，怎么知道它是否可靠呢？識(shí)別AI操作的工具無(wú)法識(shí)別人工編輯。這些工具也不會(huì)提供給所有人使用。在安全性和可用性之間存在權(quán)衡，這意味著如果我們讓所有人都可以使用它們，它們對(duì)每個(gè)人都會(huì)變得無(wú)用了，因?yàn)樵O(shè)計(jì)新欺騙技術(shù)的人會(huì)測(cè)試公開(kāi)可用的檢測(cè)器并規(guī)避其檢測(cè)。

但我們確實(shí)需要確保這些工具可以提供給全球的記者、社區(qū)領(lǐng)袖、選舉官員等相當(dāng)于第一道防線的人，并且要考慮現(xiàn)實(shí)世界的可訪問(wèn)性和使用情況。盡管在最佳情況下，檢測(cè)工具的有效性會(huì)達(dá)到85%到95%，但它們必須掌握在第一道防線的手中，而現(xiàn)在卻沒(méi)有。

因此，第一步，我一直在談?wù)撌潞髾z測(cè)。第二步，在我們的溝通、創(chuàng)造、改變和編輯中，AI 將無(wú)處不在。這不會(huì)是一個(gè)簡(jiǎn)單的二元選擇， “是 AI ”或“不是 AI ”。AI 是我們所有溝通的一部分，因此我們需要更好地了解我們所消費(fèi)內(nèi)容的配方。

有些人稱之為內(nèi)容來(lái)源和披露。技術(shù)人員一直在構(gòu)建方法來(lái)為 AI 生成的媒體添加隱形水印。

他們還在設(shè)計(jì)方式 — —我參與了這些工作 — —在一個(gè)名為C2PA的標(biāo)準(zhǔn)內(nèi)，為文件添加加密簽名的元數(shù)據(jù)。

這意味著提供有關(guān)內(nèi)容詳細(xì)信息的數(shù)據(jù)，以一種加密方式簽名，可以加強(qiáng)我們對(duì)該信息的信任。

它是使用 AI 創(chuàng)建或編輯內(nèi)容的更新記錄，涉及人類和其他技術(shù)的參與方式，以及內(nèi)容的分發(fā)方式。

基本上，它是你所看到和聽(tīng)到的融合 AI 和人類的配方和使用說(shuō)明。

這是新型 AI 媒體素養(yǎng)的關(guān)鍵組成部分。實(shí)際上，這聽(tīng)起來(lái)不應(yīng)該那么瘋狂。我們的溝通已經(jīng)朝著這個(gè)方向發(fā)展了。如果你和我一樣 — —你可以承認(rèn) — —你會(huì)瀏覽 TikTok 的“推薦”頁(yè)面，你已經(jīng)習(xí)慣看到的視頻有音頻來(lái)源、AI 濾鏡、綠幕、背景、與其他編輯的組合。

在我們今天使用的一些主要平臺(tái)上，這在某種意義上是這種透明度的初始版本。只是它尚未在互聯(lián)網(wǎng)上傳播，不可靠、不可更新，也不安全?，F(xiàn)在，這種確保真實(shí)性的基礎(chǔ)設(shè)施也存在著重大挑戰(zhàn)。

當(dāng)我們創(chuàng)造這些持久的標(biāo)記來(lái)說(shuō)明 AI 和人類的融合方式時(shí)，這些標(biāo)記會(huì)在媒體制作過(guò)程中傳播，我們需要確保它們不會(huì)損害隱私或在全球產(chǎn)生逆轉(zhuǎn)效果。

我們必須搞清楚這一點(diǎn)。我們不能強(qiáng)迫在壓迫環(huán)境中拍攝的人民記者或使用新穎的生成式 AI工具來(lái)戲仿強(qiáng)權(quán)的諷刺創(chuàng)作者……必須披露他們的身份或個(gè)人身份信息才能使用攝像頭或 ChatG-PT。

因?yàn)橹匾氖?，他們能夠保持匿名，同時(shí)能夠透明地創(chuàng)作。應(yīng)該關(guān)注的是 AI 和人類媒體制作的方式，而不是制作者的身份。

這就引出了最后一步。如果沒(méi)有從基礎(chǔ)模型和開(kāi)源項(xiàng)目到部署到系統(tǒng)、API 和應(yīng)用程序，再到我們消費(fèi)媒體和進(jìn)行溝通的平臺(tái)的責(zé)任鏈，所有這些都不會(huì)起作用。

過(guò)去15年，和很多人權(quán)界的同事一樣，我花了很多時(shí)間戰(zhàn)斗，基本上是一場(chǎng)防守戰(zhàn)，針對(duì)社交媒體的失敗。在這一代技術(shù)中，我們不能再犯同樣的錯(cuò)誤。這意味著政府需要確保 AI 的這一責(zé)任鏈中存在透明度、責(zé)任和義務(wù)。

如果沒(méi)有這三個(gè)步驟 — —為最需要的人提供檢測(cè)、尊重權(quán)利的出處和責(zé)任鏈，我們將徒勞地尋找六指手或不眨眼的眼睛。我們需要采取這些步驟。否則，我們將面臨一個(gè)情景，那就是偽造現(xiàn)實(shí)和可能視現(xiàn)實(shí)為偽造會(huì)變得越來(lái)越容易。

那樣的世界就是政治哲學(xué)家漢娜·阿倫特如此描述的世界： “無(wú)法再相信任何事情的人會(huì)無(wú)法自己做決定。他們不僅喪失了行動(dòng)的能力，還失去了思考和判斷的能力。對(duì)于這樣的人，你可以為所欲為。”我知道沒(méi)有人想要這樣的世界，而我認(rèn)為我們可以阻止世界變成那個(gè)樣子。

謝謝。