How Twitter Became a Contractor for the FBI’s Censorship Arm

In Part 1 of the Twitter Files, we learned that Twitter was working with politicians, federal agencies, and Biden’s campaign team to censor information on Twitter – specifically as related to Hunter Biden’s laptop and the ensuing New York Post story.

In Part 2, we learned that teams of Twitter employees build blacklists, prevent disfavored tweets from trending, and actively limit the visibility of entire accounts or even trending topics—all in secret, without informing users. In other words, “blacklisting” and “shadowbanning” were real, despite congressional testimony by people like Jack Dorsey claiming otherwise.

In Part 3, we revealed how Twitter employees and directors escalated their censorship campaign in January 2020… and how they effectively threw out the rulebook in favor of their own partisan ideals.

In Part 4, we learned how Twitter executives built the case for a permanent ban – something unprecedented for a world leader up to that point. More importantly, we learned how that “decision” was written in stone before it ever happened.

In Part 5, we uncovered the uncouth truth about what really happened at Twitter when they chose to permanently ban a sitting US president from one of the most-viewed social platforms in the world.

In Part 6, we learned just how closely Twitter worked with government agencies – particularly the FBI – to shut down posts and accounts… all behind closed doors. Twitter’s contact with the FBI was constant and pervasive, as if it were a subsidiary.

In Part 7, we discovered how the FBI & intelligence community discredited factual information about Hunter Biden’s foreign business dealings before the New York Post broke the story about Biden’s laptop… and how the FBI gave Twitter nearly $3.5 million for their cooperation.

In Part 8, we found out how Twitter quietly aided the pentagon’s covert online PsyOp campaign. Despite promises to shut down covert state-run propaganda networks, Twitter docs show that the social media giant directly assisted the U.S. military’s influence operations.

In Part 9, we learned about “Other Government Agencies” (which is the federal codeword for the CIA), and how that agency was intimately involved in the effort to silence dissenting opinions on Twitter and other tech/media platforms.

In Part 10, we began to unwrap the pervasive censorship associated with SARS-CoV-2, including masks, lockdowns, and vaccine safety. Both political parties worked to silence reputable doctors and scientists, which led to injury and death for millions.

In Part 11, we reviewed how and why media – including Twitter – ultimately surrendered to the intelligence community. Moreover, most of this government-mandated censorship was based on rumors and misinformation, with federal agencies unable to support their allegations with hard data.

In Part 12, we’ll learn how the FBI was able to moderate by proxy using an unrestricted “information highway” that turned Twitter into an involuntary subcontractor for the government’s censorship arm.

This story comes from Matt Taibbi. You can read the entire Twitter thread here.

In the first week of May, 2020, at the peak of Covid-19 panic, Twitter senior legal executive Stacia Cardille received a communication from the Global Engagement Center (GEC), the would-be operational/analytical arm of the U.S. State Department. Founded in the Obama years under Secretary of State Hillary Clinton, the GEC was like the State Department’s wannabe version of the NSA or the Defense Intelligence Agency.

Appended to an attachment with a long list of names was a note from the GEC — remember, these were the Trump years — that read, in part:

“We are providing these 5,500 accounts that display inorganic behavior and follow two or more of the 36 Chinese diplomatic twitter accounts that we have identified in the report. Due to the fact that these accounts follow two or more of these diplomatic accounts, and a good portion of them are newly created, we believe that they are suspicious.”

The list of “accounts that display inorganic behavior” was part of what another Twitter legal executive described as an effort by Mike Pompeo’s State Department to pull a “full court press in the media” to “hold China accountable” for “spreading misinformation about the COVID crisis.”

Just a day before, on Wednesday, May 6th, the Associated Press broke news of a four-page report by the Department of Homeland Security claiming Chinese leaders “intentionally concealed the severity” of the Covid-19 outbreak. Now the State Department, by way of GEC, was getting in on the action.

Within a day of receiving the GEC list, Twitter executives were in a lather. A high-speed examination of the accounts revealed what company executives euphemistically called “concerns.” Cardille, Trust and Safety Chief Yoel Roth, and others immediately drafted a response to the GEC:

“Thank you for sharing information… We have begun reviewing the list of 6,000 accounts that GEC provided this morning and have serious concerns…  In our initial review, we have already identified numerous accounts belonging to government entities in the Americas including Canada, NGO and human rights organizations, and journalists.”

The drama that subsequently broke out between Twitter and the State Department would prove revealing, both about the nature of the public-private “content moderation” bureaucracy, and about the internal culture at Twitter, which that year would end up rolling over in a big way for outside moderation demands, again, despite an initial show of resistance.

“WE’RE ALL ON THAT LIST”

Twitter executives seemed particularly put out by the idea that the GEC was taking someone else’s intelligence, then using the press to squeeze its way into an exclusive moderation club. The DHS circulated a report on Chinese disinformation just a few days before the GEC reached out to Twitter.

By then the company was no longer shy about working with Congress, the FBI, DHS, and the Office of the Director of National Intelligence (ODNI). However, they were not anxious to work with the GEC, which they seemed to perceive as a weak sister of the intelligence community, and also “political,” as Roth put it, which in Twitter-ese was code for “pro-Trump.” The company reportedly had similar issues with some Pentagon agencies beginning in 2017. One former defense intelligence source suggested Twitter preferred the FBI because it was “less Trumpy.”

How “Trumpy” or not the GEC was is hard to say, but it’s clear Twitter executives were opposed to letting the fledgling State agency lay hands on its magic moderation machine, preferring to “keep the circle of trust quite small” as Roth put it.

“Obviously, State is a significant voice and one we don’t want to neglect,” wrote Roth on May 6th, 2020, “but I do want us to continue to maintain a distinction between the highly trusted, valued relationships we’ve built over years with entities with considerable expertise and authority… and other parts of USG that may engage on these questions from time to time (sometimes in more political ways than others).”

Cardille argued shutting the agency out entirely was a bad idea, preferring to build “goodwill,” but agreed the GEC was amateurish, and bad news. They “cannot be trusted, particularly if they can score political points,” she said.

A cursory internal review at the company revealed the State Department list included accounts belonging to the Canadian military, Western NGOs, and journalists, including a CNN account. State’s methodology was incredibly broad, including accounts that followed “two or more” Chinese diplomats. “We’re all on that list,” joked one Twitter staffer.

Roth internally even circulated a Bellingcat tweet making fun of the GEC’s conclusions. “Oh yeah, the disinformation that U.S. sanctions are contributing to the death of Iranians during the pandemic,” sneered Aric Toler.

Twitter and GEC ended up clashing publicly that Friday, May 8th. The State Department released news to friendly journalists that “Beijing appears to be increasingly adopting Russian tactics to sow discord and spread disinformation on social media” about the origins of COVID-19.

Meanwhile, articles like CNN’s “Twitter disputes State Department claims China coordinated coronavirus disinformation accounts” presented Twitter’s side, claiming the Trump State Department included “authentic accounts” in its list of names. A GEC spokesperson replied, defensively, to CNN:

“The GEC provided Twitter with a small sample of the overall dataset that included nearly 250,000 accounts,” adding that it” was not surprising that there are authentic accounts in any sample.”

On the surface, this episode looked bad for State, and made it seem Twitter wasn’t the soft touch seemingly every other agency had found them to be. But GEC would get the last laugh, sort of, in an episode that showed that partisan conceptions of how this moderation system worked are likely off base.

“STATE/GEC, NSA, AND CIA HAVE EXPRESSED INTEREST”

When GEC sent its list of 5000+ accounts in May, the move was seen as amplifying messaging from Trump’s Secretary of State Mike Pompeo, who’d accused China of withholding information. Twitter executives agreed, with Musketeerish spirit: with a vote ahead, they wouldn’t be party to that.

“Especially as the election heats up in the coming months, introducing an actor like GEC into what has to date been a stable and (relatively) trusted group of practitioners and experts poses major risks,” is how Roth put it.

About a month after the Chinese disinformation fiasco, Roth got word via his counterpart at Facebook, Nathaniel Gleicher, that GEC wanted a seat at the regular “industry” meeting that included the FBI, DHS, and ODNI. This was too much. It was the opinion of Roth, Gleicher, and Google’s Rick Salgado that GEC’s participation should be opposed. One of the reasons given was startling:

“— The GEC’s mandate for offensive IO to promote American interests”

Eventually, Cardille chimed in, saying she “previewed” the GEC problem to the FBI. Her words resonated “with Elvis, not Laura,” i.e. with San Francisco FBI agent Elvis Chan, but not Foreign Influence Task Force chief Laura Dehmlow:

“I just spoke to the FBI… I previewed to them that they will find resistance to adding the GEC. I talked through the issues we have encountered, and also raised that the GEC/State is focused outside of the U.S., and that we should deal with U.S. elections separately. That resonated with Elvis, not Laura.”

As the 2020 election approached, the FBI, via Chan, negotiated with Roth and others to make sure every interested government agency had a seat at the table. A concerned Roth asked, “What USG agencies will be allowed” on a new Signal channel for centralizing industry briefings?

“I think the easy ones will be FBI, DHS/CISA, and ODNI,” Chan replied. “For your awareness, State/GEC, NSA, and CIA have expressed interest in being allowed on in listen mode only. Welcome your thoughts on this.”

Roth protested the GEC’s participation, noting the “press-happy” nature of the GEC, adding, “I’m also not entirely sure that every member of that group wants their phone number quite so broadly available.”

Chan replied that the Signal group would be a “one-way communication from USG to industry.” If “industry” could “rely on the FBI to be the belly button for the USG,” then “we can do that as well.”

A belly button for the USG! Chan elaborated, explaining that the FBI would essentially coordinate federal-level reports, while the DHS’s Cybersecurity and Infrastructure Security Agency would handle domestic traffic.

“We can give you everything we’re seeing from the FBI and USIC agencies,” he wrote. “CISA will know what is going on in each state.”

This, it seems, ended up being the compromise solution Twitter accepted. A significant portion of the government analytical sector — from the FBI to the CIA to the GEC — was able to at least listen in, in the fashion of someone auditing a college class, to regular “industry” intelligence briefings. This was important because, as Twitter knew full well, anyone who saw any information flow about suspect accounts could leverage them in the media.

Though not every agency got Yoel Roth’s number (which agent Chan, in a curious display of operational security, sent in a Word Document called “Signal Phone Numbers,” via an email subject-lined “List of Numbers”), every agency, Trumpy or not, got to see the intel headed to and from Twitter in those meetings. They could also all send requests via the FBI.

This led to the situation described in Part 7, in which Twitter was paid $3,415,323, essentially for acting as an overwhelmed subcontractor. Requests poured in from FBI offices all over the country, day after day, hour after hour: Las Vegas, Los Angeles, Baltimore. “Do you guys have a list of those 132 accounts action was taken against on 09/29/2020? We wanted to get process served on those accounts,” an FBI agent asked one of Twitter’s senior lawyers just before the 2020 election. “I apologize in advance for adding to your work load.”

Twitter also ended up taking in requests from every conceivable government agency, from state officials in Wyoming, Georgia, Minnesota, Connecticut, California and others to the NSA, FBI, DHS, DOD, DOJ, and many others. They even, ultimately, elevated requests from the GEC, which appeared to grab intelligence passed during one of the industry briefings and went to the media about it. Once Twitter realized the State Department planned on publicly identifying accounts like @BricsMedia and @RebelProtests as “GRU-controlled,” they backed down.

This is the background of the email described in an earlier thread, in which a Twitter executive with a CIA background explained how he previously had elected to wait for more evidence in those cases, he now thought a “more aggressive” stance by Twitter’s government “partners” made that impossible. “Our window on that is closing,” he said.

Twitter ended up receiving so many requests from so many different avenues, their executives seemed to get confused. “Hi Elvis!” wrote one, receiving a notice about something called the Northern California Regional Intelligence Center. “Is this something different than what we’ll already be receiving through the Signal channel?” Chan explained: “We will be sharing the typical threat indicators through Signal,” he said, while the NCRIC was part of a Homeland Security-based network called HISN, which would be “focused on local public safety matters (riots shootings, bombings, etc).”

“My in-box is really f— up at this point,” wrote Cardille, referencing the swarm of FBI requests.

Remember the “internal guidance” Twitter leaders circulated in late 2017, in which they formally agreed to remove any user “identified by the U.S. intelligence community” as a state-sponsored entity involved in “cyber operations”? Such “USIC” identifications by 2020 came regularly and in bulk, asserting that Russia’s Internet Research Agency was targeting Africans, South 

Americans, even African-Americans (spreading “racially derogatory content” in July of 2022, according to an “Other Government Agency” report). There was even an “OGA” warning about publicity for a book written by former Ukrainian prosecutor Viktor Shokhin, who claims he was fired at the behest of Joe Biden, for investigating his son.

In some cases the reports were incredibly short, as small as a single paragraph, saying things like “Advanced Persistent Threat (APT) cyber actors used the attached email accounts… for use in influence operations, social media collection, or social engineering.” An excel document containing 660 email addresses would then be passed to Twitter. There are multiple emails in the record in which Twitter execs express frustration that government “partners” didn’t even do the legwork of matching phone numbers or email addresses to Twitter handles, leaving that work to the company.

Not every request was honored. In one remarkable case, Twitter’s government liaison passed word from the office of Democratic congressman and Intelligence Committee then-chair Adam Schiff asking to “suspend the many accounts, including… @paulsperry, which have repeatedly spread false QAnon conspiracies.” Journalist Paul Sperry of Real Clear Investigations by an extraordinary coincidence was the reporter who revealed the name of Schiff’s “whistleblower” in the Ukrainegate affair. The coincidence was so extraordinary that even Twitter cringed at first. “We don’t do this… No, we don’t do this,” came an immediate response.

These episodes from the Twitter Files show how the digital censorship system evolved from 2017. Early on, company emails were entirely internal and requests about, say, “Russia-linked” accounts came on a case-by-case basis, in some cases through physical meetings with officials at places like the Senate Intelligence Committee.

By 2020, the moderation machine was a high-speed, formalized information highway, with federal and international requests passed through the FBI via Signal and Teleporter, and domestic asks funneled upward through Homeland Security mechanisms like HISN. The $3 million tab Shellenberger revealed Twitter was paid by the state looks like a bad deal for the company, the more you examine the documents. Twitter had to assign reams of personnel to deal with these requests, and during the election season there were clearly too many “reports” to handle.

Moreover the FBI and DHS stopped asking Twitter, and soon simply sent long lists with the expectation of fulfillment. If Twitter didn’t act fast enough, they got quick follow up emails from the Bureau. “Was action taken…? We wanted to get process served.” Or: “Any movement?”

Working through the “belly button,” Twitter was an involuntary subcontractor. And an underpaid one.