Baltimore police have arrested Dazhon Leslie Darien, the former athletic director of Pikesville High School (PHS), for allegedly impersonating the school's principal using AI software to make it seem as if he made racist and antisemitic remarks.

Darien, of Baltimore, Maryland, was subsequently charged with witness retaliation, stalking, theft, and disrupting school operations. He was detained late at night trying to board a flight at BWI Thurgood Marshall Airport. Security personnel stopped him because the declared firearm he had with him was improperly packed and an ensuing background check revealed an open warrant for his arrest.

"On January 17, 2024, the Baltimore County Police Department became aware of a voice recording being circulated on social media," said Robert McCullough, Chief of Baltimore County Police, at a streamed press conference today. "It was alleged the voice captured on the audio file belong to Mr Eric Eiswert, the Principal at the Pikesville High School. We now have conclusive evidence that the recording was not authentic.

"The Baltimore County Police Department reached that determination after conducting an extensive investigation, which included bringing in a forensic analyst contracted with the FBI to review the recording. The results of the analysis indicated the recording contained traces of AI-generated content."

McCullough said a second opinion from a forensic analyst at the University of California, Berkeley, also determined the recording was not authentic.

"Based off of those findings and further investigation, it's been determined the recording was generated through the use of artificial intelligence technology," he said.

According to the warrant issued for Darien's arrest, the audio file was shared through social media on January 17 after being sent via email to school teachers. The recording sounded as if Principal Eric Eiswert had made remarks inflammatory enough to prompt a police visit to advise on protective security measures for staff.

"It should be noted that comments in the recording spoke to [Eiswert's] alleged frustrations with test scores of African-American students and assertions that they (the African American students) were unable to 'test their way out of a paper bag,'" the warrant says.

"The recording went on to make disparaging comments about Jewish individuals and two teachers who 'should have never been hired' at the school. The recording depicted a conversation between [Eiswert] and one of PHS' Assistant Principals."

And then everyone piled in

The clip, according to the warrant, led to the temporary removal of Eiswert from his position and "a wave of hate-filled messages on social media and numerous calls to the school," and significantly disrupted school operations. Police say it led to threats against Eiswert and concerns about his safety.

Eiswert told investigators that he believes the audio clip was fake as "he never had the conversations in the recording." And he said he believed Darien was responsible due to his technical familiarity with AI and had a possible motive: Eiswert said there "had been conversations with Darien about his contract not being renewed next semester due to frequent work performance challenges."

Those challenges are described in more specific terms in the warrant. One incident involved Eiswert reprimanding Darien for firing a long-time coach without approval.

Another is that Eiswert was investigating Darien's handling of school funds. "In December of 2023, Darien was the subject of a potential theft investigation [Eiswert] initiated," the warrant says. Hence, the witness retaliation charge.

The other party heard on the recording told investigators that the conversation had not happened and said the remarks did not sound like anything Eiswert would have said.

Interviews with Darien and associates at the school suggest the file was sent to Baltimore County Public School addresses from a Gmail account, TJFOUST9. One of the people interviewed noted that Darien had "urged her to check her BCPS email account for a 'weird' message."

Thereafter, another staff member - who "had a difficult time working under [Eiswert's] leadership for various reasons" - disclosed that she forwarded the email to a PHS student's cellphone, who she knew would rapidly spread the message around various social media outlets and throughout the school." A subpoena for her former university email account indicated the file had been forwarded to news organizations and the NAACP.

During his initial interview with investigators, Darien denied knowing anyone named "TJ Foust" and denied any involvement in the creation and distribution of the recording.

Blame it on the granny

Subsequently, investigators sent subpoenas to Google and AT&T to obtain account ownership information and IP address information. They determined that the Gmail address was created on August 15, 2022 at an IP address associated with the Los Angeles residence of Darien's grandmother. They further found that the recovery phone number for the TJFOUST9 account was associated with a T-Mobile US account registered to Darien.

The specific AI tools allegedly used by Darien have not been disclosed. The warrant however indicates that the BCPS Office of Information Technology reviewed Darien's network access and found several instances where he searched for OpenAI tools via Bing Chat.

We are entering a new deeply concerning frontier

OpenAI Voice Engine, a tool for creating synthetic voices, was not publicly available at the time of the alleged crimes. It was announced on March 29, 2024. However, other companies like ElevenLabs have been offering voice cloning technology for longer.

During the press conference, John Olszewski, Baltimore County Executive, called for more caution with regard to public media.

"It is clear that we are also entering a new deeply concerning frontier as we continue to embrace emerging technology and its potential for innovation and social good," he said.

"We must also remain vigilant against those who would have used it for malicious intent. That will require us to be more aware and more discerning about the audio we hear and the images we see. We will need to be careful in our judgment." ®

https://www.theregister.com//2024/04/25/ai_voice_arrest/