Related Coverage

The College Board did, however, have information that showed a security crisis was brewing in East Asia three years ago.

The information is contained in a June 2013 PowerPoint presentation, marked "Internal Confidential." It was written after a major security breach that year in South Korea. The College Board cancelled the May 2013 sitting of the SAT there after test-prep operators allegedly obtained tests in advance.

Co-authored by Caldwell and shared with other senior College Board employees, the PowerPoint document describes broad security breaches overseas.

SAT "content theft" had been identified as a problem in South Korea, Egypt, Saudi Arabia and China, and answer keys for SAT exams were available in some of those countries, the document notes.

The document says nine of the 18 versions of tests in the entire SAT global inventory at the time "have some level of known compromise." Five of those nine compromised tests were scheduled to be administered outside the United States in the coming school year.

The breaches extended to the hour-long tests the SAT offers in specialised subjects. Those exams are often required for students applying to elite U.S. colleges. Eight of the 10 existing Mathematics Level II subject tests were compromised - three in their entirety and five in part, including two exams that had never been given anywhere, the PowerPoint shows. Ten of the 13 Biology exams were also compromised in whole or in part, including one totally new test.

To reduce opportunities for gaming the SAT and to protect uncompromised versions of the test, the College Board contemplated cutting the number of times it offered the exam in countries with security problems, the PowerPoint shows. Doing so would limit the likelihood that test-takers had seen parts of an exam before taking the test, or that an exam would leak, the document notes.

According to the PowerPoint, College Board officials also considered another approach: Push ahead with all scheduled tests in every country, regardless of the security risks. Under this scenario, one of the "benefits" listed was giving the "appearance that security situation is under control."

The option was "not recommended," in part because the organisation feared it could result in even more cheating and that "another large-scale incident could get attention of U.S. press and universities."

In an interview, Caldwell said the passage was "probably beyond poorly worded." She stressed that College Board executives ultimately rejected the option. "We consistently make these decisions for the right reasons," Caldwell said, citing the organisation's desire to balance "the security of the exam with the needs of students and our members."

"NO EVIDENCE"

The June 2013 PowerPoint shows that the College Board decided to reduce from six to four the number of test dates held in South Korea, Egypt and Saudi Arabia, where actual exam content had been illegally obtained. But it didn't do the same in China, though the document notes that a Chinese website had "compromised" four SAT exams.

Caldwell said the College Board didn't act in mainland China or Hong Kong because "we had no evidence that they had incidents of test materials being stolen." She didn't elaborate on the nature or extent of the breach by the Chinese website.

Included in the PowerPoint was a "market impact analysis" that showed how many fewer times the test would be taken if the College Board limited the number of test dates. The analysis noted that reducing the number of test sittings would enable the arch-rival ACT exam to gain market share in places where the College Board was struggling to stop cheating. That's because students may have decided to take the ACT if the SAT were offered less frequently in their countries.

Based on the figures in the document, Reuters calculated that limiting the number of sittings in China would have cost the College Board roughly $1.2 million to $1.5 million in lost revenue over the next fiscal year. The College Board declined to comment on that estimate. The organisation reported net income of $99 million on revenue of $841 million in the year ended June 2014.

The College Board confirmed that it continued to use material from at least some of the nine exams that had been compromised. Officials said the questions were used in countries other than those where the College Board found that the items had been circulating. They declined to say where or which portions of the compromised tests were reused.

The decision to offer fewer test dates in the three countries was no panacea, the College Board acknowledged in the PowerPoint. It wouldn't solve what the document calls "mobility risk." Because Asian students frequently travel abroad to take the SAT, students from South Korea might travel to Japan, for example, where they would be able to take a version of the test that had been compromised in their home country.

Test-security specialist Neal Kingston, who reviewed the PowerPoint at the request of Reuters, said he was troubled that the College Board decided to administer test material that it knew had been compromised. He also said he was perplexed that the College Board didn't limit test sittings in China, given that it knew in 2013 that a Chinese website was responsible for four breaches.

Kingston, who spent 12 years working for the College Board's security contractor, Educational Testing Service, said he was "significantly troubled by the magnitude" of the security failures documented in the PowerPoint.

"The problem is even larger than I had believed it to be," said Kingston, the director of the Achievement and Assessment Institute at the University of Kansas.

Ray Nicosia, who heads the Office of Testing Integrity for ETS, said investigators use computer analytics to help spot cheating after the fact. But the College Board and ETS declined to discuss particular security efforts. College Board spokesman Zach Goldberg said that "sharing specific details can help the organizations and individuals attempting to gain an unfair advantage."

HARVESTING THE TEST

Gaming the SAT can take many forms. One scam is time-zone cheating, in which test-takers in one part of the world feed questions and answers from the SAT to people sitting the exam later the same day. Another problem is the outright theft of test booklets. In 2015, SAT administrators began shipping booklets to and from some test centres in lock boxes.

A student doesn't need complete questions and answers from an SAT to gain an edge. Getting an advance look at a single reading passage can help a test-taker master the material and get through it more quickly, leaving more time to handle other questions. One extra correct answer can boost a score by 10 or 20 points on the 800-point scale for each section, according to a scoring table for practice tests posted online by the College Board.

The largest threat to the SAT's integrity appears to come from the test-preparation centres. The Asian cram schools have prospered by exploiting what is perhaps the College Board's biggest security weakness: its practice of taking questions and entire sections from tests that have been previously given in the United States and reusing them in later versions of the SAT, typically those offered overseas.

Recycling makes it is possible even for an honest student to take the same test twice unintentionally. (See related story, "Taking it twice.")

One reason for recycling exam material is rooted in the science of standardized testing. The College Board must ensure that scores are comparable on different versions of an exam. The reuse overseas of material previously administered in America helps achieve that.

Recycling also saves money. Developing a single version of the SAT can take up to 30 months and costs about $1 million, according to people familiar with the process.

One way to stop cram schools from exploiting recycled material would be to administer questions once, globally, and then never use them again. The industry calls these "one-and-done" tests.

College Board officials said offering a new SAT each time a test is given is unrealistic. "This is not a matter of just running another one off the assembly line," Caldwell said.

Spokesman Goldberg said the cost of using exams just once would be passed on to test-takers, who potentially would have to pay more than double the current fee.

TEST VALIDITY

It costs up to $54.50 to take the SAT; students in East Asia also pay a $53 surcharge. That's a fraction of the thousands of dollars many Asian students pay to attend test-prep centres.

The "one-and-done" approach also wouldn't solve the problem of time-zone cheating. Kingston, the test security specialist, said he still believes the College Board should cut back on the use of recycled material to combat the systemic security breaches.

"One-and-done forms, from a test validity point of view, is the best practice," he said. "They need to find the sweet spot of better reducing cheating while not exploding their costs."

Because of its reliance on recycling, the College Board needs a large inventory of various versions of the test. The more exams the organization has, the harder it is for students to predict and prepare for the version that will be given.

Chinese and South Korean test-prep operations have circumvented the College Board's defences by building repositories of past SAT questions. These archives, harvested from past exams given in the United States, focus on helping clients prepare for the SAT's reading and writing sections. East Asian students who didn't grow up speaking English find the language sections far more challenging than the math questions.

Among the companies that offer booklets of SAT questions is Sanli, the Shanghai-based test-prep centre that UCLA student Ding attended. About 8,000 students take the company's test-prep courses each year, said Peng Wu, general manager of Sanli's Shanghai branch.

He confirmed that Sanli has created booklets that include key words from answers to past SAT questions. Creating these study aids "can't be helped," Wu said. If a prep school fails to do so, students "think you aren't teaching them properly."

To get material for the jijings, Wu said some test-prep centres have "friends" in the United States who take exams and surreptitiously photograph the forms or memorize and later reconstruct the questions.

Test-prep centres in Asia collect Internet chatter, too. American students who have taken the SAT often go online to kibitz about the questions and even try to reconstruct entire exams within hours of taking the test. The College Board instructs test-takers not to discuss the exam, but many do anyway.

A FILE ON REDDIT

After the SAT was given in the United States last December, for example, someone created a text file and linked to it on the popular website reddit.com. The file contained 21 pages of questions that test-takers recalled, along with a discussion of what were purported to be the correct answers. Some parts of that same December test were reused just a month later in parts of East Asia on Jan. 23, according to people in the region's test-prep industry.

To try to stop the online discussions, the College Board sends "take-down notices" to site operators. Because the SAT exam booklets contain copyrighted material, the College Board maintains that distributing information from them without permission is illegal.

Goldberg, the spokesman, said the College Board sent 18 take-down notices to social media and community forum websites in the past year, and issued more after the redesigned SAT was given this month.

One website that regularly hosts discussions about what's on the SAT is College Confidential, highly popular with American college applicants.

Daniel Obregon, vice president of marketing and user experience at Hobsons, the company that owns College Confidential, said the site usually receives a few take-down notices from the College Board after each test date. It complies, he said, "with any requests pertaining to copyrighted material." But he said he considers online discussions of SAT questions a "grey area."

The College Board is tight-lipped about its methods in the United States. But it recently recycled test material in America.

The SAT given this January in the United States contained several of the same reading passages - including one on celebrities and another on dark matter - as the exam from June 2014, according to online discussions among test-takers after both exams.

"Ayyy... this test (June 2014) was the same as the (Jan 2016) SAT," one person wrote on College Confidential.

Enforcing copyright in China is more difficult.

Purported copies of SATs previously given in the United States, for instance, have been advertised for sale on Chinese websites such as Taobao, a popular shopping site run by e-commerce giant Alibaba. A recent search of the site turned up at least 11 vendors offering dozens of versions of the SAT over the past few years. Some of the ads included photos of purported test booklets. Other advertisers claimed to have the correct answers to past exams.

Asked about the ads, an Alibaba spokesman said it bars the listing of products that violate copyright. "We are currently taking steps to remove the infringing listings" and discipline the sellers, the spokesman said.

NEW TEST, OLD PROBLEMS

The debut of the completely redesigned SAT means that old jijings, the study guides filled with past test material, are obsolete.

This month, the College Board announced it was prohibiting non-students from taking the first sitting of the new exam in the United States. The measure was intended "to prevent security violations." As a result, prep-centre tutors from abroad who had registered to take the test were prevented from directly gathering intelligence on the new SAT's content. The policy will apply to most exam dates.

Other vulnerabilities remain. The ETS security staff is small: Although it sometimes hires contractors, about two dozen ETS security staffers handle College Board matters worldwide.

ETS last year introduced lock boxes to safeguard exams, but the system doesn't always work. After the SAT was given in Taiwan last June, staffers at the Taipei European School put the test booklets and answer sheets in lock boxes and sent them to a local ETS shipping agent. By the time the boxes reached the United States, two booklets were missing, according to emails between the school and ETS.

The boxes don't cover the entire supply chain. Two administrators who oversee SAT exams in the United States said they weren't given the boxes at their high schools. An ETS spokesman said the boxes have been introduced "in select locations."

Perhaps the biggest weakness is the College Board's plans to keep recycling versions of the new test. The risks were evident on March 5, the day the re-engineered SAT debuted in the United States.

After the exam, American test-takers went online to discuss the new test in detail. And Asian test-prep companies mobilized to take advantage.

Source: Reuter