Most standardized tests have a fee: the SAT costs $50, the GRE costs $200, and the NY Bar Exam costs $250. This year, the bar exam came at a much larger cost for recent law school graduates — their privacy.
Many in-person events have had to find ways to move to the internet this year, and exams are no exception. We’d like to think that online exams shouldn’t be a big deal. It’s 2020. We have a pretty good grasp on how security and privacy should work, and it shouldn’t be too hard to implement sensible anti-cheating features.
It shouldn’t be a big deal, but for one software firm, it really is.
The NY State Board of Law Examiners (NY BOLE), along with several other state exam boards, chose to administer this year’s bar exam via ExamSoft’s Examplify. If you’ve missed out on the Examplify Saga, following the Diploma Privilege for New York account on Twitter will get you caught up pretty quickly. Essentially, according to its users, Examplify is an unmitigated disaster. Let’s start with something that should have been settled twenty years ago.
Did They Just Email Me My Password?
Passwords are stored in plaintext. Seriously- how is this still a thing? Users report being able to call customer support and retrieve not only their usernames but their passwords as well. Others had their passwords emailed to them. If a customer support rep can read your password to you over the phone, you’ve got a real problem. It would only take a bit of social engineering for somebody to get into your account, and if you reuse passwords, then your unrelated accounts can be easily compromised. (Please, use a password manager!)
It turns out that you don’t even need to log into somebody’s account to scope out their personal info. Users uploaded government IDs to their accounts, which were promptly sent to a server and stored accessible publicly via a random URL. It’s worth noting that it seems like more of a shortcoming on the part of the NY BOLE and not ExamSoft. Thankfully, once users reported the problem, the BOLE fixed it — though the fact that it was a issue in the first place is ridiculous. More astonishingly, the NY BOLE isn’t the only exam board that had this sort of problem. The DC Bar published users’ background check documents, containing SSNs and employment histories, in addition to their IDs.
12345? Amazing, I have the same combination on my luggage!
There is a set of config files that comes bundled with each downloaded exam, and those files are, well, just plaintext. Puzzlingly, the parameters in these files, such as “isTimed” and “allowSpellChecking,” seem to have little effect in the software. Examsoft claims that modifying them will corrupt and invalidate the exam, but that doesn’t appear to be the case either.
The software also downloads the exam days before the test starts, leaving the files vulnerable to being poked and prodded by the curious-minded. These files are at least encrypted with an 18-character key, according to ExamSoft. On the exam day, the key is made public and test-takers can use it to decrypt the exam files. This isn’t the best strategy, especially when the Michigan bar exam, rather than employing an 18-character key as ExamSoft promised, used the passwords green56, purple34, and blue78 for their exam files. These horribly weak passwords are probably vulnerable to a brute-force attack.
One of Examplify’s main selling points is its ability to lock down a computer. You wouldn’t want a test-taker googling anything, or glancing at a PDF of the textbook on a second monitor. While some of the more obvious ways to cheat are blocked here, users have found a loophole that ExamSoft unsurprisingly overlooked. Mac users may be familiar with the Universal Clipboard, a feature which, when enabled, allows a user to copy something on one device and paste it on another linked to the same account. Guess what — Examplify doesn’t disable it. This is a fun loophole, since a test-taker could copy the question text, and a friend could paste it onto an iPad and copy the relevant section in the textbook for the test-taker to read. It’s unclear as to whether or not this has been fixed yet, but as of September 28th the workaround was still usable.
Users found other creative ways to get around the software lockout as well. It’s been reported that if the computer gets rebooted mid-exam, the user gets ~30s of unrestricted access to their files, as well as the internet. One user was even able to reset the timer and start the exam over by rebooting their computer.
As you can imagine, it’s not just the security that’s awful. Users report a slew of interface problems, from software freezing mid-exam to the facial tracking software (which determines whether you’ve cheated by looking away from your screen) not being able to recognize dark-skinned people.
It’s tough to dive into this sea of shortcomings without registering for the bar exam, however we were able to find a link to the Examplify installer (Windows, macOS) hosted on ExamSoft’s servers and, of course, unprotected. Go nuts.
It’s impressive that with modern advances in technology, we’ve found ways to make exams more stressful than ever before. It may be worth considering doing away with the bar exam and other standardized tests entirely. For the time being these tests are still a reality and it goes without saying, but we’ll say it anyway: even though ExamSoft and the NY BOLE seem to have made it easy to cheat, please don’t do that. Amid all of this failure, they’ve at least succeeded in one way. They’ve given us excellent examples of how not to do, well, pretty much everything.
Thanks to [Jonathan Merrin] for the tip!