AI tool review checklist

← All writing

A vendor demo answers the questions the vendor wants to answer. This is the list of questions someone on campus has to ask instead, before a tool gets approved, paid for, and quietly wired into advising, admissions, grading, or research.

It's meant for the group that actually does the reviewing: a technology committee, a privacy or security office, a library, a research office, or one department head trying to do this responsibly with no committee at all. Work down it with the vendor's answers in front of you, then run the final pass. If you want the reasoning behind the sections, the companion article covers how to write a campus AI policy people will actually follow.

Ticks are for this browsing session only. Nothing you check is stored or sent anywhere.

Before the review starts

• The problem is written down in one sentence, without naming the tool. If nobody can write that sentence, the review can stop here.
• Someone is named as the decision owner, and it isn't a committee in general.
• The people who will use the tool daily are in the room, not just the people who will sign for it.
• We know whether the campus already licenses something that does this, so we aren't buying a fourth version of the same thing.
• The review has a deadline, so "still reviewing" doesn't become the answer by default while people use the tool anyway.

1. What the tool actually does

• We can describe the workflow in plain language: what goes in, what comes out, and which human step it replaces or assists.
• We know which models sit underneath, whether they're the vendor's own or someone else's, and whether that can change without notice.
• We know whether the tool retrieves from our own documents, generates from general training, or both, because the risks differ.
• Decisions the tool influences are listed, and any decision affecting a person's record, admission, funding, or grade is flagged as high stakes.
• For every high stakes use, a human reviews before anything reaches the student, applicant, or employee, and that review is real work with time allocated to it.

2. Data handling

• We have a written answer to where our data goes: which company, which cloud, which country.
• The contract says our prompts, uploads, and outputs are not used to train the vendor's models or anyone else's.
• Retention is stated in days, not described as "as long as necessary," and we know how to request deletion.
• Subprocessors are listed, and the vendor commits to telling us before that list changes.
• Encryption in transit and at rest is confirmed, along with who at the vendor can read our content and under what circumstances.
• Logging is understood: what the tool records about user activity, who can see it, and whether it could be used to monitor staff or students.
• Breach notification terms exist, with a timeframe we can live with.
• We've asked for a current security report (SOC 2, ISO 27001, or a completed HECVAT) rather than a marketing page about security.

3. Student records, research data, and other regulated material

• We've said plainly whether student education records will touch this tool, and if so, whether the vendor qualifies as a school official under FERPA with a written agreement to match.
• If health data is involved, HIPAA coverage and a business associate agreement are settled before use, not after.
• If human subjects data is involved, the IRB has been consulted, and the consent language actually permits processing by a third party.
• Data covered by a data use agreement, license, or sponsor terms is checked against those terms, because many forbid sending data to an outside service.
• International students and staff are considered, including GDPR obligations and where processing physically happens.
• Anything export controlled or otherwise restricted is named as out of scope, in writing, with a safer path offered for that work.
• Where the answer to several of these is no, we've looked at whether a self hosted or private deployment would let the work go ahead safely.

4. Accuracy and failure

• The vendor's accuracy claims come with the test they ran, on what data, and when. A number with no method behind it doesn't count.
• We've run our own pilot on our own real cases, including the awkward ones, and written down what we found.
• We know how the tool fails: does it decline, guess confidently, or invent a citation? We've seen it fail at least once.
• Outputs can be traced back to a source when the tool claims to be answering from our documents.
• We've checked performance across the groups we serve, not just on average, so a tool that works well for most and badly for some gets caught.
• There's a way for a user to report a bad output, and a named person who reads those reports.
• We know what happens to the work if the tool is wrong: who catches it, and what the correction costs.

5. Access, accessibility, and fairness

• An accessibility conformance report (VPAT or equivalent) exists and is recent, and we've tested keyboard and screen reader use ourselves rather than trusting the document.
• The tool works for people who don't have the newest device or a fast connection.
• If students are expected to use it, the cost is covered by the institution rather than pushed onto them, or an equivalent free path exists.
• Language support matches the community, not just English.
• Opting out is possible for anyone who has a reason to, and the alternative isn't a penalty.

6. Contract, cost, and exit

• The real cost is written out for three years, including seats added later, storage, support tiers, and usage or token charges.
• We know what happens to the price at renewal, and whether a pilot discount disappears.
• The contract lets us terminate for convenience with reasonable notice.
• We can export our own content and configuration in a usable format, and we've confirmed how, not just that we're allowed to.
• Deletion at the end of the contract is specified, including copies held by subprocessors and in backups.
• Ownership of what our people create with the tool is clear and sits with us or the author, not the vendor.
• Anything the vendor promised verbally that we're relying on has made it into the written agreement.
• We've thought about what we'd do if the vendor is acquired or shuts down next year, since many of these companies are young.

7. People, training, and governance

• Guidance for users is written in a page or less, and says what to put in, what to keep out, and who to ask.
• Training is offered to the people expected to use it, not just a link to the vendor's help site.
• Disclosure expectations are settled: when a person on the other end should be told that AI was involved.
• Faculty governance has been consulted where the tool touches teaching, grading, or academic judgment.
• Someone owns the tool after launch: renewals, questions, incidents, and the decision to stop using it.
• A review date is on the calendar, ideally within a year, with the measures we'll judge it by written down now.

Final pass before you approve

Read your own notes back with these in mind. This is the pass that catches what an auditor, a reporter, or an unhappy student would catch.

• We can explain the decision to a skeptical colleague in five sentences, including what we're accepting as risk.
• Nothing in our approval depends on a claim we couldn't verify.
• The riskiest use we can imagine someone attempting is named and either allowed on purpose or ruled out in writing.
• If the tool disappeared tomorrow, we know what the work falls back to.
• The decision, the conditions attached to it, and who made it are recorded somewhere findable a year from now.
• This checklist has been saved as our template, since the next tool will raise most of the same questions.

The question underneath all of these: if this tool gets something wrong about a real student, who notices, how fast, and what happens next? A campus that can answer that is ready to approve. One that can't is running a pilot whether it calls it that or not.

A note on saying no

A review that only ever ends in approval isn't a review. Some tools should be declined, and some should be approved for one narrow use rather than campus wide. Writing down the narrow version is usually more useful than a blanket yes, because it gives the next department something to point at when they ask for the same thing.