Generative AI profile
NIST released its Generative AI Profile in 2024 to help organisations identify risks unique to generative AI and select risk management actions aligned with their goals and context.
Buying AI software looks like procurement. In practice, it is governance. A tool can look simple in a demo and still create hard questions about data processing, model training, output quality, auditability, user behaviour, and accountability.
The safest moment to ask those questions is before procurement, not after rollout. Once a tool is embedded into workflows, switching it off becomes politically and operationally harder.
The demo hides the governance surface
AI vendors are good at showing speed: faster drafting, faster search, faster summaries, faster support. Buyers should also look for control: what data goes in, where it goes, who can access it, how long it is retained, whether it trains models, how output is monitored, and who owns the workflow risk.
Seven questions before buying an AI tool
1. What data will the tool process?
List data types before approving the tool: public, internal, confidential, personal, sensitive, regulated, client/patient/employee data, trade secrets, and unpublished business information.
2. Is the data used for model training?
The answer should be explicit. If the vendor says data may be used to improve services, ask whether prompts, uploads, outputs, metadata, feedback, or embeddings are included.
3. Where is the data stored and processed?
Location matters for data protection, confidentiality, client expectations, and cross-border transfer analysis.
4. What controls exist for access, retention, and deletion?
Ask about role-based access, logs, admin controls, deletion, retention periods, export, and offboarding.
5. How does the tool support human review?
A good AI tool should make review easier, not harder. It should preserve sources, show confidence limits where relevant, expose citations, allow correction, and avoid hiding the reasoning path behind polished output.
6. What evidence can the organisation keep?
Governance depends on evidence: approval records, data-flow notes, vendor terms, risk assessments, test results, training records, and incident logs.
7. Who owns the business decision?
IT, legal, compliance, procurement, and security may all review the tool. But a business owner must still be accountable for why the tool is used, in which workflow, by whom, and under which controls.
AI vendor review pack
0/0Use a small pilot before broad rollout
The pilot should test the workflow, not only the tool. Define success criteria before users start: quality, time saved, error rate, review burden, user behaviour, data handling, and escalation.
NIST's AI RMF and Generative AI Profile are useful because they push organisations to connect risks to context. ISO/IEC 42001 adds the management-system discipline: policies, roles, processes, measurement, and improvement.
What to do next
Before the next vendor demo, prepare a one-page review sheet. If the vendor cannot answer the data, training, retention, review, and evidence questions clearly, do not treat that as a detail to resolve later. It is the core of the decision.