AI Adoption Checklist for Engineering Leaders

Most engineering teams already have an AI tool. Fewer can say what changed in lead time, defect rate, or a product metric because of it. An AI adoption checklist is a way to see that gap in about ten minutes, before the next licence renewal or the next board question about "our AI strategy".

This one is for CTOs and engineering leaders. Score the company as it works today, not as the pilot deck describes it. There is no prize for a high average. The lowest area is usually the bottleneck.

The checklist sits next to our AI automation work, where the goal is a workflow with a measurable outcome, and our AI development work, where the product or the internal tool needs engineering, evaluation, and integrations.

How to use the levels

Use the same five levels in every area:

Level	Name	What it means
0	Not started	No practice, no owner, or the activity is not allowed.
1	Experimenting	Talk, a pilot, or a few enthusiasts. No shared standard.
2	Individual use	Many people do it, each in their own way.
3	Team standard	Agreed tools, owners, and a written way of working.
4	Measured and scaled	The practice is reviewed against an outcome, and it survives staff changes.

Write down one number for each area, A to G. Then read the patterns at the end. Fix one or two areas. A programme that tries to lift all seven at once becomes a slide, not a change in delivery.

A. Strategy and business outcomes

Level	Description
0	No stated position on AI. Nobody owns the topic.
1	Leadership talks about AI. A few pilots, no budget or goals.
2	Some teams have AI goals, but those goals are not tied to business metrics.
3	A written stance (what you use, why, who owns it) and a budget. Goals exist per team.
4	AI work is tracked against business outcomes (cost, time to market, revenue) and reviewed on a regular cadence.

If this area is the low score, more tools will not help. Name an owner and one metric the initiative is allowed to move. Cost per release, cycle time on a named class of tickets, or deflection of a support queue are enough. "Usage of the assistant" is not a business outcome.

B. AI in the engineering workflow

Level	Description
0	AI tools are not used, or not allowed.
1	Some engineers use assistants on their own, mostly autocomplete and chat.
2	Most engineers use AI daily, each with their own prompts and habits.
3	Agreed tools and practices beyond coding: review, tests, docs, ticket breakdown. Shared prompts or templates exist.
4	Agents handle defined tasks end to end (test generation, migrations, pull-request triage) with human review, and the impact is measured.

Daily use is level 2. Level 3 starts when the team agrees how AI shows up in review and tests, not only in the editor. For how to choose and run coding assistants as a company standard, see corporate coding assistants.

C. Delivery foundations

This area is not an AI score. It is the system AI amplifies.

The DORA 2025 AI Capabilities Model looks at the engineering practices around AI tools, not only at who holds a licence. That is why a team can feel faster in the IDE and still ship on the same cadence.

Level	Description
0	Manual releases, little automated testing, long-lived branches.
1	Basic CI. Tests exist but coverage is patchy. Releases are stressful.
2	CI/CD for the main services. Code review is mandatory. Releases are roughly weekly.
3	Small batches, trunk-based or short-lived branches, reliable automated tests, internal templates or a platform.
4	Lead time and change failure rate are measured, and AI-generated changes go through the same gates as human changes.

If B is high and C is low, people produce more code into the same bottleneck: review queues, flaky tests, or a release that still needs a hero. Buying a stronger model does not shorten that queue.

D. Data and context for AI

Level	Description
0	Knowledge lives in people's heads and in scattered documents.
1	Some documentation exists. It is outdated or hard to find.
2	Docs, tickets, and code are reasonably organised. Someone can paste context into a tool by hand.
3	Internal knowledge (code, docs, APIs, data schemas) is connected so tools can use it with access control, for example search or retrieval.
4	Data quality and access have an owner. AI has governed access to the right internal data, and that access is monitored.

Generic answers are often a context problem, not a model problem. If the assistant cannot see the service boundaries, the ticket history, or the schema, it will invent a plausible one. Level 3 is the first point where internal AI tools can use your system instead of the public internet.

E. AI in the product

Level	Description
0	No AI features in the product, and none planned.
1	Ideas or prototypes. Nothing in front of customers.
2	One AI feature is live, often a wrapper around an LLM API, with limited monitoring.
3	Several AI features have quality checks, cost tracking, and a fallback when the model fails.
4	AI is part of the product strategy. Features are evaluated, monitored, and improved. Unit economics are known.

A demo that works on a clean sample is level 1. Level 3 means you can say what "good" is, what a call costs, and what the user sees when the model is wrong. For a picture of product functions beyond a single chat box (valuation, grounded search, agents over a data platform), see the anonymised property intelligence article.

Operational automation inside the company is a different problem from a customer-facing feature. Where that pays back first is covered in applied AI for SMB operations.

F. Risk, security, and compliance

Level	Description
0	No rules. Nobody knows what data goes into which tool.
1	Informal rules, such as "do not paste customer data".
2	A written policy and a list of approved tools.
3	Controls are in place: data classification, vendor review, logging. AI-generated code is security-reviewed.
4	AI risk sits in regular audits and regulatory readiness (EU AI Act, DORA for financial services, SOC 2, ISO 27001).

Here DORA means the EU operational-resilience regulation, not the DORA research programme in area C. Informal rules fail the day a contractor pastes a production export into a personal account. A written allow-list is the minimum before an AI feature faces customers.

Teams in regulated products can go further with the fintech regulatory and security checklist and with GRC standards by product stage.

G. People and skills

Level	Description
0	No training. The team is split on whether AI is welcome.
1	A few enthusiasts. Knowledge is not shared.
2	Occasional sessions or demos. Adoption depends on the individual.
3	Structured onboarding to the approved tools, internal champions, and a shared playbook.
4	AI skills are part of roles and hiring. The team reviews what worked and what did not.

Training without an agreed tool (area B) and without a policy (area F) produces enthusiasts who cannot share a workflow. A playbook that nobody updates is still level 2.

Patterns that matter more than the average

Pattern	What you will feel	What to do first
B high, C low	The editor feels faster. Releases do not.	Fix batch size, tests, and review gates before adding agents.
E high, F low	An AI feature is live ahead of controls.	Stop expanding the feature. Put policy, logging, and a fallback in place.
B or E high, A low	Plenty of activity, no proof of value.	Tie the work to one business metric before the next budget review.
D low	Answers are generic or invented.	Connect a bounded set of docs, schemas, and tickets. Do not boil the whole wiki.

What to do with the score

Pick the lowest area, or the pattern in the table that matches you. Spend the next quarter there. Level 4 in every column is not the goal of the first pass.

Practical next steps by bottleneck:

• A. One owner, one metric, a written stance of one page.
• B. One approved assistant and a short standard for review, tests, and docs.
• C. Smaller changes and the same pipeline for AI-written code as for human-written code.
• D. One retrieval path over code and docs that the team already trusts, with access control.
• E. Evaluation, cost, and a fallback on the feature you already shipped.
• F. An allow-list and a rule for customer data, before new tools.
• G. Onboarding and a champion for the standard you just wrote, not a lunch-and-learn with no follow-up.

If the gap is a workflow inside the company, start with AI automation. If the gap is a product feature, an internal assistant, or an integration the team will maintain, that is AI development. If you need an engineer inside the team to install the practice rather than a one-off build, a dedicated development team is the closer fit.

Tell us which area scored lowest. We will treat that score as the brief, not as a request to roll out seven workstreams.