AI in the executive team: 10 questions that take you from idea to business value

Management's most important task is not to choose a language model. It is to determine which problems are worth solving, what level of risk the business can accept, and who is responsible for the outcome.

Generative technology can provide faster access to knowledge, support document-intensive processes, and automate parts of repetitive work. Machine learning can improve, for example, forecasting, classification, and matching. But an investment is only justified when it fits a real need and can be managed over time.

The following ten questions can be used as an agenda in a management meeting. They take the discussion from general inspiration to a concrete decision basis.

Management's task is to choose the problem – not the model

Models and vendor terms change quickly. Business goals, risk appetite, and accountability cannot be changed as easily. Therefore, management should start with the processes: Where do waiting times, quality defects, manual double-work, or decisions with weak supporting data occur? Which improvements would be valuable enough to measure?

NIST describes governing, mapping, measuring, and managing as interconnected parts of risk management.[1] The EU's AI Act also imposes requirements that vary with the system's role and risk level. Management therefore needs to create both direction and control, even when technical development takes place in other parts of the organization.[2]

A successful initiative does not start with the question "which model should we buy?" but with "what outcome does the business need to achieve – and under what conditions?"

10 questions the management team should ask

1. Which business problem should be solved?

Describe the problem without using technology as the starting point. "We need a chatbot" is a solution. "Employees spend a long time on average finding the right version of a routine" is a problem that can be investigated.

Also ask what happens if you do nothing. A clear current state makes it easier to compare alternative actions. Sometimes an improved search function, an integration, or a modified process is simpler and better than a generative solution.

How to choose the right use case for your first AI project

2. Who owns the benefit?

Every initiative needs a business owner who is responsible for the outcome, not just a project manager who is responsible for delivery. The owner needs to be able to prioritize users, approve metrics, and make decisions when benefits and risks are weighed against each other.

IT can own the platform and security can set control requirements, but the business unit concerned must own the work process. If the benefit lacks an owner, the pilot will often stall when the project budget runs out.

3. What data needs to be used?

Map out which data sources are needed, who owns them, and whether they are sufficiently up-to-date and consistent. Distinguish between data used as input, data created during use, logs, and any training data.

Management also needs to know which information classes are affected. Personal data, trade secrets, security-protected information, and regulated documents may require different model and hosting choices. "Our own data" is not a single homogeneous category.

What happens to company data when you use AI?

4. What are the risks and regulatory requirements?

The risk assessment must be linked to the use case. An assistant that helps an employee find an internal routine has a different impact profile than a system that affects recruitment, credit, or access to a public service.

Assess, among other things, incorrect results, discrimination, unauthorized access, processing of personal data, information leakage, copyright, vendor lock-in, and impact on business continuity. Establish when legal, data protection, and information security need to participate.

AI Act, NIS2, DORA & GDPR – which requirements affect your AI solution?

5. Do we need an assistant, agent, or standard automation?

An assistant provides support or suggestions. An agent can plan and execute multiple steps using tools and system access. Traditional automation follows predetermined rules. More autonomy is not automatically more valuable.

Base it on four levels: answer, suggest, prepare, or execute. For each level, you need to know what access is required, what can go wrong, and where a human needs to approve.

AI assistant or AI agent – what is the difference?

6. How do we maintain human control?

Human control means more than just having a person somewhere in the process. The person must understand what needs to be reviewed, have the time and competence to do so, and be able to stop or correct the result.

Define which decisions must never be made automatically, how uncertainty should be displayed, and what the solution should do when supporting information is missing. For critical tasks, there should be clear stopping points and a path back to manual handling.

7. How do we measure value?

Select a few metrics that link the solution to the business outcomes. Examples include time per case, proportion of correct classifications, forecast error, answers with verifiable sources, reduced double-work, or improved lead time.

Measure against a baseline and include cost of control work, operations, integrations, training, and maintenance. A faster process is not better if the cost of errors or the need for post-review increases more than the time savings.

From idea to pilot: what are PoC, prototype, and pilot?

8. What competence is required?

A cross-functional team typically needs business knowledge, product ownership, data and systems expertise, as well as support from security and legal. Change management and training are often crucial for actual adoption.

The EU's requirements for AI literacy mean that organizations need to adapt competence initiatives based on how staff use and are affected by the systems.[2] A general inspirational lecture is rarely enough for roles that need to approve results, handle incidents, or maintain the solution.

9. What is required for production?

A working demo is not a production solution. Before deployment, you need stable integrations, identity and access management, logging, test environment, monitoring, cost control, incident management, and a plan for model and data changes.

Ask for a clear picture of what the pilot does not cover. Otherwise, management risks comparing the cost of a demo with the cost of a managed service.

Build or buy AI? Ready-made service, in-house development, or AI platform?

Role-based AI with Microsoft Entra ID – how access works

10. Who is responsible after launch?

Maintenance needs named owners for the product, data sources, security, and business outcomes. Determine who approves new features, monitors quality, handles incidents, and decides on model replacement.

Also set a review cycle. Models, vendor terms, legislation, and business data change. A decision that was reasonable at launch may need to be reconsidered.

Turn the questions into a decision basis

After the meeting, the management team should have more than just a list of ideas. Summarize each prioritized use case on a single page detailing the problem, target group, data, risk, metrics, owner, and proposed testing phase. Then score the ideas based on benefit, feasibility, and risk.

A first initiative should be important enough to create value but sufficiently limited to evaluate. It must be possible to terminate without significant lock-in if the assumptions do not hold.

Checklist for the next management meeting

• A problem formulated in business terms.

• A named owner of the benefit.

• Known data sources and information classes.

• An initial assessment of risk and regulations.

• Selected level: answer, suggest, prepare, or execute.

• Clear points for human control.

• Baseline and metrics for the pilot.

• Right competences and decision-making authority.

• Requirements for production and maintenance.

• A date for follow-up and the next decision.

Sources

1. NIST, AI Risk Management Framework:
   https://www.nist.gov/itl/ai-risk-management-framework

2. European Commission, AI Act:
   https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai

3. Digg, Implementing an AI policy:
   https://www.digg.se/kunskap-och-stod/regler-och-rekommendationer/regler-och-rekommendationer/infor-en-ai-policy