Why do AI pilots fail to become routine work?

Technology acceptance, task fit and the organisational complements of deployment

This retrospective review considers source versions available by 13 July 2026. The cutoff defines the evidence included; preparation and source checking took place on 21 October 2026.

Keywords: AI pilots; task fit; technology adoption; organisation; implementation

1. Deployment is a different outcome from demonstration

A demonstration answers whether a system can produce a useful-looking result under selected conditions. A working deployment must reproduce useful results across recurring tasks, fit existing responsibilities and remain economical to operate. These are different outcomes. An AI pilot can succeed as a technical demonstration while failing as a service that people can use reliably.

This review examines the organisational conditions associated with that transition. It draws on research into technology acceptance, task fit, organisational complements and AI-project failure. The question is deliberately narrower than a prediction of which organisations will succeed. The literature can identify mechanisms and observed barriers; it cannot assign a universal probability of failure to every marketing pilot.

Marketing workflows combine knowledge held in documents, tacit judgement and coordination across functions. A proposed automation may touch only one visible step while depending on several less visible ones. A system for drafting claims, for example, requires an authoritative specification and a usable approval process. Its output quality cannot be evaluated independently of whether the source material and decision rights are available when the work occurs.

2. Review method and evidence types

Targeted searches on 21 October 2026 located twelve sources covering AI implementation, information-systems adoption, organisational knowledge and diffusion. The RAND report's online methods and findings were inspected, including its sampling and definition of failure. Other research was consulted through primary author or publisher summaries and publication records. The selection is narrative and purposive, without duplicate screening, a comprehensive database protocol or quantitative synthesis.

Qualitative interviews, survey associations, economic models and field studies contribute different forms of evidence. Interviews can reveal recurring explanations and practical mechanisms, but their sample cannot automatically estimate a population failure rate. A theory can distinguish constructs without proving that each is the dominant cause in a current deployment. A trial in one workflow can establish a local effect while leaving organisational transfer unresolved.

These differences constrain the synthesis. No percentage from an industry survey is treated as an audited proportion of AI projects that fail. No historical adoption theory is assumed to be fully validated for generative AI. Instead, the review asks what each source identifies about the relationship between a task, its users and the surrounding organisation, and which observations would be necessary to test that relationship in a new setting.

3. What the AI-project interviews actually measure

RAND interviewed 65 experienced data scientists and engineers, with interviews conducted between August and December 2023. The report defined failure according to the organisation's perception and included technical and business failures. Its scope excluded projects that simply used pretrained LLMs. Respondents described misunderstood problems, inadequate data, technology-led priorities, infrastructure constraints and technical limits [1].

The sampling and question structure matter. Fifty industry participants responded from 379 contacted candidates, alongside 15 academic participants. Much of the sample consisted of nonmanagerial engineers, and the report acknowledges that this perspective may favour explanations involving leadership. These are qualitative accounts of experienced practitioners, not a random sample of all project outcomes [1].

The study supplies useful categories for investigation. It does not settle which category dominates in marketing, which applies to a simple assistant or how failure rates change with later models. A claim that most projects fail because of people, technology or data would require a representative design and definitions that support that comparison. In a practical diagnosis, several constraints can interact: unclear objectives can produce the wrong data request, and weak infrastructure can prevent users from testing a technically sound method.

Perceived failure also has multiple meanings. A tool can work technically but miss a commercial target; another can create useful information yet be discontinued because the underlying priority changes. Treating all discontinuation as technical failure overstates one mechanism. Treating all continuation as success makes continued expenditure the outcome. An evaluation needs a definition that remains stable enough to compare cases.

4. Acceptance and performance are separate constructs

Davis develops measures of perceived usefulness and perceived ease of use in technology acceptance [2]. The distinction helps explain why making a system accessible does not establish that people see value in using it. It also separates an attitude from the performance outcome that an organisation ultimately seeks. A favourable intention to use is evidence about intention under the elicitation conditions.

Goodhue and Thompson's task-technology-fit framework links utilisation and fit with individual performance [3]. A system can be easy to use while poorly suited to a task. Conversely, a potentially useful capability may be difficult to incorporate into the way work is organised. These constructs remain analytically distinct even where a survey finds them correlated.

For marketing, fit includes more than the model's ability to write. It includes access to the right evidence, the output format needed by the next person and the timing of the task. A system that produces a high-quality report after the decision deadline has a different operational value from one that delivers the same report in time. An evaluation limited to the final text would miss that distinction.

Non-use can also be rational. Users may know that a source is incomplete, that review will take longer than the task saved or that the tool cannot handle a recurring exception. Calling every instance of non-use resistance assumes the conclusion before investigating it. The empirical question is what users understand about the task and what the system changes in their actual resource requirements.

5. Knowledge and organisational complements

Cohen and Levinthal describe absorptive capacity as an organisation's ability to recognise the value of external knowledge, assimilate it and apply it. Their framework emphasises prior related knowledge and the cumulative development of capability [4]. It provides a way to analyse why access to the same technical information need not produce the same organisational result.

A marketing team evaluating model output draws on domain knowledge that the model's fluent presentation can leave implicit. Assessing whether a recommendation follows from a category's evidence requires understanding that evidence. Access to a generated answer can reduce some search or drafting work while leaving the appraisal problem intact. The relevant skill is part of the production process, even when it does not appear in the prompt.

Bresnahan, Brynjolfsson and Hitt examine information technology, workplace organisation and skilled labour as interrelated changes [5]. The historical evidence does not dictate a contemporary AI organisation chart. It does support treating technology and work organisation as connected objects of analysis. A licence-only description of an intervention can omit the practices that make its use productive.

The productivity J-curve literature similarly identifies complementary intangible investment and adjustment in the measurement of technology returns [6]. That mechanism explains why installation and realised benefit can occur at different times. It cannot be used to excuse indefinite underperformance without evidence. A delayed-return explanation should specify the complement being developed and the observable result expected when it becomes usable.

6. Diffusion across firms, functions and tasks

Worker adoption research documents the rapid spread of generative AI [7]. The microstructure-of-diffusion working paper distinguishes firm adoption, business-function deployment and worker-task use [8]. The distinction is valuable because an organisation can be an adopter while most of its tasks remain unchanged. A binary firm label discards the depth and location of use.

A pilot's reach can expand along several dimensions: more users, more task types, more markets or greater action permissions. These are different treatments. A system proven on a narrow class of internal summaries may encounter new knowledge and coordination requirements when expanded to external communications. User count alone understates the change if the work or consequence also changes.

Selective rollout complicates comparisons. Early users may volunteer, receive more support or work with cleaner data. Their experience can be informative about feasibility while overestimating ordinary performance under a wider rollout. A later group may face more difficult tasks or less implementation support. Comparing the groups without recognising those differences can misattribute the change to user motivation or technical decline.

There is also an important denominator problem in the phrase pilot-to-production conversion. A programme can count trials, teams, use cases or software systems. Some trials are intentionally exploratory and should stop when the learning objective is met. An unqualified conversion percentage obscures both the unit and the reason for stopping. The useful outcome is whether the trial produced a justified decision and whether the deployed system meets its defined requirements.

7. Data availability is a workflow property

Datasheets for datasets propose documenting motivation, composition, collection and recommended use [10]. This is relevant to implementation because a model's input data have a history and intended scope. A repository containing many files is not automatically an authoritative evidence base. The organisation must distinguish current specifications, historical decisions and material that was never approved for the proposed use.

For a recurring workflow, access must work when the user needs it. Data that can be assembled by specialists for a demonstration may remain difficult to retrieve in routine work. A successful pilot can consequently depend on hidden manual preparation. Measuring that preparation and identifying who maintains it are necessary to understand whether the process can persist without the demonstration team.

Data ownership and decision rights can also differ. A document may be technically accessible while no one has authority to declare it current. Two departments may hold inconsistent versions. Asking a model to resolve that disagreement can produce a plausible answer while leaving the underlying governance question unresolved. The evaluation should identify the authoritative input rather than reward confident reconciliation of incompatible records.

The cost of maintaining an evidence base changes with the business. Product specifications, market rules and approved language can become obsolete. A system evaluated on a frozen corpus has been tested under that corpus. A maintained service needs a process for updates and a way to detect whether an update alters prior results. The data lifecycle therefore belongs inside the workflow boundary.

8. Measurement can change the diagnosis

METR's developer experiment found that participants' perceptions of assistance differed from the measured effect on completion time [11]. That result concerns a particular early-2025 setting, but it shows why perceived usefulness and observed productivity should be measured separately. A system can feel helpful while increasing work, or feel unfamiliar while improving a measured outcome.

The customer-support research also shows that worker responses can differ within one deployment [12]. A single average may conceal who benefits and where a tool requires more support. A diagnosis based only on aggregate uptake can miss these distributions. The evaluation needs observations that preserve the relationship between task type, user experience and accepted work.

Success criteria should be capable of rejecting the intervention. If every result can be described as learning, no observation distinguishes a useful deployment from an expensive activity. Learning can be a legitimate pilot objective, but then the research question and decision it informs should be specified. A production objective, such as reducing accepted-work cost at a defined standard, requires a different outcome.

Measurement also needs a baseline that represents the existing process. Comparing a carefully supported AI pilot with an unusually weak or unfamiliar manual task can exaggerate the benefit. The alternative should include the tools and expertise that would realistically be used. This makes the result more informative even if it reduces the apparent size of the gain.

9. Governance and the transition to routine operation

NIST's AI Risk Management Framework places contextual risk management across the system lifecycle [9]. Its relevance is institutional: intended use, measurement and management remain connected after a pilot ends. The framework does not certify that a deployment is safe or effective. It supplies a structure for identifying and managing the questions that continue during operation.

A routine service needs to distinguish permission to use a capability from responsibility for the resulting decision. A person may be able to generate a recommendation without being authorised to act on it. Expanding the system's permissions changes both its possible usefulness and the consequences of error. Evidence about an internal drafting workflow therefore supplies limited information about an externally acting agent.

Operational ownership includes the ability to observe failures, revise the system and stop or revert a change. These functions have resource requirements. A pilot staffed by its builders can solve exceptions informally, while a routine service must make that work repeatable. The transition should be evaluated as a change in the support arrangement, rather than assuming that a technically working prototype already contains an operating model.

9.1 Separating learning from selection

Users who remain in a pilot can differ from users who leave it. If the final satisfaction survey includes only persistent users, an apparent improvement may partly reflect the departure of dissatisfied participants. A longitudinal assessment needs to retain information about the original cohort, participation and missing outcomes. Otherwise the reported experience gradually becomes the experience of survivors rather than the experience of the intended user group.

Practice can also move in either condition. People become familiar with a task through repetition, even without a new tool. A before-after comparison that attributes all improvement to AI can therefore combine tool effects with ordinary learning. Conversely, a trial conducted before users have learned the interface may underestimate later performance. These are design questions about the intervention and observation period; neither direction should be assumed in advance.

Support from the implementation team is itself an input. If specialists prepare every prompt, correct sources and repair failures during a pilot, the observed result includes that labour. The support may be entirely appropriate for a feasibility study. Its continuation at wider scale, however, has to be specified and costed. A transition assessment should reveal whether users can reproduce the result with the support arrangement that will actually exist in routine operation.

10. Synthesis and limitations

The evidence supports analysing deployment as a relationship between capability, task, users and organisation. Technical demonstrations, acceptance measures and economic outcomes each reveal part of that relationship. A pilot can stall because the model is unsuitable, because necessary information or infrastructure is absent, or because the working arrangement never becomes useful enough to sustain. The selected research does not establish one universal ordering of those causes.

The review is limited by purposive source selection, historical theory and largely summary-level access. RAND's interviews provide more detailed qualitative evidence but concern a defined 2023 practitioner sample and a scope that excludes simple pretrained-LLM use. Current marketing workflows need direct evaluation before their barriers or success probabilities can be inferred.

The central conclusion is that scaling evidence should describe the task, the support conditions and the criterion of sustained performance. Adoption counts alone cannot establish deployment quality. A technically successful system becomes an organisational result only when people can use it on recurring work, with the required information, responsibilities and resources in place.

Source access and appraisal register

The 12 references below were located through targeted title and topic searches on 21 October 2026. Author or publisher abstracts, official research summaries and publication records were consulted. The discussion uses the claims available in those materials; full manuscripts, supplementary analyses and underlying data were not independently appraised. Institutional guidance is treated as guidance, rather than a controlled estimate of effectiveness. Where a working paper is cited, its provisional status remains part of the interpretation. The evidence map is selective and does not estimate the prevalence of results in the complete literature.

The RAND online report was additionally inspected for participant recruitment, interview dates, scope exclusions, the organisational definition of failure and the findings. Its qualitative sample does not estimate a population pilot-failure rate. Other sources remain at summary or record level.

References