When can a smaller language model retain the performance a marketing task requires?

Scaling, distillation, adaptation and the measurement of task-specific substitution

This retrospective review considers source versions available by 20 March 2026. The cutoff defines the evidence included; preparation and source checking took place on 21 October 2026.

Keywords: small language models; distillation; quantisation; task specialisation; model evaluation

1. Model size is an incomplete description of capability

The number of parameters in a language model is easy to report and difficult to interpret in isolation. It describes the scale of one artifact, but says little about how much data it has seen, how those data were selected, whether its behaviour was adapted to a task or which numerical representation is used at deployment. A model-selection question that begins and ends with parameter count therefore compares only part of the system. For marketing work, the relevant target may be a narrow extraction task, a broad synthesis task or a constrained transformation of approved text.

This review asks when research on small and specialised models supports substituting a smaller model for a larger one. The question is conditional. It concerns retaining required performance on a specified task while changing resources, latency or operating cost. Evidence that a small model performs well on code exercises cannot establish that it will analyse customer interviews accurately. Evidence that one compressed checkpoint preserves benchmark scores cannot establish equal behaviour on every rare or safety-relevant input.

2. Method and outcome definitions

Targeted primary-source retrieval was conducted on 21 October 2026 around scaling laws, distillation, synthetic training data, low-rank adaptation, quantisation and model routing. Fourteen sources were selected to distinguish mechanisms frequently grouped under the term small model. Most studies were consulted through author abstracts and publication records. The accessible AWQ manuscript was additionally inspected for its compression and hardware discussion. The review is narrative and does not estimate a pooled size-performance curve.

Three outcomes are kept separate: task quality, resource demand and deployment performance. Task quality depends on the scoring criterion and input distribution. Resource demand includes model storage and the memory used while processing requests. Deployment performance includes latency and throughput on specified hardware. A reduction in one quantity need not produce an equal reduction in the others. The analysis preserves reported experimental comparisons without presenting historical benchmark records as a current ranking of available products.

3. What scaling research can and cannot answer

3.1 Loss under a training budget

Kaplan and colleagues describe empirical relationships between language-model cross-entropy loss, model size, dataset size and training compute [1]. These are relationships for the training regimes studied. Cross-entropy measures predictive performance over tokens; it is not a direct measure of whether a marketing recommendation is correct or useful. Scaling results can help describe the allocation of training resources while leaving the downstream decision criterion unresolved.

Hoffmann and colleagues revisit compute allocation through a large set of training runs and a compute-optimal model called Chinchilla. Their comparison demonstrates that a smaller model trained on more data can outperform substantially larger models under the studied conditions [2]. The inference is about the interaction of model size and training data. It provides no basis for reading parameter count as a complete ordering of capability, and it does not imply that adding data indefinitely preserves the same relationship.

The distinction between a training budget and a deployment budget is essential. A model that minimises training loss for a fixed training expenditure may not minimise the total resources needed across a long period of inference. Training is incurred when the model is created or adapted. Inference is repeated whenever the application is used. The number and shape of those requests determine the importance of the two stages. This accounting distinction does not require a new scaling law; it requires stating which cost is being optimised.

3.2 Data composition

DoReMi studies how the mixture of pretraining domains affects learning. Its procedure uses a small proxy model to estimate domain weights before training a larger model, with improved performance in the reported comparisons [13]. The work shows that data composition can change outcomes within a training process. It does not establish that a marketing organisation can obtain the same effect merely by adding its own documents to a prompt or by selecting a provider with a larger advertised corpus.

Data quality and task relevance are also separate claims. A clean corpus can be poorly matched to the target language, product domain or output format. A corpus matched to a task can still contain errors or private information. Model size, training amount and data composition should therefore be treated as distinct explanatory variables. Comparing two released models usually changes several at once, making it difficult to attribute an observed performance difference to size alone.

4. Distillation and restricted task distributions

4.1 Teacher and student

Hinton, Vinyals and Dean develop knowledge distillation as a way of transferring information from a model or ensemble into a more deployable model [3]. The teacher supplies a training signal rather than a guarantee of complete behavioural equivalence. A student can approximate the teacher on the training distribution while differing on inputs absent from that distribution. The relevance to model selection is the possibility of task-specific compression, not an assumption that all teacher capabilities fit into an arbitrarily small student.

DistilBERT reports a smaller language representation model that retains much of BERT's performance on its evaluated language-understanding tasks [4]. Its historical result concerns an encoder and a defined set of evaluations. It does not establish a universal compression ratio for modern conversational generators. The study remains useful because it demonstrates that a reduction in parameters need not cause a proportionate reduction in every measured capability.

Distillation also makes the teacher's role part of the provenance of a model. If training targets reflect a teacher's systematic errors, reproducing them faithfully can preserve those errors. If the target distribution is narrow, average performance may conceal rare failures. These are implications of approximating one predictive function with another, not new empirical estimates about a particular model. Evaluation against independent task labels is needed to distinguish matching a teacher from satisfying the task's actual requirements.

4.2 Narrow language environments

TinyStories constructs a synthetic corpus of simple stories and demonstrates coherent multi-paragraph generation from very small models [5]. The simplified vocabulary and narrative environment are part of the intervention. The result informs a question about the relationship between training distribution and model capacity. It cannot be transferred directly to complex consumer research, legal claims or strategic reasoning, where both the information required and the acceptable errors differ.

Textbooks Are All You Need reports a small code model trained with selected and synthetic educational-style material, evaluated on coding benchmarks [6]. Together, these studies show why a task definition matters when interpreting small-model performance. The target distribution can make some regularities easier to learn. Neither result establishes that synthetic data are always superior to human data, or that the same construction procedure produces equal gains in an unrelated marketing domain.

5. Instruction tuning and task adaptation

Ouyang and colleagues report that a much smaller instruction-tuned model was preferred to a larger base model on their evaluated prompt distribution [14]. The comparison demonstrates that post-training changes can be more important than raw size for a particular preference outcome. Human preference is itself a defined evaluation target. It does not establish complete factual accuracy, nor does it convert the result into evidence that the smaller model can replace the larger model on every task.

LoRA reduces the number of trainable parameters by adding low-rank matrices while freezing the base weights [7]. The deployed configuration can therefore be specialised without updating every original parameter. Its relevance is adaptation efficiency. Calling a LoRA-adapted large model a small model would confuse the size of the learned update with the size of the base inference system. Both quantities can matter, but they answer different engineering questions.

QLoRA combines low-rank adaptation with a quantised frozen base model to reduce fine-tuning memory requirements [8]. The reported experiments show that substantial memory savings can coexist with strong performance under specified training and evaluation conditions. The research does not imply that every four-bit model has the same quality or that a successful adaptation eliminates the need to test deployment behaviour. The numerical representation, adapter and calibration or training configuration are parts of the final artifact.

Task specialisation requires a boundary around the task. A model trained to classify a fixed set of campaign themes can be assessed against a labelled taxonomy. An assistant asked to discover entirely new themes encounters a different problem, where the answer space is open. The former may permit narrower training and a clearer acceptance criterion. Treating both as the same task merely because they process customer text obscures the evidence needed to justify substitution.

6. Quantisation: fewer bits is a different intervention

GPTQ performs post-training weight quantisation using approximate second-order information and reports preservation of evaluated model quality at low bit widths [9]. The intervention changes how the weights are represented. It does not delete the corresponding learned parameters in the way a smaller architecture would. Quantisation can therefore make a large checkpoint more practical to store or execute while leaving its nominal parameter count unchanged.

The July 2024 AWQ manuscript uses activation statistics to identify salient weight channels and applies scaling before quantisation [10]. TinyChat links that representation to an execution system. Reducing stored precision does not alone establish faster execution on a different runtime. Transfer of the reported benefit depends on the model, kernels and workload. This review retains the mechanism in that version without importing subsequent benchmark revisions.

Numerical precision is not a substitute for task validation. Small average differences on a language-model benchmark can coexist with larger differences on a specialised output format or a rare input. The relevant comparison evaluates the exact compressed artifact and its uncompressed reference under the same task conditions. If a deployment uses constrained output, retrieval or tool calls, those components should remain fixed so that a measured difference can be attributed to the model change rather than to a simultaneous application change.

7. Routing between models

RouteLLM learns routing policies from preference data to select between stronger and weaker models. The authors report cost reductions while retaining evaluated response quality in their benchmark settings [11]. This is a system-level alternative to choosing one model for every request. It depends on the router identifying requests for which the cheaper model is sufficient. The routing decision can itself be wrong, creating a new source of quality loss that a single-model benchmark does not contain.

A routing comparison must include the cost of the routing process and the handling of uncertain cases. If a request is first answered by a cheaper model and then repeated on a larger one, the total includes both attempts. If the router sends nearly every request to the larger model, it may preserve quality while yielding little resource benefit. A useful evaluation therefore reports model allocation, retained quality and end-to-end latency or cost under a common workload.

Selection and escalation can also be based on observable task properties, such as required context length or output format. That is different from inferring difficulty from a model's confidence, which may be poorly calibrated. The literature reviewed here does not establish a universal escalation rule for marketing tasks. It establishes routing as an empirical question: a policy can be compared against always using either model, with the same evaluation items and an explicit treatment of failure.

8. Measuring equivalence for a marketing task

8.1 The acceptance criterion

Substitution requires a clear statement of what must be preserved. A sentiment classifier may be judged by agreement with human-coded labels, a structured extractor by field-level correctness and a summariser by supported coverage. These outcomes cannot be substituted for one another. A model can produce more fluent summaries while omitting more decision-relevant facts. The quality measure must reflect the intended use of the output before a resource saving is interpreted as an improvement.

Average performance alone can conceal distributional differences. Inputs may vary by language, length, product category or the presence of ambiguous evidence. A narrower model can perform well on common items while failing more often on a less common segment. Reporting those segments is informative even when the overall score changes little. The point is to expose the conditions under which substitution succeeds, rather than claiming global equivalence from one aggregate value.

8.2 Evaluator effects

Zheng and colleagues examine LLM-based judging and identify position, verbosity and self-enhancement biases, alongside agreement with human preferences in their experiments [12]. This matters when an automated judge is used to compare a small model and a large model. If one produces longer answers, the evaluation can reward verbosity rather than correctness. A model-selection test needs to distinguish the desired property from the evaluator's preferences.

Pairwise preference is useful for some open-ended tasks, but it does not establish compliance with every requirement. A concise extraction can be correct even if a longer response appears more polished. A brand-writing task can meet a tone criterion while containing an unsupported product claim. Where such errors matter, explicit rubric dimensions and verifiable facts make the comparison more interpretable. The current review introduces no new scoring instrument; it identifies the measurement distinctions that the cited evaluation work makes relevant.

8.3 Independence and test contamination

A task-specific model may have seen examples similar to the evaluation material during adaptation. Performance on those examples is not an independent estimate of performance on future work. Separating training, development and final evaluation data is therefore part of the interpretation of a substitution claim. The split should also preserve the intended challenge: near-duplicate campaign descriptions divided across sets can make a task appear easier than genuinely new product or audience material.

Repeatedly adjusting a prompt or adapter against the same final test set turns that set into development data. An apparent improvement can then reflect optimisation to known examples. A convincing comparison records the selection process and evaluates the retained configuration on material not used to make the choice. This is an established distinction between model development and evaluation, rather than a claim that any particular released benchmark has been contaminated.

9. Resources per accepted output

The cheapest generated response need not be the cheapest accepted output. If one model requires more retries, manual correction or secondary verification, its lower inference charge can be offset elsewhere in the workflow. An economic comparison therefore needs a denominator tied to a usable result. This does not imply that smaller models are more costly overall. It means that a token price or a storage reduction is an incomplete measure of the task's resource demand.

Latency also depends on the application. A long-running batch classifier may prioritise throughput, while an interactive assistant may prioritise time to first response. Output length, context size and concurrent requests affect those quantities differently. A model can be faster in one workload and slower in another after batching or routing is introduced. The historical compression and routing studies report specific implementations; their speedups should remain attached to those conditions.

Resource comparisons should keep model quality visible rather than treating it as a fixed property. If a smaller model meets a documented acceptance threshold on the intended distribution, the remaining comparison can focus on deployment requirements. If it does not, a lower resource footprint does not answer the original task. This framing permits genuine substitution where the evidence supports it without converting a successful narrow application into a claim about general intelligence.

9.1 Non-inferiority and the cost of different errors

A substitution test can ask whether a candidate's quality loss is small enough to accept in exchange for a resource reduction. That requires specifying what counts as small enough before interpreting the result. A non-significant difference does not establish equivalence: a small evaluation may simply be unable to detect a practically important loss. Conversely, a statistically detectable difference on a large test can be operationally negligible. The acceptance margin belongs to the task and error consequences rather than to model size itself.

The distribution of errors can matter more than the mean difference. A system that extracts dates correctly but occasionally changes the associated product can receive a high field-level average while creating unusable records. A summariser that omits background detail differs from one that reverses a central finding. Error coding should therefore preserve the distinction between omission, unsupported addition and contradiction. The reviewed studies do not supply a universal severity weight for those categories; the evaluation must make the intended use and relevant distinctions explicit.

9.2 Stability across repeated requests

Generation settings can introduce variation even when the input and model artifact remain fixed. A quality comparison based on a single response per item measures one draw from that process. Repeated responses can reveal whether a model meets requirements consistently or only intermittently. They also change the cost denominator if the application selects the best of several attempts. A comparison should state whether the intended deployment produces one response, retries failures or generates several candidates for selection.

Selection must be evaluated as part of the system. If a larger model judges the smaller model's output, the configuration includes both models and the judge's own errors. If a human corrects every output, measured quality after correction is a workflow result rather than the unaided model's accuracy. Either arrangement can be useful, but omitting the selection and correction stage misstates what produced the result. The same principle applies to routing: the accepted answer may reflect several computational steps that must remain visible in resource accounting [11,12].

These design choices also affect generalisation. A model selected on familiar short examples may fail on a longer or ambiguous request even when the topic is unchanged. The relevant unit is the distribution of task demands, including uncertainty and exceptional cases. A narrow task can be evaluated rigorously without pretending that its successful configuration has become a general substitute for a larger model.

10. Limits of transfer and conclusion

The reviewed literature spans different architectures, training periods and tasks. Some results concern encoders, some conversational generators and others code or simple stories. Combining their reported percentages would produce a meaningless common score. Most central studies were consulted at abstract level; the review has not recreated their training runs or benchmark evaluations. The evidence supports mechanisms and conditional comparisons, while current product choice requires testing the specific available configurations.

Small-model substitution is supported when the required task performance survives a precisely specified change in architecture, training, adaptation or numerical representation. Parameter count alone cannot establish that condition. Scaling, distillation, data selection, quantisation and routing offer different routes to resource efficiency, each with its own evaluation target. For marketing, the empirical question is whether the complete configuration preserves the properties the task requires at a lower measured resource cost per accepted result.

Source access and appraisal register

Sources [1,2,3,4,5,6,7,8,9,11,12,13,14] were consulted through author abstracts and publication records. The July 2024 AWQ manuscript [10] was inspected for quantisation design and hardware-related implementation distinctions. Numerical benchmark results have not been independently reproduced. No current vendor ranking, training run or company-specific model-selection experiment is reported.

References