Course completion and satisfaction are partial evidence. Evaluate starting capability, later task performance and the workplace conditions needed to apply the lesson.
A completed course shows that a learner reached its end. It does not establish that the person can apply the lesson to a new task at work. For a defence technology business buying training software, that distinction matters when the commercial case depends on faster onboarding, fewer avoidable errors or more independent performance.
Learning transfer is the application of acquired knowledge or skill beyond the training activity. Evaluating it requires a connection between the lesson, an observable outcome and the environment in which the learner is expected to use it. The product's engagement statistics remain useful, but they describe only part of that chain.
A customer choosing a new training product may be trying to replace a classroom module, provide additional practice or reduce demand on specialist instructors. Each decision needs different evidence. A product that improves access to practice can be valuable without proving that it replaces every supervised session in the existing programme.
CDC's May 2026 evaluation planning guidance distinguishes questions about training delivery from questions about changes in participants. It recommends deciding the purpose, questions and feasible collection methods early, with a timetable that makes the findings available for the intended decision. The guidance is written for public health training; the commercial product examples here apply its evaluation distinctions to another setting.
Write an outcome that a receiving manager can recognise. For a hypothetical supplier onboarding course, the learner might need to find the correct product record, identify its release status and explain when a question requires specialist review. That is more specific than improving digital skills and more informative than recording how long the employee spent in the application.
Agree which parts of the outcome the product is intended to influence. A course can teach the process while access to the necessary records remains the employer's responsibility. If the evaluation combines those elements without distinction, a useful training product may be blamed for an unavailable workplace resource, or a weak one credited for strong supervisor support.
A high final score can reflect knowledge learners already possessed. Measure the relevant starting point where practical, using an assessment appropriate to the skill. This also helps the buyer determine whether the course serves new staff, experienced staff changing roles or people who need a focused refresher.
CDC's training effectiveness guidance distinguishes assessed learning from later workplace application. It explains that a post-training assessment alone can establish a level of proficiency without establishing improvement, and that satisfaction does not determine effectiveness. Its follow-up guidance ties timing to the subject and the opportunity to apply learning, rather than prescribing one universal interval.
For the onboarding example, use comparable tasks before and after the course without making the later assessment a memorised repetition. The questions can address the same capability while using different records. If the format changes substantially, explain why the two results remain comparable. A harder final task may be appropriate, but its score cannot be casually subtracted from an easier baseline.
Record relevant prior experience and participation. A mixed group may show a small average gain because some employees already perform well while others improve substantially. Grouping the results by the intended audience can reveal whether the product is useful for a particular role even when an overall number appears modest.
Transfer requires evidence beyond successful navigation of the training interface. Ask learners to apply the relevant principle in an equivalent setting that introduces a new example or a different presentation. The purpose is to establish whether they can use the knowledge, not merely recognise the screen they saw during instruction.
For a documentation task, this could mean selecting and explaining the appropriate record in a supervised workplace exercise. For a collaboration lesson, it could involve producing a handover that another colleague can correctly interpret. These are examples of evaluation design, not prescribed procedures for any particular defence system.
Choose evidence that matches the claimed outcome. A written explanation can reveal reasoning but may not establish practical execution. Observation can show performance but may be influenced by the presence of an instructor. A supervisor's account can provide workplace context while remaining a judgment rather than a controlled measurement. Combining suitable methods can clarify the result without pretending that every method answers the same question.
Keep the assessment criteria visible to the people interpreting the results. If two reviewers disagree, determine whether the learner's work is ambiguous or the standard is poorly specified. The evaluation may expose a gap in the organisation's own process that needs attention before the training product can be judged fairly.
A follow-up date is useful only if learners have had a realistic chance to apply the lesson. An employee assigned to a different project may retain the knowledge without encountering the relevant task. Record opportunity to use the skill separately from observed use and successful use. Those are three different outcomes.
The workplace may also require permissions, equipment, time or supervisor support that the course cannot provide. Include questions about these conditions when interpreting transfer. If staff understand the process but cannot access the approved records, another training session is unlikely to resolve the immediate barrier.
Follow-up should be proportionate to the purchase. A focused product pilot may use a small set of observed tasks and structured supervisor feedback. A larger programme can justify a broader comparison. In either case, record how many participants supplied follow-up evidence and why others did not. Missing responses should not silently become successful transfer.
When feasible, compare the new product with the training arrangement the customer would otherwise use. Keep the intended learning outcome consistent and describe differences in practice time, instructor access and supporting materials. If one group receives more of all three, the result concerns the complete package rather than the software alone.
A before-and-after improvement without a comparison group can still be useful evidence, especially for an initial pilot. It provides a weaker basis for attributing the change solely to the product because employees may also gain experience or receive other support. State that limit in the commercial account rather than inflating a local observation into a causal guarantee.
The simulation validation guide addresses the separate question of whether the model is appropriate for the intended lesson. A technically credible simulation creates conditions for training; transfer evidence shows what learners can subsequently do. Buyers need both when both claims underpin the purchase.
Report the learning objective, initial population, training delivered, assessment method and follow-up conditions alongside the results. Include incomplete training, unsuccessful assessments and unavailable follow-up evidence. This allows another business unit to judge whether the finding is relevant to its own staff and work.
Keep delivery efficiency visible too. The XR session evaluation guide examines whether learners can complete usable sessions with manageable support. A product may deliver sound learning outcomes but require more instructor assistance than the proposal assumed. That changes the capacity and cost calculation without erasing the educational result.
A credible rollout can begin with the audience and objectives for which the evidence is strongest. Additional roles, more advanced tasks or reduced supervision can be treated as extensions that need their own support. For suppliers, this creates a more durable sales argument than completion rates alone: the product has helped a defined group perform a defined task, under conditions the next customer can inspect and compare.
CDC guidance retains its public health training context. The supplier onboarding examples and commercial interpretation are BDI analysis.
Suggest a correction