Knowing the Answer vs. Doing the Work: How to Build a Blended Technical Assessment

A candidate knows which command restarts a service. They pick the right answer, move to the next question, and earn the point. 

But put that same candidate in front of a server where the service won’t start, and the picture often changes. Is it a config error? A port conflict? A permissions problem? A missing dependency? Knowing the restart command doesn’t tell you whether they can work any of that out.

That gap between knowing the answer and doing the work is one of the hardest things to capture in a technical assessment. For certification programs and hiring teams alike, the practical question is: how do we measure both?

What multiple choice does well

Multiple-choice questions became the backbone of technical testing for good reasons, and they still earn their place.

• Coverage. A candidate can answer dozens of questions in an hour, so an exam can sample a broad body of knowledge.
• Consistency. Every candidate sees the same kind of item, and scoring is objective.
• Efficiency. Items are quick to deliver, quick to score, and well understood by psychometricians.
• Foundational knowledge. Terminology, concepts, and “which option fits this scenario” judgments are things multiple choice measures well.

For many credentials, especially foundational ones, a well-built multiple-choice exam may be exactly the right tool. The question isn’t whether multiple choice works. It’s whether it measures everything the credential or role requires.

Where knowing and doing split apart

The gap tends to show up in skills that depend on working inside a live system, where the next step depends on what you just saw.

• Troubleshooting. Real problems don’t come with four answer choices. Diagnosis means forming a hypothesis, checking it, and adjusting.
• Configuration. Knowing which setting controls a behavior is different from getting a working configuration across several interdependent components.
• Multi-step work. Deploying an application or hardening a server means doing tasks in order, where one early mistake affects everything after it.
• Recovery. What a candidate does after something goes wrong often says more than whether they got it right the first time.

A multiple-choice question can ask about each of these. It can’t watch a candidate do them. When those skills are central to the job, a question alone can leave you inferring ability rather than observing it.

What a blended assessment looks like

A blended assessment combines knowledge-based questions with hands-on tasks in one exam, so each method covers what it measures best. Some technical certification programs already work this way. Canonical Academy’s Ubuntu exams use a hybrid of multiple-choice, scenario-based and performance-based questions. GIAC’s cybersecurity Practitioner exams also combine traditional multiple-choice questions with hands-on CyberLive challenges.

Picture a cloud certification exam built in two parts:

	Part one: knowledge	Part two: performance
Format	Multiple-choice and scenario questions	Tasks inside a live, configured environment
Example content	Networking, identity, storage, and security concepts	A deployment has been misconfigured; find and fix it
What it shows	Breadth of understanding	Whether the candidate can apply it under realistic conditions
Scoring	Answer key	Automated checks on the end state of the environment

Both parts roll into one result. The existing question bank stays. The hands-on tasks add evidence it couldn’t capture: what a candidate does when the system in front of them is broken.

The same idea works in hiring. A short set of knowledge questions can screen for fundamentals, and one practical task shows how a candidate approaches the actual work of the role.

Four design decisions that shape a blended exam

Adding hands-on tasks is less about technology than about a few deliberate choices made up front.

1. Put hands-on tasks where mistakes cost the most

Not every exam objective needs a lab. Start with your blueprint or job task analysis and ask: which skills, if a certified professional or new hire got them wrong, would cause real problems? Those are the strongest candidates for performance tasks.

2. Use questions for breadth, tasks for depth

Knowledge items cover many topics quickly. A single well-built task probes one skill thoroughly, but takes longer to complete. A blended exam works when each method does the job it’s suited for, rather than forcing tasks to cover the whole blueprint.

3. Decide how scores combine before you build

Can a strong score on the knowledge questions make up for a weaker hands-on performance, or should candidates show a minimum level of proficiency in both? There are two common approaches:

• Compensatory scoring: Knowledge questions and hands-on tasks feed one combined score, so stronger performance in one area can offset weaker performance in another. It’s simpler to run, but a candidate can pass without showing much practical skill.
• Conjunctive scoring: Candidates must meet a minimum in each section to pass. It protects the claim that a certified professional or new hire can do the work, but a short hands-on section, such as a single task, then carries a lot of weight on its own.

You’ll also need to decide how much weight the hands-on section carries. A task that measures a critical job skill may deserve more weight than its share of exam time suggests.

Make these decisions early, based on what the credential or role is meant to measure. Your scoring approach shapes task design, cut scores, and how you explain results to candidates.

4. Design the environment around what you’re measuring

Should candidates have internet access? Documentation? AI tools? The answer depends on the skill. Restricting resources can isolate specific knowledge. Allowing the tools people use on the job can give a more realistic view of how they work. Performance-based environments let you make that call deliberately.

Some certification vendors are already making that call in public. CIO reports that SAP’s redesigned exams include open-book portions where candidates can use resources, including AI copilots.

How it plays out: certification vs. hiring

The blended model applies to both, but the goals differ.

For certification programs, hands-on tasks strengthen the claim a credential makes. A certified professional hasn’t just passed a test about the technology; they’ve shown they can work with it. Adding tasks to an existing exam also preserves the investment already made in the question bank and its validation work.

For hiring teams, a practical exercise gives direct evidence of how a candidate handles job-relevant work. It can sit after a knowledge screen and before technical interviews, so interviewers start from what the candidate actually did rather than what their résumé claims. Because every candidate completes the same task in the same environment, comparisons are more consistent.

Where TrueAbility fits

TrueAbility makes it possible to add hands-on performance tasks to an existing assessment program without starting over. Candidates work in browser-based environments running real software, not simulations, allowing organizations to assess practical skills using the systems and tools relevant to their certification or role.

The platform handles environment provisioning, automated scoring, and teardown, while supporting integration with existing assessment delivery workflows. Your team retains control of the exam content, scoring requirements, and candidate experience.

Start with one task

Modernizing an assessment program doesn’t have to mean rebuilding it. The most practical path is often the smallest:

1. Pick one high-stakes skill from your blueprint or role.
2. Build a single hands-on task that measures it.
3. Pilot it alongside your existing assessment with a representative group of candidates.
4. Review the results, then decide whether to expand.

You don’t need to replace a proven exam to find out whether candidates can apply what they know. Start with one meaningful task and build from there. The goal isn’t to make every question hands-on. It’s to measure the skills that matter, using the method that fits each one.

Planning your 2027 certification or hiring program? Talk to us about a single-task pilot and see what a hands-on task adds to the exam you already have.

FAQ

What is a blended assessment?

An exam that combines knowledge-based questions, such as multiple choice, with hands-on performance tasks completed in a live environment.

Is performance-based testing better than multiple choice?

Neither is better in general. Multiple choice measures broad knowledge efficiently; performance tasks show whether someone can apply it. Many programs benefit from both.

Do we have to rebuild our exam to add hands-on tasks?

No. Tasks can be added alongside an existing question bank, starting with a single task.

How are hands-on tasks scored?

Typically by automated checks on the state of the environment when the candidate finishes, such as whether a service runs or a configuration is correct.