# Keeping decision criteria consistent throughout a comparison

Learn how to set fixed evaluation criteria before comparing, apply consistent scoring rules, and stop criteria drift from skewing your shortlist.

Keeping decision criteria consistent means fixing your evaluation criteria and scoring rules before you compare any options, then applying them unchanged to every candidate. Without that anchor, criteria drift sets in and your shortlist starts measuring different things at different times. The fix is a written criteria definition you can return to at every step. 

- Write your evaluation criteria down before you open the first listing.
- Score every option against the same rules, in the same order.
- When a new criterion appears, pause and apply it to all options, not just the ones left.
- Separate what you must have from what you would like, and score them differently.
- Keep a record of scores so decision consistency can be checked later.
 

## Why consistent criteria decide a fair comparison 

Consistent decision criteria are what make a comparison comparable at all. Each option is judged by the same measures, in the same way, so the differences you see reflect the options and not your mood. That is the core of decision consistency. 

When the measures shift, the shortlist stops being a comparison and becomes a series of separate judgments. A candidate reviewed in week one and a candidate reviewed in week four can end up scored on entirely different grounds. The result looks like a ranking but is not one. 

A fixed set of evaluation criteria protects you from your own attention. Memory fades unevenly, and the details that stick are not always the ones that matter. The written criteria keep the decision tied to what you said you cared about. 

## Define your criteria before comparing anything 

Criteria definition before comparing is the step that makes everything else work. Sit down with no listings open and list the qualities that would make an option right for you. Rank them, and decide how each one will be scored. 

Split the list into must-haves and preferences. A must-have is a pass or fail, and an option that fails it leaves the shortlist regardless of its other scores. Preferences get a scale, so strong and weak performers separate clearly. 

Give each criterion a weight that matches how much it matters to you. Write the weights down next to the criterion, because a weight you remember is a weight you can bend later. Be specific about what a high score looks like for each one. 

- Name the criterion in plain words anyone could apply.
- Decide whether it is a must-have or a preference.
- Set a scoring scale, such as one to five.
- Record the weight it carries in the decision.
- Define what a top score on it actually looks like.
 

## Recognizing criteria drift when it starts 

Criteria drift is the quiet replacement of your criteria with new ones as you learn more. It rarely announces itself, and it often feels like insight. The symptom is a criterion you never wrote down suddenly shaping your view of an option. 

Comparison bias works alongside drift and pushes you toward whichever option you saw first or liked most. Once a favorite forms, criteria that support it gain weight and criteria that undercut it lose weight. The scoring rules stay the same on paper while the weighting shifts in your head. 

Watch for phrases in your own reasoning like 'but this one also has' or 'on the other hand it lacks'. Those are signs that a new measure has entered. The honest response is to decide whether it belongs in the criteria, then apply it everywhere. 

## Build consistent scoring rules you can reuse 

Consistent scoring rules turn a judgment into a repeatable step. Fixed evaluation criteria only help if the same number means the same thing for every option. Write down what each point on the scale means before you score anything. 

Score one criterion across all options at a time, rather than one option across all criteria. That ordering keeps the standard fresh in your mind and reduces the pull of a single option. It also makes your notes easier to compare side by side. 

Record a short reason next to each score. A number without a reason cannot be defended later, and it hides whether you applied the criterion the same way. Reasons also reveal a criterion that turned out to be unclear. 
| Criterion | Type | Scale | What a top score means | 
| Reliability | Must-have | Pass or fail | Meets your stated requirement fully | 
| Ease of use | Preference | One to five | Needs no workaround to operate | 
| Fit with current setup | Preference | One to five | Works with what you already have |  

## Applying the same rules to every option 

Every option on the shortlist deserves the same treatment under the same rules. Score them in one sitting where you can, using the same information and the same scale. Differences in effort between options distort the result as much as differences in criteria. 

Watch the information you use. If one option has a detailed description and another has a thin one, a poor score may reflect the listing rather than the option. Note where information is missing instead of scoring it as bad. 

Revisit options in a rotating order if you must score across more than one session. Always starting with the same option gives it the benefit of your freshest attention. Rotation spreads that advantage across the field. 

## Handling a new criterion mid-shortlist 

A new criterion appearing mid-shortlist is not automatically a problem, but it must be applied to all options. Pause the comparison, add the criterion to the written list, and define how it is scored. Then rescore every option on it, not just the remaining ones. 

Ask whether the new criterion would have changed any earlier decision. If it would, revisit those options with the same care you gave the new ones. Consistency here means the timing of an idea does not decide how much it counts. 

If the new criterion is genuine but niche, consider holding it for a later decision rather than rebuilding the shortlist. Not every insight belongs in the current comparison. Drift is dangerous when it is silent, not when it is deliberate. 

## Keeping decision consistency under review 

Decision consistency improves when you can check it after the fact. A simple record of criteria, weights, scores and reasons gives you something to audit. It also shows where a rule was bent or a criterion was quietly dropped. 

Compare your final notes against your original criteria list before committing. Any measure that shaped the outcome but never appeared on the list deserves an explanation. Any listed criterion that vanished deserves the same scrutiny. 

Save the record with the chosen option, so a later change can be judged against the same standard. Keeping the criteria and the scores together protects you from rewriting history to fit the choice you already made. 

## Questions About Consistent Comparison Criteria 

### How do I keep decision criteria consistent when I learn something new? 

You keep decision criteria consistent by adding the new measure to the written list and applying it to every option. Rescore the candidates already reviewed before you continue. The timing of a new idea should not decide how much weight it carries. 

### What is criteria drift in a comparison? 

Criteria drift is the gradual replacement of your stated criteria with new ones during a comparison. It feels like learning, but it quietly changes the standard between one option and the next. Catching it requires a written criteria list you check against. 

### Should I score all options on one criterion or one option on all criteria? 

Score all options on one criterion at a time. That order keeps the standard fresh and makes the scoring rules easier to apply evenly. It also lets you see every option side by side on the measure that matters at that moment. 

### How do I stop comparison bias from favoring the first option I saw? 

You reduce comparison bias by fixing the criteria and weights before you open any listing. Rotate the order in which you review options if the work spans several sessions. Written scores with reasons make favoritism easier to spot. 

### What counts as fixed evaluation criteria? 

Fixed evaluation criteria are the measures, weights and scoring rules you set in advance and keep unchanged across every option. They include must-haves that pass or fail and preferences scored on a scale. The list is written down, so it can be checked rather than remembered.
