How to interpret disagreement between experts

Expert disagreement is informative only after you establish what each assessment actually evaluates. Compare the question, date, evidence, assumptions and meaning of any score. Different conclusions may reflect different information or different judgments; they do not automatically imply that one expert is incompetent or that the midpoint is the correct answer.

First check that the questions match

One author may assess whether a technology works in a pilot, while another assesses whether a security is attractively priced. A third may focus on a two-year execution risk. These conclusions cannot be compared as if they answered one identical question.

Begin by rewriting each view in a short sentence: "Given this evidence, the author concludes this about this object over this period." If you cannot complete that sentence, read the fuller analysis before relying on its headline or score.

The comparison method below is an Orbitrum educational adaptation. It does not replace the platform's published scoring methodology or prescribe how individual experts must weigh their judgments.

Make uncertainty part of the conclusion

EFSA's scientific methods guidance treats uncertainty analysis as identifying limitations in knowledge and explaining their implications for an assessment. Its context is scientific risk assessment, not investment selection. We use that principle to organize a reader's questions, not to claim an EFSA-approved financial method. EFSA Scientific Committee, 2018

A 2025 Comment in Nature Human Behaviour argues for reporting information readers can use to assess uncertainty. It is an expert commentary, not a randomized trial showing that a particular format improves investment outcomes. Only its public preview is used here. Cruz-Cano and Allison, 2025

Instead of asking only which author sounds more confident, ask which part of the conclusion is observed, inferred or still unresolved. Confidence of expression is not itself the missing evidence.

Build a comparison sheet

Use the same fields for both assessments so that a difference does not disappear inside different presentation styles.

This worksheet is proposed by the Academy. It is not a statistically validated ranking of experts.

Identify the kind of disagreement

A disagreement about a reported fact calls for checking the original source, period and unit. A disagreement about meaning calls for examining the inference. A disagreement about assumptions calls for making the alternative assumptions explicit.

There can also be a difference in objectives. An author studying technical feasibility may rationally emphasize different evidence from a reader studying liquidity or financial risk. Do not force those objectives into one scale and then treat the scale as neutral.

Finally, newer evidence may explain a difference. A report published before a material event should not be judged as if its author had already seen that event. Equally, an old view should not be presented as a current conclusion without checking its date.

Do not turn a score into a probability

A score of 80 out of 100 does not, by itself, mean an 80% chance of success. That interpretation would require an explicitly defined probabilistic quantity and evidence about how such estimates perform. Read the actual scale and method.

Fictional example: one author awards 90 for progress against a technical roadmap; another awards 60 for commercial attractiveness. Their arithmetic midpoint is 75, but the calculation does not create a meaningful common assessment. The inputs answer different questions.

Even when two scores use the same published rubric, the explanations may reveal shared evidence, different assumptions or unresolved gaps. An aggregate can summarize inputs; it does not make the supporting evidence stronger by itself.

Count independent evidence, not repeated headlines

If two analyses both depend on one company announcement, the announcement has not become two independent observations. Record the shared source before describing a consensus.

Similarly, an affiliation should be disclosed and considered, but it does not logically prove every claim is false. Examine the evidence and the potential influence separately. Avoid replacing critical reading with either automatic trust or automatic dismissal.

Write a fair synthesis

A useful synthesis can be short: "The authors agree on A. They disagree on B because they use different assumptions about C. Evidence D would help resolve the difference. The available material does not yet establish E."

Attribute each view to its author and date. Do not rewrite an expert's conclusion to make the discussion appear more harmonious. If a quotation is used, preserve its context and permissions.

For authors, state your update conditions before the next outcome is known. For readers, revisit the specific claim rather than treating a later success or failure as proof that every part of the original assessment was right or wrong.

Apply this on Orbitrum

Open the available analyses linked from a project or expert profile. Compare the scope and dates before comparing scores, and consult the published methodology where relevant. Not every project has multiple accessible evaluations; missing or paywalled work should not be summarized as if you had read it.

Use the general due-diligence guide to structure your own notes, or the applied filing case to practice separating reported facts and interpretation.

Questions about conflicting assessments

Should I follow the highest-scoring expert or average all the scores?

This guide recommends neither as an automatic rule. First establish what the scores mean and whether they are comparable. Then examine the underlying reasoning and the limits of the evidence.

Does disagreement mean the research is low quality?

Not necessarily. Determine whether the difference comes from the question, evidence, assumptions or reasoning. Unexplained inconsistency and transparent disagreement are not the same thing.

Does stating uncertainty make an assessment useless?

No. An assessment can identify what is supported and which missing evidence matters. The purpose is to bound the conclusion, not to promise certainty that the sources cannot provide.