Documentation
Evaluation logic, scoring criteria, product assumptions, limitations, and open product risks.
How the tool works
Lexi evaluates content for retrieval fitness, not general writing quality. Its purpose is to estimate how reliably a modern language-model-driven retrieval system can identify, interpret, and reuse information from a page or section.
The tool works in two stages.
Stage 1: Intent analysis
Stage 1 is not scored. It exists to anchor the rest of the evaluation to what the content is actually trying to cover.
For each input, Lexi determines:
- —Parent topic: the broad subject domain.
- —Subtopic angle: the specific semantic angle or use case the page is addressing.
- —Schema types: the most specific likely Schema.org types that fit the page.
- —Expected entities: the core concepts, actors, objects, or terms that a strong page on this topic should address.
This stage matters because retrieval quality depends heavily on whether the content is judged against the correct semantic frame. A page about “beta invite systems” and a page about “API authentication” may overlap in terminology, but they should not be evaluated against the same entity expectations.
What Stage 1 is doing conceptually
Lexi first tries to answer:
- —What is this page really about?
- —What kind of thing is it trying to explain?
- —What named or implied concepts should appear if the page is complete?
- —What semantic commitments does the page appear to make?
The output of this stage becomes the reference frame for scoring. In practice, that means strong writing can still score poorly if it is judged to omit key entities for its inferred topic.
Why users should care
If the inferred topic is correct, the downstream scoring is usually useful and targeted.
If the inferred topic is wrong, the rest of the analysis can be directionally misleading. For that reason, Stage 1 should be treated as the first validity check on any evaluation. Users should verify that Lexi has identified the page correctly before acting on the score.
Stage 2: Scored evaluation
Once intent is established, Lexi evaluates the content in semantic chunks.
A semantic chunk is defined as:
- —one heading section
- —plus the paragraphs that belong to that section
Each chunk is scored independently, then rolled up into a page-level result.
This design reflects how retrieval systems often operate in practice: they do not always preserve full-document context. They frequently work with partial passages, chunks, or extracted spans. Because of that, Lexi emphasizes whether a section can stand on its own.
What Stage 2 is measuring
For each chunk, Lexi asks questions like:
- —Does this section contain the entities it should contain?
- —Are the relationships between those entities explicit?
- —Can the chunk be understood without relying on surrounding sections?
- —Does it read as factual and extractable, rather than promotional or evasive?
The key principle is local extractability. If an important claim is only understandable after reading three earlier sections, Lexi will usually treat that as a weakness.
Why this chunk-based approach exists
Many documents are coherent for human readers because humans naturally carry forward context. Retrieval systems are less reliable in that respect. A model may encounter only one chunk, or a chunk may be extracted without the context that made it meaningful in the full page.
Lexi therefore favors writing that makes complete claims within a bounded section.
Page structure
Extracted unit
Must stand alone — no surrounding context
Output format
Lexi produces two output layers:
- —a streaming narrative explanation
- —a structured JSON block
The narrative explains what the tool sees and why it is assigning scores. The JSON provides machine-readable structure for rendering, storage, QA, or downstream processing.
This dual-output model serves two audiences at once: humans, who need diagnostic interpretation, and systems, which need normalized structured fields.
Scoring system
Lexi uses a weighted scoring framework designed for retrieval-oriented analysis.
Score scale and formula
Each sub-signal is scored on a 0–10 scale. Each criterion score is calculated as a weighted average of its sub-signals. The final page score is calculated as:
The score ceiling is 97. Scores above 97 are never produced.
This ceiling is intentional. It prevents the interface from implying that a page is “perfect,” and it preserves headroom for exceptional performance without normalizing inflated scores.
Scoring philosophy
Lexi is intentionally conservative at the high end.
A 9.0–9.5 sub-score is reserved for genuinely exceptional execution. It should not be awarded for merely competent or even very good content. This matters because many scoring systems become uninformative when they cluster too much content in the 9+ range.
Lexi also recognizes that some sub-signals have natural ceilings depending on topic type. For example, a topic that does not require multi-hop reasoning should not be expected to earn a perfect score on chain-based reasoning sub-signals. In those cases, a 7–8 may represent fully adequate performance rather than a deficiency.
What the score is for
The score is meant to be:
- —a compressed summary of retrieval-oriented structural quality
- —a way to compare drafts, sections, or revisions
- —a prioritization aid for editing
The score is not meant to be:
- —a measure of factual correctness
- —a universal measure of writing quality
- —a direct predictor of search ranking or AI answer inclusion
Users should treat the score as a diagnostic index, not a guarantee.
Scoring criteria
Lexi scores content across four criteria.
Entity Coverage
Entity Coverage measures how completely the content addresses the entities that should appear for the inferred topic and subtopic.
This criterion reflects a simple retrieval reality: if the relevant concepts are absent, a retrieval system has less evidence to connect the content to the right questions.
Sub-signals
Interpretation
A page can fail Entity Coverage even if it sounds polished. Lexi is not satisfied by elegant prose that leaves out the core terms, actors, concepts, or objects needed to establish semantic completeness.
A single mention does not count as strong coverage. The entity has to be developed enough that a retrieval system could plausibly connect the page to related questions about that entity.
Typical failure patterns
- —Important entities never named explicitly
- —Core concepts mentioned once but never explained
- —Secondary concepts omitted, making the page too narrow
- —The page assumes prior knowledge and skips basic entity grounding
What to do when this score is weak
Inspect whether the content actually names and explains the entities a reader or model would expect. Improvements usually involve:
- —explicitly naming missing entities
- —expanding treatment of thinly mentioned concepts
- —adding supporting concepts that clarify the core topic
Relationship Clarity
Relationship Clarity measures whether entities are connected through explicit, extractable relationships within a single chunk.
Lexi is looking for clear subject-verb-object structures and qualified claims that survive extraction. It is not enough that a human reader could infer the relationship by combining scattered clues.
Sub-signals
Interpretation
This criterion is central to Lexi because retrieval systems often work best when claims are explicit and local. A chunk that says “Invite codes create a signed session token valid for seven days” is stronger than a chunk that separately mentions invite codes, session tokens, and validity period without tying them together.
Directionality matters. “A depends on B” is not the same as “B depends on A.” If the sentence leaves agency or dependency unclear, extraction reliability falls.
Condition specificity also matters. Relative claims such as “much faster,” “better,” or “significantly lower” are weak unless they specify compared to what.
Typical failure patterns
- —Relationships implied but not stated
- —Ambiguous verbs or missing agents
- —Conditions omitted
- —Cause and effect blurred
- —Important logic split across multiple chunks
What to do when this score is weak
Rewrite key claims so the section itself states what is acting, what it acts on, under what conditions, and with what consequence. Explicitness usually helps more than stylistic elegance here.
Messaging Clarity
Messaging Clarity measures whether each chunk is self-contained, intelligible, and unambiguous.
This criterion reflects the fact that a chunk may be extracted out of context. The opening sentence, key referents, conditions, and definitions therefore need to function locally.
Sub-signals
Interpretation
A section opener like “This allows more flexibility” is weak unless the same chunk names what “this” refers to. Human readers may recover the referent from previous context, but extraction systems may not.
Lexi allows normal within-paragraph pronoun use if the subject is named in the first sentence of that paragraph. The concern is not pronouns themselves; it is unresolved reference at chunk boundaries.
Definition adequacy is audience-relative. A highly technical page can assume more prior knowledge than a general explainer, but it still has to define terms enough for the intended semantic task.
Typical failure patterns
- —Section opens with “this,” “it,” or “they”
- —Important condition words lack baselines
- —Definitions are too thin for the topic
- —The chunk depends heavily on the preceding section to make sense
What to do when this score is weak
Strengthen chunk openings, restate the named subject earlier, define key terms locally, and anchor relative phrases with concrete reference points.
Neutrality
Neutrality measures whether the content reads as extractable explanatory material rather than promotional or hedged copy.
This criterion is an overlay because it modifies trust and reuse potential rather than defining the topic itself.
Sub-signals
Interpretation
Retrieval systems are often more confident reusing neutral, factual material than overtly promotional copy. Lexi therefore treats heavy persuasion, self-positioning, or unsupported marketing language as a structural drawback for citability.
Attribution can improve this score. Clean but unsourced material may still perform reasonably, but specific attribution can increase perceived reliability and claim portability.
Typical failure patterns
- —Marketing superlatives with no evidence
- —Explanatory copy dominated by first-person brand framing
- —Excessive hedging that prevents confident extraction
- —CTA language embedded in informational paragraphs
What to do when this score is weak
Separate persuasive copy from explanatory copy, remove unsupported superlatives, reduce first-person brand framing, and strengthen attribution where possible.
Score bands
Lexi maps final scores into qualitative bands.
| Band | Label | Interpretation |
|---|---|---|
| 88–97 | Exceptionally well-optimised | Content is already highly optimized for retrieval. Remaining issues are usually narrow or marginal. Strong structural performance — not perfection. |
| 70–87 | Well-structured, minor gaps | Solid foundations with room for targeted improvement. Most content in this band is already usable for retrieval. A strong outcome in Lexi’s scoring philosophy. |
| 50–69 | Retrievable but significant gaps | Workable but structurally inconsistent. Retrieval may succeed in some contexts and fail in others. Often the range where targeted revision has the biggest payoff. |
| 0–49 | Systematic issues affecting citability | Fundamental structural problems. The page may be difficult to interpret reliably at the chunk level, or may omit too much of the expected semantic content. |
How to interpret bands
Bands are not moral judgment or literary quality. They are statements about retrieval reliability.
A page may convert well, sound good, and still underperform in Lexi because it is optimized for persuasion rather than extractable explanation.
Output structure
Lexi produces a narrative and a structured JSON payload. The structured output includes the following fields.
Stage 1 fields
These fields document the semantic frame Lexi used for the evaluation.
Page-level scoring fields
Criterion-level fields
For each criterion, Lexi returns:
This allows both human review and front-end rendering of detailed score components.
Sub-signal fields
For each sub-signal under a criterion, Lexi returns:
This provides fine-grained explainability. It makes it possible to distinguish, for example, between weak entity presence and weak entity prominence, which may require different edits.
Flagged items
Lexi can also return flagged problem instances containing:
These tie diagnostics back to concrete parts of the source content.
Why the output is structured this way
The structure supports several use cases:
- —UI scorecards and diagnostics
- —future reporting or export workflows
- —quality assurance and regression testing
- —model behavior inspection
- —revision workflows at section level
The JSON is not just an implementation detail. It is part of the product’s explainability model.
Product assumptions
Lexi relies on a number of assumptions. These assumptions are not incidental; they shape what the score means and where it can mislead users.
The model can infer the page's true intent
Lexi assumes the model can correctly identify the page's parent topic, subtopic angle, likely schema types, and expected entities. This assumption is necessary because the scoring framework depends on topic-relative expectations.
What this means for users
If the page is mixed-purpose, poorly signposted, or semantically ambiguous, Lexi may anchor to the wrong interpretation. In that case, the scoring may be internally consistent but externally misaligned with the author's intention. Users should always sanity-check Stage 1 before trusting Stage 2.
Heading-based chunking is a valid unit of evaluation
Lexi assumes a heading section plus its associated paragraphs is a useful approximation of a retrievable unit. This is a practical design choice, not a universal truth about how all content is consumed.
What this means for users
Some content formats depend on layout, tables, captions, sidebars, footnotes, or visual sequencing. Those formats may be underspecified when reduced to heading-plus-paragraph chunks. Lexi may therefore under-credit content whose meaning emerges from structure outside the chunk.
Retrieval systems prefer explicit local relationships
Lexi assumes that retrieval systems are more reliable when claims are explicit within a single chunk than when they must be inferred across distant passages. This assumption drives the strong weighting of Relationship Clarity and Messaging Clarity.
What this means for users
Lexi will often prefer explicit, slightly repetitive writing over elegant, cumulative longform writing. Users optimizing for Lexi may need to repeat named subjects or restate relationships more often than a traditional editor would prefer.
Retrieval quality is separable from general writing quality
Lexi assumes that a page can be strong for retrieval while being ordinary for style, and vice versa. This is a foundational product assumption. Without it, the scoring framework would collapse into a generic writing evaluator.
What this means for users
A low score does not necessarily mean the page is bad for humans. It means the page is less structurally dependable for retrieval, extraction, citation, and reuse.
Neutrality improves extractability
Lexi assumes that neutral, factual phrasing is more reusable than overtly promotional or heavily hedged phrasing. This is why Neutrality appears as an overlay criterion.
What this means for users
Marketing content may be penalized even when it is doing its job well from a conversion perspective. Lexi's idea of "better" is specific to retrieval-oriented explanatory usefulness.
The selected model can judge consistently enough for scoring
The scoring system is currently configured around claude-sonnet-4-20250514. Lexi assumes the model is capable of making sufficiently stable judgments about topic framing, entity expectations, chunk extractability, and sub-signal diagnostics.
What this means for users
Scores are model-mediated. A different model, or even future changes to the same model family, may change the character of the output. Users should not treat the score as if it came from a deterministic rules engine.
The weights represent a useful theory of retrieval fitness
Lexi assumes that the chosen weights meaningfully reflect what matters most for retrieval optimization: Entity Coverage (0.30), Relationship Clarity (0.30), Messaging Clarity (0.25), Neutrality (0.15).
What this means for users
The output reflects a framework, not a universal law. It is an opinionated scoring system designed around a specific view of what makes content citable and extractable.
Limitations
Lexi’s limitations should be explicit because many users will otherwise over-interpret the result.
It does not verify factual accuracy
Lexi evaluates structure, clarity, and extractability. It does not check whether the claims are true.
User impact
A factually wrong page can still score well if it is well structured. Users should not treat Lexi as a fact-checker.
It is not a ranking predictor
Lexi does not prove that a page will rank better in search or appear more often in AI-generated answers.
User impact
The tool gives structural guidance, not guaranteed visibility outcomes. Retrieval fitness may help, but the score is not a promise of distribution.
Scores are model-mediated rather than fully deterministic
The same framework can still produce some variation across runs, versions, or edge cases.
User impact
Users should expect directional consistency, not exact lab-grade repeatability. Small score changes should be interpreted carefully, especially near thresholds.
Topic identification can fail on ambiguous pages
Stage 1 may anchor to the wrong parent topic or subtopic angle when the input is broad, blended, or weakly signposted.
User impact
Downstream scoring may be well reasoned for the wrong frame. This is one of the most important failure modes to watch.
Chunk-based scoring can penalize strong longform writing
Some writing relies intentionally on buildup, transitions, and cumulative context.
User impact
Content optimized for human narrative flow may underperform in Lexi compared with content optimized for local extractability.
The framework prefers explicitness and may over-penalize brevity
A concise statement can be perfectly adequate for a human reader while still being under-specified for retrieval.
User impact
Writers may need to be more explicit, repetitive, or definitional than they would otherwise prefer.
Promotional or brand-led pages are at a structural disadvantage
Neutrality penalizes unsupported superlatives, first-person brand voice, and embedded CTAs in explanatory sections.
User impact
Commercially effective marketing pages may score lower because the tool values citation-like extractability over persuasion.
The score ceiling can be misunderstood
Because the maximum reported score is 97, some users may incorrectly assume that even excellent pages are somehow incomplete.
User impact
97 is an intentional cap, not a system failure or hidden penalty.
Not all topics support all sub-signals equally
Some topics naturally involve dense chains and conditionality; others do not.
User impact
Lower sub-scores in certain areas do not always imply a defect. Sometimes they reflect the actual structure of the topic.
Theme preferences are local to the browser
Theme state is stored in localStorage.
User impact
A user's preferred theme will not automatically carry across devices or browsers.
BYO mode is session-scoped
In BYO mode, the API key is stored in sessionStorage.
User impact
The key is tied to the current browser session. Users may need to re-enter it in future sessions or on other devices.
Browser-direct API usage has trust implications
BYO mode involves browser-direct API interaction.
User impact
Even if the key is not sent to Lexi servers and is only stored in session storage, some users may still be uncomfortable entering an API key into a browser-based interface. This is a product trust consideration, not only a technical one.
Sponsored mode is rate-limited
Sponsored evaluation and recommendation endpoints are capped per session, and invite redemption is capped per IP.
User impact
Heavy users may hit temporary usage limits and need to wait before continuing.
Invite handling is simple, but not infinitely scalable
Invite codes are managed through environment configuration.
User impact
This works well for beta access control but is not the same as a full entitlement or account-management system.
No database-backed session record is assumed
Sponsored sessions are self-contained and HMAC-signed rather than backed by a database record.
User impact
This simplifies operations but may limit advanced account features such as fine-grained revocation, session history, auditability, or detailed entitlement logic unless further systems are added.
Lexi is an opinionated scoring framework, not a universal arbiter
The framework reflects a specific theory of retrieval optimization.
User impact
Users should use Lexi as a decision-support tool, not a final authority on content quality.
Open product risks
The following risks are not just theoretical; they follow directly from how Lexi is designed.
Risk: score overinterpretation
Users may treat the score as more objective, predictive, or comprehensive than it really is. This is especially likely when a numeric output appears alongside detailed diagnostics, because numbers create a false sense of precision.
Why this risk exists
- —The system looks rigorous
- —The score is normalized and weighted
- —Users naturally want a single answer
- —Retrieval performance in the real world is multifactorial, but the product surfaces a compact number
What could go wrong
- —Users optimize for the number instead of the diagnostic substance
- —Teams compare pages too simplistically
- —Stakeholders treat the score as a KPI without understanding its boundaries
Mitigation
- —Explain the framework clearly
- —Position the score as a summary, not a guarantee
- —Keep diagnostics prominent
- —Explain the 97 ceiling and strict top-end philosophy
Risk: misalignment between retrieval quality and brand goals
Lexi may encourage changes that improve extractability but reduce narrative quality, persuasion, or brand distinctiveness.
Why this risk exists
The framework explicitly favors explicitness, local self-containment, and neutral explanatory tone. Many brand and marketing systems favor elegance, momentum across sections, emotional language, and differentiation through voice.
What could go wrong
- —Copy becomes repetitive or flat
- —Explanatory accuracy improves while conversion performance drops
- —Teams over-optimize informational sections without protecting commercial goals
Mitigation
- —Position Lexi as one optimization layer
- —Encourage selective use on explanatory content
- —Distinguish between pages meant to convert and pages meant to be cited
Risk: model drift
Changes in the underlying model may change how Lexi interprets topics, entities, or chunk quality.
Why this risk exists
The evaluation depends on model judgment at multiple stages: intent inference, expected entity generation, sub-signal scoring, and diagnostics. Any model change can alter these judgments.
What could go wrong
- —Scores shift over time for similar content
- —Historical comparisons become noisier
- —Users lose trust if outputs feel unstable
Mitigation
- —Version the scoring system clearly
- —Display the model version used
- —Avoid direct comparison across major model or rubric changes without caveat
Risk: ambiguous input handling
Mixed-intent or weakly structured pages may be framed incorrectly at Stage 1.
Why this risk exists
The product requires a coherent semantic frame before scoring. Real-world pages do not always provide one.
What could go wrong
- —The wrong entities are expected
- —The wrong relationship standards are applied
- —The resulting score is coherent but mis-targeted
Mitigation
- —Surface Stage 1 outputs prominently
- —Encourage users to validate them before acting on the rest of the result
- —Potentially provide mechanisms to constrain or correct topic framing in future versions
Practical interpretation guidance
Taken together, Sections 1–8 imply a clear operating principle for users.
Lexi is most useful when it is treated as a retrieval-structure diagnostic system. It is least useful when treated as a universal content judge, a truth verifier, or a guaranteed predictor of downstream visibility.
Strongest use cases
- —Comparing drafts before and after revision
- —Identifying chunk-level weaknesses in explanatory content
- —Improving citability and extractability for documentation, explainers, and knowledge pages
Weakest use cases
- —Judging factual correctness
- —Predicting traffic or ranking outcomes
- —Optimizing highly branded persuasive copy as if retrieval fitness were the only objective
The right way to use Lexi is to read the score as a summary, the Stage 1 framing as a validity check, and the diagnostics as the main source of editing value.