# Methodology ## 1. Purpose of the simulation LLM Ambiguity Lab v1 demonstrates a **control policy for task-resolution ambiguity**. It is designed to make the transition from uncertain input to an execution decision inspectable and reproducible. The simulator does not attempt to reproduce the hidden internal state of an LLM. It does not call an LLM API. Its analysis is generated by deterministic JavaScript rules in `index.html`. Canonical project URL: https://deshimarusakaguchi.com/llm-ambiguity-lab/ **Creator and responsible maintainer:** Deshimaru Sakaguchi ## 1.1 Development provenance The first prototype arose from a real context-loss incident during a Gemini Deep Research session on September 4, 2026. A research plan had been prepared, followed by the short approval-style input `ありがとう。進めてください😊`. After a technical interruption, the working context was lost while the approval message remained. The subsequent Deep Research run treated the missing-context situation itself as the problem to analyze and produced a report and interactive SPA prototype. The public v1 methodology deliberately separates itself from that prototype where the prototype made claims that were not adequately supported for public release. In particular, v1: - removes the prototype's fixed numerical performance-improvement visualization unless a traceable source or reproducible experiment is supplied; - does not present heuristic scores as LLM-internal confidence; - does not describe model-internal mechanisms as if they had been measured by the simulator; - treats `Learn` as conceptual because v1 has no persistent memory mechanism; and - uses explicit hard resolution rules in addition to an illustrative score. Gemini is referenced only to document development provenance. The public v1 specification, maintenance, publication, and responsibility are attributed to Deshimaru Sakaguchi. ## 2. Unit of analysis The input is a natural-language **user instruction**. The simulator attempts to represent the instruction using the following fields: | Field | Meaning | |---|---| | `input` | Normalized user instruction. | | `context_resolution_score` | Operational simulation score from 0.00 to 1.00. | | `resolution_state` | Current control state. | | `target_task` | Operation the user appears to request. | | `target_object` | Object, artifact, topic, or process upon which the task operates. | | `context_grounding` | `grounded`, `partial`, or `ungrounded`. | | `ambiguity_type` | `none`, `lexical`, `referential`, `intent_task`, or `mixed`. | | `output_format` | Explicit or unspecified output form. | | `missing_parameters` | Information still needed for stronger resolution. | | `plausible_interpretations` | Materially different candidate interpretations, when useful. | | `assumption_disclosure` | A non-critical default explicitly disclosed before proceeding. | | `recommended_action` | Control action selected by the simulation policy. | | `recommended_clarification` | Focused clarification question when required. | | `explanation` | Short explanation of the simulated decision. | ## 3. Ambiguity taxonomy ### 3.1 Lexical ambiguity A word or expression permits multiple plausible meanings and the available context does not sufficiently constrain the intended one. The v1 interface explains this category, but the free-text heuristic is not a general lexical-disambiguation engine. ### 3.2 Referential ambiguity The requested operation may be identifiable while the object of that operation remains unresolved. Typical signals include expressions such as `it`, `that`, `this`, `those`, or an underspecified reference to something `above`, `previous`, or `earlier`. ### 3.3 Intent / task ambiguity The subject may be identifiable while the operation, revision goal, constraint, or expected output remains under-specified. Mixed cases may contain more than one ambiguity class. The machine-readable taxonomy is in `data/ambiguity-taxonomy.json`. ## 4. DCRL control loop ### Detect Identify the task, target object, grounding status, missing parameters, referential signals, and candidate interpretations before selecting an execution path. ### Clarify If missing information could materially change what the agent does, pause execution and ask a bounded question. The goal is not to ask for the entire request again, but to recover the parameter that changes the decision. ### Resolve Combine explicit instruction, available contextual references, clarification, and safe defaults into an operational task specification. `Resolve` means *sufficiently specified under the current policy*. It does not mean the simulator has recovered the user's objectively true internal intention. ### Learn A production implementation may preserve corrections, recurring ambiguity patterns, or established preferences when an appropriate memory mechanism exists. **The public v1 simulator does not implement persistent learning.** `Learn` is conceptual in v1. The structured phase specification is in `data/dcrl-spec.json`. ## 5. Preset reference scenarios The six preset inputs are deterministic reference cases. Exact text matches bypass the free-text heuristic and return predefined result objects. The reference scenarios are: 1. clearly grounded; 2. minor condition missing; 3. referential ambiguity; 4. intent / task ambiguity; 5. mixed ambiguity; 6. severe context loss. This design makes the core behavior reproducible even when the free-text heuristic is intentionally simple. ## 6. Free-text heuristic Free-text analysis is deterministic and illustrative. ### 6.1 Normalization Input is trimmed and repeated whitespace is collapsed. ### 6.2 Recognized task verbs The v1 implementation recognizes: `summary/summarize` behavior is driven by the literal verb `summarize`; the implemented list is: - `summarize` - `rewrite` - `revise` - `translate` - `compare` - `explain` - `extract` - `convert` - `list` - `analyze` `improve` and `fix` are additionally interpreted as revision-like requests. ### 6.3 Target-object detection The simulator first recognizes a small set of explicit contextual phrases, including references to the DCRL explanation, previous or earlier sections, and previous reports, messages, drafts, artifacts, or answers. For other simple inputs, v1 also performs a limited extraction of the explicit phrase following a recognized task verb. Unresolved pronouns such as `it`, `that`, `this`, `those`, and `them` are not accepted as resolved target objects. This is a deterministic convenience rule, not general semantic parsing. ### 6.4 Recognized output-format signals - `json` - `markdown` - `table` - `bullets` - `prose` - `code` - `csv` ### 6.5 Generic continuation signals - `continue` - `proceed` - `go ahead` - `carry on` - `keep going` ### 6.6 Vague modification signals - `make it better` - `improve it` - `fix it` - `clean it up` - `do something useful` These vocabularies are implementation features, not general linguistic rules. ## 7. Score construction The free-text heuristic begins at: ```text 0.35 ``` Positive signals: ```text +0.25 recognized task +0.20 identifiable target object +0.10 explicit output format +0.10 explicit scope or constraint signal ``` Negative signals: ```text -0.25 unresolved referential expression -0.20 generic continuation without identifiable task or target -0.15 vague modification request without a defined revision goal ``` The raw result is clamped to `0.00–1.00`. The score is then constrained by semantic resolution rules. A numeric score cannot override a missing execution-critical field. ## 8. Hard resolution rules ### Rule 1: target task unresolved If `target_task` cannot be identified: - state: `PENDING_SPECIFICATION` - score ceiling: `0.39` - autonomous execution: blocked ### Rule 2: target object unresolved If the operation can be identified but the simulator cannot identify what the operation applies to: - state: `CLARIFICATION_REQUIRED` - score constrained to `0.40–0.69` - clarification is required before execution If an unresolved referential expression is present, `referent` is also reported as missing. ### Rule 3: non-critical output format missing If task and target object are resolved but no output format is specified: - state: `READY_WITH_DISCLOSED_ASSUMPTION` - score constrained to `0.70–0.89` - v1 uses a concise prose response as the illustrative safe default - the assumption must be disclosed ### Rule 4: execution-critical fields resolved If task, target object, and requested format are sufficiently specified under the heuristic: - state: `READY` - score floor: `0.90` ## 9. Context-resolution policy bands | Score band | State | Default control action | |---|---|---| | `0.90–1.00` | `READY` | Proceed. | | `0.70–0.89` | `READY_WITH_DISCLOSED_ASSUMPTION` | Disclose safe default, then proceed. | | `0.40–0.69` | `CLARIFICATION_REQUIRED` | Pause and ask a bounded clarification. | | `0.00–0.39` | `PENDING_SPECIFICATION` | Block autonomous execution and request specification. | These bands are illustrative policy boundaries. They are not calibrated probabilities or empirically universal constants. See `data/resolution-policy.json` for the machine-readable policy. ## 10. Transparency and reproducibility The v1 design intentionally favors a small deterministic heuristic over an LLM-backed classifier because its primary purpose is architectural explanation. A given exact preset always returns the same predefined result. A given free-text input returns the same result as long as the v1 rules remain unchanged. This reproducibility should not be confused with general language-understanding accuracy. ## 11. What the method does not establish The methodology does not establish that: - the score corresponds to model confidence; - the simulator recovers the user's true mental intention; - the taxonomy is exhaustive; - the policy thresholds are optimal; - DCRL improves task accuracy by a particular percentage; - DCRL eliminates hallucinations; - the v1 heuristic is suitable as a production classifier. See [limitations.md](limitations.md) for the full boundary statement.