Problem Statement
The goal is to create a structured workflow that produces a stable, trustworthy response-format specification for grounded answering with clear citations. The workflow must define answer sections, citation rules, uncertainty language, fallback behavior, and an evaluation checklist. It must avoid hallucinated citations and maintain a calm, helpful tone.
Steps
Ingest project parameters
INGESTCollect the domain description, question types, source types, and definition of what a valid citation includes. Parse all inputs carefully without inventing missing details.
Retrieve and analyze source structure
EXECUTEReview the properties of the provided source types—document titles, sections, and URLs—to understand how citations should be formatted and validated.
Generate response format specification
EXECUTEDraft the required output sections: answer template, citation rules, uncertainty phrasing guidelines, fallback behavior, and evaluation checklist. Ensure tone is trustworthy, calm, and helpful.
Validate grounding and non-hallucination rules
EXECUTECheck that the generated formats do not hallucinate citation structures, include 'I don’t know' behavior when evidence is weak, and obey safety constraints.
Test specification using example question
EXECUTERun the specification on a realistic example question and verify that citation rules, uncertainty guidance, and fallback behaviors operate as intended.
Prompt Configurations
Response Format Drafting Prompt
You are drafting a response-format specification for grounded answering. Using the provided domain, question types, source types, and citation definition, generate: 1. An answer template with clearly separated sections. 2. Citation formatting rules based strictly on the provided source metadata (doc title, section, URL). Do not invent citations. 3. Clear phrasing guidelines for communicating uncertainty or partial confidence. 4. Fallback rules for situations where retrieval evidence is weak or absent, including safe 'I don’t know' behavior. 5. A short evaluation checklist. Tone: trustworthy, calm, helpful. Constraints: Do not hallucinate citations or source details. Only use information explicitly provided. If something is missing, state the assumption explicitly.
Example Inputs/Outputs
Domain: healthcare guidelines. Question types: definition and procedure questions. Sources: PDF manuals with titles, section names, and URLs. Citation definition: title + section + link. Goal: produce a format for grounded answers with safe fallback and uncertainty phrasing.
The workflow produces a structured response-format specification that includes: (1) an answer template with sections such as Direct Answer, Supporting Evidence, Citations, and Confidence; (2) citation rules requiring exact manual title, section name, and public URL; (3) uncertainty language such as 'Based on available sources' or 'The evidence does not clearly state'; (4) fallback behavior instructing the assistant to answer 'I don’t know' when no valid section is found; and (5) an evaluation checklist ensuring no hallucinated citations.