trivia: strengthen prompt to prevent hallucinated/incoherent questions
Lint / Shell (shellcheck) (push) Successful in 9s
Lint / JS (eslint) (push) Successful in 10s
Lint / Python (ruff) (push) Successful in 10s
Lint / Python deps (pip-audit) (push) Successful in 1m11s
Lint / Secret scan (gitleaks) (push) Successful in 8s

The previous system prompt was basically empty. Now it explicitly:
- Requires the answer to be unambiguously correct
- Bans vague, ambiguous, or invented facts
- Requires plausible-but-wrong distractors
- Includes a concrete example of a good question
- Tells the model to pick a simpler topic if unsure

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-04-22 21:51:55 -04:00
parent 8bbcc0530f
commit bc50e8205a
+18 -11
View File
@@ -1019,13 +1019,23 @@ async def _generate_trivia_question(category: str) -> dict | None:
+ "; ".join(f'"{q}"' for q in recent[-10:]) + "; ".join(f'"{q}"' for q in recent[-10:])
+ "." + "."
) if recent else "" ) if recent else ""
prompt = ( system_prompt = (
"You are a trivia question writer. Respond with ONLY a valid JSON object — no markdown, no explanation.\n"
'Format: {"q": "question", "options": ["A answer", "B answer", "C answer", "D answer"], "answer": 0}\n'
"where answer is the 0-based index of the correct option.\n\n"
"Rules for a good trivia question:\n"
"- Ask about a single, specific, verifiable fact. Do not ask vague or ambiguous questions.\n"
"- The correct answer must be unambiguously correct. If you are not confident, pick a different topic.\n"
"- Wrong options must be plausible but clearly wrong — not trick answers, not obviously absurd.\n"
"- The question must be grammatically correct and make sense on its own.\n"
"- Do NOT ask questions where the answer depends on interpretation or opinion.\n"
"- Do NOT invent facts. If unsure, ask about something simpler and more certain.\n\n"
"Example of a good question:\n"
'{"q": "What is the chemical symbol for gold?", "options": ["Au", "Ag", "Fe", "Cu"], "answer": 0}'
)
user_prompt = (
f"Generate a trivia question about {topic}." f"Generate a trivia question about {topic}."
+ avoid_clause + + avoid_clause
" Respond with ONLY a JSON object, no markdown, no explanation. "
'Format: {"q": "question text", "options": ["A text", "B text", "C text", "D text"], "answer": 0} '
"where answer is the 0-based index of the correct option. "
"The question should be clear, factual, and have exactly one correct answer."
) )
try: try:
timeout = aiohttp.ClientTimeout(total=60) timeout = aiohttp.ClientTimeout(total=60)
@@ -1036,11 +1046,8 @@ async def _generate_trivia_question(category: str) -> dict | None:
"model": ASK_MODEL, "model": ASK_MODEL,
"stream": False, "stream": False,
"messages": [ "messages": [
{ {"role": "system", "content": system_prompt},
"role": "system", {"role": "user", "content": user_prompt},
"content": "You are a trivia question generator. Respond with only valid JSON, nothing else.",
},
{"role": "user", "content": prompt},
], ],
}, },
) as response: ) as response: