Skip to content

< CurriculumModel APIs · 8 of 46 ·62 · Level 01, LLM APIs

Ask Again When the JSON Is Wrong

medium · implement · LLM Fundamentals

When a model's reply fails validation, tell it what was wrong and ask again. A model that left out a field will nearly always supply it once it is told which field.

Implement ask_for_json(llm, prompt, schema, max_attempts=3). validate(reply, schema) is provided: it returns (obj, None) when the reply holds a JSON object matching the schema, and (None, error) when it does not, where error is a short sentence such as the field 'age' is missing.

  1. Start a conversation with one user message: the prompt, unchanged.
  2. Call llm.messages.create(messages=...) and validate the reply's text.
  3. If it is valid, return the object. Make no further calls.
  4. If it is not, grow the conversation by two messages: the model's reply, as it was, with role "assistant"; then a user message that includes the error text and asks for the corrected JSON. Ask again with the whole conversation.
  5. After max_attempts calls without a valid reply, return None.

The wording of your correction is yours; the tests check that the error text is in it. The model is scripted, so the tests read exactly what you sent on each attempt.