Solved: JSON Parsing & Schema Validation Errors in OpenAI Function Calling

You have deployed a feature using OpenAI’s function calling capabilities. It works perfectly in staging. Then, in production, your error monitoring platform lights up with SyntaxError: Unexpected token ' in JSON at position... or ZodError: Unrecognized key.

The LLM returned a Python dictionary with single quotes instead of valid JSON. Or perhaps it hallucinated a parameter that doesn't exist in your TypeScript interface.

When building deterministic systems on top of probabilistic models, strict schema validation is the single most common failure point. This article details the root cause of these failures and provides a production-grade TypeScript solution using Zod, JSON5, and Recursive Healing.

The Root Cause: Why Models Fail at JSON

To fix the problem, we must understand why it happens. LLMs are not logic engines; they are autoregressive token predictors.

1. The "Pythonic" Bias

LLMs are heavily trained on Python code. In Python, {'key': 'value'} is a valid dictionary. In JSON, strictly {"key": "value"} is allowed. When a model's "temperature" introduces variance, it may slip into Python syntax, using single quotes or trailing commas. JSON.parse() in JavaScript is unforgiving and will throw a fatal error immediately.

2. Context Window Drift

OpenAI’s response_format: { type: "json_object" } mode significantly reduces syntax errors but does not guarantee schema compliance. The model might generate valid JSON that violates your business rules (e.g., returning a string instead of an array, or hallucinating a field named user_age when your schema defines age).

3. The Validation Gap

Most developers blindly trust the arguments string returned by the API.

// ❌ Dangerous Pattern
const args = JSON.parse(toolCall.function.arguments); 
database.save(args); // Crashes if args doesn't match the schema

This code assumes the LLM is perfect. It is not. You need a defensive layer that handles both malformed syntax and malformed schemas.

The Solution: A Resilient Parsing Architecture

We will implement a robust execution pipeline that performs three specific actions:

Lenient Parsing: Use a parser that accepts trailing commas and single quotes.
Strict Validation: Use Zod to enforce types at runtime.
Auto-Correction (Reflexion): If validation fails, feed the error back to the LLM so it fixes itself.

Prerequisites

Install the necessary dependencies. We use json5 for lenient parsing and zod for schema definition.

npm install openai zod json5 zod-to-json-schema

Step 1: Define the Robust Schema

Do not write raw JSON schemas manually. They are verbose and prone to error. Use Zod to define your interface, then convert it.

import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";

// 1. Define the strictly typed schema
const UserProfileSchema = z.object({
  fullName: z.string().describe("The user's full legal name"),
  age: z.number().int().min(18).describe("Age in years"),
  tags: z.array(z.string()).describe("List of professional skills"),
  status: z.enum(["active", "inactive"]).default("active"),
});

// 2. Extract the TypeScript type automatically
type UserProfile = z.infer<typeof UserProfileSchema>;

// 3. Convert to OpenAI-compatible JSON Schema
const openAIFunctionSchema = {
  name: "update_user_profile",
  description: "Updates the user's profile based on unstructured input",
  parameters: zodToJsonSchema(UserProfileSchema),
};

Step 2: The Lenient Parser

Standard JSON.parse is too brittle for LLM outputs. We use json5 to handle "human-like" JSON errors (single quotes, trailing commas) before validation.

import JSON5 from "json5";

function safeParseJSON<T>(jsonString: string, schema: z.ZodSchema<T>): 
  | { success: true; data: T } 
  | { success: false; error: string } 
{
  // Phase 1: Syntax Correction
  let parsedRaw: any;
  try {
    // Tries standard JSON first (fastest)
    parsedRaw = JSON.parse(jsonString);
  } catch (e) {
    try {
      // Fallback: JSON5 handles single quotes, trailing commas, etc.
      parsedRaw = JSON5.parse(jsonString);
    } catch (parseError) {
      return { 
        success: false, 
        error: `Fatal Syntax Error: ${(parseError as Error).message}` 
      };
    }
  }

  // Phase 2: Schema Validation
  const validationResult = schema.safeParse(parsedRaw);

  if (!validationResult.success) {
    // Format Zod errors into a readable string for the LLM
    const issues = validationResult.error.issues
      .map((i) => `Field '${i.path.join(".")}' has issue: ${i.message}`)
      .join("; ");
    return { success: false, error: `Schema Validation Error: ${issues}` };
  }

  return { success: true, data: validationResult.data };
}

Step 3: The Self-Correcting Execution Loop

This is the most critical part of the solution. If the parsing fails, we do not crash. We send the error message back to OpenAI as a system message. The model will analyze the error and re-generate the correct JSON.

import OpenAI from "openai";

const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

async function extractUserData(userInput: string, maxRetries = 2): Promise<UserProfile | null> {
  const messages: OpenAI.Chat.ChatCompletionMessageParam[] = [
    { role: "system", content: "You are a data extraction assistant." },
    { role: "user", content: userInput },
  ];

  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    console.log(`Attempt ${attempt + 1}...`);

    const response = await client.chat.completions.create({
      model: "gpt-4-turbo", // or gpt-3.5-turbo-0125
      messages: messages,
      tools: [{ type: "function", function: openAIFunctionSchema }],
      tool_choice: { type: "function", function: { name: "update_user_profile" } },
    });

    const toolCall = response.choices[0]?.message?.tool_calls?.[0];

    if (!toolCall) {
      console.error("No tool call generated.");
      return null;
    }

    const argumentsString = toolCall.function.arguments;
    console.log("Raw LLM Output:", argumentsString);

    // Validate using our robust parser
    const result = safeParseJSON(argumentsString, UserProfileSchema);

    if (result.success) {
      console.log("✅ Success:", result.data);
      return result.data;
    }

    // ❌ FAILURE CASE: Prepare for retry
    console.warn("⚠️ Validation Failed:", result.error);

    if (attempt < maxRetries) {
      // Add the assistant's bad response to history
      messages.push(response.choices[0].message);
      
      // Add a system message explaining EXACTLY what went wrong
      messages.push({
        role: "tool",
        tool_call_id: toolCall.id,
        content: `Error parsing arguments: ${result.error}. Please fix the JSON format and schema violations.`,
      });
    }
  }

  throw new Error("Failed to extract valid data after max retries.");
}

// Example Usage
(async () => {
  const badInput = "My name is 'John Doe', age is twenty, skills: React, Node.";
  // Note: "twenty" is a string, schema requires number. This will trigger auto-correction.
  try {
    const profile = await extractUserData(badInput);
    console.log("Final Profile:", profile);
  } catch (err) {
    console.error(err);
  }
})();

Deep Dive: Why This Works

The JSON5 Advantage

In the example above, if the LLM outputs { 'fullName': 'John' } (single quotes), JSON.parse throws a syntax error. JSON5 parses it successfully. This alone resolves roughly 40% of function calling errors in production.

Semantic Error Feedback

Look at how we handle the Zod error:

`Field '${i.path.join(".")}' has issue: ${i.message}`

If the LLM provides "age": "twenty", Zod flags it as Expected number, received string. By feeding this specific error message back to the LLM, we turn the LLM into a debugger. It sees its mistake ("Ah, I sent a string, I need an integer") and corrects it in the next token generation pass.

Strict Typing

By using type UserProfile = z.infer<typeof UserProfileSchema>, your application code is type-safe. You receive autocomplete in your IDE for profile.fullName, and you have a runtime guarantee that profile exists and matches the shape your database expects.

Edge Cases and Pitfalls

1. Large Payload Truncation

If your schema is massive, the generated JSON might exceed the output token limit, resulting in invalid (cut-off) JSON.

Fix: Use zod to split large schemas into smaller, sequential function calls, or increase max_tokens.

2. Recursive Hallucinations

Occasionally, a model gets stuck in a loop, making the same schema error repeatedly.

Fix: Implement the maxRetries counter shown in the code above. Never create an infinite while loop with an LLM.

3. PII Leakage in Logs

When logging argumentsString for debugging, be aware that it may contain PII (Personally Identifiable Information). Ensure your logging infrastructure sanitizes these strings in production environments.

Conclusion

Reliability in LLM applications isn't about hoping the model gets it right; it's about architecting for when the model gets it wrong. By combining JSON5 for syntax leniency, Zod for schema enforcement, and a Self-Correction Loop, you transform a fragile demo into a resilient production system.

The code provided here drops directly into any modern TypeScript backend. Stop parsing blindly and start validating.

Programming Tutorials

Search This Blog