skip to content
ainoya.dev

Developing a Web UI for Controlling LLM JSON Output

/ 5 min read

llm , json , web ui , output control , schema , openai , gemini

Developing a Web UI for Controlling LLM JSON Output

Introduction

Controlling LLM (Large Language Model) output to get the desired results often requires meticulous specification of output formats and content. This is especially crucial when integrating LLM outputs into other systems, where JSON format control becomes essential.

While Gemini and OpenAI APIs allow specifying JSON schema as output options, existing JSON schema builders often have a steep learning curve due to their ability to handle all possible structures. This article introduces a Web UI that simplifies the process by applying constraints to the JSON schema output format, enabling intuitive schema building through a user-friendly GUI.

JSON Schema generator for LLM JSON formattted output

OpenAI’s JSON Schema Option

Many LLM APIs, including OpenAI’s, offer JSON schema options to control output in JSON format. JSON schema is a format for defining the structure of JSON data, using keywords like type, properties, and required to specify data types, properties, and mandatory fields.

For instance, you can specify the following JSON schema to output information about a person, including their name and age, in JSON format:

{
  "type": "object",
  "properties": {
    "name": {
      "type": "string"
    },
    "age": {
      "type": "integer"
    }
  },
  "required": ["name", "age"]
}

However, manually writing JSON schema can be quite tedious, especially when dealing with complex data structures.

Building JSON Schema with a Web UI

The developed Web UI, implemented using React, features a GUI that allows for intuitive JSON schema building through drag-and-drop operations. Users can easily create a JSON schema by arranging the necessary elements on the GUI and configuring their properties.

Benefits of the Web UI

This Web UI offers several advantages:

  • Schema Definition without LLM: No need to rely on LLMs for schema generation; developers can design schemas according to their specific needs.
  • Intuitive Schema Building: The GUI-based operation simplifies schema building, even for users unfamiliar with JSON schema.
  • Easy Adjustments: The schema can be easily edited on the GUI, allowing for quick adjustments and modifications.
  • Support for Multiple LLMs: Compatible with JSON schema options for various LLMs, including OpenAI and Gemini.
  • Schema Sharing: Created schemas can be shared easily via URL.

Web UI Use Cases

The Web UI can be applied to various use cases. Here are a few examples:

You can freely experiment with the UI at https://ainoya.github.io/llm-output-format-gen. NOTE: Operations are implemented only on the front-end to prevent sending and recording data to the server.

Let’s consider a task that involves summarizing an article and simultaneously assigning topics to classify it. We’ll define the JSON output structure using the GUI.

You can configure three output types:

  • Text: Text format, used for outputting summaries, for example.
  • Single Select: Single selection format, used for classifying content, for example.
  • Multi Select: Multiple selection format, used for assigning topics, for example.

The following screenshot shows an example:

schema definition

Clicking the Generate Schema button outputs the JSON schema:

generate schema

Copy the schema to the clipboard using Copy to clipboard and let’s create a prompt.

Prompt example:

please read the text and output with following a JSON schema:

# Text

<Text>
...copy and paste from an article in https://ainoya-dev.pages.dev/posts/llm-application-security-practice-markdown-img-tag/
</Text>

# Output Schema

```json
{
  "type": "object",
  "additionalProperties": false,
  "properties": {
    "result": {
      "type": "object",
      "additionalProperties": false,
      "properties": {
        "items": {
          "type": "array",
          "additionalProperties": false,
          "items": {
            "oneOf": [
              {
                "type": "object",
                "additionalProperties": false,
                "properties": {
                  "type": {
                    "const": "text"
                  },
                  "key": {
                    "const": "summary"
                  },
                  "description": {
                    "const": "summarize following text."
                  },
                  "payload": {
                    "type": "object",
                    "properties": {
                      "value": {
                        "type": "string"
                      }
                    },
                    "required": [
                      "value"
                    ]
                  }
                },
                "required": [
                  "type",
                  "key",
                  "description",
                  "payload"
                ]
              },
              {
                "type": "object",
                "additionalProperties": false,
                "properties": {
                  "type": {
                    "const": "multipleSelect"
                  },
                  "key": {
                    "const": "tags"
                  },
                  "description": {
                    "const": "Topics of following text."
                  },
                  "payload": {
                    "type": "object",
                    "properties": {
                      "value": {
                        "type": "array",
                        "items": {
                          "type": "string",
                          "enum": [
                            "sport",
                            "technology"
                          ]
                        }
                      }
                    },
                    "required": [
                      "value"
                    ]
                  }
                },
                "required": [
                  "type",
                  "key",
                  "description",
                  "payload"
                ]
              }
            ]
          }
        }
      },
      "required": [
        "items"
      ]
    }
  },
  "required": [
    "result"
  ]
}

With GPT-4, this prompt generates the following result, formatted for use with the EAV pattern:

{
  "result": {
    "items": [
      {
        "type": "text",
        "key": "summary",
        "description": "summarize following text.",
        "payload": {
          "value": "The article discusses the potential security risks of rendering markdown images in LLM (Large Language Model) applications. It highlights how embedding images using markdown can expose sensitive data through image URLs, which may be exploited by malicious actors. The article provides methods to mitigate these risks, such as disabling image embedding entirely or whitelisting trusted hosts. Example TypeScript code is provided to demonstrate these approaches. The importance of implementing robust security measures, especially in applications dealing with user-generated content (UGC), is emphasized."
        }
      },
      {
        "type": "multipleSelect",
        "key": "tags",
        "description": "Topics of following text.",
        "payload": {
          "value": ["technology"]
        }
      }
    ]
  }
}

The definition used in this example can be shared as a URL using the Share URL feature. Here’s the URL for the schema shown in the example.

Future Development

The Web UI is still under development, and the following tasks remain:

Support for OpenAI’s Structured Outputs: Currently, the UI doesn’t support specifying options for OpenAI’s Structured Outputs because structured output doesn’t support the oneOf notation in JSON schema. It also doesn’t support the schema specification option for Gemini (Vertex AI). Only JSON output specification is possible via API options; schemas are used by directly inputting them into the prompt.

Caveats

There are reports that forcing complex JSON outputs or structured outputs on LLMs can degrade output quality. In simple tasks, based on my observations, the impact of quality degradation was minimal.

Conclusion

This article introduced a Web UI developed for controlling LLM JSON output. This UI simplifies the process of controlling LLM outputs in JSON format, making it easier to integrate them into various systems.

References