skip to content
ainoya.dev

Enhancing Tools with OpenAI API: Insights from Past Projects

/ 2 min read

openai , graphql , looker , streamlit , langchain

Early in the previous year, following the launch of OpenAI’s GPT-3.5, I initiated a project to improve existing tools’ user experience using the OpenAI API. This article focuses on two significant instances from this project.

Transforming Unstructured Data into Structured, Machine-Readable Formats

One of the key features I focused on was the ability to easily convert unstructured data into structured, machine-readable formats like JSON. This capability was utilized in two distinct applications, both of which were rapidly prototyped using LangChain and Streamlit.

Example 1: ainoya/graphql-gpt

Converting Natural Language Queries into GraphQL

  • The aim was to transform natural language questions into GraphQL queries.
  • In-context learning was employed, embedding the GraphQL schema within the prompt, to facilitate this conversion.
  • Additionally, the implementation included executing the resulting queries and retrieving results from a GraphQL server. A protective layer for query execution would likely be necessary to prevent data retrieval beyond the intended scope in real-world applications.

Example 2: ainoya/gpt-looker

Natural Language to Looker Queries

  • Similar to the first example, this project involved converting natural language questions into Looker queries.
  • LookML schemas were embedded in the prompt for in-context learning responses.
  • Due to the size of LookML definitions, there was a challenge of context overflow when included directly in the prompt. This was addressed by preprocessing LookML into segments, converting them into embeddings, and storing them on Chroma. During a query, the most relevant schema was searched and retrieved, a technique known as Retrieval Augmented Generation (RAG).
  • Given that Looker is a Google product, I anticipate similar functionalities may be implemented in tools like Gemini by Google DeepMind.

Conclusion

The experimentation highlighted the astonishing ease with which unstructured data processing, previously demanding specialized machine learning expertise, can now be implemented. Recent developments, like the Code Interpreter by OpenAI, further simplify the implementation of such AI-enhanced service experiences, promising widespread adoption and continuous improvement in the field.