skip to content
ainoya.dev

Automating My Note Tagging with LLMs

/ 2 min read

llm , automation , tagging , obsidian , vertex ai , gemini , typescript , note taking

I’ve been using tags to organize and connect my notes locally. This has been especially helpful when navigating related information within Obsidian. While tagging has been useful, it’s also been a manual and tedious process. I previously considered using morphological analysis to extract keywords as tags, but the results were often noisy and didn’t capture the core meaning of my notes.

Recently, I started experimenting with LLMs to see if they could help. My idea was to feed my notes into an LLM and use prompts to extract the most important topics, which could then be used as tags. The results were promising!

I implemented this solution in my PocketMD project, leveraging Vertex AI to generate tags and append them to the frontmatter of my notes. Both Vertex AI and Gemini offer the ability to specify the output format using JSON and schemas. However, I found that this option is only available with Gemini Pro and not Gemini Flash. Here’s an example of the schema definition:

  const schema: ResponseSchema = {
    type: FunctionDeclarationSchemaType.OBJECT,
    properties: {
      tags: {
        // string array
        type: FunctionDeclarationSchemaType.ARRAY,
        // @ts-ignore
        items: {
          type: FunctionDeclarationSchemaType.STRING,
        },
      },
    },
  };

I encountered an error when trying to add a type to the array elements using the TypeScript types provided by @google-cloud/vertexai. I had to use @ts-ignore as a temporary workaround.

It’s worth noting that advanced JSON schema features like allOf and oneOf didn’t work for API validation, both with Vertex AI and OpenAI’s structure output (as of August 25, 2024).

While the LLM generally adhered to the specified output format, I still needed to refine the values using prompts. For instance, even when specifying that tags should start with a hashtag, the LLM occasionally missed this or added extra double quotes. I ended up using regular expressions to clean up the generated output.

This implementation has significantly improved my workflow. I can now automatically tag both new and existing notes with a simple script, making it much easier to connect ideas and revisit past information. This has been a game-changer for organizing my thoughts and staying on top of my ever-growing collection of notes.

References