Using Google Calendar with Natural Language via LangChain

By Andrej Staš

Embarking on a journey to gain efficient management of Google Calendar events with the LangChain framework

In the modern era, where digital calendars hold the threads of our scheduled lives, and with AI trending and growing in prominence, being able to manage your Google Calendar using natural language beckoned us into exploration. The objective was clear and ambitious: to seamlessly integrate voice instructions into software, allowing for intuitive and efficient management of Google Calendar events through natural language commands.

The challenge

Despite the existence of advanced virtual assistants like Google Assistant, Alexa, and Siri, a significant gap persists. These platforms, while proficient in understanding and executing basic commands, often falter when it comes to intricate integration within diverse software environments. The challenge lay in transcending the capabilities of these existing solutions, crafting a system adept at not only comprehending natural language commands but also flawlessly executing them within the context of Google Calendar management.

The journey

Embarking on this journey, we envisioned a system unbounded by the limitations of current technologies, one that synergises impeccably with Google Calendar, transforming verbal instructions into precise actions, and making calendar management a breeze. This vision propelled us into a comprehensive exploration of potential solutions, each with its own merits.

The objective

Our objective transcended mere voice command recognition. We sought to create a robust system capable of seamless integration with any software environment, ensuring that managing Google Calendar events with natural language commands is not just a possibility, but a seamless and intuitive reality. This endeavour led us through various avenues, from considering Large Language Models (LLMs) to exploring frameworks like LangChain, each holding the promise of realising our vision.

In the following sections, we will delve deeper into the options we evaluated, the challenges we encountered, and the solution that ultimately stood out, marking the culmination of our quest for seamless, natural language-based Google Calendar management.

Considering a Large Language Model (LLM)

Initially, we contemplated utilising a Large Language Model (LLM), such as the ones exposed via OpenAI APIs, to manage Google Calendar events using natural language commands. Despite the LLM’s prowess in understanding and generating human-like text, we encountered a significant hurdle. LLMs, by default, cannot interact with external systems like Google Calendar. They can simulate conversations about creating or retrieving calendar events but cannot execute these tasks, as they lack access to a user’s Google Calendar. This limitation curtailed their utility for our project.

Embracing a framework over a LLM

This challenge led us to explore frameworks that operate over a Large Language Model. Frameworks like LangChain transcend the limitations of standard LLMs. They offer robust options, including the integration of plugins capable of accessing external APIs and executing functions beyond the reach of a typical LLM. This feature enables interaction with external data sources like Google Calendar, allowing for the manipulation of data based on the model’s input and output. This capability unlocked a realm of possibilities, making such frameworks an invaluable asset for our project.

Choosing LangChain

After a comprehensive assessment of our options, we concluded that LangChain was the optimal solution for our needs. In the subsequent sections, we will delve deeper into LangChain’s functionality and elucidate how we leveraged it to develop an interactive and dynamic application. This application not only processes natural language but also adeptly manipulates real-world data.

What is LangChain?

LangChain is a framework developed for creating applications powered by language models. It offers two versatile platforms for development: the Python version and a JavaScript version.

Objective of LangChain

The primary objective of LangChain is to streamline the creation of applications that not only harness the power of language models but also dynamically interact with diverse data sources and environments. This functionality allows LangChain to establish seamless connections with various data sources, fostering dynamic interactions and the development of more agile, data-aware, and agent-based applications.

Features of LangChain

LangChain is equipped with an array of pre-built modules, including schema, models, prompts, indexes, memory, chains, and agents, simplifying the application development process.

Flexibility and extensibility

The essence of LangChain’s prowess lies in its remarkable flexibility and extensibility. It grants developers the liberty to employ LangChain in a multitude of ways, enabling the implementation of unique functionalities tailored to meet specific project requirements. This adaptability underscores LangChain’s commitment to providing a robust and versatile framework for developing sophisticated language model-powered applications.

Understanding and utilising Tools in LangChain

In the ecosystem of LangChain, a “tool” emerges as a pivotal element in the seamless development and operation of language model applications. It stands as an abstraction around a function, meticulously crafted to synergise with a language model. Defined by a singular text input and output, a tool in LangChain epitomises seamless communication, bridging the gap between the language model and external interfaces.

Active components

Contrary to being mere passive elements, tools in LangChain are active entities, dynamically enabling language models to interact with their multifaceted environment. They are engineered to execute specific tasks, amplifying the language model’s capacity to establish connections with external resources, fetch pertinent information, or adeptly manipulate data.

Enhancing interaction and functionality

By empowering language models to actively engage with their surroundings and perform diverse tasks, tools in LangChain significantly augment the interaction and functionality of language model applications. They play a crucial role in enhancing the efficiency, effectiveness, and dynamism of applications, ensuring they operate optimally within the intended environment.

Example: Calculator tool

LangChain is not just a framework but a comprehensive ecosystem, equipped with a diverse array of built-in tools such as aws_lambda, google_custom_search, IFTTT_webhook, SQL Toolkit, web browser, and Zapier. This extensive toolkit broadens the horizon of potential applications, enabling developers to create more functional and interactive language model applications.

Utilising the Calculator tool

Take the Calculator tool as an illustrative example. Unlike a standalone language model that might only approximate mathematical computations, the Calculator tool in LangChain delivers precise and accurate results for mathematical expressions.

How does the Calculator tool work?

Here is a basic implementation of a “Calculator” tool included in LangChain:

import { Parser } from "expr-eval";
import { Tool } from "langchain/tools";

export class Calculator extends Tool {
    constructor() {
        super(...arguments);
        this.name = "calculator";
        this.description = "Useful for getting the result of a math expression. The input to this tool should be a valid mathematical expression that could be executed by a simple calculator.";
    }
    /** @ignore */
    async _call(input) {
        try {
            return Parser.evaluate(input).toString();
        }
        catch (error) {
            return "I don't know how to do that.";
        }
    }
}

And here it is in use:

import { initializeAgentExecutorWithOptions } from "langchain/agents";
import { OpenAI } from "langchain/llms/openai";
import { Calculator } from "langchain/tools/calculator";

const model = new OpenAI({
  temperature: 0,
  openAIApiKey: OPENAI_API_KEY,
});

const tools = [
  new Calculator(),
  //...additional tools added in to the agent
];

const agent = await initializeAgentExecutorWithOptions(tools, model, {
  agentType: "zero-shot-react-description",
  verbose: true,
});

const input = `what is the square root of 100 * 3`;

const createResult = await agent.call({ input });

// Returns "The square root of 100 * 3 is 30"

When a user prompts LangChain to calculate a mathematical expression, the system quickly discerns the computational nature of the request and engages the Calculator tool based on its description. This tool is specifically dedicated to processing mathematical expressions, ensuring the delivery of exact and reliable results. The definition of the “Calculator” tool within a code snippet instructs LangChain about the expected input in the description. By extending the base “Tool” class and overriding the _call method, the Calculator tool is empowered to execute when called upon. This _call method evaluates the input mathematical expression and returns the result, or an error message if the expression is invalid.

This streamlined process exemplifies how tools within the LangChain framework can significantly enhance the capabilities of a language model, ensuring precise and efficient task execution.

Our tool: create and retrieve events from Google Calendar

In our case, we leveraged this functionality to create a GoogleCalendar tool. This tool, specifically designed to interact with Google Calendar, has the ability to:

• Create events from a user’s calendar

• Retrieve events from a user’s calendar

By integrating this tool with our LangChain application, we were able to enhance the application’s capabilities, allowing it to not just process and generate language, but also to interact with real-world data, making it a more useful and practical assistant for users.

A case study in utilising LangChain tools

Initially, this tool had a classification functionality that determined one of two different types of user prompts:

• Creating an event

• Retrieving an event from Google Calendar

The tool had to classify the type of user request before performing the appropriate action. This setup, although functional, had its challenges, including latency in responses and occasional infinite loops due to the dual functionality.

Upon further analysis, we realised a more efficient approach would be to split the GoogleCalendar tool into two distinct tools, each dedicated to a specific function — one for creating events and another for retrieving events. This separation allowed the LangChain agent to handle the classification of user prompts, directing the request to the appropriate tool based on the user’s intent.

By leveraging LangChain’s flexibility, we were able to refine our tool’s design, resulting in faster response times and eliminating the issue of infinite cycles. This example illustrates the power of LangChain tools and how they can be optimised to provide more efficient and effective solutions.

// Tool for creating events
class GoogleCalendarCreateTool extends GoogleCalendarBase {
        constructor() {
            super(...arguments);
              this.name = "google_calendar_create";
            this.description = `A tool for creating Google Calendar events and meetings....
    Input example...
    Output example...`;
          }

    // This is the function that is run when the tool is called
    async _call(query) {
        const auth = await this.getAuth() // Authenticate with Google API
        const model = this.getModel() // Get the current language model

        // Run function to create an event
        return await runCreateEvent(query)
    }
}

// Tool for viewing events
class GoogleCalendarViewTool extends GoogleCalendarBase {
        constructor() {
            super(...arguments);
            this.name = "google_calendar_view";
            this.description = `A tool for retrieving Google Calendar events and meetings....
    Input example...
    Output example...`;
      }

      // This is the function that is run when the tool is called
      async _call(query) {
          const auth = await this.getAuth() // Authenticate with Google API
          const model = this.getModel() // Get the current language model

          // Run function to view events
          return await runViewEvents(query)
      }
}

In the new version of the tool, we split the Google Calendar tool into two separate tools: one for creating events (GoogleCalendarCreateTool) and another for viewing events (GoogleCalendarViewTool). Now, the tool selection and therefore the action (creating or viewing events) is performed externally by LangChain, leading to increased performance and reliability. The system can now more quickly select the appropriate tool without the overhead of an internal classification step, improving the speed of operation and the overall user experience.

Input/output examples for tools

To illustrate how these tools interact with user prompts, let’s look at some examples of inputs and outputs for both the GoogleCalendarCreateTool and the GoogleCalendarViewTool.

GoogleCalendarCreateTool

The GoogleCalendarCreateTool takes natural language prompts for creating events in Google Calendar. Here’s an example:

Input: “Create a new event tomorrow morning at 8am called ‘breakfast with friend’.”

In this input, the user is instructing the tool to create a new event. The tool parses the details from the prompt: the date (tomorrow), time (8am), and the title of the event (‘breakfast with friend’).

Output: “The event has been scheduled.”

This output message indicates that the event was created successfully in the user’s Google Calendar.

GoogleCalendarViewTool

The GoogleCalendarViewTool, on the other hand, retrieves events from the user’s Google Calendar. Here’s an example:

Input: “What are my meetings tomorrow?”

In this case, the user is asking for information about their scheduled meetings for tomorrow. The tool will retrieve this data from the user’s Google Calendar.

Output: “You have three meetings tomorrow:

1. ‘Team Sync’ at 10:00 AM

2. ‘Project Review’ at 2:00 PM

3. ‘Client Call’ at 4:30 PM.”

The output will list all events (meetings) scheduled for the specified date, providing the title and time for each event.

These examples illustrate the interaction between the LangChain tools and the user prompts. They highlight the ability of the tools to interpret natural language instructions, perform actions, and respond in a helpful and understandable way.

Practical application of the Google Calendar tools

Imagine the endless possibilities when developing an application that seamlessly integrates Google Calendar functionality. Whether it’s a task management platform, a health and wellness app, a social media scheduler, or a custom-built smart home system, incorporating the ability to schedule, view, and manage calendar events using natural language commands can revolutionise the user experience.

Unlocking new possibilities with LangChain

By integrating LangChain with the GoogleCalendarCreateTool and GoogleCalendarViewTool into your application, you unveil a world of innovative user interactions. These tools, designed for intuitive and accessible calendar management, allow users to effortlessly handle their events through natural language instructions.

Practical scenarios

Consider a task management platform where a user types, “Schedule a project review meeting for the upcoming Monday at 10 AM.” The GoogleCalendarCreateTool springs into action, scheduling the event in the user’s Google Calendar. Or envision a health and wellness app where a user inquires, “What are my scheduled workouts for this week?” The GoogleCalendarViewTool retrieves and presents this information, ensuring the user stays informed and organised.

Enhancing accessibility with speech-to-text integration

We can expand the utility of these tools by integrating them with speech-to-text capabilities, allowing users to manage their Google Calendar using only their voice. This enhancement not only simplifies the process but also amplifies accessibility, making calendar management a breeze for users with visual impairments or those unable to use traditional input methods. It’s a boon for users who are multitasking or prefer hands-free interaction with their devices.

By incorporating our Google Calendar tools into your applications, you can offer users a highly convenient and intuitive way to manage their time and tasks. The integration of these tools enables effortless scheduling and management of events, leading to improved productivity and a more seamless user experience.

Comparing prompts: Google Calendar tool vs. Google Assistant

When it comes to retrieving calendar events using natural language prompts, both the Google Calendar tool developed with LangChain and Google Assistant offer unique capabilities and challenges. Here are some points from our work where we compared both tools on specific prompts.

Example prompts and responses

Creating a reminder, using the following prompt:

Create a reminder to walk the dog at half three

• GoogleCalendarTool v1: Good, except wrong day and created an event not a reminder:

  • Returns { result: { output: 'N/A' } } but with "text": "The reminder to walk the dog has been successfully created.\nFinal Answer: N/A" in an earlier log

  • Creates an event from 15:30 to 16:30 the next day titled “Walk the dog”

• GoogleCalendarTool v2: Good, except it created an event, not a reminder:

  • Returns { result: { output: 'I have successfully created a reminder for walking the dog at half three using the google_calendar_create tool.' } }

  • Creates an event from 15:30 to 16:30 today titled “Walk the dog”

  • We tried various other times: half one,half eleven, half nine, and all events were entered at the next occurrence of that time from that day (so half nine created the event at 21:00)

• Google Assistant: Good result, except wrong time and poor casing of title:

  • Pops up a confirmation box saying “Got it, I’ll remind you today at 3pm” and “Reminder set: Walk the dog, today at 3:00pm” (title was sentence case in confirmation box)

  • Created a “reminder” titled “walk the dog” (title is all lowercase in actual calendar) at 3pm (not 3:30pm)

Delete a meeting in your calendar, using the following prompt:

Create a two hour event titled "Launch planning" this time next week

• GoogleCalendarTool v1: Completed with the correct day but the wrong time:

  • Returns { result: { output: 'The event was successfully created and the link to the event was returned.' } }

  • Created an event titled “Launch planning” from 10am to 12pm on the following Wednesday

• GoogleCalendarTool v2:

  • Returns { result: { output: 'The event has been successfully created.' } }

  • Created an event titled “Launch Planning”, but created it for 10:42am – 12:42pm: one hour out, as if it applied UTC time as BST), and one day too early

• Google Assistant: Complete failure:

  • Returns Google search results, where the top result is an article “How to create a successful product launch”

  • It’s not clear what search terms it actually searched for

  • Has a few suggested alternate actions but they’re all just related search terms

Both tools exhibit strengths and weaknesses in handling prompts for calendar management. Our tool, especially its version 1, tends to offer more structured and detailed outputs. However, it may face challenges with complex queries and technical limitations. Google Assistant provides concise and clear responses but may sometimes lack in understanding and processing the requests effectively. The tailored and extendable nature of the Google Calendar tool with LangChain showcases significant potential for meeting diverse and specific user needs.

Contributing our GoogleCalendar tools to LangChain

In the spirit of collaborative development and enhancing the LangChain ecosystem, we are excited to contribute two pivotal tools to the LangChain community: the GoogleCalendarCreateTool and the GoogleCalendarViewTool. These tools are crafted with precision to seamlessly integrate Google Calendar functionality into applications, enabling natural language processing for scheduling and viewing calendar events.

GoogleCalendarCreateTool

The GoogleCalendarCreateTool is designed to simplify the event creation process in Google Calendar. Users can effortlessly schedule events using natural language commands, eliminating the need for manual date and time entry. This tool interprets the user’s command, translates it into an actionable request, and schedules the event in Google Calendar, ensuring accuracy and efficiency in event management.

GoogleCalendarViewTool

On the other hand, the GoogleCalendarViewTool is engineered to provide users with an easy and intuitive way to view their scheduled events. By simply asking about their schedule, users can retrieve detailed information about their upcoming events, enhancing their ability to manage their time effectively.

Enhancing LangChain’s ecosystem

By contributing these tools to LangChain, we aim to enrich the ecosystem, providing developers and users with advanced functionalities for Google Calendar management. The integration of these tools into the LangChain framework allows for the development of more interactive, accessible, and user-friendly applications.

Collaborative development

We believe in the power of community, collaborative development and open source. Contributing the GoogleCalendarCreateTool and GoogleCalendarViewTool to LangChain was an obvious next step for us at NearForm once we had completed our experiments.

Conclusion

Through our journey with LangChain, we have found it to be a promising and potent framework for developing applications with precise expected outcomes. In contrast to models like AutoGPT, which solely relies on a user’s prompt for its operation, LangChain provides a platform where we can define a more structured approach to natural language processing and application interactions.

Our experience in building the Google Calendar tool provided us with numerous insights, notably on how to approach classifications and tool structures within LangChain. We discovered the benefits of dividing functionality into smaller, more focused tools as opposed to creating a monolithic tool that tries to do it all. This segmentation not only improves performance but also reduces complexity and potential error points.

However, we recognise that there is room for improvement. Our solution is not perfect, and we continue to refine it, tackling its shortcomings and optimising its performance.

Our work on these tools has now been reviewed and merged into LangChain.js, available from release 0.0.159. You can view the documentation here