LangChain - the Guide of LLMs

An astonishingly popular new library - what is it and what problem does it solve?

Warning (Feb 2024 edit) - this post was written when LangChain was a hot, new library. It has since then matured and changed a lot. However, the main premise of this article is still valid, so I’ll leave it be.

---

With LLMs such as BARD, GPT-4 or Claude constantly hitting every tech headline, many companies started wrapping them into their core product offerings. Turns out that what powers many AI features under the hood is one open source library.

The core idea

LLMs are absolutely great at producing text ...which may or may not be accurate - they hallucinate a lot. They also cannot act on their own or use additional tooling like web search. If we gave LLMs additional sources of knowledge or computation AND forced them to use them each time they produce an answer, then both of these problems could be mitigated.

That's the core idea behind LangChain - partially automated prompt engineering.

But how does it work?

First, LangChain takes your initial prompt - like a question or an action to perform. Then, instead of immediately querying LLM to give you an answer, it forces the LLM to generate intermediate reasoning steps that could lead then to a true answer ("let's think it through step-by-step"). The LLM plans their answer ahead and then constantly loops to find it.

At each loop it is presented with the possible tools and sources that they can use to find the answer. These tools may consists of a Google Search, Python interpreter, Bash terminal, a database of internal documents, a cache, an external API - anything. Additionally, after each Action performed, a LLM also produces an internal Thought (!). This thought could be "I need more data - maybe I'll use Google?", "I only partially answered the question", "I think I know the final answer" and more.

Only after the LLM "thinks" that their job is complete, you get the answer.

Here’s an example of the library in action, taken from their docs:

 'Question: What is the difference between the Illinois and Missouri orogeny?',
------------------- loop
 'Thought 1: I need to search Illinois and Missouri orogeny, and find the difference between them.',
 'Action 1: Search[Illinois orogeny]',
 'Observation 1: The Illinois orogeny is a hypothesized orogenic event that occurred in the Late Paleozoic either in the Pennsylvanian or Permian period.',
------------------- loop
 'Thought 2: The Illinois orogeny is a hypothesized orogenic event. I need to search Missouri orogeny next and find its details.',
 'Action 2: Search[Missouri orogeny]',
 'Observation 2: The Missouri orogeny was a major tectonic event that occurred in the late Pennsylvanian and early Permian period (about 300 million years ago).',
------------------- loop
 'Thought 3: The Illinois orogeny is hypothesized and occurred in the Late Paleozoic and the Missouri orogeny was a major tectonic event that occurred in the late Pennsylvanian and early Permian period. So the difference between the Illinois and Missouri orogeny is that the Illinois orogeny is hypothesized and occurred in the Late Paleozoic while the Missouri orogeny was a major']

>>> retrun answer

Under the hood ...it is all just a clever prompt (on top of the one you initially provided) running in a loop:

Absolutely ingenious!

Metaphorically, it's like a teacher that forces the student to carefully plan how a task can be solved, gather additional knowledge and then to constantly reevaluate the state of their task, before they give the final result.

Of course, you can add your own tools and customize the prompt for a LLM.

Wait, isn’t that what ChatGPT Plugins are?

Essentially ...yes. However, this library predates their Plugins. Turns out the concept works so well that OpenAI implemented it too.

Is the library for ChatGPT only?

No! That’s the crucial difference.

While ChatGPT has gotten absolutely mind blowing publicity lately, it isn't the only LLM available on the market. The model that LangChain uses can be arbitrary. Companies such as Anthropic, Cohere or (soon) Google also have astonishingly great LLMs capable of many things.

TL;DR and where to find LangChain?

LangChain queries a LLM in a loop with a clever prompt and lets it use predefined tools and utilities to better solve incoming tasks. Give it a shot here.

Do you know which LLM-related library I'll cover next? A hint - it recently rebranded their initial name. 🤫