Thinking, Fast and Slow, with LLMs and PDDL
ChatGPT is never shy at pretending to perform deep thought, but — like our brain — might need additional tools to reason accurately
“ChatGPT can make mistakes. Check important info.” is now written right underneath the prompt, and we all got used to the fact that ChatGPT stoically makes up anything from dates to entire references. But what about basic reasoning? Looking at a simple tower rearranging task from the early days of Artificial Intelligence (AI) research, we will show how large language models (LLM) reach their limitations and introduce the Planning Domain Definition Language (PDDL) and symbolic solvers to make up for it. Given that LLMs are fundamentally probabilistic, it is likely that such tools will be built-in to future versions of AI agents, combining common sense knowledge and razor-sharp reasoning. To get the most out of this article, set up your own PDDL environment using VS Code’s PDDL extension and planutils planner interface and work along with the examples.
In a large language model (LLM), every character is literally conditioned on all previous characters of its response as well as the user’s prompt. Trained with almost everything that has ever been written, LLMs have become not only omniscient, but even witty. Yet, it usually does not take long to figure out that LLMs are very lazy and essentially refuse to really think about a problem. This can be illustrated with a simple example from “blocks…
