Thought-Augmented Reasoning through Buffer of Thoughts (BoT)

Single-query reasoning methods can be implemented using frameworks such as Chain-of-Thought (CoT), which improves LLM reasoning through a series of intermediate reasoning steps, and Few-shot Prompting, which aims at providing query-relevant examples to support response generation. 

Multi-query reasoning methods employ the use of query decomposition into a set of sub-questions. These sub-questions are answered through LLM and the combined knowledge is used to answer the original query. Graph-of-Thoughts (GoT) and Tree-of-Thoughts (ToT) are some examples of multi-query reasoning frameworks. GoT utilises the idea of an arbitrary graph to model the information generated by an LLM, termed LLM thoughts, and combines these arbitrary LLM thoughts into synergistic outcomes. Tree-of-Thought (ToT) on the other hand, generalises over the CoT framework and allows a language model to perform decision-making by considering multiple reasoning paths.

Comparison between CoT, ToT and GoT Methods

Single-query and multi-query reasoning processes are limited because of their reasoning structures and examples. These limitations can be avoided using the Buffer of Thoughts (BoT) approach which implements a meta-buffer, a library containing a series of high-level thoughts (thought-template). These high-level thoughts are refined from different problem-solving processes and are shareable across multiple tasks. For each problem, a relevant thought template is used with a specialised reasoning structure for effective thought-augmented reasoning. BoT also uses a buffer manager for dynamically updating the meta-buffer, enhancing it as more problems are solved. 

Comparison between Single-query, Multi-query and BoT

BoT improves reasoning accuracy and efficiency using informative historical reasoning structures, without needing to build from scratch. The process from thought-retrieval to thought instantiation is the same as the human thought process, enabling the LLM to address similar problems consistently, significantly improving the model's robustness and precision. 

BoT Reasoning Process

Components under BoT

Buffer of Thoughts implements three primary components to implement the thought-augmented reasoning process: 

Problem Distiller - During the reasoning phase, LLMs suffer from three primary challenges - extraction of vital information, understanding potential constraints and applying accurate reasoning. BoT uses a problem distiller for extracting problem-specific information along with the relevant constraints. The key elements extracted from input tasks, using a problem distiller, are the parameters and variables for problem-solving, the objectives of the input problem and their corresponding constraints. This information is crucial for decomposing problems and making them easier for the subsequent components to operate. 

Meta Buffer - This is a library containing a series of high-level thoughts in the form of thought templates. These templates are obtained from problem-solving processing by the buffer manager. The most relevant thought template is retrieved from the meta buffer depending on the problem. 

Buffer Manager - Summarises the entire problem-solving process and distils high-level thoughts for increasing the capacity of meta-buffer. It uses a three-step process - identifying and summarising the problem challenges, describing the solution steps and providing a proposition of a solution template that can be reused on similar problems. 

Buffer Manager’s Operation Example

Evaluation Benchmarks

BoT achieves significant performance improvements over previous prompting methods across multiple challenging benchmarks such as Game of 24, and Checkmate-in-One. 

BoT Benchmarks 

In terms of inference time, BoT is considerably less than conventional multi-query methods such as ToT.  

Logarithmic Inference Time Comparison

The evaluation of the trade-off between model size and performance with Llama3-8B and Llama3-70B on three challenging benchmarks shows that BoT+Llama3-8B has the potential to surpass the Llama3-70B model. 

The trade-off between model size and performance

The ablation study based on disabling the problem distiller shows an accuracy decline on benchmark problems such as Game of 24. When the meta buffer is disabled, both Llama3-70B and GPT-4 models show a decline in performance as well. 

Ablation study of problem distiller

Ablation study of meta buffer

Final Words

Buffer of Thoughts is a valuable approach for LLMs tackling complex tasks that require multi-step reasoning or information retrieval using thought-augmented reasoning. This significantly increases the accuracy, efficiency and robustness of LLMs. The BoT approach demonstrates state-of-the-art performance on ten challenging task benchmarks and offers great value for further research. 

References

1. Buffer of Thoughts Research Paper
2. BoT Git Repo
3. Chain-of-Thought Prompting Method
4. Graph-of-Thoughts
5. Few-shot Learning with Prompting Methods
6. Reasoning with Language Model Prompting