Paper Notes: LLM Powered Autonomous Agents

LLM POWERED AUTONOMOUS AGENTS

1. AGENT SYSTEM OVERVIEW

• 计划：任务分解，自我学习

• 记忆：短期（上下文），长期（信息获取）

• 工具：APIs for 额外的信息

2. PLANNING

2.1 Task Decomposition

• 链式思维：

The model is instructed to “think step by step” to utilize more test-time computation to decompose hard tasks into smaller and simpler steps.

 

• 树型思维：

by exploring multiple reasoning possibilities at each step

 

2.2 Self-Reflection

• 自省，在迭代中通过复盘来进行能力提升。

• ReAct

integrates reasoning and acting within LLM by extending the action space to be a combination of task-specific discrete actions and the language space. 融合任务行动空间和语言空间。

 

• Reflection

a framework to equips agents with dynamic memory and self-reflection capabilities to improve reasoning skills. Reflexion has a standard RL setup, in which the reward model provides a simple binary reward and the action space follows the setup in ReAct where the task-specific action space is augmented with language to enable complex reasoning steps. 强化学习机制+LLM 实现可理解 self-reflection。

(View Source)

• Chain of Hindsight

encourages the model to improve on its own outputs by explicitly presenting it with a sequence of past outputs, each annotated with feedback. 事后复盘

 

3. MEMORY

3.1 Memory Types

• Sensory Memory（早期，图像/声音）：

This is the earliest stage of memory, providing the ability to retain impressions of sensory information (visual, auditory, etc) after the original stimuli have ended.

 

• Short-Term Memory（STM）：

It stores information that we are currently aware of and needed to carry out complex cognitive tasks such as learning and reasoning.

 

• Long-Term Memory (LTM) :
• Explicit/Declarative memory: life events, facts and concepts
• Implicit/Procedural memory: unconscious and involves skills and routines that are performed automatically

3.2 Maximum Inner Product Search(MIPS)

• 一些算法（暂时不关注这部分内容，不是我的领域）：
• LSH
• ANNOY
• HNSW
• FAISS
• ScaNN

4.TOOL USE

• HuggingGPT

5. CASES