Lost in the Middle problem

[rossjillian]

The LLM forgets details in the middle of the prompt. Rather than dumping all text into one prompt, do something more intelligent to structure information search or summarize prior information. This weakness has been pointed out in the research: https://arxiv.org/html/2404.02060v1

One possible solution would be to optimize prompt engineering. Another possible solution is a multi-agent system (https://arxiv.org/pdf/2306.03314), in which we have specialized submodels, i.e. a cog.yaml specialist, a predict.py specialist, a debugging specialist, etc. Possible multi-agent frameworks include:

• AutoGen: https://github.com/microsoft/autogen
• CrewAI: https://github.com/crewAIInc/crewAI
• WorkGPT: https://github.com/team-openpm/workgpt

A proposed multi-agent structure is:

• file agent - reads files, orders by importance, and extracts relevant parts of the cog.yaml agent and predict.py agent
• cog.yaml agent - generates cog.yaml using extracted files, working examples, and PyPi grounding. Will test by trying to run cog run.
• predict.pyagent - generates predict.py using extracted files and working examples. Will test by trying to run cog predict.
• critic agent (optional, costly but could save development time) - reviews generated cog.yaml and predict.py before they're run
• debugging agent - reviews errors from cog run or cog predict and synthesizes feedback back to its respective agent
• weights agent

Concerns of using multi-agent systems include getting stuck in loops and the potential for high costs.

[joaquinariasco-lab]

Hi,

I read your thoughts on the "Lost in the Middle" problem and your proposed multi-agent structure for Cog/Predict. Your point about the Critic Agent being "costly but time-saving" is spot on, it’s the classic dilemma of agentic workflows.

I am building the Global Agentic Economy (GAEP), a protocol designed specifically to manage the concerns you mentioned: loops and high costs. Our system allows you to set "Micro-Budgets" for each specialized agent (like your Debugging or Critic agents). If an agent gets stuck in a loop, the economic constraint triggers a circuit breaker before your API bill explodes. Essentially, we are building the Financial Orchestration that makes your proposed structure viable for production.

Since you're looking into AutoGen and CrewAI, I’d love to share how we’re adding an Economic Layer to those exact frameworks to prevent the "high cost" risk you identified.

Would you be open to a brief chat on how economic guardrails can make multi-agent systems more reliable?

Best,
Joaquin