[Feature] Expose local file path in Feishu inbound messages to enable agent file tools

[huer512]

🎯 The Goal / Use Case

When users send non-image files (documents, audio, video, etc.) via Feishu, PicoClaw currently:

• Downloads the file to a local temp path (e.g. /tmp/picoclaw_media/...), and
• Stores it in MediaStore, but
• Does not expose the local path to the agent in the message text.

As a result, the agent cannot easily use tools like read_file to open the file that the user just sent, because it has no way to discover the corresponding local path from the conversation context.

The goal is to:

• Let the agent see and use the local filesystem path of inbound Feishu files, so it can read, analyze, or transform user-uploaded files via existing file tools (e.g. read_file).

💡 Proposed Solution

For Feishu inbound messages of type:

• file
• audio
• media (video)

Include the resolved local filesystem path in the human-visible message content in a structured tag format.

For example:

• File with filename:
  report.pdf [file:/tmp/picoclaw_media/om_xxx-filekey.pdf]

• File without filename:
  [file:/tmp/picoclaw_media/om_xxx-filekey]

Similarly for audio / video:

• recording.ogg [audio:/tmp/picoclaw_media/...]
• video.mp4 [video:/tmp/picoclaw_media/...]

This gives the agent a stable, parseable way to locate the exact file that was received.

🛠 Potential Implementation (Optional)

Current Feishu inbound flow (simplified):

• handleMessageReceive:
  • content := extractContent(messageType, rawContent)
  • mediaRefs := c.downloadInboundMedia(...) → returns []string of media:// refs
  • content = appendMediaTags(content, messageType, mediaRefs)

In handleMessageReceive (for MsgTypeFile, MsgTypeAudio, MsgTypeMedia), after resolving mediaRefs, instead of only appending a generic [file] / [audio] / [video] tag, derive the local path via MediaStore.Resolve(ref) and build structured tags:

• Get store := c.GetMediaStore()
• For each ref in mediaRefs:
  • localPath, err := store.Resolve(ref)
  • Build tags like: [file:localPath], [audio:localPath], [video:localPath]
• Append these tags to content:
  • If there is filename text already:
    content = content + " " + strings.Join(parts, " ")
  • Else:
    content = strings.Join(parts, " ")
• Fallback: if Resolve fails or mediaRefs is empty, keep current appendMediaTags behavior.

This keeps behavior unchanged for images and for cases where the local path is not available, while exposing extra power when it is.

🚦 Impact & Roadmap Alignment

• This is a Core Feature
• This is a Nice-to-Have / Enhancement
• This aligns with the current Roadmap

Rationale:

• File understanding and manipulation is a core capability for personal agents.
• Enabling agents to directly read user-uploaded files via existing tools is central to many workflows (code review, document analysis, data inspection, etc.).

🔄 Alternatives Considered

1. Add a dedicated tool like resolve_media_ref exposed to the agent

   • The agent would call a tool with media://... and receive the local path.
   • This is cleaner from a separation-of-concerns perspective but requires adding and wiring a new tool interface.
   • It also assumes the agent always remembers or can see the raw media:// refs, which is not always true.

2. Embed file content directly as base64 for all file types

   • Similar to how images are converted to data:image/...;base64,....
   • This can be very heavy for large documents/binaries and is not ideal for tools like read_file that want a real path.

3. Store the path only in metadata, not in message content

   • Safer from a user-facing perspective, but the LLM/agent would still lack a direct, parseable reference in the text channel.

The proposed [file:/path] approach is the simplest way to make this power available to the agent immediately, leveraging existing tools.

💬 Additional Context

• Feishu inbound media is already downloaded to a temp directory via downloadResource (e.g. /tmp/picoclaw_media), and registered in MediaStore with media:// refs.
• For images, a separate pipeline converts refs to base64 data URLs for multi-modal LLMs; this proposal focuses specifically on non-image file types (documents, audio, video), where path-based tools like read_file are more appropriate.
• The structured tag format [file:/absolute/path] is:
  • Easy for the agent to parse,
  • Backward-compatible with existing plain-text content, and
  • Does not change the underlying MediaStore or storage semantics.

[dataCenter430]

Hi, @huer512 thanks for reporting this.
I'd love to work on this.
cc: @alexhoshina

[alexhoshina]

Hi, @dataCenter430 I'd like to first understand the solution you plan to implement.

[alexhoshina]

The MediaStore system was originally designed by me. My idea was to shield the actual file paths from the agent layer, but in hindsight, this might have been over-engineered. I'd like to hear your thoughts on this.

[dataCenter430]

Hi, @dataCenter430 I'd like to first understand the solution you plan to implement.

Hi, @alexhoshina thanks for your messages.
Hmm, based on the analysis of the codebase, I have one approach for this feature:

• modify appendMediaTags for non-image types (file, audio, media) to resolve the media:// ref back to its local path and embed it in the tag. Concretely, the function signature gains access to the MediaStore, and the output changes from: report.pdf [file] to: report.pdf [file:/tmp/picoclaw_media/msgID-filekey.pdf]
  Fallback to the current generic [file] / [audio] / [video] if resolution fails or the store is nil. Images are left untouched (they already work well via base64).
  This is a ~20-line change in one function, fully backward-compatible.

The MediaStore system was originally designed by me. My idea was to shield the actual file paths from the agent layer, but in hindsight, this might have been over-engineered. I'd like to hear your thoughts on this.

MediaStore as internal plumbing (ref tracking, lifecycle, cross-channel) , well-designed. Path-shielding at the agent boundary, an abstraction that costs more than it protects.

[dataCenter430]

what do you think of it?
shall I go ahead and implement it?

[alexhoshina]

I want to have dereferencing happen at the agent layer. Would that be more universal? However, based on my original conception, most multimodal models can handle images, so I think image dereferencing can happen anywhere. For formats like PDF or ZIP that LLMs cannot process, perhaps we don't perform dereferencing. Instead, we let the model pass them to tools when calling them, and the tools handle the dereferencing. But this architecture might cause issues with skill support. What do you think?

[dataCenter430]

Your instinct about the skill issue is right, and that's the dealbreaker.
If non-image files stay as media:// refs and tools handle dereferencing, then: every file-capable tool needs MediaStore access and media:// resolution logic, every skill/prompt that involves files needs to teach the agent "pass media:// refs to tools, not paths," and any third-party or user-authored skill that touches files must understand this protocol. That's a lot of implicit coupling to a concept the LLM has no natural understanding of.
Compare with just exposing the path: the agent sees /tmp/picoclaw_media/report.pdf, calls read_file("/tmp/picoclaw_media/report.pdf"). No special protocol, no tool changes, skills work naturally with file paths because paths are a universal concept LLMs already understand.
My take: dereferencing at the agent layer is the right instinct for locality, but the split you're describing (images = dereference, others = defer to tools) creates two media protocols that skills must navigate. The simpler move is: dereference images to base64 (as today), dereference non-image files to paths (not content), both at the agent layer. The agent gets a path it can pass to any tool, no tool needs MediaStore awareness, skills just work.

[alexhoshina]

@dataCenter430 Thank you for taking the time to chat with me! I've learned a lot and given it a lot of thought! I'll assign this issue to you. Thank you so much for your enthusiasm!
So we'll dereference all references at the agent layer?

[dataCenter430]

@alexhoshina You're welcome.

So we'll dereference all references at the agent layer?

Yes. All dereferencing at the agent layer, in resolveMediaRefs. Two outcomes based on type:

• Images: base64 data URL (current behavior, unchanged)
• Everything else :local file path injected into the message content
  One place, one pass, all channels benefit. Tools stay dumb, skills stay simple.

I'm so happy to contribute to picoclaw with you, thanks for your assignment. 👍