[Feature Request]:I am working with GraphRag I have added the support for the AWS s3 and other infra. Can we integrate that feature here? or is it exclusive to the Microsoft products?

[algosmith]

Do you need to file an issue?

• I have searched the existing issues and this feature is not already filed.
• My model is hosted on OpenAI or Azure. If not, please look at the "model providers" issue and don't file a new one here.
• I believe this is a legitimate feature request, not just a question. If this is a question, please use the Discussions area.

Is your feature request related to a problem? Please describe.

I am working with GraphRag I have added the support for the AWS s3 and other AWS products. Now it take the input from AWS s3 and save the output files there.

Can we integrate that feature here in code ? or is it exclusive to the Microsoft products?

Describe the solution you'd like

No response

Additional context

No response

[p971607]

Hi @algosmith, May I ask if you have integrated any other products besides AWS? I'm using GCP's Vertex AI, and I've been modifying the source code myself, but I've been having constant issues. It's been two days of sitting in front of the computer, checking logs, fixing errors, running again, only to face new errors. I'm feeling very discouraged - opening the computer to code has become something frightening, even with AI assistance. Am I just too stupid? If you have successfully integrated GCP's Vertex AI, especially using service accounts (with project_id and location), and got it running perfectly, I would be incredibly grateful if you could let me take a look. I'm literally in tears about this.

[algosmith]

No, we are building mainly with aws. So I don't have the code to integrate the GCP's vertex AI. I agree code is bit complex. You can search github repos maybe someone implemented what you are looking for.

[p971607]

@algosmith, After spending another day looking at the code, I have gained a better understanding. I now think that GCP might not be that crucial, and the AWS changes should be similar. I would like to know which parts you modified, so I can use that as a reference to modify the GCP part. Since I'm still having some difficulties getting it to work independently, could I take a look at your code?

[algosmith]

I will let you know. Give me a day. My repo is private. Is there any way I can share it privately? I don't want to make it public.

[basitanees]

Hi @algosmith, I am trying to implement s3 storage, if you have already implemented, I would be grateful if you could share your repo with me. My email is basitcod@gmail.com

[natoverse]

With the v3 release we have introduced a streamlined factory pattern across all of the major subsystems so it is very straightforward to implement, register, and use your own backend for storage, vectors, etc. We're not going to implement any new built-ins because it becomes a lot of maintenance, but I encourage you to post links to your implementations if you have them.