[BUG] S3.client settings missing from list of repository-s3 settings

[aphilippejolivel]

Describe the bug

I have a problem right now using Oracled Bucket with the remote storage

Exception occurred while uploading remote index paths for 
   indexes=[top_queries-2025.11.30-85662/OlMiRO3eQDmMREEf0_gBQg],[.ql-datasou....e3-15363/hCmCFo-gS22nIaqfCUnpnA]
     
        Suppressed: org.opensearch.core.common.io.stream.NotSerializableExceptionWrapper: completion_exception:
   software.amazon.awssdk.services.s3.model.S3Exception: AWS chunked encoding not supported. (Service: S3, Status Code: 501, 
   Request ID: fra-1:qd07EryGbB2TqNOyZ_YhB5w9EK5Kz34JZtDo_U12-cEFjiJFUr6x67MWO8DUoSrB) (SDK Attempt Count: 1)
     
        Caused by: org.opensearch.core.common.io.stream.NotSerializableExceptionWrapper: s3_exception: AWS chunked encoding not
   supported. (Service: S3, Status Code: 501, Request ID: 
   fra-1:qd07EryGbB2TqNOyZ_YhB5w9EK5Kz34JZtDo_U12-cEFjiJFUr6x67MWO8DUoSrB) (SDK Attempt Count: 1)
                at software.amazon.awssdk.services.s3.model.S3Exception$BuilderImpl.build(S3Exception.java:113) ~[?:
                at software.amazon.awssdk.services.s3.model.S3Exception$BuilderImpl.build(S3Exception.java:61) ~[?:

So I tried to disable the encoding as per the setting here in my opensearch.yml (helm chart)

OpenSearch/plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3ClientSettings.java

Line 246 in b7f013f

static final Setting.AffixSetting<Boolean> DISABLE_CHUNKED_ENCODING = Setting.affixKeySetting(

but got an error

SettingsException[unknown setting [s3.client.default.disable_chunked_encoding] please check that any required plugins are installed, or check the breaking changes documentation for removed settings]

it seems similar to the problem fixed by

#19788

Is it possible to expose all settings available in the client from the opensearch.yml file as tehre is no other way to modify them with the helm chart ?

Related component

Plugins

To Reproduce

1. add s3.client.default.disable_chunked_encoding: true to your opensearch.yml

Expected behavior

Clean start of the app

Additional Details

Plugins
repository-s3

Host/Environment (please complete the following information):

• Kubernetes 1.34.1
• Helm chart version 3.3.2

[Gautam-aman]

Hi , I’d like to work on this issue as a first contribution.

What I understand so far:
The setting s3.client..disable_chunked_encoding is defined in S3ClientSettings.java but isn’t added to the plugin’s accepted client-settings list. That’s why adding it in opensearch.yml causes unknown setting.

What I plan to do:
Add DISABLE_CHUNKED_ENCODING into the CLIENT_SETTINGS list so it becomes a valid client-setting.
Update documentation to reflect that this can be configured via opensearch.yml .

If this approach looks good to maintainers/owners of the repository-s3 plugin, I will submit a PR soon. Thanks!

[aphilippejolivel]

Imho it would maybe benefit from a larger fix and expose all the settings currently missing ( unless there is a problem with modifying one of them )

• REQUEST_TIMEOUT_SETTING
• CONNECTION_TIMEOUT_SETTING
• CONNECTION_TTL_SETTING
• MAX_CONNECTIONS_SETTING
• MAX_SYNC_CONNECTIONS_SETTING
• CONNECTION_ACQUISITION_TIMEOUT
• MAX_PENDING_CONNECTION_ACQUIRES
• DISABLE_CHUNKED_ENCODING

[andrross]

Catch All Triage - 1 2

[msfroh]

@Gautam-aman -- I will assign this issue to you.

May I make a suggestion for how to approach it in a broader way?

Add a (package-private) static method to S3ClientSettings, like:

static Collection<Setting<?>> getAllClientSettings() {
  return List.of(
    ACCESS_KEY_SETTING,
    SECRET_KEY_SETTING,
    SESSION_TOKEN_SETTING,
    ...
  );
}

Then add a test in S3ClientSettingsTest that uses reflection to make sure that all settings are there:

    public void testGetAllClientSettings() throws IllegalAccessException {
        List<Setting<?>> allSettings = new ArrayList<>();
        for (Field field : S3ClientSettings.class.getDeclaredFields()) {
            if (Modifier.isStatic(field.getModifiers()) && Setting.class.isAssignableFrom(field.getType())) {
                allSettings.add((Setting<?>) field.get(null));
            }
        }
        assertEquals(Set.copyOf(allSettings), Set.copyOf(S3ClientSettings.getAllClientSettings()));
    }

Then, S3RepositoryPlugin can just call S3ClientSettings.getAllClientSettings() to get all the client settings. (A similar approach could be used for settings from S3Repository.)

What do you think?

[Gautam-aman]

Thanks for the review and guidance on the issue.

I noticed that #20376 is also addressing the same problem. My PR follows the
same single-source-of-truth approach for S3 client settings and includes
registration of disable_chunked_encoding along with a regression test.

Happy to adjust, rebase, or close this PR if #20376 is preferred.
Please let me know how you’d like to proceed.

[msfroh]

Thanks @Gautam-aman! I'll review that PR and we can go from there.