-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Closed
Labels
IndexingIndexing, Bulk Indexing and anything related to indexingIndexing, Bulk Indexing and anything related to indexingenhancementEnhancement or improvement to existing feature or requestEnhancement or improvement to existing feature or requestv2.16.0Issues and PRs related to version 2.16.0Issues and PRs related to version 2.16.0v3.0.0Issues and PRs related to version 3.0.0Issues and PRs related to version 3.0.0
Description
Is your feature request related to a problem? Please describe.
Currently it is possible to set dynamic: "strict" which prevents adding fields that do not have a static mapping in the "properties" but it also disables usage of dynamic_templates.
For example
{
"dynamic": "strict",
"properties": {
"url": { "type": "keyword" },
},
"dynamic_templates": [
{
"custom_dates": {
"match": "date__*",
"mapping": {
"type": "date",
},
},
},
],
}This will disallow documents like
{ "bad": "something" }while allowing
{ "url": "https://example" }but it does not allow
{ "date__created": "2023-11-20T16:11:20.598Z" }which is unfortunate 😞
Describe the solution you'd like
Option like "dynamic": "templates" which allows the date__created and url but disallows the bad field would be really useful to avoid accidental data in the index.
Describe alternatives you've considered
There's a hacky workaround. One can map any remaining fields to a "broken mapping"
{
"properties": {
"url": { "type": "keyword" },
},
"dynamic_templates": [
{
"dates": {
"match": "date__*",
"mapping": {
"type": "date",
},
},
},
{
"reject_rest": {
"match": "*",
"mapping": {
// Something invalid according to opensearch
"__NOT_ALLOWED__": "__NOT_ALLOWED__"
},
},
},
],
}And this works like described but emits bit weird error messages.
unknown parameter [__NOT_ALLOWED__] on mapper [bad] of type [null]
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
IndexingIndexing, Bulk Indexing and anything related to indexingIndexing, Bulk Indexing and anything related to indexingenhancementEnhancement or improvement to existing feature or requestEnhancement or improvement to existing feature or requestv2.16.0Issues and PRs related to version 2.16.0Issues and PRs related to version 2.16.0v3.0.0Issues and PRs related to version 3.0.0Issues and PRs related to version 3.0.0