Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .env
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@ OPENSEARCH_PORT=9200
OPENSEARCH_PROTOCOL=https
OPENSEARCH_JAVA_OPTS=-Xms1g -Xmx1g

# Anonymous Authentication
# Set to true to allow access to OpenSearch/Dashboards without login
OPENSEARCH_ANONYMOUS_AUTH=false

# OpenSearch Dashboards Configuration
OPENSEARCH_DASHBOARDS_VERSION=3.6.0
OPENSEARCH_DASHBOARDS_HOST=opensearch-dashboards
Expand Down
27 changes: 25 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,10 @@ OPENSEARCH_PASSWORD='My_password_123!@#'
OPENSEARCH_HOST=opensearch
OPENSEARCH_PORT=9200

# Anonymous Authentication
# Set to true to allow access to OpenSearch/Dashboards without login
OPENSEARCH_ANONYMOUS_AUTH_ENABLED=false

# OpenTelemetry Collector Configuration
OTEL_COLLECTOR_VERSION=0.143.0
OTEL_COLLECTOR_HOST=otel-collector
Expand Down Expand Up @@ -937,13 +941,31 @@ When modifying OpenSearch credentials:

Data Prepper uses a template (`pipelines.template.yaml`) with placeholders processed at container startup via `command:` in docker-compose.yml. No manual credential edits needed in pipeline configs.

### Anonymous Authentication

Anonymous auth is controlled by `OPENSEARCH_ANONYMOUS_AUTH_ENABLED` in `.env` (default: `false`). When enabled, users can access OpenSearch Dashboards without logging in.

The setting is injected at container startup via `sed` into two templates:
- `docker-compose/opensearch/opensearch-security/config.template.yml` → OpenSearch security plugin config
- `docker-compose/opensearch-dashboards/opensearch_dashboards.template.yml` → Dashboards config

Additionally, `savedObjects.permission.enabled` is conditionally set in the Dashboards config at container startup: `false` when anonymous auth is enabled (so anonymous users can access workspaces created by the init script), and `true` (the default) when anonymous auth is disabled. This version of OSD does not support per-workspace permission grants via the API, so without disabling this setting anonymous users get 403 on all workspace-scoped API calls.

The init script sets the `defaultWorkspace` UI setting after creating the Observability Stack workspace, so all users (including anonymous) land directly in the workspace instead of seeing a workspace picker.

Anonymous users can browse data, view, create, and modify saved objects (visualizations, dashboards, saved queries), explore traces and service maps, run queries, and access the REST API without credentials. They cannot delete existing saved objects or perform admin operations.

Modify access is required because Dashboards persists UI settings on every page load via `update` and `bulk` writes to its system indices. Without these permissions the page fails with 403 errors. Since UI settings and saved objects share the same indices, this also allows modification of existing saved objects.

**Important**: Toggling `OPENSEARCH_ANONYMOUS_AUTH_ENABLED` requires `docker compose down -v` (not just `restart`) because OpenSearch applies security configuration to an internal index on first startup. The `-v` flag removes all stored data (traces, logs, saved dashboards) to force reinitialization.

### Configuration File Locations

- **OpenSearch**: No custom config file - uses environment variables in docker-compose.yml
- **OpenSearch**: Environment variables in docker-compose.yml + `docker-compose/opensearch/opensearch-security/config.template.yml` (anonymous auth injected at startup)
- **OpenTelemetry Collector**: `docker-compose/otel-collector/config.yaml`
- **Data Prepper**: `docker-compose/data-prepper/pipelines.template.yaml` (credentials injected at startup) and `docker-compose/data-prepper/data-prepper-config.yaml`
- **Prometheus**: `docker-compose/prometheus/prometheus.yml`
- **OpenSearch Dashboards**: `docker-compose/opensearch-dashboards/opensearch_dashboards.yml`
- **OpenSearch Dashboards**: `docker-compose/opensearch-dashboards/opensearch_dashboards.template.yml` (credentials, anonymous auth, and `savedObjects.permission.enabled` injected at startup)
- **Environment Variables**: `.env` file in repository root

### Index Management
Expand All @@ -967,6 +989,7 @@ When adding new services, consider adding health checks if they depend on other

Development configuration includes:
- OpenSearch security enabled with default admin/admin credentials
- Anonymous authentication disabled by default (enable via `OPENSEARCH_ANONYMOUS_AUTH_ENABLED=true` in `.env`)
- SSL certificate verification disabled for development
- CORS enabled for all origins
- No network isolation
Expand Down
36 changes: 36 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -310,6 +310,42 @@ The interactive installer prompts "Customize OpenSearch credentials?" — enter

**How it works:** `.env` is the single source of truth for credentials. OpenSearch, Dashboards, and the init script read from `.env` via environment variables. Data Prepper uses a [template](docker-compose/data-prepper/pipelines.template.yaml) with `OPENSEARCH_USER`/`OPENSEARCH_PASSWORD` placeholders that are injected via `sed` at container startup — no manual config edits needed. OpenSearch uses HTTPS with self-signed certificates, so use `-k` flag with curl commands.

### Anonymous Authentication

By default, OpenSearch Dashboards requires login with credentials. You can enable anonymous authentication to allow users to access Dashboards without a login prompt — useful for demos, workshops, or shared development environments.

**To enable anonymous access**, set in `.env`:
```env
OPENSEARCH_ANONYMOUS_AUTH_ENABLED=true
```

Then restart the stack:
```bash
docker compose down -v
docker compose up -d
```

> **Warning:** The `-v` flag removes all stored data (traces, logs, saved dashboards). This is required because OpenSearch applies security configuration (roles, role mappings) to an internal index on first startup — restarting without `-v` won't update the security settings.

**What anonymous users can do:**
- Browse and search all data (traces, logs, metrics)
- View existing dashboards, visualizations, and saved queries
- Create and modify visualizations, dashboards, saved queries, and index patterns
- Explore trace analytics and service maps
- Run PPL and SQL queries
- Access the OpenSearch REST API without credentials (e.g., `curl -k https://localhost:9200/_cat/indices`)

**What anonymous users cannot do:**
- Delete existing dashboards, visualizations, or saved objects
- Write data to OpenSearch indices
- Perform admin operations (cluster settings, security configuration, user management)

> **Why modify is allowed:** OpenSearch Dashboards requires `update` and `bulk` write permissions on its system indices to persist UI settings (theme, date format, default index) on every page load. Without these permissions the page fails with 403 "Unable to update UI setting" errors. Because UI settings and saved objects share the same system indices, granting the permissions Dashboards needs to function also allows modification of existing saved objects. Deletion is still blocked.

Admin operations still require full credentials. When disabled (the default), all users must authenticate via the login page.

**Toggling back to require login:** Set `OPENSEARCH_ANONYMOUS_AUTH_ENABLED=false` in `.env` and restart with `docker compose down -v && docker compose up -d`. Note that the `-v` flag removes all stored data (traces, logs, saved dashboards) — this is required because OpenSearch applies security configuration to an internal index on first startup.

## Resource Requirements

| Configuration | Memory Usage | Recommended Minimum |
Expand Down
7 changes: 6 additions & 1 deletion docker-compose.local-opensearch-dashboards.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,15 @@ services:
pull_policy: always
command: >
/bin/bash -c "
ANON_AUTH='${OPENSEARCH_ANONYMOUS_AUTH_ENABLED}';
if [ \"$$ANON_AUTH\" != 'true' ]; then ANON_AUTH='false'; fi;
if [ \"$$ANON_AUTH\" = 'true' ]; then SO_PERM='false'; else SO_PERM='true'; fi;
cp /tmp/opensearch_dashboards.template.yml /tmp/opensearch_dashboards.yml &&
sed -i 's|OPENSEARCH_ANONYMOUS_AUTH_ENABLED|'$$ANON_AUTH'|g' /tmp/opensearch_dashboards.yml &&
sed -i 's|SAVED_OBJECTS_PERMISSION_ENABLED|'$$SO_PERM'|g' /tmp/opensearch_dashboards.yml &&
sed -i 's|OPENSEARCH_HOSTS|${OPENSEARCH_PROTOCOL}://${OPENSEARCH_HOST}:${OPENSEARCH_PORT}|g' /tmp/opensearch_dashboards.yml &&
sed -i 's|OPENSEARCH_USER|${OPENSEARCH_USER}|g' /tmp/opensearch_dashboards.yml &&
sed -i 's|OPENSEARCH_PASSWORD|${OPENSEARCH_PASSWORD}|g' /tmp/opensearch_dashboards.yml &&
sed -i 's|OPENSEARCH_USER|${OPENSEARCH_USER}|g' /tmp/opensearch_dashboards.yml &&
cp /tmp/opensearch_dashboards.yml /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml &&
cd /usr/share/opensearch-dashboards &&
exec ./opensearch-dashboards-docker-entrypoint.sh opensearch-dashboards"
Expand Down
11 changes: 11 additions & 0 deletions docker-compose.local-opensearch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,20 @@ services:
# Initial admin password (required for OpenSearch 2.12+)
- "OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_PASSWORD}"
- plugins.query.datasources.encryption.masterkey=BTqK4Ytdz67La1kShIKV3Pu9
command: >
/bin/bash -c "
ANON_AUTH='${OPENSEARCH_ANONYMOUS_AUTH_ENABLED}';
if [ \"$$ANON_AUTH\" != 'true' ]; then ANON_AUTH='false'; fi;
cp /tmp/opensearch-security/config.template.yml /usr/share/opensearch/config/opensearch-security/config.yml &&
sed -i 's|OPENSEARCH_ANONYMOUS_AUTH_ENABLED|'$$ANON_AUTH'|g' /usr/share/opensearch/config/opensearch-security/config.yml &&
exec ./opensearch-docker-entrypoint.sh"
volumes:
# Persist data across container restarts
- opensearch-data:/usr/share/opensearch/data
# Custom security config for anonymous authentication (template processed at startup)
- ./docker-compose/opensearch/opensearch-security/config.template.yml:/tmp/opensearch-security/config.template.yml
- ./docker-compose/opensearch/opensearch-security/roles.yml:/usr/share/opensearch/config/opensearch-security/roles.yml
- ./docker-compose/opensearch/opensearch-security/roles_mapping.yml:/usr/share/opensearch/config/opensearch-security/roles_mapping.yml
ports:
# REST API endpoint
- "${OPENSEARCH_PORT}:9200"
Expand Down
1 change: 1 addition & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,7 @@ services:
- OPENSEARCH_DASHBOARDS_PROTOCOL=${OPENSEARCH_DASHBOARDS_PROTOCOL}
- PROMETHEUS_HOST=${PROMETHEUS_HOST}
- PROMETHEUS_PORT=${PROMETHEUS_PORT}
- OPENSEARCH_ANONYMOUS_AUTH_ENABLED=${OPENSEARCH_ANONYMOUS_AUTH_ENABLED}
volumes:
- ./docker-compose/opensearch-dashboards/init/init-opensearch-dashboards.py:/init.py
- ./docker-compose/opensearch-dashboards/saved-queries-traces.yaml:/config/saved-queries-traces.yaml
Expand Down
26 changes: 26 additions & 0 deletions docker-compose/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,30 @@ Then access your cloud Dashboards URL directly — no local container needed.

For production environments with valid certificates, enable verification in each of these places.

## Anonymous Authentication

By default, users must log in to access OpenSearch Dashboards. To skip the login page (useful for demos or workshops), enable anonymous authentication in `.env`:

```env
OPENSEARCH_ANONYMOUS_AUTH_ENABLED=true
```

Then restart:
```bash
docker compose down -v
docker compose up -d
```

> **Warning:** The `-v` flag removes all stored data (traces, logs, saved dashboards). This is required because OpenSearch applies security configuration (roles, role mappings) to an internal index on first startup. Without `-v`, the security settings are not reinitialized and the change won't take effect.

Anonymous users can browse all data, view, create, and modify saved objects (visualizations, dashboards, saved queries, index patterns), explore traces and service maps, run queries, and access the OpenSearch REST API without credentials. They cannot delete existing saved objects or perform admin operations — those still require credentials.

> **Note:** Modify access is required because OpenSearch Dashboards persists UI settings (theme, date format, default index) on every page load via `update` and `bulk` writes to its system indices. Without these permissions the page fails with 403 errors. Since UI settings and saved objects share the same indices, this also allows modification of existing saved objects. Deletion is still blocked.

Set `OPENSEARCH_ANONYMOUS_AUTH_ENABLED=false` (the default) to require login for all users. Restart with `docker compose down -v && docker compose up -d` to apply. Note that the `-v` flag removes all stored data (traces, logs, saved dashboards) — this is required because OpenSearch applies security configuration to an internal index on first startup.

**Troubleshooting:** If toggling `OPENSEARCH_ANONYMOUS_AUTH_ENABLED` doesn't take effect, make sure you used `docker compose down -v` (not just `docker compose restart` or `docker compose down` without `-v`). The `-v` flag is required to reinitialize OpenSearch's security configuration.

## Security Warning

⚠️ **This configuration is for development only!**
Expand All @@ -308,13 +332,15 @@ Security considerations:
- SSL certificate verification is disabled for development ease
- Permissive CORS settings
- No network isolation between services
- Anonymous authentication is disabled by default (enable via `OPENSEARCH_ANONYMOUS_AUTH_ENABLED=true` in `.env`)

For production use:
- Change default passwords
- Enable proper SSL/TLS with valid certificates
- Configure proper authentication and authorization
- Implement network policies
- Review and harden all security settings
- Keep anonymous authentication disabled

Never use this configuration in production without proper hardening.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
PROMETHEUS_PORT = os.getenv("PROMETHEUS_PORT", "9090")
_opensearch_protocol = os.getenv("OPENSEARCH_PROTOCOL", "https")
OPENSEARCH_ENDPOINT = f"{_opensearch_protocol}://{os.getenv('OPENSEARCH_HOST', 'opensearch')}:{os.getenv('OPENSEARCH_PORT', '9200')}"
ANONYMOUS_AUTH_ENABLED = os.getenv("OPENSEARCH_ANONYMOUS_AUTH_ENABLED", "false").lower() == "true"

def wait_for_dashboards():
"""Wait for OpenSearch Dashboards to be ready"""
Expand Down Expand Up @@ -232,7 +233,7 @@ def create_prometheus_datasource(workspace_id):

payload = {
"name": datasource_name,
"allowedRoles": [],
"allowedRoles": ["all_access", "opendistro_security_anonymous_role"] if ANONYMOUS_AUTH_ENABLED else ["all_access"],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be if OPENSEARCH_ANONYMOUS_AUTH_ENABLED?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nvm I see this set at L18, please disregard

"connector": "prometheus",
"properties": {
"prometheus.uri": prometheus_endpoint,
Expand Down Expand Up @@ -427,6 +428,45 @@ def create_opensearch_datasource(workspace_id):
return None


def set_default_workspace(workspace_id):
"""Set the default workspace so all users land here on login.

When workspace.enabled is true, users see a workspace picker on first load.
Setting defaultWorkspace directs all users (including anonymous) straight
to the Observability Stack workspace instead.

Returns True on success, False on failure or skip.
"""
if not workspace_id or workspace_id == "default":
print("⏭️ Skipping default workspace (using default)")
return False

print(f"⭐ Setting default workspace: {workspace_id}")

url = f"{BASE_URL}/api/opensearch-dashboards/settings"
payload = {"changes": {"defaultWorkspace": workspace_id}}

try:
response = requests.post(
url,
auth=(USERNAME, PASSWORD),
headers={"Content-Type": "application/json", "osd-xsrf": "true"},
json=payload,
verify=False,
timeout=10,
)

if response.status_code == 200:
print("✅ Default workspace set")
return True
else:
print(f"⚠️ Failed to set default workspace: {response.status_code} {response.text}")
return False
except requests.exceptions.RequestException as e:
print(f"⚠️ Error setting default workspace: {e}")
return False


def set_default_index_pattern(workspace_id, pattern_id):
"""Set the default index pattern"""
print(f"⭐ Setting default index pattern: {pattern_id}")
Expand Down Expand Up @@ -1253,6 +1293,9 @@ def main():
else:
workspace_id = create_workspace()

# Direct all users (including anonymous) to this workspace on login
set_default_workspace(workspace_id)

# Create index patterns (idempotent - will skip if already exist)
# Titles must match exactly what the APM plugin expects
logs_schema_mappings = '{"otelLogs":{"timestamp":"time","traceId":"traceId","spanId":"spanId","serviceName":"resource.attributes.service.name"}}'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,13 @@ opensearch.pingTimeout: 3000
# Allow requests from any origin for development
opensearch.requestHeadersAllowlist: [authorization, securitytenant]

# Anonymous authentication - skip login page for browse access
# Anonymous users can view all data and create/modify saved objects (visualizations, dashboards, saved queries)
# but cannot delete existing saved objects or perform admin operations
# Modify access is required — Dashboards persists UI settings via update/bulk writes on page load
# Set OPENSEARCH_ANONYMOUS_AUTH_ENABLED=true in .env to enable
opensearch_security.auth.anonymous_auth_enabled: OPENSEARCH_ANONYMOUS_AUTH_ENABLED

# Multi-tenancy configuration
# Disabled for simpler development setup
opensearch_security.multitenancy.enabled: false
Expand Down Expand Up @@ -79,6 +86,14 @@ explore.discoverTraces.enabled: true
explore.discoverMetrics.enabled: true
explore.agentTraces.enabled: true
workspace.enabled: true
# Saved-object-level permission checks.
# When anonymous auth is enabled, this must be false so anonymous users can access
# workspaces created by the init script. Without this, only the workspace owner (admin)
# can access workspace-scoped API calls — anonymous users get 403.
# This version of OSD does not support per-workspace permission grants via the API.
# When anonymous auth is disabled, this is true (default) to preserve workspace permissions.
# Value is computed at container startup from OPENSEARCH_ANONYMOUS_AUTH_ENABLED.
savedObjects.permission.enabled: SAVED_OBJECTS_PERMISSION_ENABLED
data_source.enabled: true
data_source.ssl.verificationMode: none
datasetManagement.enabled: true
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# OpenSearch Security Plugin Configuration
# Controls anonymous authentication for OpenSearch Dashboards access without login
# Set OPENSEARCH_ANONYMOUS_AUTH_ENABLED=true in .env to enable
# NOTE: Applying partial config.yml is not supported. Must include entire default config.
# Only anonymous_auth_enabled and basic_internal_auth_domain are active;
# unused auth domains (kerberos, proxy, JWT, client cert, LDAP) have been removed.

_meta:
type: "config"
config_version: 2
config:
dynamic:
http:
anonymous_auth_enabled: OPENSEARCH_ANONYMOUS_AUTH_ENABLED
xff:
enabled: false
internalProxies: "192\\.168\\.0\\.10|192\\.168\\.0\\.11"
authc:
basic_internal_auth_domain:
description: "Authenticate via HTTP Basic against internal users database"
http_enabled: true
transport_enabled: true
order: 4
http_authenticator:
type: "basic"
challenge: true
authentication_backend:
type: "intern"
Loading
Loading