NPE in `efm2` after upgrading to 4.0.0

[bahaa]

Describe the bug

After upgrading from 3.3.0 to 4.0.0, efm2 plugin throws NullPointerException.

Expected Behavior

No Exception is thrown as the upgrade is backwards compatible.

What plugins are used? What other connection properties were set?

failover2,efm2

Current Behavior

NPE is thrown with this root cause:

Caused by: java.lang.NullPointerException: Cannot invoke "software.amazon.jdbc.plugin.efm.v2.HostMonitorServiceV2Impl$HostMonitorKey.toString()" because "this.monitorKey" is null
	at software.amazon.jdbc.plugin.efm.v2.HostMonitorServiceV2Impl.getSnapshotState(HostMonitorServiceV2Impl.java:188) ~[aws-advanced-jdbc-wrapper-4.0.0.jar:na]
	at software.amazon.jdbc.util.WrapperUtils.addSnapshotState(WrapperUtils.java:839) ~[aws-advanced-jdbc-wrapper-4.0.0.jar:na]
	at software.amazon.jdbc.util.WrapperUtils.addSnapshotState(WrapperUtils.java:826) ~[aws-advanced-jdbc-wrapper-4.0.0.jar:na]
	at software.amazon.jdbc.plugin.efm.base.HostMonitoringConnectionBasePlugin.getSnapshotState(HostMonitoringConnectionBasePlugin.java:289) ~[aws-advanced-jdbc-wrapper-4.0.0.jar:na]
	at software.amazon.jdbc.ConnectionPluginManager.getSnapshotState(ConnectionPluginManager.java:638) ~[aws-advanced-jdbc-wrapper-4.0.0.jar:na]
	at software.amazon.jdbc.util.WrapperUtils.addSnapshotState(WrapperUtils.java:839) ~[aws-advanced-jdbc-wrapper-4.0.0.jar:na]
	at software.amazon.jdbc.util.WrapperUtils.addSnapshotState(WrapperUtils.java:826) ~[aws-advanced-jdbc-wrapper-4.0.0.jar:na]
	at software.amazon.jdbc.util.WrapperUtils.collectState(WrapperUtils.java:754) ~[aws-advanced-jdbc-wrapper-4.0.0.jar:na]
	at software.amazon.jdbc.util.WrapperUtils.extendWithContext(WrapperUtils.java:659) ~[aws-advanced-jdbc-wrapper-4.0.0.jar:na]
	at software.amazon.jdbc.util.WrapperUtils.executeWithPlugins(WrapperUtils.java:363) ~[aws-advanced-jdbc-wrapper-4.0.0.jar:na]
	at software.amazon.jdbc.wrapper.ConnectionWrapper.isValid(ConnectionWrapper.java:649) ~[aws-advanced-jdbc-wrapper-4.0.0.jar:na]
	at com.zaxxer.hikari.pool.PoolBase.checkValidationSupport(PoolBase.java:483) ~[HikariCP-7.0.2.jar:na]
	at com.zaxxer.hikari.pool.PoolBase.checkDriverSupport(PoolBase.java:466) ~[HikariCP-7.0.2.jar:na]
	at com.zaxxer.hikari.pool.PoolBase.setupConnection(PoolBase.java:434) ~[HikariCP-7.0.2.jar:na]
	at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:378) ~[HikariCP-7.0.2.jar:na]
	at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:210) ~[HikariCP-7.0.2.jar:na]
	at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:488) ~[HikariCP-7.0.2.jar:na]
	at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:576) ~[HikariCP-7.0.2.jar:na]
	... 30 common frames omitted

Reproduction Steps

Just configure the wrapper to connect to a local Postgres server using version 4.0.0 with efm2 enabled without any customizations.

Possible Solution

No response

Additional Information/Context

It seems like monitorKey field in HostMonitorServiceV2Impl is access before being initialized. It get initialized only after startMonitoring() is called, but getSnapshotState() is called before startMonitoring() is called from WrapperUtils.

The AWS Advanced JDBC Wrapper version used

4.0.0

JDK version used

25

Operating System and version

macOS 26.4.1

[amasson88]

I see the same issue in junit tests with Flyway and local postgres container

[sergiyvamz]

Hi @bahaa @amasson88

Thank you for reporting this issue. We have just merged in a fix. Could you kindly checkout our snapshot build and let us know if the issue still persists?

Thank you!

[bahaa]

Hi @sergiyvamz

I tried the snapshot version and the NPE is gone now, but instead we get the following exception:

SQL State  : null
Error Code : 0
Message    : java.lang.RuntimeException: Unable to identify the given connection: 'localhost', please ensure the correct host list provider is specified. The host list provider in use is: 'software.amazon.jdbc.hostlistprovider.ConnectionStringHostListProvider@352e4b6d'.

	at org.flywaydb.core.internal.jdbc.JdbcUtils.openConnection(JdbcUtils.java:70) ~[flyway-core-11.14.1.jar:na]
	at org.flywaydb.core.internal.jdbc.JdbcConnectionFactory.<init>(JdbcConnectionFactory.java:76) ~[flyway-core-11.14.1.jar:na]
	at org.flywaydb.core.FlywayExecutor.execute(FlywayExecutor.java:142) ~[flyway-core-11.14.1.jar:na]
	at org.flywaydb.core.Flyway.migrate(Flyway.java:186) ~[flyway-core-11.14.1.jar:na]
	at org.springframework.boot.flyway.autoconfigure.FlywayMigrationInitializer.afterPropertiesSet(FlywayMigrationInitializer.java:67) ~[spring-boot-flyway-4.0.6.jar:4.0.6]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1864) ~[spring-beans-7.0.7.jar:7.0.7]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1813) ~[spring-beans-7.0.7.jar:7.0.7]
	... 17 common frames omitted
Caused by: java.sql.SQLException: java.lang.RuntimeException: Unable to identify the given connection: 'localhost', please ensure the correct host list provider is specified. The host list provider in use is: 'software.amazon.jdbc.hostlistprovider.ConnectionStringHostListProvider@352e4b6d'.
	at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62) ~[na:na]
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) ~[na:na]
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:483) ~[na:na]
	at software.amazon.jdbc.util.WrapperUtils.createInstance(WrapperUtils.java:519) ~[aws-advanced-jdbc-wrapper-4.0.1-20260508.215554-14.jar:na]
	at software.amazon.jdbc.util.WrapperUtils.wrapExceptionIfNeeded(WrapperUtils.java:639) ~[aws-advanced-jdbc-wrapper-4.0.1-20260508.215554-14.jar:na]
	at software.amazon.jdbc.plugin.failover2.FailoverConnectionPlugin.dealWithOriginalException(FailoverConnectionPlugin.java:345) ~[aws-advanced-jdbc-wrapper-4.0.1-20260508.215554-14.jar:na]
	at software.amazon.jdbc.plugin.failover2.FailoverConnectionPlugin.execute(FailoverConnectionPlugin.java:253) ~[aws-advanced-jdbc-wrapper-4.0.1-20260508.215554-14.jar:na]
	at software.amazon.jdbc.ConnectionPluginManager.lambda$execute$5(ConnectionPluginManager.java:320) ~[aws-advanced-jdbc-wrapper-4.0.1-20260508.215554-14.jar:na]
	at software.amazon.jdbc.ConnectionPluginManager.lambda$null$3(ConnectionPluginManager.java:259) ~[aws-advanced-jdbc-wrapper-4.0.1-20260508.215554-14.jar:na]
	at software.amazon.jdbc.ConnectionPluginManager.executeWithTelemetry(ConnectionPluginManager.java:223) ~[aws-advanced-jdbc-wrapper-4.0.1-20260508.215554-14.jar:na]
	at software.amazon.jdbc.ConnectionPluginManager.lambda$makePluginChainFunc$4(ConnectionPluginManager.java:258) ~[aws-advanced-jdbc-wrapper-4.0.1-20260508.215554-14.jar:na]
	at software.amazon.jdbc.ConnectionPluginManager.executeWithSubscribedPlugins(ConnectionPluginManager.java:208) ~[aws-advanced-jdbc-wrapper-4.0.1-20260508.215554-14.jar:na]
	at software.amazon.jdbc.ConnectionPluginManager.execute(ConnectionPluginManager.java:317) ~[aws-advanced-jdbc-wrapper-4.0.1-20260508.215554-14.jar:na]
	at software.amazon.jdbc.util.WrapperUtils.executeWithPlugins(WrapperUtils.java:334) ~[aws-advanced-jdbc-wrapper-4.0.1-20260508.215554-14.jar:na]
	at software.amazon.jdbc.wrapper.ConnectionWrapper.isValid(ConnectionWrapper.java:649) ~[aws-advanced-jdbc-wrapper-4.0.1-20260508.215554-14.jar:na]
	at com.zaxxer.hikari.pool.PoolBase.checkValidationSupport(PoolBase.java:483) ~[HikariCP-7.0.2.jar:na]
	at com.zaxxer.hikari.pool.PoolBase.checkDriverSupport(PoolBase.java:466) ~[HikariCP-7.0.2.jar:na]
	at com.zaxxer.hikari.pool.PoolBase.setupConnection(PoolBase.java:434) ~[HikariCP-7.0.2.jar:na]
	at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:378) ~[HikariCP-7.0.2.jar:na]
	at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:210) ~[HikariCP-7.0.2.jar:na]
	at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:488) ~[HikariCP-7.0.2.jar:na]
	at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:576) ~[HikariCP-7.0.2.jar:na]
	at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:97) ~[HikariCP-7.0.2.jar:na]
	at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:111) ~[HikariCP-7.0.2.jar:na]
	at org.flywaydb.core.internal.jdbc.JdbcUtils.openConnection(JdbcUtils.java:65) ~[flyway-core-11.14.1.jar:na]
	... 23 common frames omitted
	Suppressed: software.amazon.jdbc.exceptions.SnapshotStateException: 
State snapshot: 
  pluginService: 
    properties: 
      password: ***
      database: test
      wrapperPlugins: failover2,efm2
      skipFailoverOnInterruptedThread: true
      readerHostSelectorStrategy: roundRobin
      user: test
    originalUrl: jdbc:postgresql://localhost:5432/test
    currentConnection: org.postgresql.jdbc.PgConnection@237cd264
    currentHostSpec: HostSpec@51cd2d2 [hostId=null, host=localhost, port=5432, WRITER, AVAILABLE, weight=100, null]
    isInTransaction: false
    dialect: software.amazon.jdbc.dialect.PgDialect
    pooledConnection: null
  pluginManager: 
    plugins: 
      software.amazon.jdbc.plugin.failover2.FailoverConnectionPlugin: 
        properties: 
          password: ***
          database: test
          wrapperPlugins: failover2,efm2
          skipFailoverOnInterruptedThread: true
          readerHostSelectorStrategy: roundRobin
          user: test
        failoverTimeoutMsSetting: 300000
        failoverMode: STRICT_WRITER
        telemetryFailoverAdditionalTopTraceSetting: false
        failoverReaderHostSelectorStrategySetting: random
        closedExplicitly: false
        isClosed: false
        isInTransaction: false
        rdsUrlType: OTHER
        skipFailoverOnInterruptedThread: true
      software.amazon.jdbc.plugin.efm.v2.HostMonitoringConnectionPluginV2: 
        properties: 
          password: ***
          database: test
          wrapperPlugins: failover2,efm2
          skipFailoverOnInterruptedThread: true
          readerHostSelectorStrategy: roundRobin
          user: test
        monitoringHostSpec: null
        monitorService: 
          failureDetectionTimeMillis: 30000
          failureDetectionIntervalMillis: 5000
          failureDetectionCount: 3
          monitorKey: null
        isEnabled: true
      software.amazon.jdbc.plugin.DefaultConnectionPlugin: (blank)
    defaultConnProvider: 
      software.amazon.jdbc.DriverConnectionProvider: 
        driver: org.postgresql.Driver
        targetDriverClassName: org.postgresql.Driver
    effectiveConnProvider: null
  storageService: 
  monitorService: 
    monitorCaches: 
      software.amazon.jdbc.plugin.efm.v2.HostMonitorV2Impl: 
        monitors: 

Removing efm2 plugin fixes the issue. It seems to me that efm2 doesn't work with non RDS connections (local) anymore.

Thank you very much!

[sergiyvamz]

Hi @bahaa,

Thank you for the follow-up and the detailed stack trace - that's very helpful.

You're right that efm2 should work with any URL, including non-RDS ones like localhost. We've identified the root cause - the connection identification logic introduced in 4.0.0 fails to match the DB-reported address (e.g., 127.0.0.1) against the connection string hostname (localhost) in the topology. A fix is on the way that will gracefully fall back to the connection string host when identification can't resolve a match.

A couple of notes for your configuration to consider:

• failover2 plugin with a local/single-instance database — the failover plugin is designed for Aurora/RDS clusters where there are multiple nodes to fail over to. When connecting to a standalone Postgres instance (like localhost), failover has no cluster topology to work with, so it adds overhead without providing benefit. You may want to use just efm2 for connection health monitoring without failover2.

• HikariCP + auroraConnectionTracker plugin — if you're using HikariCP as your connection pool (which it looks like from the stack trace), we recommend adding the auroraConnectionTracker plugin to your configuration. It helps the wrapper properly track and invalidate pooled connections when node state changes, which improves the overall reliability of the connection pool. This suggestion may be not much important since you don't use Aurora/RDS cluster.

We'll update this issue once the fix is available in a snapshot build.

[bahaa]

Hi @sergiyvamz,

Thanks for the advice. We use the wrapper because we use Aurora/RDS. We use the local/single-instance only to run the tests. We prefer to run the tests with the Wrapper and plugins configured like production.

We will definitely add auroraConnectionTracker plugin to our configuration.

Thanks again.

[cnocula-peg]

I have a similar error and stacktrace after upgrading to 4.0.x. Though for us it is not related to localhost, but an actual host name with a configuration unchanged since 2+ years.

Rootcause:

Caused by: java.lang.RuntimeException: Unable to identify the given connection: 'xxxx-cluster-dev-xxxxx.cluster-xxxx.eu-central-1.rds.amazonaws.com', please ensure the correct host list provider is specified. The host list provider in use is: 'software.amazon.jdbc.hostlistprovider.RdsHostListProvider@563cddb1'.
	at software.amazon.jdbc.plugin.efm.base.HostMonitoringConnectionBasePlugin.getMonitoringHostSpec(HostMonitoringConnectionBasePlugin.java:265)
	at software.amazon.jdbc.plugin.efm.base.HostMonitoringConnectionBasePlugin.execute(HostMonitoringConnectionBasePlugin.java:158)
	at software.amazon.jdbc.ConnectionPluginManager.lambda$execute$5(ConnectionPluginManager.java:320)
	at software.amazon.jdbc.ConnectionPluginManager.lambda$null$3(ConnectionPluginManager.java:259)
	at software.amazon.jdbc.ConnectionPluginManager.executeWithTelemetry(ConnectionPluginManager.java:223)
	at software.amazon.jdbc.ConnectionPluginManager.lambda$makePluginChainFunc$4(ConnectionPluginManager.java:258)
	at software.amazon.jdbc.ConnectionPluginManager.lambda$null$2(ConnectionPluginManager.java:261)
	at software.amazon.jdbc.plugin.failover2.FailoverConnectionPlugin.execute(FailoverConnectionPlugin.java:249)
	... 186 common frames omitted

Unfortunately I can't give much more detail than that, because for us it looks like it came out of nowhere. The service had an uptime of 4+ days. We have two datasources configured to an Aurora PostgreSQL . One of them suddenly started to throw the exception, while the other was fine. It also looks like a simple restart of the service helped.

Our Config (similar for the other datasource):

      hikari:
        pool-name: XXXXPool
        maximum-pool-size: 35
        data-source-properties:
          clusterId: XXXXCluster
          wrapperPlugins: iam,auroraConnectionTracker,failover2,efm2
          wrapperDialect: aurora-pg
          iamRegion: ${aws.region}
        exception-override-class-name: software.amazon.jdbc.util.HikariCPSQLException

[sergiyvamz]

Hi @bahaa

We have now introduced this fix in version 4.0.1 of the driver.
https://github.com/aws/aws-advanced-jdbc-wrapper/releases/tag/4.0.1

Thank you!

[sergiyvamz]

Hi @cnocula-peg

Would you be able to confirm your issue on the latest 4.0.1? If the issue persists I'll need driver logs (FINEST/TRACE level please) for period of time preceding the issue, or a full stack exception. Please make sure that exception isn't cut and includes complete snapshot state details. I hope that would be enough to understand the root cause of the issue.

[bahaa]

@sergiyvamz I can confirm that everything is working as expected now with v4.0.1.

Thank you so much.