[inputs.sqlserver] SQLServerDatabaseIO produces no metrics when running as a Windows service, works correctly with --test

[dancastrolopez-or]

Environment

Telegraf: 1.37.0
OS: Windows Server 2019 Standard 10.0.17763.0
SQL Server: 2019 (MSSQL 15), named instance
Output plugin: outputs.prometheus_client, metric_version = 2
database_type: SQLServer
Description

The measurement sqlserver_database_io is visible in --test mode but never appears in the /metrics endpoint when Telegraf runs as a Windows service. All other SQLServer queries (sqlserver_performance, sqlserver_waitstats, etc.) work correctly in both modes.

Steps to reproduce

1. Install Telegraf 1.37.0 as a Windows service
2. Configure inputs.sqlserver with database_type = "SQLServer"
3. Run telegraf.exe --config telegraf.conf --test → sqlserver_database_io metrics appear
4. Start the Telegraf service → query http://localhost:9273/metrics → sqlserver_database_io is absent
   What was verified

• health_metric shows 13/13 attempted and successful queries — SQLServerDatabaseIO is included based on github
• No errors or warnings in debug logs related to sqlserver
• outputs.file with namepass = ["sqlserver_database_io"] produces an empty file
• sys.dm_io_virtual_file_stats returns 23 rows when executed as a scheduled task in session 0 (NT AUTHORITY\SYSTEM)
• sys.dm_os_volume_stats works for all files (CROSS APPLY returns no missing entries)
• NT AUTHORITY\SYSTEM has VIEW SERVER STATE granted
• No metrics dropped (metrics_dropped = 0)
• tagexclude has no effect
• interval = "120s" on inputs.sqlserver has no effect
  Minimal config

[[outputs.prometheus_client]]
  listen = ":9273"
  metric_version = 2
  export_timestamp = true
 
[[inputs.sqlserver]]
  servers = ["Server=localhost;Port=1433;app name=telegraf;log=1;"]
  database_type = "SQLServer"

Expected behavior

sqlserver_database_io metrics appear in /metrics like all other sqlserver_* measurements.

Actual behavior

sqlserver_database_io is completely absent from /metrics. No error is logged. The query is reported as successful by health_metric.

Note

This issue appears similar to #3349. The problem is not specific to this single server but most (90%) of the others servers running the same Telegraf version and config expose sqlserver_database_io correctly.

[srebhan]

@dancastrolopez-or can you please post the missing (redacted) output metric when running with --test!?

[dancastrolopez-or]

Hello yes,
here some sample i had to censore

`

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=master,environment=production,file_type=ROWS,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=master,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=D:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\DATA\master.mdf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=D: read_bytes=26722304i,read_latency_ms=2089i,reads=590i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=1933312i,write_latency_ms=271i,writes=188i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=tempdb,environment=production,file_type=ROWS,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=tempdev,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=D:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\DATA\tempdb.mdf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=D: read_bytes=8486912i,read_latency_ms=796i,reads=259i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=8798208i,write_latency_ms=408i,writes=398i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=tempdb,environment=production,file_type=ROWS,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=logicalfile02,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=D:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\DATA\tempdb_2.ndf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=D: read_bytes=10444800i,read_latency_ms=192i,reads=166i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=10403840i,write_latency_ms=166i,writes=160i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=tempdb,environment=production,file_type=ROWS,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=logicalfile03,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=D:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\DATA\tempdb_3.ndf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=D: read_bytes=10149888i,read_latency_ms=196i,reads=164i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=10108928i,write_latency_ms=163i,writes=158i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=tempdb,environment=production,file_type=ROWS,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=logicalfile04,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=D:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\DATA\tempdb_4.ndf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=D: read_bytes=10330112i,read_latency_ms=252i,reads=169i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=10289152i,write_latency_ms=175i,writes=160i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=model,environment=production,file_type=ROWS,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=modeldev,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=D:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\DATA\model.mdf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=D: read_bytes=18489344i,read_latency_ms=1338i,reads=233i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=303104i,write_latency_ms=49i,writes=37i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=CustomDB1,environment=production,file_type=ROWS,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=logicalfile05,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=D:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\DATA\CustomDB1_file05.ndf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=D: read_bytes=57196544i,read_latency_ms=3053i,reads=1397i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=221184i,write_latency_ms=25i,writes=22i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=CustomDB1,environment=production,file_type=ROWS,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=logicalfile06,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=D:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\DATA\CustomDB1_file06.ndf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=D: read_bytes=106831872i,read_latency_ms=8301i,reads=6680i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=15171584i,write_latency_ms=70917i,writes=287i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=CustomDB1,environment=production,file_type=ROWS,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=logicalfile07,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=D:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\DATA\CustomDB1_file07.ndf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=D: read_bytes=1539973120i,read_latency_ms=209202i,reads=50762i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=315908096i,write_latency_ms=1029323i,writes=4063i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=CustomDB1,environment=production,file_type=ROWS,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=logicalfile08,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=D:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\DATA\CustomDB1_file08.ndf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=D: read_bytes=1462484992i,read_latency_ms=73318i,reads=22675i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=356851712i,write_latency_ms=522041i,writes=3163i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=CustomDB1,environment=production,file_type=ROWS,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=logicalfile09,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=D:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\DATA\CustomDB1_file09.ndf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=D: read_bytes=434176i,read_latency_ms=22i,reads=11i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=0i,write_latency_ms=0i,writes=0i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=CustomDB1,environment=production,file_type=ROWS,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=logicalfile10,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=D:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\DATA\CustomDB1_file10.ndf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=D: read_bytes=458752i,read_latency_ms=83i,reads=14i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=0i,write_latency_ms=0i,writes=0i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=CustomDB2,environment=production,file_type=ROWS,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=CustomDB2,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=D:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\DATA\CustomDB2.mdf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=D: read_bytes=4718592i,read_latency_ms=885i,reads=365i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=434176i,write_latency_ms=233i,writes=53i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=msdb,environment=production,file_type=ROWS,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=MSDBData,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=D:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\DATA\MSDBData.mdf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=D: read_bytes=71917568i,read_latency_ms=7253i,reads=676i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=2981888i,write_latency_ms=21158i,writes=308i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=CustomDB1,environment=production,file_type=ROWS,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=logicalfile11,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=D:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\DATA\CustomDB1_file11.mdf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=D: read_bytes=64307200i,read_latency_ms=6141i,reads=3574i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=16138240i,write_latency_ms=115772i,writes=1355i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=CustomDB3,environment=production,file_type=ROWS,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=CustomDB3,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=D:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\DATA\CustomDB3.mdf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=D: read_bytes=3063808i,read_latency_ms=1103i,reads=190i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=90112i,write_latency_ms=119i,writes=11i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=CustomDB3,environment=production,file_type=LOG,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=logicalfile12,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=E:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\Data\CustomDB3_log.ldf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=E: read_bytes=1012224i,read_latency_ms=70i,reads=10i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=111616i,write_latency_ms=36i,writes=45i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=CustomDB1,environment=production,file_type=LOG,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=logicalfile13,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=E:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\Data\CustomDB1_log.ldf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=E: read_bytes=1217024i,read_latency_ms=486i,reads=60i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=1449321472i,write_latency_ms=260623i,writes=191132i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=msdb,environment=production,file_type=LOG,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=MSDBLog,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=D:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\DATA\MSDBLog.ldf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=D: read_bytes=1118720i,read_latency_ms=361i,reads=36i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=1497088i,write_latency_ms=2116i,writes=599i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=CustomDB2,environment=production,file_type=LOG,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=logicalfile14,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=E:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\Data\CustomDB2_log.ldf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=E: read_bytes=1129472i,read_latency_ms=159i,reads=35i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=509440i,write_latency_ms=166i,writes=150i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=model,environment=production,file_type=LOG,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=modellog,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=D:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\DATA\modellog.ldf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=D: read_bytes=1036800i,read_latency_ms=122i,reads=16i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=186368i,write_latency_ms=92i,writes=67i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=tempdb,environment=production,file_type=LOG,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=templog,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=E:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\Data\templog.ldf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=E: read_bytes=1007616i,read_latency_ms=88i,reads=7i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=624448000i,write_latency_ms=21941i,writes=11289i 1775655883000000000

sqlserver_database_io,csu=CUSTOMER-NAME,customer_managed_id=CUST0001,database_name=master,environment=production,file_type=LOG,host=SQLSERVER01,instance=SQLSERVER01,logical_filename=mastlog,measurement_db_type=SQLServer,os_family=Windows,os_version=Microsoft\ Windows\ Server\ 2019\ Standard\ 10.0.17763.0,parameter=managed_services,physical_filename=D:\Microsoft\ SQL\ Server\MSSQL15.INSTANCE\MSSQL\DATA\mastlog.ldf,sql_instance=SQLSERVER01:INSTANCE,volume_mount_point=D: read_bytes=599040i,read_latency_ms=133i,reads=27i,rg_read_stall_ms=0i,rg_write_stall_ms=0i,write_bytes=6625792i,write_latency_ms=1956i,writes=1841i 1775655883000000000
`