Service Fabric Support - Load App Insights connection string from environment variable

[WhitWaldo]

Hello - I'm working to transition my applications away from putting connection strings directly in my configuration variables and instead using the managed key vault references. These expose the secret value from the Key Vault as environment variables instead of the configuration approach.

Are there any plans to or is there current support for loading the connection string out from an environment variable? Thank you!

[xiaomi7732]

If I understand it correctly, it seems like we only need to accept configurations from environment variables here.

Loop in @karolz-ms for comments.

[karolz-ms]

Hello! We do not have any plans around loading configuration data from environment variables. You can construct the EventFlow pipeline in code and use environment variables there, but then you give up on the whole configuration mechanism, so it is not a great workaround.

Adding support for reading configuration values from environment variables would be a good enhancement though, and I would probably argue for making it the core feature of EventFlow configuration, benefitting all inputs and outputs. E.g. one could say "server=${DATABASE_SERVER}" inside the configuration document and EventFlow would use the DATABASE_SERVER environment variable to substitute the value before the configuration setting is used. This would require some design of course, and would be a breaking change, so we would probably make it an opt-in feature.

@xiaomi7732 @WhitWaldo please share your thoughts

[xiaomi7732]

Thanks for your ask Karol. Without deep investigation, based on the IConfiguration pattern established by .NET (.NET Core) over time, I am leaning toward fully leveraging it, and that opens up possibilities for all kinds of configuration providers (Env, KV, etc.).

Specific to ServiceFabric, the service fabric configuration file should be provided as an SFConfigureFileProvider.

As a side effect, as long as the client uses IConfiguraiton, it works.

[WhitWaldo]

In my use-case, I'm pulling the event source values from Service Fabric and surfacing them in Application Insights. Right now I'm using the functionality to pull the App Insights connection string from the Service Fabric configuration so being able to opt into sourcing that from environment variables instead would allow SF to populate that variable instead itself making it all a little less exposed and easier to change centrally (e.g. from the Key Vault secret instead of in a pile of configuration files).

I'd be perfectly fine with that being an opt-in functionality.

[karolz-ms]

Good discussion, guys. I am actually very glad that @xiaomi7732 reminded me of IConfiguration because after browsing through some EventFlow tests I think what Whit wants can be made to "just work" by slightly modifying the example we have in the documentation.

1. To enable getting configuration settings from environment variables, enable environment variables provider in addition to JSON file provider. So, the second line should read something like

    IConfiguration config = new ConfigurationBuilder()
        .AddJsonFile(configFilePath)
        .AddEnvironmentVariables()
        .Build();

2. For ServiceFabric it might be necessary to use ServiceFabricDiagnosticPipelineFactory and call ApplyFabricConfigurationOverrides(), instead of vanilla DiagnosticPipelineFactory. If you are using ServiceFabric, you are probably using that already.

3. The example shows you how to set a connection string property on an EventHub output; you would modify it to set it on the output you are using. The value will be populated by the environment provider into the root configuration, so to set it for the EventFlow output you will do something like

   myOutput["connectionString"] = config["SERVICE_FABRIC_CONNECTION_STRING_VAR"];

Let us know please if this does work for you.