Allow adding "external argv"s to be parsed alongside/before command line

[casey]

Maintainer's notes

• This is related to Layering env, args, and config #3113, Designing for layered configs #2763

---

I wrote up a sketch of how this might look here:

https://github.com/casey/clap-config

Basically, it would be pretty cool to generate a simple config file parser from a clap argument parser. There's a longer description of some use cases in the readme, but it would be great for projects which are configurable with both the command line and a config file, and for helping projects add config file configuration in addition to command line configuration.

[kbknapp]

This relates to #251

I'm not against this at all, but I don't have the bandwidth to implement it just yet. I'll keep it on the issue tracker. I'd say supporting YAML and TOML would be fine.

The part I'd still need fleshed out is whether a user can use both the command line and this config file as well? Does one override the other? What about arguments that accept multiple values, and are passed both in the command line and config files, do they get overridden or appended to?

[casey]

I think that a MergeType enum would work, something like:

enum MergeType {
  Override, // replace values with those from config file
  Defaults, // use values from config file only where not given on command line
  Conflict. // complain if any value is provided in both places
}

...

matches.merge_from_config(config, MergeType::Override);

For arguments that accept multiple values, I think that until someone has a specific use case for merging values from both places, it's probably best to only allow overriding or ignoring.

[ssokolow]

This is actually something I'm going to have to deal with in a project soon. (In Python, I tend to dodge the issue by hard-coding my defaults at the top of the only (or top-level) .py file and telling users to just edit them there).

While I'm not familiar with clap internals, I don't see why it would be necessary to over-complicate things. Let's start by separating what clap needs out from a more generic "config file for my program" solution.

1. Standard format: So people can write frontends easily, use something like JSON or TOML. (Ideally, a lightweight parser, since clap already results in my experimental "smallest possible libstd-using Rust binaries" tests hitting a wall at 156KiB (static/musl) or 129KiB (dynamic/glibc) with nothing but clap and some stub argument definitions... and it'd be nice to include config file support.)
2. String-oriented key-value store: As I see it, the only use clap has for a configuration file is as an alternative representation of the command-line arguments, so...
   1. the only need to support data types beyond key="string" pairs is to allow things like key=1 as more intuitive alternatives to key="1". (Being able to have syntax highlighting for data types correspond to the data's actual/conceptual/intuitive type is also a nice thing to have when editing manually.)
   2. It's only necessary to support non-flat data structures as a comfortable shorthand for things clap has built-in support for producing from key=value pairs, like a list equivalent to --append=this --append=that.
   3. Positional arguments generally aren't explicitly supported in configuration files since the situations where giving them defaults are so uncommon and esoteric. (eg. Allowing "target_dir" to be optional in fancy_cp <file> [...] [target_dir] would be an example where it'd actually make sense but be esoteric.)
3. Simple precedence: I've never actually seen a situation where MergeType is worth the trouble. Generally, a much better solution is to just use simple linear precedence:
   1. Make parsing a command-line and parsing a file have the same API so you can call either first and then merge the other. (This also has the benefit that it makes it really easy to build precedence chains for config files like the "system-wide for vendor, system-wide for application, user-specific for vendor, user-specific for application" one that Qt's QSettings API implements.)
   2. The only situation I can see where Override wouldn't be frustrating and/or confusing is when a --foo-from-file= argument comes after a --foo= argument in the command-line... so just add "all from file" and "specific field from file" as argument types and call it a day. (ie. --foo=A --foo=B_trumps_A --foo-from-file=contents_trump_B --foo=winner.)
   3. The only situation where I've seen Conflict making sense is within the same source (ie. Complaining about specifying two values within the same config file or on the same command line.), so supporting it in an intuitive way is orthogonal to supporting config files. (The key being to understand what purpose Conflict actually serves. Forcing people to modify config files for a one-off run is never a good design (it encourages human error), so Conflict only really makes practical sense as a way to say that a single source is internally contradictory.)

With all of that said, what I'd suggest is that it's not really the config file parsing you need to focus on, but providing a clearly documented API that allows something like (human_readable_source_identifier, parsed_data_structure) to be fed into clap so we can feed in JSON/TOML/whatever at our leisure. (The human-readable source identifier would be used for error messages.)

That would also allow clap defaults to be embedded into larger JSON/TOML/etc. configuration files without complicating clap.

Heck, with a little thought, the "from command-line arguments" functions you already have could just be parsers on the same tier as JSON/TOML/etc. which just feed into the API I'm proposing.

[BurntSushi]

The only situation where I've seen Conflict making sense is within the same source (ie. Complaining about specifying two values within the same config file or on the same command line.), so supporting it in an intuitive way is orthogonal to supporting config files.

I'm not sure I really buy this. If I have a flag --foo that takes a value N, then it seems reasonable for me to want to assign a default value to it in a config file, but retain the right to want to override it on the actual command line. Depending on how the --foo flag was defined, this might not be allowed in the form of --foo X --foo Y, so a config file might need special handling?

[ssokolow]

I'm not sure I really buy this. If I have a flag --foo that takes a value N, then it seems reasonable for me to want to assign a default value to it in a config file, but retain the right to want to override it on the actual command line. Depending on how the --foo flag was defined, this might not be allowed in the form of --foo X --foo Y, so a config file might need special handling?

You misunderstand. I made only two claims and neither was that Conflict should be mandatory within a single source. Those two claims are:

1. Conflict never produces a worthwhile behaviour when applied to two pieces of data from different sources.
2. Regardless of how MergeType behaviour is handled, it is entirely internal to the parsing of each individual source.

What I'm arguing is that:

1. If we're doing fine without MergeType now, there's nothing about adding support for configuration files which adds it to the list of requirements.
2. If we need MergeType urgently and implement it before config file support, no beneficial functionality can be gained by expanding that "within a single source" implementation to also operate between multiple sources.

Hence, orthogonal.

First, let's discuss the "never produces worthwhile behaviour" point, starting with a few examples:

1. If ~/.user_default can't override /etc/global_default, then that's bad.
2. If --one-time-setting can't override either of those, then that's bad.
3. If ~/.defaults_a and ~/.defaults_b don't have a defined precedence order, you're forcing users to manage pointless busywork in the best case and, in the worst case, you wind up with a "FAT table copies 1 and 2 disagree? Which one did you intend?" situation.

This derives from two issues:

1. If at all possible, you want to have a single, authoritative source for settings to limit the potential for human error.
2. Multiple sources of configuration are introduced specifically to take advantage of defined-precedence overriding to allow customizing in progressively narrower scopes. (eg. site, user, project, task)

Or, let's come at it from the other direction and address Override, Defaults, and Conflict as options.

Defaults needs no justification. When applied to reconcile data from two different sources, it implements the behaviour configuration files serve in every case I've ever encountered.

Conflict is the hardest to justify because:

• If you're the administrator, it just cripples the utility of the config file if you can't put anything into it which you might want to override on a task-by-task basis. (And then people end up implementing another layer of default handing using a wrapper which passes in --foo=false unless you call it with --foo=true.)
• If you're not the administrator, it means "You can't specify --foo=true because /etc/config_defaults specified foo=false".

The former case is counter-productive and the latter sounds like trying to kitchen-sink some kind of user permissions system into clap when that's a job for code which runs after clap is done.

Finally, Override is just Defaults with the ordering swapped, so it's only justifiable in the one case where you can't swap the order: The command-line arguments, which must always be the last in the chain.

The problem is that, by their very nature, command-line arguments are also the most suitable for one-off overrides, which means that there is no justifiable reason to use Override to silently ignore what the user specified on the command-line because some config file somewhere disagrees.

Therefore, it's much better to just make defined-precedence overriding an inherent, unavoidable feature of how conflicts between multiple sources are resolved. It's the de facto standard way to handle things and it's the only solution which addresses all of the concerns I brought up.

...it's also easy for people to conceptualize. For example, in Python (since it's pseudocode-like):

matches = dict()
matches.update(parse_global_config())
matches.update(parse_local_config())
matches.update(parse_command_line_args())

Therefore, the concept of MergeType is only meaningful within the process of parsing a single source, because merging input from multiple sources should always Defaults.

We may indeed need to discuss the MergeType idea more, but the conclusion we come to has no bearing on a well-designed multi-source implementation.

1. Parse command-line input
2. Parse one or more config files (optionally with a source file specified in step 1 via --config-file or the like)
3. Merge the results using a simple "When in doubt, the last to speak wins" algorithm.

The only potential kink I see is to at least warn users that step 3 might reset config_file so it's not the actual value used if the config file itself contains a config_file value.

The beauty of this approach is that it's made of very small, very extensible pieces and you can get a lot of utility early on, then amend it later. Here are some example steps:

1. Split the argv parsing out from the rest so that anyone can implement the connecting interface on top of a JSON/TOML/etc. parser. (Now, even if they have to manually hook up the config parser and manually merge things, users don't have to reinvent clap's "apply the validation" code in between those two steps.)
2. Add a simple function for merging two parse results. (Now users don't have to manually merge entry-by-entry anymore.)
3. Anything else you think is necessary.

What I'm envisioning is that the argv-parsing side of the interface would take a reference it could use to query the schema-validation side to disambiguate things that would be inherent in the framing of some formats but not others. For example:

• Please resolve this key name (so argv can resolve command-line shorthands like -s without complaint while config files can print a deprecation message or exit with a failure message asking the user if they meant sidechannel_data=... in config.cfg)
• What type of value does this key take? (so parsers can perform any additional transformations needed, like argv deciding whether --foo --bar actually means --foo=--bar)
• Does this key accept a list of values? (so argv can use --foo=this --foo=that --foo=those while the internal representation it outputs can be closer to the {"foo": ["this", "that", "those"]} that JSON would use.)

[BurntSushi]

@ssokolow Frankly, I'm having a really hard time following any of what you're saying. In particular, I find it hard to tie what you're saying to a concrete user experience. I'm probably missing some important context. In any case, I discussed some of the issues I personally see with config files: BurntSushi/ripgrep#196 (comment)

[ssokolow]

Ugh. That's the worst kind of response because it's a perfectly fair one, yet it's the hardest to formulate a meaningful answer to. Give me an hour or two to do "morning routine" stuff and I'll see what I can do to come at it from a different angle and simplify it so that, at the very least, maybe we can figure out where the disconnect is.