We run kafkacat with a wrapper script that reads from a librdkafka configuration file also shared by other applications. It specifies the broker list with the standard metadata.broker.list key. But kafkacat requires this be duplicated in the -b command line argument.
For compatibility, if -b is provided, it should override the -X value. But I don't see any issues that could arise by allowing the brokers to be specified with -X if -b was not used.