Proposal - Development of CLI utility to interface with SOCA configuration

[mcrozes]

This proposal is linked to the migration of SOCA configuration to AWS System Manager Parameter Store: #120

For the first version, the objective is to ship SOCA with its socactl utility for configuration management. Eventually, our goal is to extend this tool to support log management, queue parameters as well as the ability of display the entire configuration in the web-interface.

• socactl will be available on the main node (for SOCA admins) with read/write permissions.
• socactl will be available on all other nodes with read permissions only

Our goal is to reduce configuration errors by removing manual edits for the various configuration files used by SOCA.

Examples:

View all SOCA environment configuration keys

./socactl config get --key "/" --output json
{
    "/configuration/AuthProvider": "openldap",
    "/configuration/ComputeNodeIAMRole": "soca-dev10-ComputeNodeRole7A9ECFBB-TEmWl348zZw7",
    "/configuration/ComputeNodeSecurityGroup": "sg-0dab283558f1510b6",
    "/configuration/ControllerIAMRole": "soca-dev10-ControllerRole10B665F1-jtTHALXKA7na",
    "/configuration/DCVAllowBareMetal": false,
[... OUTPUT TRUNCATED ..] 
}

Note: you can also use ./socactl config snapshot to get a dump of your entire configuration tree.

For our example, let's imagine we want to limit the number of EC2 subnets for compute provisioning to only 2.

First, we query the SOCA configuration to determine the current subnet IDs we have configured:

./socactl config get --key "/configuration/PrivateSubnets"
['subnet-083c658ed3994602d', 'subnet-02a7be43d11f42ecf', 'subnet-0c93c372dc59dbb34'

Let's assume we have repurposed subnet-0c93c372dc59dbb34 for another project and we do not want to use it for compute provisioning anymore.
We can use config set to automatically update the value of the /configuration/PrivateSubnets key.

./socactl config set --key "/configuration/PrivateSubnets" --value "['subnet-083c658ed3994602d', 'subnet-02a7be43d11f42ecf']"
Success: /soca/soca-dev10/configuration/PrivateSubnets has been updated successfully with version: 2

Let's now confirm the value has been updated correctly:

./socactl config get --key "/configuration/PrivateSubnets"
['subnet-083c658ed3994602d', 'subnet-02a7be43d11f42ecf']

You can get the entire configuration history for this key using config history

 ./socactl config history --key "/configuration/PrivateSubnets" --output json
{
    "1": {
        "Version": 1,
        "Value": "['subnet-083c658ed3994602d', 'subnet-02a7be43d11f42ecf', 'subnet-0c93c372dc59dbb34']",
        "LastModifiedDate": "2024-06-06 13:38:39.217000+02:00"
    },
    "2": {
        "Version": 2,
        "Value": "['subnet-083c658ed3994602d', 'subnet-02a7be43d11f42ecf']",
        "LastModifiedDate": "2024-06-07 20:48:54.273000+02:00"
    }
}

If needed, you can easily rollback to a previous healthy configuration

./socactl config rollback --key "/configuration/PrivateSubnets"
No --version specified, Rollback to the previous version
RollBack to: {'Version': 1, 'Value': "['subnet-083c658ed3994602d', 'subnet-02a7be43d11f42ecf', 'subnet-0c93c372dc59dbb34']", 'LastModifiedDate': datetime.datetime(2024, 6, 6, 13, 38, 39, 217000, tzinfo=tzlocal())} Confirm (use --force to skip)? (Yes, No): Yes
Success: /soca/soca-dev10/configuration/PrivateSubnets has been updated successfully with version: 3

Finally, let's verify the value is now back to its original state and our SOCA is configured with 3 private subnets

./socactl config get --key "/configuration/PrivateSubnets"
['subnet-083c658ed3994602d', 'subnet-02a7be43d11f42ecf', 'subnet-0c93c372dc59dbb34']

We are also thinking about adding some input verification. For example, expected values for /configuration/PrivateSubnets is a List[str] and anything else will be rejected.

We hope this utility will simplify configuration management for SOCA admins.

[mcrozes]

Closed, pushed to production as of 24.10 https://awslabs.github.io/scale-out-computing-on-aws-documentation/documentation/architecture/socactl/