Python Config group and selection

I have a situation where a program that runs for different zones of geographics, with overridden options. I would like to have a config-management solution that goes as follows for the mentioned example.

src/configs  ├─ zones  │   |─ zone1.yaml  │   └─ zone2.yaml  └─ config.yaml  

zones/zone1.yaml

zone:     name: zone_A  path:      exp_name : exp_1  scalling: minmax  

zones/zone2.yaml

zone:     name: zone_B  path:      exp_name : run_1  feature_engineering:      pca: True  

config.yaml

path:      parent: "../data"      result_path: "//"      input_path: "//input_data"  scalling: standard  feature_engineering:      calc_trend: True      pca: False  

If there is a way to select the zone and its config is merged with config.yaml, that can be used for the application. I have tried hydra (by Facebook), which does the other way around (ie, have full set of configurations in the respective zone.yaml file in config group and use a default.yaml to select the zone to run). Adobe's himl works as expected when we have single zone.yaml file but not provide an option to select between the sub-configs (zone.yaml) files.

Expected configuration on choosing zone1.yaml

zone:     name: zone_A  path:      exp_name : exp_1  scalling: minmax  path:      parent: "../data"      result_path: "..data/zone_A/exp_1"      input_path: "..data/zone_A/input_data"  feature_engineering:      calc_trend: True      pca: False  

PS: This way spliting the configuration helps in playing around zone specific settings (which vary by each zone) and avoid scrolling over 4500 lines of configurations.

https://stackoverflow.com/questions/66774168/python-config-group-and-selection March 24, 2021 at 11:05AM