Configuration with Template::HashExpand

By Ben Tilly ([email protected])

Work originally sponsored by ZipRecruiter

What is Configuration?

• Software needs to know about its environment
• A hash of keys and values is sufficient
• This differs in production and development (and even by developer)
• Tends towards repetitive code (we will help this)
• Easy problem, lots of possible solutions
• Nobody agrees on how to do it

Approach #1, multiple files

• Load data from a series of files
• Here is sample code:

  
    use Config::Any;
    # ... code here.
    my $raw_config_list
        = Config::Any‐>load_files({files => \@config_path, use_ext => 1});
    my $config = {map %$_, map values %$_, @$raw_config_list};
  

• This will accept data in many formats

Approach #1, tradeoffs

• Data can come in different formats
• Configuration can be reused by other programs, which might not be in Perl
• No temptation to put production data in source control
• But adding new config variables that require customization is hard

Approach #2, unified config file

• No need to leave Perl
• Have a giant hash %PARAM with every possible configuration
• At run time, select the right configuration
• Use $PARAM{$developer} = {%{$PARAM{sandbox}}, ...} to limit repetition.

Approach #2, tradeoffs

• Have full access to Perl while generating config
• Seeing other sandboxes helps you create your own
• You can add a new config key/value pair and fix sandboxes
• But you generate configs you don't need at runtime
• But the config file tends to be ugly

Approach #3, external build

• Build the exact right configuration
• Can use various tools (make, puppet, chef...)
• Can be set up in an infinite number of ways
• In the end you somehow need to put in the same data, get same result as other approaches

Approach #3, tradeoffs

• Most flexible possible solution
• Eat your deployment dogfood
• Best run-time efficiency
• But force an annoying build step on developers
• But lot of cognitive overhead

Configuration structure

• All approaches lend themselves to similar setups
• Let's use a 3-level version with production, sandbox and individual developer levels
• The production level has sane defaults for everything
• The sandbox level has sane overrides for a sandbox
• The individual developer level has overrides to use the developer's directories, urls, email address, etc.

Where is the repetition?

• Many config variables need similar alterations, for instance all file paths start off with /home/[% user %]/, and all emails go to the developer
• If you want to override one line of a nested data structure, you need to copy the whole thing in every config
• This is simple, but has to be repeated for every developer
• This makes configuration files longer, and more error-prone to edit

Example of developer's overrides

 {
    cron_dir => '/home/btilly/sandbox/files/cron',
    tmp_dir  => '/home/btilly/sandbox/files/tmp',
    email    => {
      support_email  => 'Support <[email protected]>',
      pager_email    => 'Pagers <[email protected]>',
    },
    session => {
      cookie_name  => 'example_session_btilly',
      expires      => 60 * 60 * 24 * 14,
      expires_long => 60 * 60 * 24 * 60,
      storage      => '/home/btilly/sandbox/files/session',
      unlink_on_exit => 0,
    },
  }

Templating could help

 {
    default_email  => '[% user %]@example.com',
    root_files_dir => '/home/[% user %]/sandbox/files',
    cron_dir => '[% root_files_dir %]/cron',
    tmp_dir  => '[% root_files_dir %]/tmp',
    email    => {
      support_email  => 'Support <[% default_email %]>',
      pager_email    => 'Pagers <[% default_email %]>',
    },
    session => {
      cookie_name  => 'example_session_[% user %]',
      expires      => 60 * 60 * 24 * 14,
      expires_long => 60 * 60 * 24 * 60,
      storage      => '[% root_files_dir %]/session',
      unlink_on_exit => 0,
    },
  }