configargparse
A drop-in replacement for argparse that allows options to also be set via config files and/or environment variables.
Downloads: 0 (30 days)
Description
ConfigArgParse
--------------
.. image:: https://img.shields.io/pypi/v/ConfigArgParse.svg?style=flat
:alt: PyPI version
:target: https://pypi.python.org/pypi/ConfigArgParse
.. image:: https://img.shields.io/pypi/pyversions/ConfigArgParse.svg
:alt: Supported Python versions
:target: https://pypi.python.org/pypi/ConfigArgParse
.. image:: https://static.pepy.tech/badge/configargparse/week
:alt: Downloads per week
:target: https://pepy.tech/project/configargparse
.. image:: https://img.shields.io/badge/-API_Documentation-blue
:alt: API Documentation
:target: https://bw2.github.io/ConfigArgParse/
Overview
~~~~~~~~
Applications with more than a handful of user-settable options are best
configured through a combination of command line args, config files,
hard-coded defaults, and in some cases, environment variables.
Python's command line parsing modules such as argparse have very limited
support for config files and environment variables, so this module
extends argparse to add these features.
Available on PyPI: http://pypi.python.org/pypi/ConfigArgParse
Features
~~~~~~~~
- command-line, config file, env var, and default settings can now be
defined, documented, and parsed in one go using a single API (if a
value is specified in more than one way then: command line >
environment variables > config file values > defaults)
- config files can have .ini or .yaml style syntax (eg. key=value or
key: value)
- user can provide a config file via a normal-looking command line arg
(eg. -c path/to/config.txt) rather than the argparse-style @config.txt
- one or more default config file paths can be specified
(eg. ['/etc/bla.conf', '~/.my_config'] )
- all argparse functionality is fully supported, so this module can
serve as a drop-in replacement (verified by argparse unittests).
- env vars and config file keys & syntax are automatically documented
in the -h help message
- new method :code:`print_values()` can report keys & values and where
they were set (eg. command line, env var, config file, or default).
- lite-weight (no 3rd-party library dependencies except (optionally) PyYAML)
- extensible (:code:`ConfigFileParser` can be subclassed to define a new
config file format)
- unittested by running the unittests that came with argparse but on
configargparse, and using tox to test with Python 3.5+
Example
~~~~~~~
*config_test.py*:
Script that defines 4 options and a positional arg and then parses and prints the values. Also,
it prints out the help message as well as the string produced by :code:`format_values()` to show
what they look like.
.. code:: py
import configargparse
p = configargparse.ArgParser(default_config_files=['/etc/app/conf.d/*.conf', '~/.my_settings'])
p.add('-c', '--my-config', required=True, is_config_file=True, help='config file path')
p.add('--genome', required=True, help='path to genome file') # this option can be set in a config file because it starts with '--'
p.add('-v', help='verbose', action='store_true')
p.add('-d', '--dbsnp', help='known variants .vcf', env_var='DBSNP_PATH') # this option can be set in a config file because it starts with '--'
p.add('vcf', nargs='+', help='variant file(s)')
options = p.parse_args()
print(options)
print("----------")
print(p.format_help())
print("----------")
print(p.format_values()) # useful for logging where different settings came from
*config.txt:*
Since the script above set the config file as required=True, lets create a config file to give it:
.. code:: py
# settings for config_test.py
genome = HCMV # cytomegalovirus genome
dbsnp = /data/dbsnp/variants.vcf
*command line:*
Now run the script and pass it the config file:
.. code:: bash
DBSNP_PATH=/data/dbsnp/variants_v2.vcf python config_test.py --my-config config.txt f1.vcf f2.vcf
*output:*
Here is the result:
.. code:: bash
Namespace(dbsnp='/data/dbsnp/variants_v2.vcf', genome='HCMV', my_config='config.txt', v=False, vcf=['f1.vcf', 'f2.vcf'])
----------
usage: config_test.py [-h] -c MY_CONFIG --genome GENOME [-v] [-d DBSNP]
vcf [vcf ...]
Args that start with '--' (eg. --genome) can also be set in a config file
(/etc/app/conf.d/*.conf or ~/.my_settings or specified via -c). Config file
syntax allows: key=value, flag=true, stuff=[a,b,c] (for details, see syntax at
https://goo.gl/R74nmi). If an arg is specified in more than one place, then
commandline values override environment variables which override config file
values which override defaults.
positional arguments:
vcf variant file(s)
optional arguments:
-h, --help show this help message and exit
-c MY_CONFIG, --my-config MY_CONFIG
config file path
--genome GENOME path to genome file
-v verbose
-d DBSNP, --dbsnp DBSNP
known variants .vcf [env var: DBSNP_PATH]
----------
Command Line Args: --my-config config.txt f1.vcf f2.vcf
Environment Variables:
DBSNP_PATH: /data/dbsnp/variants_v2.vcf
Config File (config.txt):
genome: HCMV
Special Values
~~~~~~~~~~~~~~
Under the hood, configargparse handles environment variables and config file
values by converting them to their corresponding command line arg. For
example, "key = value" will be processed as if "--key value" was specified
on the command line.
Also, the following special values (whether in a config file or an environment
variable) are handled in a special way to support booleans and lists:
- :code:`key = true` is handled as if "--key" was specified on the command line.
In your python code this key must be defined as a boolean flag
(eg. action="store_true" or similar).
- :code:`key = [value1, value2, ...]` is handled as if "--key value1 --key value2"
etc. was specified on the command line. In your python code this key must
be defined as a list (eg. action="append").
Config File Syntax
~~~~~~~~~~~~~~~~~~
Only command line args that have a long version (eg. one that starts with '--')
can be set in a config file. For example, "--color" can be set by putting
"color=green" in a config file. The config file syntax depends on the constructor
arg: :code:`config_file_parser_class` which can be set to one of the provided
classes: :code:`DefaultConfigFileParser`, :code:`YAMLConfigFileParser`,
:code:`ConfigparserConfigFileParser` or to your own subclass of the
:code:`ConfigFileParser` abstract class.
*DefaultConfigFileParser* - the full range of valid syntax is:
.. code:: yaml
# this is a comment
; this is also a comment (.ini style)
--- # lines that start with --- are ignored (yaml style)
-------------------
[section] # .ini-style section names are treated as comments
# how to specify a key-value pair (all of these are equivalent):
name value # key is case sensitive: "Name" isn't "name"
name = value # (.ini style) (white space is ignored, so name = value same as name=value)
name: value # (yaml style)
--name value # (argparse style)
# how to set a flag arg (eg. arg which has action="store_true")
--name
name
name = True # "True" and "true" are the same
# how to specify a list arg (eg. arg which has action="append")
fruit = [apple, orange, lemon]
indexes = [1, 12, 35 , 40]
*YAMLConfigFileParser* - allows a subset of YAML syntax (http://goo.gl/VgT2DU)
.. code:: yaml
# a comment
name1: value
name2: true # "True" and "true" are the same
fruit: [apple, orange, lemon]
indexes: [1, 12, 35, 40]
colors:
- green
- red
- blue
*ConfigparserConfigFileParser* - allows a subset of python's configparser
module syntax (https://docs.python.org/3.7/library/configparser.html). In
particular the following configparser options are set:
.. code:: py
config = configparser.ArgParser(
delimiters=("=",":"),
allow_no_value=False,
comment_prefixes=("#",";"),
inline_comment_prefixes=("#",";"),
strict=True,
empty_lines_in_values=False,
)
Once configparser parses the config file all section names are removed, thus all
keys must have unique names regardless of which INI section they are defined
under. Also, any keys which have python list syntax are converted to lists by
evaluating them as python code using ast.literal_eval
(https://docs.python.org/3/library/ast.html#ast.literal_eval). To facilitate
this all multi-line values are converted to single-line values. Thus multi-line
string values will have all new-lines converted to spaces. Note, since key-value
pairs that have python dictionary syntax are saved as single-line strings, even
if formatted across multiple lines in the config file, dictionaries can be read
in and converted to valid python dictionaries with PyYAML's safe_load. Example
given below:
.. code:: py
# inside your config file (e.g. config.ini)
[section1] # INI sections treated as comments
system1_settings: { # start of multi-line dictionary
'a':True,
'b':[2, 4, 8, 16],
'c':{'start':0, 'stop':1000},
'd':'experiment 32 testing simulation with parameter a on'
} # end of multi-line dictionary value
.......
# in your configargparse setup
import configargparse
import yaml
parser = configargparse.ArgParser(
config_file_parser_class=configargparse.ConfigparserConfigFileParser
)
parser.add_argument('--system1_settings', type=yaml.safe_load)
args = parser.parse_a