Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: proper env variable naming #69

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 70 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ Windows registry.
The library focuses on providing a lot of compile-time configurability
and extensibility with a strong adherence to the DRY principle.

Items marked with `TODO` are envisioned for future incorporation but are currently not implemented.

Let's illustrate the API with a highly annotated example. Our configuration
might be described in a separate module looking like this:

Expand Down Expand Up @@ -405,31 +407,83 @@ provide a `help` command and the following additional switches:

## Handling of environment variables and config files

After parsing the command line options, the default behavior of Confutils is
to try to fill any missing options by examining the contents of the environment
variables plus two per-user and system-wide configuration locations derived from
the program name. If you want to use Confutils only as a command-line processor
or a config file parser for example, you can supply an empty/nil value to the
`cmdLine`, `envTable` or `configFileEnumerator` parameters of the `load` call.
If you want your application to use also as parameter env. variables or config files you have to specify configure
the `confutils` for it using the `load` parameter `secondarySources` where you define fallback sources.

Currently, you can use:
- `Envvar` (`confutils/envvar/envvar_serialization`) for environment variables. The `InputFile` defines prefix of the env. variable names
- `Toml` (`toml_serialization`) for Toml config
- `Winreg` (`confutils/envvar/winreg_serialization`) for Window's registries.

More specifically, the `load` call supports the following parameters:
Example how such configuration could look like:

#### `cmdLine`, `envTable`
```nim
import confutils/envvar/envvar_serialization
import toml_serialization
import
confutils/toml/defs as confTomlDefs,
confutils/toml/std/net as confTomlNet,
confutils/toml/std/uri as confTomlUri


type
Nested = object
field: string

YourConf* = object
configFile* {.
desc: "Loads the configuration from a TOML file"
defaultValueDesc: "none"
defaultValue: InputFile.none
name: "config-file" }: Option[InputFile]

The command-line parameters and the environment table of the program.
logLevel* {.
defaultValue: "INFO"
desc: "Sets the log level",
name: "log-level" }: string

nested* {.
name: "nested" }: Nested

let config = YourConf.load(
secondarySources = proc (config: YourConf, sources: auto) =
sources.addConfigFile(Envvar, InputFile "codex")

# The YourConf type has configFile attribute of type Option[InputFile], which the user can use
# to configure its own custom configuration file that will be loaded
if config.configFile.isSome:
sources.addConfigFile(Toml, config.configFile.get)
)
```

### Env. variable names

When using the env. variables, `confutils` transform the configuration parameters name in following matter:

- The specified prefix is applied
- The parameter name are transformed to uppercase
- All spaces or dashes are replaced with underscore

In the above example the parameter `logLevel` with name `log-level` will be possible to specify under
env. variable named `CODEX_LOG_LEVEL`. Moreover, if the configuration type contains nested objects
then the key names are joined with `_` as well. In the above example the `YourConf.nested.field` would be
accessible using `CODEX_NESTED_FIELD`.

> **Warning!** Conflicts of env. variable names are not detected! If you had field in the above example called
> `nested-field` it would also resolve to env. variable named `CODEX_NESTED_FIELD`!

#### `cmdLine`

The command-line parameters of the program.
By default, these will be obtained through Nim's `os` module.

#### `EnvValuesFormat`, `envVarsPrefix`
#### TODO `EnvValuesFormat`, `envVarsPrefix`

A nim-serialization format used to deserialize the values of environment
variables. The default format is called `CmdLineFormat` and it uses the
same `parseCmdArg` calls responsible for parsing the command-line.

The names of the environment variables are prefixed by the name of the
program by default. They are matched in case-insensitive fashion and
certain characters such as `-` and `_` are ignored.

#### `configFileEnumerator`
#### TODO `configFileEnumerator`

A function responsible for returning a sequence of `ConfigFilePath` objects.
To support heterogenous config file types, you can also return a tuple of
Expand All @@ -449,7 +503,7 @@ from the following files:
/etc/{appName}.{ConfigFileForamt.extension}
```

#### `ConfigFileFormat`
#### TODO `ConfigFileFormat`

A [nim-serialization](https://github.com/status-im/nim-serialization) format
that will be used by default by Confutils.
Expand Down
11 changes: 10 additions & 1 deletion confutils/envvar/utils.nim
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import
os,
strutils,
stew/byteutils, stew/ptrops

type
Expand Down Expand Up @@ -84,10 +85,18 @@ template uTypeIsRecord*[N, T](_: type array[N, T]): bool =
false

func constructKey*(prefix: string, keys: openArray[string]): string =
## Generates env. variable names from keys and prefix following the
## IEEE Open Group env. variable spec: https://pubs.opengroup.org/onlinepubs/000095399/basedefs/xbd_chap08.html

var size = prefix.len
for i in 0..<keys.len:
inc(size, keys[i].len)
inc(size, keys[i].len + 1)

result = newStringOfCap(size)
result.add prefix

for x in keys:
result.add "_"
result.add x

return result.toUpperAscii.multiReplace(("-", "_"), (" ", "_"))
2 changes: 1 addition & 1 deletion tests/test_config_file.nim
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ proc readValue(r: var WinregReader, value: var GraffitiBytes) {.used.} =

proc testConfigFile() =
suite "config file test suite":
putEnv("prefixdata-dir", "ENV VAR DATADIR")
putEnv("PREFIX_DATA_DIR", "ENV VAR DATADIR")

test "basic config file":
let conf = TestConf.load(secondarySources = proc (config: TestConf, sources: auto) =
Expand Down
37 changes: 26 additions & 11 deletions tests/test_envvar.nim
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import
os,
std/options,
unittest2,
../confutils/envvar/envvar_serialization,
Expand All @@ -7,7 +8,7 @@ import
const
commonPrefix = "Nimbus"

template readWrite(key: string, val: typed) =
template readWriteTest(key: string, val: typed) =
test key:
setValue(key, val)
var outVal: type val
Expand All @@ -20,16 +21,30 @@ proc testUtils() =
Apple

suite "envvar utils test suite":
readWrite("some number", 123'u32)
readWrite("some number 64", 123'u64)
readWrite("some bytes", @[1.byte, 2.byte])
readWrite("some int list", @[4,5,6])
readWrite("some array", [1.byte, 2.byte, 4.byte])
readWrite("some string", "hello world")
readWrite("some enum", Apple)
readWrite("some boolean", true)
readWrite("some float32", 1.234'f32)
readWrite("some float64", 1.234'f64)
readWriteTest("some number", 123'u32)
readWriteTest("some number 64", 123'u64)
readWriteTest("some bytes", @[1.byte, 2.byte])
readWriteTest("some int list", @[4,5,6])
readWriteTest("some array", [1.byte, 2.byte, 4.byte])
readWriteTest("some string", "hello world")
readWriteTest("some enum", Apple)
readWriteTest("some boolean", true)
readWriteTest("some float32", 1.234'f32)
readWriteTest("some float64", 1.234'f64)

test "nested structure is read correctly":
type
NestedObject = object
someInt: uint32

TestedObject = object
nested: NestedObject

os.putEnv("NIMBUS_NESTED_SOMEINT", "7b000000")

let x = Envvar.decode(commonPrefix, TestedObject)
check x.nested.someInt == 123


proc testEncoder() =
type
Expand Down