String interpolation of values that aren't strings + validation

[hanschen]

I want to use a value that isn't a string from one option in another option, and validate both options.

Example

config.ini:

[section]
total_cores = 1
cores_app1 = $total_cores

configspec.ini:

[section]
total_cores = integer
cores_app1 = integer

main.py:

from configobj import ConfigObj
from validate import Validator


if __name__ == "__main__":
    config = ConfigObj("config.ini", configspec="configspec.ini",
                       interpolation='Template')
    is_valid = config.validate(Validator())

Expected outcome:

>>> config
>>> ConfigObj({'section': {'total_cores': 1, 'cores_app1': 1}})

Actual outcome:

TypeError                                 Traceback (most recent call last)
/data/tmp/configobjtest/test.py in <module>()
      6     config = ConfigObj("config.ini", configspec="configspec.ini",
      7                        interpolation='Template')
----> 8     is_valid = config.validate(Validator())

/usr/lib/python3.5/site-packages/configobj.py in validate(self, validator, preserve_errors, copy, section)
   2308                 section.comments[entry] = configspec.comments.get(entry, [])
   2309                 section.inline_comments[entry] = configspec.inline_comments.get(entry, '')
-> 2310             check = self.validate(validator, preserve_errors=preserve_errors, copy=copy, section=section[entry])
   2311             out[entry] = check
   2312             if check == False:

/usr/lib/python3.5/site-packages/configobj.py in validate(self, validator, preserve_errors, copy, section)
   2261             else:
   2262                 missing = False
-> 2263                 val = section[entry]
   2264 
   2265             ret_true, ret_false = validate_entry(entry, configspec[entry], val, 

/usr/lib/python3.5/site-packages/configobj.py in __getitem__(self, key)
    555         if self.main.interpolation:
    556             if isinstance(val, six.string_types):
--> 557                 return self._interpolate(key, val)
    558             if isinstance(val, list):
    559                 def _check(entry):

/usr/lib/python3.5/site-packages/configobj.py in _interpolate(self, key, value)
    547                 engine = self._interpolation_engine = class_(self)
    548         # let the engine do the actual work
--> 549         return engine.interpolate(key, value)
    550 
    551 

/usr/lib/python3.5/site-packages/configobj.py in interpolate(self, key, value)
    350         # Back in interpolate(), all we have to do is kick off the recursive
    351         # function with appropriate starting values
--> 352         value = recursive_interpolate(key, value, self.section, {})
    353         return value
    354 

/usr/lib/python3.5/site-packages/configobj.py in recursive_interpolate(key, value, section, backtrail)
    334                 else:
    335                     # Further interpolation may be needed to obtain final value
--> 336                     replacement = recursive_interpolate(k, v, s, backtrail)
    337                 # Replace the matched string with its final value
    338                 start, end = match.span()

/usr/lib/python3.5/site-packages/configobj.py in recursive_interpolate(key, value, section, backtrail)
    324 
    325             # Now start the actual work
--> 326             match = self._KEYCRE.search(value)
    327             while match:
    328                 # The actual parsing of the match is implementation-dependent,

TypeError: expected string or bytes-like object

Changing total_cores = integer to total_cores = string solves the issue, but gives the wrong type for config['section']['total_cores'].

I don't know the code well, but I would expect it to call str() on the value of total_cores (i.e., 1) before attempting to use it in the string interpolation of cores_app1.

Tested with both default and Template string interpolation with ConfigObj 5.0.6 and Python 2.7.9, 2.7.11, and 3.5.1.

[jhermann]

After template injection, the same coercion like after parsing should be applied, I guess.

[WeatherGod]

I am running into a similar issue where I have a list of strings assigned to one item that I would like to have it interpolated into another entry. I can devote some resources to fixing this (I need this feature for a project at work). If you point me in the right direction, I'll see what I can do for this.

[WeatherGod]

Ok, I see a real quick and easy solution, but it does have some peculiarities:

diff --git a/src/configobj/__init__.py b/src/configobj/__init__.py
index 3f6eac0..7badf3e 100644
--- a/src/configobj/__init__.py
+++ b/src/configobj/__init__.py
@@ -281,7 +281,7 @@ class InterpolationEngine(object):
                     replacement = v
                 else:
                     # Further interpolation may be needed to obtain final value
-                    replacement = recursive_interpolate(k, v, s, backtrail)
+                    replacement = recursive_interpolate(k, str(v), s, backtrail)
                 # Replace the matched string with its final value
                 start, end = match.span()
                 value = ''.join((value[:start], replacement, value[end:]))

So, if you have a config like so:

foo = 1, 2, 3
bar = this list $foo

you get "this list ['1', '2', '3']" if you don't specify a config spec of int_list() for foo. If you do specify that, then you get "this list [1, 2, 3]", which one would more likely want. Another possible desired result would be "this list 1, 2, 3", which would require keeping the original string data and more invasive changes.

[WeatherGod]

Ah-ha! If I refactor the _write_line() method so that there is a _write_value() method that does the specific coercion of the value into a string, and use that instead of str(v), you get this list 1, 2, 3, regardless of whether a config spec was applied or not!

[WeatherGod]

yeah, the more I think about this, the more I like the solution. I'll put together a PR with this solution along with unit tests tomorrow.

[robdennis]

optimistically putting this in 5.1.0 hoping that #169 helps me get my arms around it