Adding dumps and doc creation coverage for tomlkit

[sg3-141-592]

Improving the coverage for the tomlkit by adding coverage for the dumps fuzzer. This is based on my existing improvements to the toml fuzzer #9834 , using a modified version of the atheris_dict.py approach to creating random dictionaries.

Doing some local runs this takes the coverage

• Functions statically reachable by fuzzers 26% ->37%
• Cyclomatic complexity statically reachable by fuzzers 31% -> 39%

[sg3-141-592]

Hey @DavidKorczynski , I've removed the document creation for now while I work out a more elegant way to generate them.

Following from an earlier comment, I've had some time to refine and improve the random dictionary generation, test it, make sure the results are deterministic and make it a self contained library pypi - dictgen. This is because there's good potential for using it to cover the other dumps methods for YAML, JSON, TOML encoders etc.

Do you mind having another look, and seeing if you think we can use this library to drive coverage for the tomlkit dumps method?

[DavidKorczynski]

Do you mind having another look, and seeing if you think we can use this library to drive coverage for the tomlkit dumps method?

Yeah, I'll aim go over this tomorrow.

[DavidKorczynski]

Could you move this to the Dockerfile?

[sg3-141-592]

Have moved this to the Dockerfile and tested locally running using introspector. All working.

[DavidKorczynski]

lgtm with the minor nit above.

Have you compared dictgen to a simple construct e.g.

d = json.loads(random_fuzz_str)
if isinstance(d, dict):
  # use the dict as fuzz input

in terms of how much code coverage is achieved?

[sg3-141-592]

I tried an initial json.dumps comparison but it's not able to generate all of the toml datatypes like datetime so I used tomlkit.loads to be a bit fairer.

fdp = atheris.FuzzedDataProvider(input_bytes)
try:
    test_data = tomlkit.loads(fdp.ConsumeUnicodeNoSurrogates(sys.maxsize))
    tomlkit.api.dumps(test_data, sort_keys=fdp.ConsumeBool())
except Exception as e:
    if "tomlkit.exceptions." in str(e.__class__):
        pass
    else:
        raise(e)

For 100k runs we get total project coverage around 59%, and with the dictgen we get 67%. And for the main files involved in dumps (_utils.py, containers.py, items.py) its 42%, compared to 60% with dictgen. So not too much in it, with a good corpus it might catch up.