Refactor pedersen generator initialisation

[iAmMichaelConnor]

At the moment, the configuration for the number of generator indices (and sub indices) is done with two constexpr global variables.

barretenberg/cpp/src/barretenberg/crypto/generators/generator_data.cpp

Line 9 in c6b26c4

constexpr size_t num_default_generators = 64;

This has a few downsides:

• Since they're global, they slow down initialisation of bberg; even if you don't need to use pedersen in your application. (@kevaundray I think you've stated this as a problem Noir has encountered in the past).
• Configuration has to be done by tinkering with these constexpr variables, so can't be done programmatically, varying depending on the application.
• For an example configuration of 16 generators and 32 sub-indices, we end up with a lot of wastage.
  • Many pedersen commits/hashes will only need a handful of sub-indices. E.g. Merkle membership hashing only needs two; not 32. So we end up 'using up' precious, scarce* generator points for no reason. *We're limited in the number of generators we can initialise, so careful instantiation would be better.

Suggestion for improvement:

• Don't initialise generators as globals. Have an initialise_generators function which must be called (by the application using bberg as a library) to generate points. Give the option of holding these points in memory, or return the points to the application (more discussion needed here, please).
• Allow the 'configuration' of the generators to be expressed more granularly, e.g.:
  • std::map<int, size_t> num_sub_indices; a mapping from some enum (whose underlying types are int by default) of generator indices to the number of sub indices required per index.
  • So if you had an enum:

    enum GeneratorIndices {
        MERKLE_TREE = 1,
        NULLIFIER,
        // etc...
    };
    
    

    You could specify the number of sub-indices needed per enum value. Sorry about the pseudo syntax here:

    std::map<int, size_t> num_sub_indices = {
        GeneratorIndices::MERKLE_TREE: 2,
        GeneratorIndices::NULLIFIER: 3,
        // etc...
    };
    

[kevaundray]

Yeah I'd prefer to have a process to generate these points into a context and then I can feed the context to whatever function requires it. This would require a bigger refactor though if all functions are expecting a global

[kevaundray]

@iAmMichaelConnor This seems related to #338

[iAmMichaelConnor]

@iAmMichaelConnor This seems related to #338

Yeah, this issue's proposal should reduce the memory footprint for both applications.
We can probably have a method for 'granular' initialisation of generators as I propose above (for aztec 3 where we know exactly how many we need for each domain separated pedersen hash), and a 'coarse' initialisation which matches what Noir is used to today (i.e. the constants that were previously being used).

[iAmMichaelConnor]

Blocked - need to identify stakeholders and get them on a call to decide best approach.

[codygunton]

To be closed by AztecProtocol/aztec-packages#2592