vdk-impala: Add optional parameter for staging table prefix

[sbuldeev]

What:
Adding the option to pass a staging table prefix as well as code refactoring and readme file enhancements

Why:
It is linked to the issue

Signed-off-by: Stefan Buldeev [email protected]

[antoniivanov]

I left a few questions in the source issue trying to understand the problem. I will take a look at the PR after we that.

[antoniivanov]

Please add a test covering this new scenario - e.g execute_template being called twice with different schemas and same table name..

There are multiple reasons for why Adding a test to verify a bug fix is considered good practice:

1. The most obvious one - ensuring the bug is fixed: Adding a test that verifies the bug fix confirms that the issue has been resolved and will not recur.
2. Preventing regression in the future: A regression occurs when a new change breaks a previously working functionality. Adding a test that verifies a bug fix helps prevent regression by ensuring that the bug is not reintroduced in the future which tends to happen.
3. Providing documentation: Adding a test that verifies a bug fix serves as a form of documentation for the code. The test provides a clear and concise example of the expected behavior of the code, which can be used as a reference for future development.
4. Improving code quality: By writing a test that verifies the bug fix, you need to consider the full scope of the bug and its potential impact on the code. This helps improve the overall quality of the code by identifying potential issues that may have been missed.

[sbuldeev]

Actually this change is in order to improve performance instead of fixing a bug.
Referring to the current logic, if there are same table names in different schemas and if we try to check both tables in a same staging schema this is what will happen:

Lets say the tables are called vmc_schema.esx and vsphere_schema.esx. I would call the staging schema "check_schema".
When processing vmc_schema.esx a new table will be created with the name check_schema.vdk_check_esx. This table will be a copy of vmc_schema.esx. After that if we want to process data to vsphere_schema.esx, the template will check if table named
"check_schema.vdk_check_esx" exists and if yes it will check if its structure matches the structure of vsphere_schema.esx. If yes the check_schema.vdk_check_esx table will be reused for checking purposes. If no the check_schema.vdk_check_esx will be recreated to match vsphere_schema.esx and after that the checks will be executed.

With that said I assume we can have same table names in different schema reusing same staging schema for checks. But if the structure of these tables is different(which I guess would be most of the times), the shared staging table between them will be dropped and created on each processing run, which I want to prevent with this change.