Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vdk-core: print query duration #804

Merged
merged 4 commits into from
Apr 15, 2022
Merged

vdk-core: print query duration #804

merged 4 commits into from
Apr 15, 2022

Conversation

mrMoZ1
Copy link
Contributor

@mrMoZ1 mrMoZ1 commented Apr 14, 2022

why: Users requested clearer examples on how long given operations took:
"- In concrete case problems was with SQL query , it would be nice if we can
print profile details (especially if query fails)"

what: Added new log statements that show how long an SQL query took.

testing: Added unit tests.

Signed-off-by: Momchil Zhivkov [email protected]

Signed-off-by: mrMoZ1 <[email protected]>
@mrMoZ1 mrMoZ1 force-pushed the person/mzhivkov/sql-timing branch from 2195a5c to 7a5e4a3 Compare April 14, 2022 15:48
@mrMoZ1 mrMoZ1 merged commit b9552cb into main Apr 15, 2022
@mrMoZ1 mrMoZ1 deleted the person/mzhivkov/sql-timing branch April 15, 2022 08:36
antoniivanov added a commit that referenced this pull request Apr 18, 2022
Currently we cannot track the full execution of a query (before and
after it) . This is necessary in order to be able to generate proper
lineage events (query start and end) or to track its duration (we had to
currently had to add this to vdk-core
(#804) but that's
really better implemented as plugin (think aspect-oriented programming).

In the future we should consider adding hook for fetch (fetchMany,
fetchAll) as well as it's there where the data is returned (plugins can
take stats - count number of rows, validate sensitive columns, do result
validation tests, etc.)

Testing Done: unit tests, functional test.

Signed-off-by: Antoni Ivanov <[email protected]>
antoniivanov added a commit that referenced this pull request Apr 18, 2022
Currently we cannot track the full execution of a query (before and
after it) . This is necessary in order to be able to generate proper
lineage events (query start and end) or to track its duration (we had to
currently had to add this to vdk-core
(#804) but that's
really better implemented as plugin (think aspect-oriented programming).

In the future we should consider adding hook for fetch (fetchMany,
fetchAll) as well as it's there where the data is returned (plugins can
take stats - count number of rows, validate sensitive columns, do result
validation tests, etc.)

Testing Done: unit tests, functional test.

Signed-off-by: Antoni Ivanov <[email protected]>
antoniivanov added a commit that referenced this pull request May 2, 2022
Currently we cannot track the full execution of a query (before and
after it) . This is necessary in order to be able to generate proper
lineage events (query start and end) or to track its duration (we had to
currently had to add this to vdk-core
(#804) but that's
really better implemented as plugin (think aspect-oriented programming).

In the future we should consider adding hook for fetch (fetchMany,
fetchAll) as well as it's there where the data is returned (plugins can
take stats - count number of rows, validate sensitive columns, do result
validation tests, etc.)

Testing Done: unit tests, functional test.

Signed-off-by: Antoni Ivanov <[email protected]>
antoniivanov added a commit that referenced this pull request May 3, 2022
Currently we cannot track the full execution of a query (before and
after it) . This is necessary in order to be able to generate proper
lineage events (query start and end) or to track its duration (we had to
currently had to add this to vdk-core
(#804) but that's
really better implemented as plugin (think aspect-oriented programming).

In the future we should consider adding hook for fetch (fetchMany,
fetchAll) as well as it's there where the data is returned (plugins can
take stats - count number of rows, validate sensitive columns, do result
validation tests, etc.)

Testing Done: unit tests, functional test.

Signed-off-by: Antoni Ivanov <[email protected]>
antoniivanov added a commit that referenced this pull request May 4, 2022
Currently we cannot track the full execution of a query (before and
after it) . This is necessary in order to be able to generate proper
lineage events (query start and end) or to track its duration (we had to
currently had to add this to vdk-core
(#804) but that's
really better implemented as plugin (think aspect-oriented programming).

In the future we should consider adding hook for fetch (fetchMany,
fetchAll) as well as it's there where the data is returned (plugins can
take stats - count number of rows, validate sensitive columns, do result
validation tests, etc.)

Testing Done: unit tests, functional test.

Signed-off-by: Antoni Ivanov <[email protected]>
antoniivanov added a commit that referenced this pull request May 4, 2022
Currently we cannot track the full execution of a query (before and
after it) . This is necessary in order to be able to generate proper
lineage events (query start and end) or to track its duration (we had to
currently had to add this to vdk-core
(#804) but that's
really better implemented as plugin (think aspect-oriented programming).

In the future we should consider adding hook for fetch (fetchMany,
fetchAll) as well as it's there where the data is returned (plugins can
take stats - count number of rows, validate sensitive columns, do result
validation tests, etc.)

Testing Done: unit tests, functional test.

Signed-off-by: Antoni Ivanov <[email protected]>
antoniivanov added a commit that referenced this pull request May 4, 2022
Currently we cannot track the full execution of a query (before and
after it) . This is necessary in order to be able to generate proper
lineage events (query start and end) or to track its duration (we had to
currently had to add this to vdk-core
(#804) but that's
really better implemented as plugin (think aspect-oriented programming).

In the future we should consider adding hook for fetch (fetchMany,
fetchAll) as well as it's there where the data is returned (plugins can
take stats - count number of rows, validate sensitive columns, do result
validation tests, etc.)

Testing Done: unit tests, functional test.

Signed-off-by: Antoni Ivanov <[email protected]>
antoniivanov added a commit that referenced this pull request May 4, 2022
Currently we cannot track the full execution of a query (before and
after it) . This is necessary in order to be able to generate proper
lineage events (query start and end) or to track its duration (we had to
currently had to add this to vdk-core
(#804) but that's
really better implemented as plugin (think aspect-oriented programming).

In the future we should consider adding hook for fetch (fetchMany,
fetchAll) as well as it's there where the data is returned (plugins can
take stats - count number of rows, validate sensitive columns, do result
validation tests, etc.)

Testing Done: unit tests, functional test.

Signed-off-by: Antoni Ivanov <[email protected]>
antoniivanov added a commit that referenced this pull request May 4, 2022
Currently we cannot track the full execution of a query (before and
after it) . This is necessary in order to be able to generate proper
lineage events (query start and end) or to track its duration (we had to
currently had to add this to vdk-core
(#804) but that's
really better implemented as plugin (think aspect-oriented programming).

In the future we should consider adding hook for fetch (fetchMany,
fetchAll) as well as it's there where the data is returned (plugins can
take stats - count number of rows, validate sensitive columns, do result
validation tests, etc.)

Testing Done: unit tests, functional test.

Signed-off-by: Antoni Ivanov <[email protected]>
antoniivanov added a commit that referenced this pull request May 4, 2022
Currently we cannot track the full execution of a query (before and
after it) . This is necessary in order to be able to generate proper
lineage events (query start and end) or to track its duration (we had to
currently had to add this to vdk-core
(#804) but that's
really better implemented as plugin (think aspect-oriented programming).

In the future we should consider adding hook for fetch (fetchMany,
fetchAll) as well as it's there where the data is returned (plugins can
take stats - count number of rows, validate sensitive columns, do result
validation tests, etc.)

Testing Done: unit tests, functional test.

Signed-off-by: Antoni Ivanov <[email protected]>
antoniivanov added a commit that referenced this pull request May 21, 2022
Currently we cannot track the full execution of a query (before and
after it) . This is necessary in order to be able to generate proper
lineage events (query start and end) or to track its duration (we had to
currently had to add this to vdk-core
(#804) but that's
really better implemented as plugin (think aspect-oriented programming).

In the future we should consider adding hook for fetch (fetchMany,
fetchAll) as well as it's there where the data is returned (plugins can
take stats - count number of rows, validate sensitive columns, do result
validation tests, etc.)

Testing Done: unit tests, functional test.

Signed-off-by: Antoni Ivanov <[email protected]>
antoniivanov added a commit that referenced this pull request Jun 3, 2022
Currently we cannot track the full execution of a query (before and
after it) . This is necessary in order to be able to generate proper
lineage events (query start and end) or to track its duration (we had to
currently had to add this to vdk-core
(#804) but that's
really better implemented as plugin (think aspect-oriented programming).

In the future we should consider adding hook for fetch (fetchMany,
fetchAll) as well as it's there where the data is returned (plugins can
take stats - count number of rows, validate sensitive columns, do result
validation tests, etc.)

Testing Done: unit tests, functional test.

Signed-off-by: Antoni Ivanov <[email protected]>
antoniivanov added a commit that referenced this pull request Jun 3, 2022
Currently we cannot track the full execution of a query (before and
after it) . This is necessary in order to be able to generate proper
lineage events (query start and end) or to track its duration (we had to
currently had to add this to vdk-core
(#804) but that's
really better implemented as plugin (think aspect-oriented programming).

In the future we should consider adding hook for fetch (fetchMany,
fetchAll) as well as it's there where the data is returned (plugins can
take stats - count number of rows, validate sensitive columns, do result
validation tests, etc.)

Testing Done: unit tests, functional test.

Signed-off-by: Antoni Ivanov <[email protected]>
antoniivanov added a commit that referenced this pull request Jun 3, 2022
Currently we cannot track the full execution of a query (before and
after it) . This is necessary in order to be able to generate proper
lineage events (query start and end) or to track its duration (we had to
currently had to add this to vdk-core
(#804) but that's
really better implemented as plugin (think aspect-oriented programming).

In the future we should consider adding hook for fetch (fetchMany,
fetchAll) as well as it's there where the data is returned (plugins can
take stats - count number of rows, validate sensitive columns, do result
validation tests, etc.)

Testing Done: unit tests, functional test.

Signed-off-by: Antoni Ivanov <[email protected]>
antoniivanov added a commit that referenced this pull request Jun 4, 2022
Currently we cannot track the full execution of a query (before and
after it) . This is necessary in order to be able to generate proper
lineage events (query start and end) or to track its duration (we had to
currently had to add this to vdk-core
(#804) but that's
really better implemented as plugin (think aspect-oriented programming).

In the future we should consider adding hook for fetch (fetchMany,
fetchAll) as well as it's there where the data is returned (plugins can
take stats - count number of rows, validate sensitive columns, do result
validation tests, etc.)

Testing Done: unit tests, functional test.

Signed-off-by: Antoni Ivanov <[email protected]>
antoniivanov added a commit that referenced this pull request Jun 4, 2022
Currently we cannot track the full execution of a query (before and
after it) . This is necessary in order to be able to generate proper
lineage events (query start and end) or to track its duration (we had to
currently had to add this to vdk-core
(#804) but that's
really better implemented as plugin (think aspect-oriented programming).

In the future we should consider adding hook for fetch (fetchMany,
fetchAll) as well as it's there where the data is returned (plugins can
take stats - count number of rows, validate sensitive columns, do result
validation tests, etc.)

Testing Done: unit tests, functional test.

Signed-off-by: Antoni Ivanov <[email protected]>
antoniivanov added a commit that referenced this pull request Jun 4, 2022
Currently we cannot track the full execution of a query (before and
after it) . This is necessary in order to be able to generate proper
lineage events (query start and end) or to track its duration (we had to
currently had to add this to vdk-core
(#804) but that's
really better implemented as plugin (think aspect-oriented programming).

In the future we should consider adding hook for fetch (fetchMany,
fetchAll) as well as it's there where the data is returned (plugins can
take stats - count number of rows, validate sensitive columns, do result
validation tests, etc.)

Testing Done: unit tests, functional test.

Signed-off-by: Antoni Ivanov <[email protected]>
antoniivanov added a commit that referenced this pull request Jun 4, 2022
Currently we cannot track the full execution of a query (before and
after it) . This is necessary in order to be able to generate proper
lineage events (query start and end) or to track its duration (we had to
currently had to add this to vdk-core
(#804) but that's
really better implemented as plugin (think aspect-oriented programming).

In the future we should consider adding hook for fetch (fetchMany,
fetchAll) as well as it's there where the data is returned (plugins can
take stats - count number of rows, validate sensitive columns, do result
validation tests, etc.)

Testing Done: unit tests, functional test.

Signed-off-by: Antoni Ivanov <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants