Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

daml ledger export: export all parties #10588

Merged
merged 3 commits into from
Aug 17, 2021
Merged

daml ledger export: export all parties #10588

merged 3 commits into from
Aug 17, 2021

Conversation

aherrmann-da
Copy link
Contributor

Closes #10436

Adds an --all-parties flag to the Daml ledger export command. If set the tool will query all known parties from the ledger and generated an export of the ledger state as seen by these parties.

Pull Request Checklist

  • Read and understand the contribution guidelines
  • Include appropriate tests
  • Set a descriptive title and thorough description
  • Add a reference to the issue this PR will solve, if appropriate
  • Include changelog additions in one or more commit message bodies between the CHANGELOG_BEGIN and CHANGELOG_END tags
  • Normal production system change, include purpose of change in description

NOTE: CI is not automatically run on non-members pull-requests for security
reasons. The reviewer will have to comment with /AzurePipelines run to
trigger the build.

@aherrmann
Add test-case to ConfigSpec for output type
f6e140f 
changelog_begin
changelog_end
@aherrmann
Add an --all-parties flag to ledger export
e6099d2 
changelog_begin
* [Daml export] You can now set the ``--all-parties`` option to generate
  a ledger export as seen by all known parties.
changelog_end
@aherrmann
Update docs
30a4a98
@aherrmann-da aherrmann-da added this to the Daml Script dumps milestone Aug 16, 2021
cocreature
cocreature approved these changes Aug 17, 2021
View reviewed changes
Copy link
Contributor

@cocreature cocreature left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

daml-script/export/src/main/scala/com/daml/script/export/Config.scala
@@ -30,6 +31,13 @@ final case class ExportScript(
damlScriptLib: String,
) extends ExportType

// This is a product rather than a sum, so that we can raise
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could consider using that only during config parsing and switching to a different type afterwards but probably not worth the complexity.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I wish scopt had something like optparse-applicative's Alternative instance.

@aherrmann-da aherrmann-da merged commit d924404 into main Aug 17, 2021
@aherrmann-da aherrmann-da deleted the export-parties branch August 17, 2021 12:12
azure-pipelines bot pushed a commit that referenced this pull request Aug 18, 2021
release 1.17.0-snapshot.20210817.7604.0.0c187853
6f4d40d 
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.

@nickchapman-da is in charge of this release.

Commit log:
```
0c18785 test: test for platform independent dars (#10535)
3822a8c ledger-api-test-tool: Split TransactionServiceIT into lots of suites. (#10585)
3cc9de5 Introduce dependency on Oracle image version (#10597)
f8c0a35 rewrite trigger docs to follow gsg (#10509)
aec601f Fix badly versioned empty tx (#10596)
d924404 daml ledger export: export all parties (#10588)
fb09b72 Improve divulgence warning message. (#10595)
fd9d872 Upgrade to Oracle 19.12 (#10589)
386965c Upgrade Flyway to v7. (#10594)
654d2ee Bump url-parse to address dependabot alerts (#10593)
01b8e30 Simplify loading of logback file (#10592)
19bfdbe update NOTICES file (#10591)
5f7a369 [JSON-API] Perf gatling MultiUserQueryScenario (#10422)
56059f3 Upgrade path-parse to 1.0.7 (#10587)
5c80252 ledger-grpc: Fix the directory paths. [KVL-1005] (#10586)
9db5ccf Normalize transactions & values as a separate pass (#10524)
99e1d78 [Integrity Checker] Implemented pairwise Update normalizer (#10530)
cc37bc3 Set ErrorInfo metadata flag for definite_answer [KVL-1005] (#10583)
147e8f2 Skip subject in changelog check (#10584)
244262d Handle imports for builtin types correctly in Daml export (#10575)
1fc281f ContractStorageBackend consolidation [DPP-462] (#10402)
0cbaa15 Codeowners: Notify committer maintainers on API changes. (#10557)
6492ceb JSON API: log ledger connection errors at every attempt (#10581)
933b58e Extract grpc statuses to separate module [KVL-1005] (#10582)
0ef8944 Use explicit types to track failures when submitting a request for execution [KVL-1005] (#10567)
4430f52 changelog_begin (#10579)
dc4629f update NOTICES file (#10580)
8b0a0e7 update NOTICES file (#10578)
4b8b67a Upgrade Scalatest to v3.2.9. (#10576)
41e60f7 Upgrade to Scala 2.12.14 and 2.13.6. (#10573)
c447898 Fix display of unhandled exceptions in the scenario service (#10572)
4e1a90d Enable --incompatible_remote_results_ignore_disk (#10571)
d183ecc rotate release duty after 1.17.0-snapshot.20210811.7560.0.4f9de4ba (#10556)
76ecb44 update compat versions for 1.17.0-snapshot.20210811.7565.0.f1a55aa4 (#10563)
86a03fa Bump bazel max jvm memory (#10569)
1cc136c update NOTICES file (#10565)
c69880c ledger-api-test-tool: Enforce test naming standards. (#10562)
ee34d0f Track command - use types for error handling instead of grpc statuses [KVL-1005] (#10503)
93c25f3 Release 1.17 snapshot (#10560)
```
Changelog:
```
- [Ledger API Test Tool] The ``TransactionServiceIT`` test suite has
  been split into many test suites. If you are including or excluding
  it, you will need to use the new test suite names, or you can use
  "TransactionService" as a prefix for all of them.
  If you are including or excluding individual tests, you will need to
  update your arguments with the new test suite. You can find the new
  test suite by running the test tool with the ``--list-all``
  flag and looking for the test's short identifier. The short
  identifiers have not changed, with the exception of
  ``TXNoContractKey``, which has been renamed to ``CKNoContractKey`` and
  is now in the ``ContractKeysIT`` test suite.
* [Daml export] You can now set the ``--all-parties`` option to generate
  a ledger export as seen by all known parties.
ledger-api-client - Propagate definite_answer as metadata in the GRPC response for submit/submitAndWait
[JSON API] Ledger connection errors are now logged at every attempt
akka-bindings: `LedgerClientBinding.commands` now returns a flow of `Either[CompletionFailure, CompletionSuccess]` instead of `Completion` for clearer error handling. For backwards compatiblity the new return type can be turned back into a `Completion` using `CompletionResponse.toCompletion`
```

CHANGELOG_BEGIN
CHANGELOG_END
nickchapman-da pushed a commit that referenced this pull request Aug 18, 2021
@azure-pipelines
release 1.17.0-snapshot.20210817.7604.0.0c187853 (#10605)
f8a1820 
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.

@nickchapman-da is in charge of this release.

Commit log:
```
0c18785 test: test for platform independent dars (#10535)
3822a8c ledger-api-test-tool: Split TransactionServiceIT into lots of suites. (#10585)
3cc9de5 Introduce dependency on Oracle image version (#10597)
f8c0a35 rewrite trigger docs to follow gsg (#10509)
aec601f Fix badly versioned empty tx (#10596)
d924404 daml ledger export: export all parties (#10588)
fb09b72 Improve divulgence warning message. (#10595)
fd9d872 Upgrade to Oracle 19.12 (#10589)
386965c Upgrade Flyway to v7. (#10594)
654d2ee Bump url-parse to address dependabot alerts (#10593)
01b8e30 Simplify loading of logback file (#10592)
19bfdbe update NOTICES file (#10591)
5f7a369 [JSON-API] Perf gatling MultiUserQueryScenario (#10422)
56059f3 Upgrade path-parse to 1.0.7 (#10587)
5c80252 ledger-grpc: Fix the directory paths. [KVL-1005] (#10586)
9db5ccf Normalize transactions & values as a separate pass (#10524)
99e1d78 [Integrity Checker] Implemented pairwise Update normalizer (#10530)
cc37bc3 Set ErrorInfo metadata flag for definite_answer [KVL-1005] (#10583)
147e8f2 Skip subject in changelog check (#10584)
244262d Handle imports for builtin types correctly in Daml export (#10575)
1fc281f ContractStorageBackend consolidation [DPP-462] (#10402)
0cbaa15 Codeowners: Notify committer maintainers on API changes. (#10557)
6492ceb JSON API: log ledger connection errors at every attempt (#10581)
933b58e Extract grpc statuses to separate module [KVL-1005] (#10582)
0ef8944 Use explicit types to track failures when submitting a request for execution [KVL-1005] (#10567)
4430f52 changelog_begin (#10579)
dc4629f update NOTICES file (#10580)
8b0a0e7 update NOTICES file (#10578)
4b8b67a Upgrade Scalatest to v3.2.9. (#10576)
41e60f7 Upgrade to Scala 2.12.14 and 2.13.6. (#10573)
c447898 Fix display of unhandled exceptions in the scenario service (#10572)
4e1a90d Enable --incompatible_remote_results_ignore_disk (#10571)
d183ecc rotate release duty after 1.17.0-snapshot.20210811.7560.0.4f9de4ba (#10556)
76ecb44 update compat versions for 1.17.0-snapshot.20210811.7565.0.f1a55aa4 (#10563)
86a03fa Bump bazel max jvm memory (#10569)
1cc136c update NOTICES file (#10565)
c69880c ledger-api-test-tool: Enforce test naming standards. (#10562)
ee34d0f Track command - use types for error handling instead of grpc statuses [KVL-1005] (#10503)
93c25f3 Release 1.17 snapshot (#10560)
```
Changelog:
```
- [Ledger API Test Tool] The ``TransactionServiceIT`` test suite has
  been split into many test suites. If you are including or excluding
  it, you will need to use the new test suite names, or you can use
  "TransactionService" as a prefix for all of them.
  If you are including or excluding individual tests, you will need to
  update your arguments with the new test suite. You can find the new
  test suite by running the test tool with the ``--list-all``
  flag and looking for the test's short identifier. The short
  identifiers have not changed, with the exception of
  ``TXNoContractKey``, which has been renamed to ``CKNoContractKey`` and
  is now in the ``ContractKeysIT`` test suite.
* [Daml export] You can now set the ``--all-parties`` option to generate
  a ledger export as seen by all known parties.
ledger-api-client - Propagate definite_answer as metadata in the GRPC response for submit/submitAndWait
[JSON API] Ledger connection errors are now logged at every attempt
akka-bindings: `LedgerClientBinding.commands` now returns a flow of `Either[CompletionFailure, CompletionSuccess]` instead of `Completion` for clearer error handling. For backwards compatiblity the new return type can be turned back into a `Completion` using `CompletionResponse.toCompletion`
```

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <[email protected]>
tudor-da pushed a commit that referenced this pull request Aug 24, 2021
@aherrmann-da @aherrmann @tudor-da
daml ledger export: export all parties (#10588)
0272b21 
* Add test-case to ConfigSpec for output type

changelog_begin
changelog_end

* Add an --all-parties flag to ledger export

changelog_begin
* [Daml export] You can now set the ``--all-parties`` option to generate
  a ledger export as seen by all known parties.
changelog_end

* Update docs

Co-authored-by: Andreas Herrmann <[email protected]>
tudor-da pushed a commit that referenced this pull request Aug 24, 2021
@azure-pipelines @tudor-da
release 1.17.0-snapshot.20210817.7604.0.0c187853 (#10605)
8ba2e9c 
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.

@nickchapman-da is in charge of this release.

Commit log:
```
0c18785 test: test for platform independent dars (#10535)
3822a8c ledger-api-test-tool: Split TransactionServiceIT into lots of suites. (#10585)
3cc9de5 Introduce dependency on Oracle image version (#10597)
f8c0a35 rewrite trigger docs to follow gsg (#10509)
aec601f Fix badly versioned empty tx (#10596)
d924404 daml ledger export: export all parties (#10588)
fb09b72 Improve divulgence warning message. (#10595)
fd9d872 Upgrade to Oracle 19.12 (#10589)
386965c Upgrade Flyway to v7. (#10594)
654d2ee Bump url-parse to address dependabot alerts (#10593)
01b8e30 Simplify loading of logback file (#10592)
19bfdbe update NOTICES file (#10591)
5f7a369 [JSON-API] Perf gatling MultiUserQueryScenario (#10422)
56059f3 Upgrade path-parse to 1.0.7 (#10587)
5c80252 ledger-grpc: Fix the directory paths. [KVL-1005] (#10586)
9db5ccf Normalize transactions & values as a separate pass (#10524)
99e1d78 [Integrity Checker] Implemented pairwise Update normalizer (#10530)
cc37bc3 Set ErrorInfo metadata flag for definite_answer [KVL-1005] (#10583)
147e8f2 Skip subject in changelog check (#10584)
244262d Handle imports for builtin types correctly in Daml export (#10575)
1fc281f ContractStorageBackend consolidation [DPP-462] (#10402)
0cbaa15 Codeowners: Notify committer maintainers on API changes. (#10557)
6492ceb JSON API: log ledger connection errors at every attempt (#10581)
933b58e Extract grpc statuses to separate module [KVL-1005] (#10582)
0ef8944 Use explicit types to track failures when submitting a request for execution [KVL-1005] (#10567)
4430f52 changelog_begin (#10579)
dc4629f update NOTICES file (#10580)
8b0a0e7 update NOTICES file (#10578)
4b8b67a Upgrade Scalatest to v3.2.9. (#10576)
41e60f7 Upgrade to Scala 2.12.14 and 2.13.6. (#10573)
c447898 Fix display of unhandled exceptions in the scenario service (#10572)
4e1a90d Enable --incompatible_remote_results_ignore_disk (#10571)
d183ecc rotate release duty after 1.17.0-snapshot.20210811.7560.0.4f9de4ba (#10556)
76ecb44 update compat versions for 1.17.0-snapshot.20210811.7565.0.f1a55aa4 (#10563)
86a03fa Bump bazel max jvm memory (#10569)
1cc136c update NOTICES file (#10565)
c69880c ledger-api-test-tool: Enforce test naming standards. (#10562)
ee34d0f Track command - use types for error handling instead of grpc statuses [KVL-1005] (#10503)
93c25f3 Release 1.17 snapshot (#10560)
```
Changelog:
```
- [Ledger API Test Tool] The ``TransactionServiceIT`` test suite has
  been split into many test suites. If you are including or excluding
  it, you will need to use the new test suite names, or you can use
  "TransactionService" as a prefix for all of them.
  If you are including or excluding individual tests, you will need to
  update your arguments with the new test suite. You can find the new
  test suite by running the test tool with the ``--list-all``
  flag and looking for the test's short identifier. The short
  identifiers have not changed, with the exception of
  ``TXNoContractKey``, which has been renamed to ``CKNoContractKey`` and
  is now in the ``ContractKeysIT`` test suite.
* [Daml export] You can now set the ``--all-parties`` option to generate
  a ledger export as seen by all known parties.
ledger-api-client - Propagate definite_answer as metadata in the GRPC response for submit/submitAndWait
[JSON API] Ledger connection errors are now logged at every attempt
akka-bindings: `LedgerClientBinding.commands` now returns a flow of `Either[CompletionFailure, CompletionSuccess]` instead of `Completion` for clearer error handling. For backwards compatiblity the new return type can be turned back into a `Completion` using `CompletionResponse.toCompletion`
```

CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines DAML Build <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cocreature cocreature cocreature approved these changes

@bame-da bame-da Awaiting requested review from bame-da

@nemanja-da nemanja-da Awaiting requested review from nemanja-da

@stefanobaghino-da stefanobaghino-da Awaiting requested review from stefanobaghino-da

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
Daml Script dumps
Development

Successfully merging this pull request may close these issues.

daml ledger export: feature request - export ALL parties
3 participants
@aherrmann-da @cocreature @aherrmann