v0.10.1

Small bug fix release to address a REST API packaging issue.

Bug Fixes

• Fix REST bundled dependencies and prepare for v0.10.1 #732

Contributors

We'd like to thank all the contributors who worked on this release!

• @steven-sheehy

v0.10.0

In preparation for Hedera Node release 0.5.0, we're releasing v0.10.0 to support the latest version of HAPI. The changes include renaming Claims to LiveHash and new response codes. One important HAPI change is the addition of a topicRunningHashVersion to the transaction record. This change was necessary as the way the topic running hash is calculated is changing with the release of 0.5.0. As a result, the Hedera Mirror Node added this new field to its database and a migration is ran to populate it with either the new or old version depending upon the release date of 0.5.0.

Unfortunately, this necessitated adding a required field hedera.mirror.topicRunningHashV2AddedTimestamp to control this behavior and will fail on startup if this is not populated. This is just a temporary measure. Once Hedera Node 0.5.0 is released to testnet and mainnet we will update this so it's automatically populated with the known date.

Other changes include adding Google PubSub support to publish JSON representing the Transaction and TransactionRecord protobuf to a message queue for external consumption. We've also added REST API metrics and added Traefik as an API gateway for our helm chart.

Breaking changes

We've had to remove our event stream support. This area of the code was never enabled and was untested and was incurring technical debt without providing any benefit. If it becomes necessary in the future, we can re-add it within our newly refactored framework.

The new /api/v1/topics/:id alpha REST API that was added in 0.9 has been changed to /api/v1/topics/:id/messages. This change was made to align the API with the other topic message APIs as it refers to the messages entity and not the topic entity.

Enhancements

• Bump versions for v0.10.0 #730
• Port running hash version to V0.10 #727
• Prepare for v0.10.0-rc1 #708
• Add Traefik API gateway #693
• REST API swagger metrics #690
• Pubsub exporter for Mirror Node #686
• Push docker images on master and cleanup old images #683
• Non-fee transfers : move txn type check to implementation #681
• Add updateEntity() and updateTransaction() to TransactionHandler #680
• Store PostgreSQL passwords in a Kubernetes Secret #678
• Remove jdbc connection params not applicable on Postgres connection #671
• Remove Events #670
• HederaETL: PubSub exporter #665
• REST API Metrics #661
• Kubernetes: API Gateway #651

Bug Fixes

• Fix address book updates that span transactions #701
• Topic Message REST API Missing Path #699
• Fix topic messages REST route #698
• Fix Spring Cloud Kubernetes running locally #674
• Implement async rest api routes and centralized error handling #663
• REST API crashes when database error #631

Contributors

We'd like to thank all the contributors who worked on this release!

• @apeksharma
• @Nana-EC
• @steven-sheehy

v0.9.1

Small bug fix release to address not being able to handle address book updates that span multiple transactions.

Bug Fixes

• Address book fix #696
• Rework address book file update #58
• Address book parsing fails when file update spans transactions #695

Contributors

We'd like to thank all the contributors who worked on this release!

• @steven-sheehy

v0.9.0

This release contains another new REST API for our consensus service. You can now retrieve all topic messages in a particular topic, with additional filtering by sequence number and consensus timestamp. Here's an example:

GET /api/v1/topic/7?sequencenumber=gt:2&timestamp=lte:1234567890.000000006&limit=2

{
  "messages": [
    {
      "consensus_timestamp": "1234567890.000000003",
      "topic_id": "0.0.7",
      "message": "bWVzc2FnZQ==",
      "running_hash": "cnVubmluZ19oYXNo",
      "sequence_number": 3
    },
    {
      "consensus_timestamp": "1234567890.000000004",
      "topic_id": "0.0.7",
      "message": "bWVzc2FnZQ==",
      "running_hash": "cnVubmluZ19oYXNo",
      "sequence_number": 4
    }
  ],
  "links": {
     "next": "/api/v1/topic/7?sequencenumber=gt:2&timestamp=lte:1234567890.000000006&timestamp=gt:1234567890.000000004&limit=2"
  }
}

The other big feature of this release is Kubernetes support. We've created a Helm chart that can be used to deploy a highly available Mirror Node with a single command. This feature is still under heavy development as we work towards converting our current deployments to this new approach.

Enhancements

• Prepare for v0.9.0 #685
• Bump version for v0.9.0-rc1 #662
• Add pod priority class #650
• Add Pod Priority Class #649
• Add getEntityId() to Transaction Handler #645
• Implement Topic Messages collection REST API endpoint #644
• Add TransactionBody to RecordItem #639
• Kubernetes #637
• Add rest api code coverage #629
• Updated HCS Acceptance tests for easy running by devops #626
• REST API code coverage #623
• Topic messages REST API #618
• Google Blockchain ETL Design #560
• Investigate requester pays buckets #357
• Kubernetes Deployment #346

Bug Fixes

• Fix NPE when parsing autoRenewAccount #682
• 0.9 cherry picks and v0.9.0-rc2 #676
• Fix NPE due to proxyAccountID #664
• Fix REST API packaging issues #648
• Extend timeout in subscribeverifysequence to retrieve messages #643
• Investigate ConsensusControllerTest.subscribeVerifySequence failures #640
• Fixing REST spec tests causing bigint overflow #636
• Store surefire logs in artifacts for debugging of failures #635
• Fix REST Monitor security vulnerabilities #633
• Fix REST security vulnerability #632
• Fix REST tests in CircleCI #630
• CircleCI tests sporadically fail #621

Contributors

We'd like to thank all the contributors who worked on this release!

• @apeksharma
• @Nana-EC
• @steven-sheehy

v0.8.1

Small bug fix release to fix a packaging issue.

Bug Fixes

• Fix REST bundled dependencies #641
• Fix REST bundled dependencies part 2 #642

Contributors

We'd like to thank all the contributors who worked on this release!

• @steven-sheehy

v0.8.0

Mirror node v0.8.0 is here! We're made great strides in making the mirror node easier to run and manage. In particular, we added support for requester pays buckets. This will allow anyone to run a mirror node as long as they are willing to pay for the cost to retrieve the data. Currently only Hedera and a few partners have access to the bucket, so enabling this will open up that data to our community. We are still working on a migration of the buckets to this model, so stay tuned.

We also added two new experimental REST APIs to retrieve HCS data. Firstly, we added /api/v1/topic/message/${consensusTimestamp} to retrieve a topic message by its consensus timestamp. Secondly, we added /api/v1/topic/${topic}/message/${seqNum} to retrieve a particular topic message by its sequence number from a topic. These APIs are considered alpha and may changed or be removed in the future. We also dramatically increased test coverage for the REST APIs and squashed some bugs in the process.

For our GRPC API, we had to switch from R2DBC to Hibernate to reach the scale and stability that we needed. In doing so, we can now support a lot more concurrent subscribers at a higher throughput. It should also finally put to rest any stability concerns with the GRPC component.

There are a few breaking changes that we had to make. We now no longer write and store record or balance files to the filesystem after they are inserted into the database. If you still need these files, you can set hedera.mirror.parser.balance.keepFiles and hedera.mirror.parser.record.keepFiles to true.

Also, we moved the persist properties to be grouped under a new path. That is we moved options like hedera.mirror.parser.record.persistTransactionBytes to hedera.mirror.parser.record.persist.transactionBytes. Please update your local configuration accordingly.

Enhancements

• Bump version for v0.8.0 #634
• Add support for Requester Pays S3 buckets #622
• Bump version for v0.8.0-rc1 #620
• Increase rest api integration coverage to match api offerings 2 #617
• Topic message by timestamp REST API #616
• Support REST API retrieval of topic message by topic id and sequence #615
• Topic message by sequence number REST API #614
• Add /topic/message/:consensusTimestamp API #613
• Increase rest api integration coverage to match api offerings #612
• Update copyright to 2020 #607
• Make monitor API configurable in dashboard #606
• REST Integration tests improvements 2 replacement #603
• Remove GRPC monitor #602
• Rename parsed directory and persist properties #601
• Updated integrationDbOps.js with more detailed setup logic #598
• Configuration to delete parsed files #596
• Update RecordFileParser.loadRecordFile() to match design #587
• Make monitoring endpoint configurable in dashboard #584
• Remove GRPC monitor #583
• Switch from R2DBC to Hibernate #544
• REST API integration tests for all parameters #412
• Don't keep parsed files #259

Bug Fixes

• Fix NPE in Downloader #619
• NullPointerException in Downloader #611
• Added logic to fail if all missing messages were not retrieved during a subscription #610
• Switch from R2DBC to Hibernate #597
• Subscribe topic should return error if no missing messages #589
• Stalled clients for historical data at higher page size #558

Contributors

We'd like to thank all the contributors who worked on this release!

• @apeksharma
• @Nana-EC
• @steven-sheehy

v0.7.0

0.7.0 focuses on refactoring the record file parsing to decouple the parsing from the persisting of data. This refactoring is laying the groundwork for additional performance improvements and allowing additional downstream components to register for notification of the transactions.

Enhancements

• Bump to version 0.7.0 #599
• REST Integration tests improvements 1 #590
• Slight performance and acceptance test improvements and corrections #588
• Move file handling logic from RecordItemParser to RecordFileParser #585
• JUnit parser performance tests for 1k tps record files #582
• Rename RecordFileLogger to RecordItemParser #581
• Make RecordFileLogger implementation of RecordItemListener #580
• Prepare RecordFileParser for file init/complete logic. #579
• Postgres Writer #3 : Move transaction and batching #578
• Use RecordItem in tests #577
• Postgres Writer #2 #576
• Postgres Writer #1 : move ctl and non-fee transfers #575
• Rename RecordFileLogger to RecordItemParser #567
• [parser refactor] Split PostgresWritingRecordParsedItemHandler out of RecordFileLogger #566
• Add Parser interfaces and domains. #565
• [Parser refactor] Add interfaces and domains #564
• Parser performance tests #559
• Parser re-design #553

Bug Fixes

• Fix check topic exists flag #593
• Check topic exists flag doesn't work #592
• Fix subscribe topic with end time not completing #543
• Subscribe topic with end time not completing #535

Contributors

We'd like to thank all the contributors who worked on this release!

• @apeksharma
• @Nana-EC
• @steven-sheehy

v0.6.0

This release was mainly focused on enhancing the stability and performance of the mirror node. We improved the transaction ingestion speed from 600 to about 4000 transactions per second. At the same time, we greatly improved the resiliency and performance of the GRPC module. We also added acceptance tests to test out HCS end to end.

Breaking Change

Please note that one potentially breaking change in this release is to reject subscriptions to topics that don't exist. This avoids the server having to poll repeatedly until it is created and taking up resources for a topic that may never exist. It is expected that clients or the SDK will poll periodically after creating a topic until that topic makes its way to the mirror node. This functionality is hidden behind a feature flag but will slowly be rolled out over the next month.

Enhancements

• Bump version to v0.6.0 #574
• Bump version to 0.6.0-rc1 #555
• Split historical and incoming message retrieval #551
• Split historical and incoming message retrieval #542
• Reject subscriptions to topics that don't exist #541
• Add non_fee_transfers table. #534
• Acceptance test improvements to support subscribe and publish only scenarios #532
• Monitor script/service to bounce hedera-mirror-grpc when it gets stuck. #528
• GRPC Pinger #527
• Optionally break out fees explicitly in the rest-api results. #516
• End to End HCS Test #497

Bug Fixes

• Fix shared poller's buffer #556
• Specify Bounded Executor for NettyServerBuilder #552
• RecordFileLogger performing far too many queries against t_entities #550
• RecordFileLogger performing far too many queries against t_entities. #547
• Set additional netty properties and memory limits #545
• Set maxConcurrentCallsPerConnection to prevent excessive calls #540
• Implement query throttling on grpc connections #536
• Fix limit set logic and use test start time instead of fixed time #526
• Handle start and end time arithmetic overflow cases on topic subscribe #523
• HCS GRPC Server throwing ArithmeticException: long overflow errors #521
• Record Parser unable to keep up #512
• Reject subscriptions to topics that don't exist #498

Contributors

We'd like to thank all the contributors who worked on this release!

• @apeksharma
• @mike-burrage-hedera
• @Nana-EC
• @steven-sheehy

v0.6.0-rc1

Fix shared poller's buffer (#556)

Signed-off-by: Steven Sheehy <[email protected]>

v0.5.5

Bug Fixes

• Fix subscribeTopic with end time in the past not retrieving more than page size #531
• Fix clients occasionally stalling under heavy load

Contributors

We'd like to thank all the contributors who worked on this release!

• @steven-sheehy