refactor: change important metrics on port 8123 to be prometheus compatible

[skosito]

Description

As stated in issue, some of the metrics exposed in telemetry server endpoints should be refactored to be prometheus compatible as the other zetaclient metrics.

Moved to metrics:

• number of utxo (from status endpoint)
• version
• last start time
• last scanned block number
• last core block number

Still in telemetry, seems like it is more appropriate than metrics:

• p2p id
• ip address
• hot key burn rate (this is exposed both in metrics and telemetry, but it is stored in telemetry so i left it there)

looks like this:

version (named info so it is same as go_info)

# HELP zetaclient_info Information about Zetaclient environment
# TYPE zetaclient_info gauge
zetaclient_info{version="v12.3.0-rc-40-g56c63693"} 1

for reference, go_info looks like this:

# HELP go_info Information about the Go environment.
# TYPE go_info gauge
go_info{version="go1.20.14"} 1

last start time (using SetToCurrentTime from prometheus on zetaclient start)

# HELP zetaclient_last_start_timestamp_seconds Start time in Unix time
# TYPE zetaclient_last_start_timestamp_seconds gauge
zetaclient_last_start_timestamp_seconds 1.710267963269181e+09

utxo_number (not sure if it supposed to be count, but in telemetry it was more like gauge, so i kept gauge):

# HELP zetaclient_utxo_number Number of UTXOs
# TYPE zetaclient_utxo_number gauge
zetaclient_utxo_number 3

last scanned block number per chain (gauge, maybe it can be used to send alerts if its not increasing)

# HELP zetaclient_last_scanned_block_number Last scanned block number per chain
# TYPE zetaclient_last_scanned_block_number gauge
zetaclient_last_scanned_block_number{chain="btc_regtest"} 197
zetaclient_last_scanned_block_number{chain="goerli_localnet"} 172

last core block number (same as above)

# HELP zetaclient_last_core_block_number Last core block number
# TYPE zetaclient_last_core_block_number gauge
zetaclient_last_core_block_number 124

Closes: #1772

Type of change

Metrics refactor

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Include instructions and any relevant details so others can reproduce.

• Tested CCTX in localnet
• Tested in development environment
• Go unit tests
• Go integration tests
• Tested via GitHub Actions

Checklist:

• I have added unit tests that prove my fix feature works

[github-actions]

!!!WARNING!!!
nosec detected in the following files: zetaclient/bitcoin/bitcoin_client.go

Be very careful about using #nosec in code. It can be a quick way to suppress security warnings and move forward with development, it should be employed with caution. Suppressing warnings with #nosec can hide potentially serious vulnerabilities. Only use #nosec when you're absolutely certain that the security issue is either a false positive or has been mitigated in another way.

Only suppress a single rule (or a specific set of rules) within a section of code, while continuing to scan for other problems. To do this, you can list the rule(s) to be suppressed within the #nosec annotation, e.g: /* #nosec G401 */ or //#nosec G201 G202 G203
Broad #nosec annotations should be avoided, as they can hide other vulnerabilities. The CI will block you from merging this PR until you remove #nosec annotations that do not target specific rules.

Pay extra attention to the way #nosec is being used in the files listed above.

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 5 lines in your changes are missing coverage. Please review.

Project coverage is 44.84%. Comparing base (7ed70a0) to head (ba42692).

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #1885      +/-   ##
===========================================
+ Coverage    44.62%   44.84%   +0.22%     
===========================================
  Files          219      219              
  Lines        12771    12708      -63     
===========================================
  Hits          5699     5699              
+ Misses        6630     6567      -63     
  Partials       442      442              

Files	Coverage Δ
zetaclient/metrics/metrics.go	77.77% <ø> (ø)
zetaclient/evm/evm_client.go	12.51% <0.00%> (ø)
zetaclient/bitcoin/bitcoin_client.go	13.28% <0.00%> (ø)
zetaclient/metrics/telemetry.go	0.00% <0.00%> (ø)

[CharlieMc0]

The formatting looks good based on your comments. This will be helpful. Thanks!

[lumtis]

LGTM just one minor comment