Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop #3828

Merged
merged 73 commits into from
Jul 18, 2024
Merged

Develop #3828

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
e1ec84a
Added SIGTERM to sigintTrap
dszlachta Apr 30, 2024
f7b929c
Revert "Added SIGTERM to sigintTrap"
dszlachta Apr 30, 2024
6209b40
Added context to scrape methods and simplified returned values
dszlachta May 6, 2024
4ec9197
Merge branch 'develop' into feature/handle_sigterm
tjayrush Jun 2, 2024
a698b60
Merge branch 'develop' into feature/handle_sigterm
tjayrush Jun 5, 2024
ef27261
Merge branch 'develop' into feature/handle_sigterm
tjayrush Jun 6, 2024
6f4d93b
Fixes build
tjayrush Jun 6, 2024
09d2dc3
Cleanup and quit
dszlachta Jun 6, 2024
906ceb1
Make sure we only call covalent once
dszlachta Jun 7, 2024
a1021f0
Merge pull request #3768 from TrueBlocks/bugfix/slurp_all
tjayrush Jun 7, 2024
4c1243d
Merge pull request #3730 from TrueBlocks/feature/handle_sigterm
tjayrush Jun 7, 2024
faa0db8
chore: make function comment match function name
threehonor Jun 12, 2024
1008d2d
Moved scrape.pid
dszlachta Jun 13, 2024
92b33d6
Merge pull request #3784 from threehonor/master
tjayrush Jun 13, 2024
c3cce93
Merge pull request #3785 from TrueBlocks/feature/move_scrape.pid_to_t…
dszlachta Jun 13, 2024
bd29195
chore: make function comment match function names
fudancoder Jun 13, 2024
e39d62c
Merge pull request #3786 from fudancoder/master
tjayrush Jun 28, 2024
f19d5fc
Bump braces from 3.0.2 to 3.0.3 in /sdk/typescript
dependabot[bot] Jun 28, 2024
14c291c
Merge pull request #3794 from TrueBlocks/dependabot/npm_and_yarn/sdk/…
tjayrush Jun 28, 2024
f91398a
Updates go.mod and go.sum throughout
tjayrush Jul 1, 2024
f6b98d7
Removes petnames from all code as unused
tjayrush Jul 1, 2024
e890cbe
Updates python and Typescript SDKs
tjayrush Jul 1, 2024
43183a8
Minimal changes to render better markdown from auto code
tjayrush Jul 1, 2024
3326961
Removes gRpc from daemon and names as unused
tjayrush Jul 1, 2024
6f3c7a8
Better UX for MakeBackup functionality
tjayrush Jul 1, 2024
bc77b91
Small update to SDK
tjayrush Jul 1, 2024
c32c97b
Adds a target for docker usage
tjayrush Jul 1, 2024
9944219
Adds version 1 of namesManager example app
tjayrush Jul 1, 2024
df61132
Updates goMods
tjayrush Jul 1, 2024
f1cd047
Enables name capability but does not implement it in globals
tjayrush Jul 1, 2024
7c795b1
Enables Names capability. If --verbose is given, any exported address…
tjayrush Jul 1, 2024
882ff11
Rewrites the names package to be more robust and remove unused code. …
tjayrush Jul 1, 2024
0ac892c
Implements better crudiness
tjayrush Jul 1, 2024
1c5a79d
Updates testing and adds some new test cases
tjayrush Jul 1, 2024
29963bc
Adds a ton of new names
tjayrush Jul 1, 2024
4580de6
Updates tests
tjayrush Jul 1, 2024
4314145
Merge pull request #3797 from TrueBlocks/names-epic-1
tjayrush Jul 2, 2024
261fd71
Fixed panic when slurping Covalent
dszlachta Jul 5, 2024
37987e1
Merge pull request #3798 from TrueBlocks/bugfix/slurp_covalent_panic
dszlachta Jul 5, 2024
c7cb33b
Feature/3783 replace viper (#3796)
dszlachta Jul 9, 2024
c4aaab6
Updates go.mod and go.sum throughout.
tjayrush Jul 14, 2024
86b5801
Merge pull request #3811 from TrueBlocks/3810-update-gomod-files
tjayrush Jul 14, 2024
c88120b
Modifies go.mod so we can make changes to local SDK
tjayrush Jul 15, 2024
c1e1453
Adds a new type, RenderCtx, that holds a CancelContext and the Cancel…
tjayrush Jul 15, 2024
3381954
Pushing the rCtx down from the SDK into chifra. Still not implemented…
tjayrush Jul 15, 2024
3429e84
Pushes the render context further down towards the Handlers. Still no…
tjayrush Jul 15, 2024
f218580
Pushes the render contexts down into the Handlers, but still does not…
tjayrush Jul 15, 2024
b9629fe
Replaces existing contexts and cancel functions with those now found …
tjayrush Jul 15, 2024
4f88330
For all concurrent loops for rendering data, add a test for the conso…
tjayrush Jul 15, 2024
ba58b68
Adds an example that shows how to use the cancel context
tjayrush Jul 15, 2024
5d85d67
Final cleaning, including sdkFuzzer
tjayrush Jul 15, 2024
95593eb
Merge pull request #3812 from TrueBlocks/feature/cancel-contexts-1
tjayrush Jul 15, 2024
b75502e
Closes issue #3815
tjayrush Jul 15, 2024
6e8fe80
Cleans up creation of new examples
tjayrush Jul 16, 2024
39264d0
New example, but it is not yet implemented.
tjayrush Jul 16, 2024
cc61a68
Modifies StreamMany to accept rCtx and if rCtx.ModelChan is non-nil, …
tjayrush Jul 16, 2024
7297ba2
Expands example to test names streaming also
tjayrush Jul 16, 2024
9963f03
Removes ProgressChan from RenderCtx as unused
tjayrush Jul 16, 2024
4e6c5e7
Enhances streaming example
tjayrush Jul 16, 2024
11e623b
Documents the example
tjayrush Jul 16, 2024
5671631
Adds func (a Address) CheckSum() string
tjayrush Jul 16, 2024
fa45147
Fixes a previously unknown bug
tjayrush Jul 17, 2024
1b5d3f1
Updating withStreaming example
tjayrush Jul 17, 2024
842803d
Updates go.mod files
tjayrush Jul 17, 2024
ae4652c
Final commit
tjayrush Jul 17, 2024
59a94eb
Much better balanceChart example
tjayrush Jul 17, 2024
c97fbb2
Adds simpleStreaming example and updates simple example's README
tjayrush Jul 18, 2024
616de9a
Removes simpleStreaming example
tjayrush Jul 18, 2024
baf63b4
Adjustments to balanceChart example
tjayrush Jul 18, 2024
3c62543
Updates go.mods
tjayrush Jul 18, 2024
16c1b81
Merge pull request #3822 from TrueBlocks/3814-epic-streaming-sdk
tjayrush Jul 18, 2024
e417b50
Last post before merge to master
tjayrush Jul 18, 2024
e50d207
Merge pull request #3827 from TrueBlocks/final/for-release
tjayrush Jul 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Documents the example
  • Loading branch information
tjayrush committed Jul 16, 2024
commit 11e623b8ca9f3b95a3f3e2fd97ccc95a08518273
56 changes: 25 additions & 31 deletions examples/withStreaming/streamExport.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,57 +2,51 @@ package main

import (
"fmt"
"os"
"time"

"github.com/TrueBlocks/trueblocks-core/sdk"
"github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/output"
"github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/types"
)

// TestStreamExport demonstrates the SDK streaming feature by creating a Rendering
// Context with both Model and Error channels. It also demonstrates canceling the
// context (and therefore the rendering) by using the Cancel feature.
func TestStreamExport() {
opts := sdk.ExportOptions{
Addrs: []string{"0x1f98431c8ad98523631ae4a59f267346ea31f984"}, // Uniswap V3 Factory
Unripe: true,
Unripe: true, // to the head of the chain
RenderCtx: output.WithChannels(),
}
opts.Globals.Cache = true
opts.Globals.Cache = true // Write to the cache

go func() {
// Sleep for 3 seconds and then cancel the rendering context
time.Sleep(3 * time.Second)
opts.RenderCtx.Cancel()
}()

if len(os.Args) > 1 {
opts.RenderCtx = nil
txs, _, err := opts.Export()
if err != nil {
fmt.Println(err)
return
}

for _, tx := range txs {
fmt.Printf("%d\t%d\n", tx.BlockNumber, tx.TransactionIndex)
}
} else {
go func() {
for {
select {
case model := <-opts.RenderCtx.ModelChan:
if tx, ok := model.(*types.Transaction); !ok {
fmt.Println("Not a transaction")
} else {
fmt.Printf("%d\t%d\n", tx.BlockNumber, tx.TransactionIndex)
}
case err := <-opts.RenderCtx.ErrorChan:
fmt.Println("Error returned by fetchData:", err)
go func() {
for {
select {
case model := <-opts.RenderCtx.ModelChan:
// We got a transaction. Show it. Note that the cast will always succeed
if tx, ok := model.(*types.Transaction); ok {
fmt.Printf("%d\t%d\n", tx.BlockNumber, tx.TransactionIndex)
} else {
fmt.Println("Not a transaction")
}
case err := <-opts.RenderCtx.ErrorChan:
fmt.Println("Error returned by fetchData:", err)
// os.Exit(1)
}
}()
_, _, err := opts.Export()
if err != nil {
fmt.Println(err)
return
}
}()

// Generate the stream...
_, _, err := opts.Export()
if err != nil {
fmt.Println(err)
return
}
}
13 changes: 10 additions & 3 deletions examples/withStreaming/streamNames.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,28 +6,35 @@ import (
"github.com/TrueBlocks/trueblocks-core/sdk"
"github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/logger"
"github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/output"
"github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/types"
)

// TestStreamNames demonstrates the SDK streaming feature for names by
// creating a Rendering Context with channels.
func TestStreamNames() {
opts := sdk.NamesOptions{
RenderCtx: output.WithChannels(),
}

go func() {
// start a go routine that listens to the model and error channels
// and prints the names as they stream in
for {
select {
case name := <-opts.RenderCtx.ModelChan:
if n, ok := name.(*sdk.Name); !ok {
logger.Fatal("This should never happen")
} else {
if n, ok := name.(*types.Name); ok {
fmt.Printf("%s\t%s\t%s\n", n.Tags, n.Address, n.Name)
} else {
logger.Fatal("This should never happen")
}
case err := <-opts.RenderCtx.ErrorChan:
fmt.Println("Error returned by fetchData:", err)
// os.Exit(1)
}
}
}()

// Generate the stream...
if _, _, err := opts.Names(); err != nil {
fmt.Println(err)
}
Expand Down
5 changes: 5 additions & 0 deletions examples/withStreaming/streamProgress.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ import (
"github.com/schollz/progressbar/v3"
)

// TestProgressReporting is a simple example just demonstrating how to use
// the progress bar to report progress of the stream.
func TestProgressReporting() {
nBlocks := 1000
bar := newProgressBar(nBlocks)
Expand All @@ -34,12 +36,15 @@ func TestProgressReporting() {
}
}()

// This will stream the blocks. Note that they are LightBlocks
// which is what BlocksHashes returns.
if _, _, err := opts.BlocksHashes(); err != nil {
panic(err)
}
bar.Finish()
}

// newProgressBar creates a new progress bar with the given count
func newProgressBar(cnt int) *progressbar.ProgressBar {
return progressbar.NewOptions(cnt,
progressbar.OptionSetWriter(os.Stdout), //you should install "github.com/k0kubun/go-ansi"
Expand Down
Loading