You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Before looping though sites and imts we pre-calculate the weights, branch_hash_table, and component_branches
weights numpy array
branch_hash_table List[List[str]] (1_000_000[4])
component_branches: List[HazardComponentBranch]
We want to distribute calc_aggregation() across processes with MultiProcess. We should share the common objects to avoid copying the arrays and lists (may not need to bother with component_branches as it can be calculated on the fly quite quickly)
Note that for the full NSHM_v1.0.4 the weight array and hash table are quite small and perhaps this is not necessary:
chrisdc@glacier:~/.../DEV/toshi-hazard-post$ NZSHM22_THS_LOCAL_DIR=/data/toshi_hazard_store/pq-CDC2/ poetry run thp aggregate demo/hazard_v2.toml
warning openquake module dependency not available, maybe you want to install
with nzshm-model[openquake]
Toshi Hazard Post: hazard curve aggregation
=================================================
2024-04-26 15:33:15,329 - toshi_hazard_post.version2.aggregation - INFO - getting sites . . .
2024-04-26 15:33:15,329 - toshi_hazard_post.version2.aggregation - INFO - getting logic trees . . .
2024-04-26 15:33:15,406 - toshi_hazard_post.version2.aggregation - INFO - calculating weights and branch hash table . . .
2024-04-26 15:33:15,406 - toshi_hazard_post.version2.logic_tree - DEBUG - generating composite branches
2024-04-26 15:33:43,714 - toshi_hazard_post.version2.aggregation - INFO - time to build weight array and hash table 28.31 seconds
2024-04-26 15:33:43,714 - toshi_hazard_post.version2.aggregation - INFO - Size of weight array: 7MB
2024-04-26 15:33:43,714 - toshi_hazard_post.version2.aggregation - INFO - Size of hash table: 8MB
The text was updated successfully, but these errors were encountered:
Before looping though sites and imts we pre-calculate the
weights
,branch_hash_table
, andcomponent_branches
weights
numpy arraybranch_hash_table
List[List[str]] (1_000_000[4])component_branches
: List[HazardComponentBranch]We want to distribute
calc_aggregation()
across processes withMultiProcess
. We should share the common objects to avoid copying the arrays and lists (may not need to bother withcomponent_branches
as it can be calculated on the fly quite quickly)Note that for the full
NSHM_v1.0.4
the weight array and hash table are quite small and perhaps this is not necessary:The text was updated successfully, but these errors were encountered: