-
I want each daily partition of my downstream asset to depend on three partitions of the upstream asset:
|
Beta Was this translation helpful? Give feedback.
Replies: 2 comments
-
You can accomplish this with the from dagster import AssetIn, DailyPartitionsDefinition, TimeWindowPartitionMapping, asset
@asset(partitions_def=DailyPartitionsDefinition(start_date="2020-01-01"))
def asset1():
...
@asset(
partitions_def=DailyPartitionsDefinition(start_date="2020-01-01"),
ins={"asset1": AssetIn(partition_mapping=TimeWindowPartitionMapping(start_offset=-2))},
)
def asset2(asset1):
... |
Beta Was this translation helpful? Give feedback.
-
What happens to the data on expiry? Does it, along with the runs associated logs/events get garbage collected upon being orphaned? Also, is it possible to put this in the partition definition itself? I.e have asset1 be an asset where the data has a static TTL and all derived data is subsequently destroyed as well? |
Beta Was this translation helpful? Give feedback.
You can accomplish this with the
start_offset
argument onTimeWindowPartitionMapping
: