This repository has been archived by the owner on Dec 9, 2024. It is now read-only.
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Add support union type Signed-off-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * Fixed test Signed-off-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * Fixed test Signed-off-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * Fixed lint Signed-off-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * Fixed tests Signed-off-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * Updated tests Signed-off-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * Update Union to use tagged Unions Signed-off-by: maximsmol <[email protected]> * Update to use string tags (part 1) Signed-off-by: maximsmol <[email protected]> * Working implementation, update tests Signed-off-by: maximsmol <[email protected]> * Fixes, more tests Signed-off-by: maximsmol <[email protected]> * Fix incorrect unwrapped literal-union matching, update test Signed-off-by: maximsmol <[email protected]> * Fix duplicate tag handling, add tests for collections containing unions Signed-off-by: maximsmol <[email protected]> * Fix type hint test Signed-off-by: maximsmol <[email protected]> * Add implicit wrapping union type tests Signed-off-by: maximsmol <[email protected]> * Add union ambiguity tests Signed-off-by: maximsmol <[email protected]> * Fixup tests, make TypeTransformerFailed compatible with all old exception types Signed-off-by: maximsmol <[email protected]> * Fixup models + add tests Signed-off-by: maximsmol <[email protected]> * Implement changed design Signed-off-by: maximsmol <[email protected]> * Fix PR comments Signed-off-by: maximsmol <[email protected]> * Remote entrypoint serialize (flyteorg#733) Signed-off-by: Emirhan Karagül <[email protected]> Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: maximsmol <[email protected]> * Fix lint error in remote.py (flyteorg#755) Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Signed-off-by: maximsmol <[email protected]> * Support enum in dataclass (flyteorg#753) * Add support enum in dataclass Signed-off-by: Kevin Su <[email protected]> * Update test Signed-off-by: Kevin Su <[email protected]> * Fixed lint Signed-off-by: Kevin Su <[email protected]> * Fix subworkflow and launch plan FlyteRemote behavior (flyteorg#751) Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: maximsmol <[email protected]> * Extras: Shell task (flyteorg#747) Signed-off-by: maximsmol <[email protected]> * Add support FlyteSchema in dataclass (flyteorg#722) * schema in dataclass Signed-off-by: Kevin Su <[email protected]> * Added tests Signed-off-by: Kevin Su <[email protected]> * Fixed lint Signed-off-by: Kevin Su <[email protected]> * Updated tests Signed-off-by: Kevin Su <[email protected]> * Updated tests Signed-off-by: Kevin Su <[email protected]> * Fixed lint Signed-off-by: Kevin Su <[email protected]> * updated Signed-off-by: Kevin Su <[email protected]> * updated Signed-off-by: Kevin Su <[email protected]> * updated Signed-off-by: Kevin Su <[email protected]> * Remove workflow_execution.py (flyteorg#758) Signed-off-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * Get raw input/output from remote execution (flyteorg#675) * [wip] for feast demo Signed-off-by: Ketan Umare <[email protected]> * clean up a bit Signed-off-by: Yee Hing Tong <[email protected]> * add a test and move where constructor is called Signed-off-by: Yee Hing Tong <[email protected]> * remove unneeded import Signed-off-by: Yee Hing Tong <[email protected]> * add a part of a test Signed-off-by: Yee Hing Tong <[email protected]> * Added tests Signed-off-by: Kevin Su <[email protected]> * Fixed lint Signed-off-by: Kevin Su <[email protected]> * typo Signed-off-by: Kevin Su <[email protected]> Co-authored-by: Yee Hing Tong <[email protected]> Co-authored-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * Fix mypy errors in flytekit/types (flyteorg#757) Signed-off-by: Lisa <[email protected]> Signed-off-by: Kevin Su <[email protected]> * Remote client failed to fetch FlytePickle object (flyteorg#764) * Fetch pickle value from flytekit remote Signed-off-by: Kevin Su <[email protected]> * Fix tests Signed-off-by: Kevin Su <[email protected]> * Remove default value Signed-off-by: Kevin Su <[email protected]> * Add support FlyteFile in dataclass (flyteorg#725) * Add support Flyte File and directory in dataclass Signed-off-by: Kevin Su <[email protected]> * Fixed tests Signed-off-by: Kevin Su <[email protected]> * Fixed tests Signed-off-by: Kevin Su <[email protected]> * Updated Signed-off-by: Kevin Su <[email protected]> * Updated Signed-off-by: Kevin Su <[email protected]> * Fixed tests Signed-off-by: Kevin Su <[email protected]> * add task_resolver arg to @task decorator (flyteorg#765) Signed-off-by: Niels Bantilan <[email protected]> Signed-off-by: maximsmol <[email protected]> * Copy metadata into map task from underlying (flyteorg#766) Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: maximsmol <[email protected]> * Support for delayed annotations (flyteorg#760) Signed-off-by: Stefan Nelson-Lindall <[email protected]> Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: maximsmol <[email protected]> * Complex dataclass unit tests (flyteorg#773) Signed-off-by: Ketan Umare <[email protected]> Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: maximsmol <[email protected]> * remote_source lost on serialization of @dataclass_json with FlyteFile (flyteorg#774) * remote_source lost on serialization of @dataclass_json with FlyteFile Signed-off-by: Kevin Su <[email protected]> * updated tests Signed-off-by: Kevin Su <[email protected]> * updated tests Signed-off-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * Single-task execution FlyteRemote sync (flyteorg#778) Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: maximsmol <[email protected]> * Logging updates (flyteorg#775) Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: maximsmol <[email protected]> * fix: doc-requirements.txt to reduce vulnerabilities (flyteorg#779) The following vulnerabilities are fixed by pinning transitive dependencies: - https://snyk.io/vuln/SNYK-PYTHON-LXML-2316995 Signed-off-by: maximsmol <[email protected]> * Add cache_serialize parameter to tasks (flyteorg#673) * added cache_serialize parameter for basic tasks Signed-off-by: Daniel Rammer <[email protected]> * fixed typo Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl version Signed-off-by: Daniel Rammer <[email protected]> * changed flyteidl requirements everywhere Signed-off-by: Daniel Rammer <[email protected]> * remove flyteidl version requires in setup.py so we can use develop Signed-off-by: Daniel Rammer <[email protected]> * removed flyteidl git repos from a variety of requirements packages Signed-off-by: Daniel Rammer <[email protected]> * updated variable discovery_serializable to cache_serializable Signed-off-by: Daniel Rammer <[email protected]> * updated requirements Signed-off-by: Daniel Rammer <[email protected]> * fixed TaskMetadata _cache_serializable variable name Signed-off-by: Daniel Rammer <[email protected]> * propgating cache_serialize parameter through to tasks Signed-off-by: Daniel Rammer <[email protected]> * added cache_serializable to SdkRawContainerTask Signed-off-by: Daniel Rammer <[email protected]> * fixing cache_serializable variable propogation issues Signed-off-by: Daniel Rammer <[email protected]> * added documentation Signed-off-by: Daniel Rammer <[email protected]> * added unit tests for cache_serialize metadata Signed-off-by: Daniel Rammer <[email protected]> * linter added spaces in unit tests Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: maximsmol <[email protected]> * When using the `task` and `workflow` decorator, correctly wrap the fu… (flyteorg#780) * When using the `task` and `workflow` decorator, correctly wrap the function This enables tooling such as docstring search tools to unwrap the object and show the correct docstring. Signed-off-by: Bernhard Stadlbauer <[email protected]> * Remove blackshark copyright header Signed-off-by: Bernhard Stadlbauer <[email protected]> * Fix broken great expectations test Signed-off-by: Bernhard Stadlbauer <[email protected]> * Add test for stacked decorators Signed-off-by: Bernhard Stadlbauer <[email protected]> Co-authored-by: Bernhard Stadlbauer <[email protected]> Co-authored-by: Bernhard Stadlbauer <[email protected]> Signed-off-by: maximsmol <[email protected]> * Add option to flyte-cli for specifying root certificate (flyteorg#783) Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: maximsmol <[email protected]> * Add validation check to cacert switch (flyteorg#787) Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: maximsmol <[email protected]> * Remove pytz constraint (flyteorg#786) * Remove pytz constraint from setup.py Signed-off-by: Eduardo Apolinario <[email protected]> * Regenerate requirements files Signed-off-by: Eduardo Apolinario <[email protected]> * Put pytz back Signed-off-by: Eduardo Apolinario <[email protected]> * make requirements.txt Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Signed-off-by: maximsmol <[email protected]> * Lint: remove f-string misuse (flyteorg#788) Signed-off-by: Eduardo Apolinario <[email protected]> Signed-off-by: maximsmol <[email protected]> * Pyarrow greater than 4.0.0 (flyteorg#790) * pyarrow>=4.0.0 Signed-off-by: Eduardo Apolinario <[email protected]> * Regenerate requirements. Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Signed-off-by: maximsmol <[email protected]> * Support python 3.10 (flyteorg#791) * [wip] Support python 3.10 Signed-off-by: Eduardo Apolinario <[email protected]> * Add Dockerfile.py310 Signed-off-by: Eduardo Apolinario <[email protected]> * Stringify python version Signed-off-by: Eduardo Apolinario <[email protected]> * Skip flytekit-modin plugin tests on 3.10 Signed-off-by: Eduardo Apolinario <[email protected]> * Add 3.9 and 3.10 to list of supported version in plugins Signed-off-by: Eduardo Apolinario <[email protected]> * Comment why flytekit-modin is not running on 3.10 and disable fail-fast in plugin tests Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> * add `with_overrides` to map task (flyteorg#794) * add with_overrides Signed-off-by: Samhita Alla <[email protected]> * remove Resources Signed-off-by: Samhita Alla <[email protected]> Signed-off-by: maximsmol <[email protected]> * bump docsearch version (flyteorg#805) Signed-off-by: Samhita Alla <[email protected]> Signed-off-by: maximsmol <[email protected]> * update docs for new navbar theme (flyteorg#806) Signed-off-by: maximsmol <[email protected]> * fix requirment.txt github issue (flyteorg#810) Signed-off-by: Yuvraj <[email protected]> Signed-off-by: maximsmol <[email protected]> * Add sphinx panels (flyteorg#815) Signed-off-by: Ketan Umare <[email protected]> Signed-off-by: maximsmol <[email protected]> * Schema overhaul (flyteorg#785) Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Yee Hing Tong <[email protected]> * Parent workflow serialization fails when calling a launch plan with fixed inputs (flyteorg#814) Signed-off-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * Fix sagemaker plugin (flyteorg#817) Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Signed-off-by: maximsmol <[email protected]> * Remove legacy API (flyteorg#807) Signed-off-by: Yee Hing Tong <[email protected]> * Add Bigquery plugin (flyteorg#789) * Add bigquery plugin Signed-off-by: Kevin Su <[email protected]> * Update dependency Signed-off-by: Kevin Su <[email protected]> * update get_custom Signed-off-by: Kevin Su <[email protected]> * Add structured dataset Signed-off-by: Kevin Su <[email protected]> * Add structured dataset Signed-off-by: Kevin Su <[email protected]> * Updated comment Signed-off-by: Kevin Su <[email protected]> * Add BQ in GA Signed-off-by: Kevin Su <[email protected]> * alphabetical order Signed-off-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * Fixed flytekit-papermill ImportError (flyteorg#818) Signed-off-by: Kevin Su <[email protected]> * Add support for string-format-like sytax for shell task (flyteorg#792) * POC: Add support for f-string like sytax for shell task This commit is a proof of concept adding f-string like syntax for shell_tasks. This supports using nested types for script inputs, such as data classes. This change was motivated by the desire to combine shell_tasks that have multiple inputs with map_tasks which only support tasks with a single input. This commit is only a starting point, since it makes some changes to the shell_task API (adds a template_style field), and modifies some of the default behavior for ease of implementation (e.g. throwing an error when there are unused input arguments). Signed-off-by: Zach Palchick <[email protected]> * Drop support for old/regex style for doing string interpolation Signed-off-by: Zach Palchick <[email protected]> Signed-off-by: maximsmol <[email protected]> * Expose configured RawOutputPrefix during execution (flyteorg#813) * Expose configured RawOutputPrefix during execution Signed-off-by: Kevin Su <[email protected]> * Remove sdk_runnable.py and spark_task.py Signed-off-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * Add SecretsManager back to old import location (flyteorg#820) Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: maximsmol <[email protected]> * Add some tests (flyteorg#819) Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: maximsmol <[email protected]> * Fixed flaky spark test (flyteorg#821) Signed-off-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * fix: plugins/flytekit-greatexpectations/requirements.txt to reduce vulnerabilities (flyteorg#823) The following vulnerabilities are fixed by pinning transitive dependencies: - https://snyk.io/vuln/SNYK-PYTHON-IPYTHON-2348630 Signed-off-by: maximsmol <[email protected]> * fix: plugins/flytekit-papermill/requirements.txt to reduce vulnerabilities (flyteorg#825) The following vulnerabilities are fixed by pinning transitive dependencies: - https://snyk.io/vuln/SNYK-PYTHON-IPYTHON-2348630 Signed-off-by: maximsmol <[email protected]> * fix: requirements-spark2.txt to reduce vulnerabilities (flyteorg#826) The following vulnerabilities are fixed by pinning transitive dependencies: - https://snyk.io/vuln/SNYK-PYTHON-IPYTHON-2348630 Signed-off-by: maximsmol <[email protected]> * fix: requirements.txt to reduce vulnerabilities (flyteorg#824) The following vulnerabilities are fixed by pinning transitive dependencies: - https://snyk.io/vuln/SNYK-PYTHON-IPYTHON-2348630 Signed-off-by: maximsmol <[email protected]> * Intratask checkpointing (flyteorg#771) * wip - intratask checkpointing Signed-off-by: Ketan Umare <[email protected]> * sync checkpointer with tests Signed-off-by: Ketan Umare <[email protected]> * Checkpinter in entrypoint Signed-off-by: Ketan Umare <[email protected]> * checkpoint in progress Signed-off-by: Ketan Umare <[email protected]> * wip Signed-off-by: Ketan Umare <[email protected]> * Intratask checkpointer Signed-off-by: Ketan Umare <[email protected]> * Checkpoint updated Signed-off-by: Ketan Umare <[email protected]> * Intra-task checkpointing Signed-off-by: Ketan Umare <[email protected]> * Test and entrypoint updated Signed-off-by: Ketan Umare <[email protected]> * lint fixed Signed-off-by: Ketan Umare <[email protected]> * test fixes Signed-off-by: Ketan Umare <[email protected]> * fmt Signed-off-by: Ketan Umare <[email protected]> * updated entrypoint Signed-off-by: Ketan Umare <[email protected]> * updated Signed-off-by: Ketan Umare <[email protected]> * update Signed-off-by: Ketan Umare <[email protected]> * print Signed-off-by: Ketan Umare <[email protected]> * updated Signed-off-by: Ketan Umare <[email protected]> * SyncCheckpointer working Signed-off-by: Ketan Umare <[email protected]> * updated Signed-off-by: Ketan Umare <[email protected]> * update Signed-off-by: Ketan Umare <[email protected]> * fixed import problems Signed-off-by: Ketan Umare <[email protected]> * fixed test Signed-off-by: Ketan Umare <[email protected]> * fixed imports Signed-off-by: Ketan Umare <[email protected]> * fixed lints and errors Signed-off-by: Ketan Umare <[email protected]> * lint fix Signed-off-by: Ketan Umare <[email protected]> * addressed comments Signed-off-by: Ketan Umare <[email protected]> Signed-off-by: maximsmol <[email protected]> * Support reading subset column (flyteorg#822) * Support StructuredDatasetDecoder read subset column Signed-off-by: Kevin Su <[email protected]> * Added tests Signed-off-by: Kevin Su <[email protected]> * Fixed tests Signed-off-by: Kevin Su <[email protected]> * Fixed typo Signed-off-by: Kevin Su <[email protected]> * Updated tests Signed-off-by: Kevin Su <[email protected]> * [pr into flyteorg#822] (flyteorg#827) Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Kevin Su <[email protected]> * [pr into flyteorg#822] Final update to structured dataset column subsetting (flyteorg#828) Signed-off-by: Yee Hing Tong <[email protected]> Co-authored-by: Yee Hing Tong <[email protected]> * Fix spark regression (flyteorg#830) Signed-off-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * Update argument setting for in fast registered, dynamically generated, pod tasks (flyteorg#835) Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: maximsmol <[email protected]> * `ctx` Context can be used within shell tasks - to access context vars and secrets (flyteorg#832) * Adding context to a substitutable parameter in shell task Signed-off-by: Ketan Umare <[email protected]> * Support for secrets in context Signed-off-by: Ketan Umare <[email protected]> * addressed comments Signed-off-by: Ketan Umare <[email protected]> Signed-off-by: maximsmol <[email protected]> * Expose Checkpoint as a top-level interface in flytekit (flyteorg#839) Signed-off-by: Ketan Umare <[email protected]> Signed-off-by: maximsmol <[email protected]> * Parse duration field from flyteidl to `flytekit.models.execution.ExecutionClosure` (flyteorg#829) * Parse duration field from flyteidl to `flytekit.models.execution.ExecutionClosure` Signed-off-by: Bernhard Stadlbauer <[email protected]> * Add test for execution closure Signed-off-by: Bernhard Stadlbauer <[email protected]> * Add tests to Flyte remote Signed-off-by: Bernhard Stadlbauer <[email protected]> * Split execution test into with output and with error Signed-off-by: Bernhard Stadlbauer <[email protected]> Co-authored-by: Bernhard Stadlbauer <[email protected]> Signed-off-by: maximsmol <[email protected]> * Gate new Structured Dataset feature & remove old config objects (flyteorg#831) Signed-off-by: Yee Hing Tong <[email protected]> * Fixing out of order for conditional outputs (flyteorg#843) Signed-off-by: maximsmol <[email protected]> * Set default values to map task template (flyteorg#841) * Set sane defaults in map task templates Signed-off-by: Eduardo Apolinario <[email protected]> * Remove unused method Signed-off-by: Eduardo Apolinario <[email protected]> * Put ArrayJob.from_dict back Signed-off-by: Eduardo Apolinario <[email protected]> * Define parallelism=0 as unbounded Signed-off-by: Eduardo Apolinario <[email protected]> * Remove special case to handle 0 Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Signed-off-by: maximsmol <[email protected]> * Regenerate requirements files - dependencies dropping support for python 3.7 (flyteorg#838) * Regenerate requirements files Signed-off-by: Eduardo Apolinario <[email protected]> * Put restrictions on numpy and pandas versions Signed-off-by: Eduardo Apolinario <[email protected]> * Use --use-deprecated=legacy-resolver Signed-off-by: Eduardo Apolinario <[email protected]> * use pip==22.0.3 everywhere Signed-off-by: Eduardo Apolinario <[email protected]> * Remove --use-deprecated=legacy-resolver Signed-off-by: Eduardo Apolinario <[email protected]> * Relax click Signed-off-by: Eduardo Apolinario <[email protected]> * Regenerated plugins requirements Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> * TypeAnnotation (flyteorg#759) * feat: support for annotated simple + list Signed-off-by: Kenny Workman <[email protected]> * feat: addition of annotation att to Signed-off-by: Kenny Workman <[email protected]> * feat: core obj Signed-off-by: Kenny Workman <[email protected]> * feat: proto model Signed-off-by: Kenny Workman <[email protected]> * feat: testing suite Signed-off-by: Kenny Workman <[email protected]> * fix: more stable typing introspection Signed-off-by: Kenny Workman <[email protected]> * fix: strip legacy Signed-off-by: Kenny Workman <[email protected]> * fix: explicitly allow only one annotation Signed-off-by: Kenny Workman <[email protected]> * feat: direct type transformer tests Signed-off-by: Kenny Workman <[email protected]> * fix: there and back test Signed-off-by: Kenny Workman <[email protected]> * fix: typing_extensions for get_origin Signed-off-by: Kenny Workman <[email protected]> * fix: more semantic list generic unwrap Signed-off-by: Kenny Workman <[email protected]> * fix: tmp requirements file with custom idl Signed-off-by: Kenny Workman <[email protected]> * fix: nits Signed-off-by: Kenny Workman <[email protected]> * feat: semantic error for unsupported complex literals Signed-off-by: Kenny Workman <[email protected]> * fix: but Signed-off-by: Kenny Workman <[email protected]> * feat: more tests ;) Signed-off-by: Kenny Workman <[email protected]> * fix: imports Signed-off-by: Kenny Workman <[email protected]> * fix: complex annotations Signed-off-by: Kenny Workman <[email protected]> * fix: temp requirements files for unit tests Signed-off-by: Kenny Workman <[email protected]> * fix: lint bug Signed-off-by: Kenny Workman <[email protected]> * fix: tmp setup.py Signed-off-by: Kenny Workman <[email protected]> * fix: use typing_extensions Signed-off-by: Kenny Workman <[email protected]> * fix: typing_extensions for annotated Signed-off-by: Kenny Workman <[email protected]> * fix: typing_ext Signed-off-by: Kenny Workman <[email protected]> * fix: plugin tmp requirements Signed-off-by: Kenny Workman <[email protected]> * fix: bump requirements Signed-off-by: Kenny Workman <[email protected]> * fix: doc requirements Signed-off-by: Kenny Workman <[email protected]> * fix: whitespace Signed-off-by: Kenny Workman <[email protected]> * fix: bump flytekit Signed-off-by: Kenny Workman <[email protected]> * fix: numpy version Signed-off-by: Kenny Workman <[email protected]> * fix: lint Signed-off-by: Kenny Workman <[email protected]> * fix: pandas version Signed-off-by: Kenny Workman <[email protected]> * fix: bump requirements Signed-off-by: Kenny Workman <[email protected]> * fix: test import Signed-off-by: Kenny Workman <[email protected]> * fix: flake8 lint Signed-off-by: Kenny Workman <[email protected]> * fix: merge Signed-off-by: Kenny Workman <[email protected]> * fix: requirements Signed-off-by: Kenny Workman <[email protected]> * fix: requirements Signed-off-by: Kenny Workman <[email protected]> * fix: lint Signed-off-by: Kenny Workman <[email protected]> * fix: papermill req Signed-off-by: Kenny Workman <[email protected]> * fix: req Signed-off-by: Kenny Workman <[email protected]> * Remove singleton from structured dataset transformer engine (flyteorg#848) Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Kevin Su <[email protected]> * Assign input and output to FlyteWorkflowExecution (flyteorg#842) Signed-off-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * Add reference entities to FlyteTask and FlyteLaunchPlan (flyteorg#850) Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: maximsmol <[email protected]> * Fix fast registration error (flyteorg#851) Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Samhita Alla <[email protected]> Signed-off-by: maximsmol <[email protected]> * Add support for local execute in pod task (flyteorg#852) Signed-off-by: ggydush-fn <[email protected]> Signed-off-by: maximsmol <[email protected]> * Add anonymous retry (flyteorg#854) Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: maximsmol <[email protected]> * fix: formatting, linting, typing_extensions * fix: do not use SDK types Signed-off-by: maximsmol <[email protected]> * fix: update test comment Signed-off-by: maximsmol <[email protected]> * fix: also check literal type castability when tags match Signed-off-by: maximsmol <[email protected]> * Point flyteidl to maxim's fork in CI and requirements files Signed-off-by: Eduardo Apolinario <[email protected]> * [Core feature] Add Raw AWS Batch Task (flyteorg#782) * Init plugin Signed-off-by: Kevin Su <[email protected]> * Fixed lint Signed-off-by: Kevin Su <[email protected]> * address comment Signed-off-by: Kevin Su <[email protected]> * Fixed typo Signed-off-by: Kevin Su <[email protected]> * Updated AWS config * Fixed lint Signed-off-by: Kevin Su <[email protected]> * Added comment Signed-off-by: Kevin Su <[email protected]> * Update config Signed-off-by: Kevin Su <[email protected]> * Fixed tests Signed-off-by: Kevin Su <[email protected]> * Fixed tests Signed-off-by: Kevin Su <[email protected]> * Fixed tests Signed-off-by: Kevin Su <[email protected]> * use pyflyte execute Signed-off-by: Kevin Su <[email protected]> * Fixed tests Signed-off-by: Kevin Su <[email protected]> * Fixed tests Signed-off-by: Kevin Su <[email protected]> * Added comment Signed-off-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * Add structured dataset encoder/decoder in fsspec plugin (flyteorg#849) Signed-off-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * Delete unnecessary auth configuration (flyteorg#858) Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: maximsmol <[email protected]> * Fixed format alias in Flytekit docs (flyteorg#844) * Fixed format alias Fixed docs for file format alias that weren't rendered properly. A warning popped up stating 'ignore' is deprecated, and to use 'ignore_paths' instead. Signed-off-by: SmritiSatyanV <[email protected]> * Bump idl (flyteorg#862) Signed-off-by: Yee Hing Tong <[email protected]> * Updated authoring.rst (flyteorg#863) * Updated authoring.rst Added directive Rephrased sentence * Fixed build error Signed-off-by: SmritiSatyanV <[email protected]> * test-build-1 Signed-off-by: SmritiSatyanV <[email protected]> Signed-off-by: maximsmol <[email protected]> * Updated authoring.rst (flyteorg#866) Added the directive correctly Signed-off-by: SmritiSatyanV <[email protected]> Signed-off-by: maximsmol <[email protected]> * Change docs for HTMLPage type to say HTMLPage instead of PNGImage (flyteorg#868) Signed-off-by: maximsmol <[email protected]> * Revisit StructuredDatasetDecoder interface (flyteorg#865) Signed-off-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * Remove legacy mentions in contributing guide (flyteorg#870) Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: maximsmol <[email protected]> * Add GCS protocol in the structured dataset (flyteorg#869) Signed-off-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * Make fetched entities callable within workflows (flyteorg#867) Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: maximsmol <[email protected]> * Caching of offloaded objects (flyteorg#762) * Remove flyteidl from install_requires Signed-off-by: Eduardo Apolinario <[email protected]> * Expose hash in Literal Signed-off-by: Eduardo Apolinario <[email protected]> * Set hash in TypeEngine Signed-off-by: Eduardo Apolinario <[email protected]> * Modify cache key calculation to take hash into account Signed-off-by: Eduardo Apolinario <[email protected]> * Opt-in PandasDataFrameTransformer Signed-off-by: Eduardo Apolinario <[email protected]> * Add unit tests Signed-off-by: Eduardo Apolinario <[email protected]> * Iterate using a flyteidl branch Signed-off-by: Eduardo Apolinario <[email protected]> * Regenerate requirements files Signed-off-by: Eduardo Apolinario <[email protected]> * Regenerate requirements files Signed-off-by: Eduardo Apolinario <[email protected]> * Move _hash_overridable to StructureDatasetTransformerEngine Signed-off-by: Eduardo Apolinario <[email protected]> * Move HashMethod to flytekit.core.hash Signed-off-by: Eduardo Apolinario <[email protected]> * Fix `unit_test` make target Signed-off-by: Eduardo Apolinario <[email protected]> * Split `unit_test` make target in two lines Signed-off-by: Eduardo Apolinario <[email protected]> * Add assert to structured dataset compatibility test Signed-off-by: Eduardo Apolinario <[email protected]> * Remove TODO Signed-off-by: Eduardo Apolinario <[email protected]> * Regenerate plugins requirements files pointing to the right version of flyteidl. Signed-off-by: Eduardo Apolinario <[email protected]> * Set hash as a property of the literal Signed-off-by: Eduardo Apolinario <[email protected]> * Install plugins requirements in CI. Signed-off-by: Eduardo Apolinario <[email protected]> * Add hash.setter Signed-off-by: Eduardo Apolinario <[email protected]> * Install flyteidl directly Signed-off-by: Eduardo Apolinario <[email protected]> * Revert "Regenerate plugins requirements files pointing to the right version of flyteidl." This reverts commit c2dbb54. Signed-off-by: Eduardo Apolinario <[email protected]> * wip - Add support for univariate lists Signed-off-by: Eduardo Apolinario <[email protected]> * Add support for lists of annotated objects Signed-off-by: Eduardo Apolinario <[email protected]> * Revamp generation of cache key (to cover case of literals collections and maps) Signed-off-by: Eduardo Apolinario <[email protected]> * Leave TODO for warning Signed-off-by: Eduardo Apolinario <[email protected]> * Revert "Add support for lists of annotated objects" This reverts commit 4b5f608. Signed-off-by: Eduardo Apolinario <[email protected]> * Revert "wip - Add support for univariate lists" This reverts commit adaa448. Signed-off-by: Eduardo Apolinario <[email protected]> * Remove docstring Signed-off-by: Eduardo Apolinario <[email protected]> * Add flyteidl>=0.23.0 Signed-off-by: Eduardo Apolinario <[email protected]> * Remove mentions to branch flyteidl@add-hash-to-literal Signed-off-by: Eduardo Apolinario <[email protected]> * Bump flyteidl in plugins requirements Signed-off-by: Eduardo Apolinario <[email protected]> * Regenerate plugins requirements again Signed-off-by: Eduardo Apolinario <[email protected]> * Restore papermill/requirements.txt Signed-off-by: Eduardo Apolinario <[email protected]> * Point flytekitplugins-spark to the offloaded-objects-caching branch in papermill tests Signed-off-by: Eduardo Apolinario <[email protected]> * Set flyteidl>=0.23.0 in papermill dev-requirements Co-authored-by: Eduardo Apolinario <[email protected]> * FlyteRemote fetch of conditional nodes (flyteorg#772) Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: maximsmol <[email protected]> * Removed root logger (flyteorg#871) * Removed root logger Signed-off-by: Kevin Su <[email protected]> * Updated logger Signed-off-by: Kevin Su <[email protected]> * Fixed lint Signed-off-by: Kevin Su <[email protected]> * Updated log level Signed-off-by: Kevin Su <[email protected]> * Updated logger Signed-off-by: Kevin Su <[email protected]> * Updated logger Signed-off-by: Kevin Su <[email protected]> * Updated logger Signed-off-by: Kevin Su <[email protected]> * lint fixed Signed-off-by: Kevin Su <[email protected]> Signed-off-by: maximsmol <[email protected]> * Fix flytekit_compatibility/test_schema_types.py test Signed-off-by: Eduardo Apolinario <[email protected]> Signed-off-by: maximsmol <[email protected]> * make lint Signed-off-by: Eduardo Apolinario <[email protected]> * fix: annotated type conversion error Signed-off-by: maximsmol <[email protected]> * fix: _are_types_castable based on tests Signed-off-by: maximsmol <[email protected]> * fix: test failing if using random order Signed-off-by: maximsmol <[email protected]> * Merge branch 'master' into maximsmol/union_type Signed-off-by: maximsmol <[email protected]> * fix: merge issue Signed-off-by: maximsmol <[email protected]> * fix: requirements Signed-off-by: maximsmol <[email protected]> * fix: schema transformer error Signed-off-by: maximsmol <[email protected]> * fix: test Signed-off-by: maximsmol <[email protected]> * fix: merge issue Signed-off-by: maximsmol <[email protected]> * fix: union + annotated behavior Signed-off-by: maximsmol <[email protected]> * Regenerate requirements Signed-off-by: Eduardo Apolinario <[email protected]> * Bump requirements in plugins Signed-off-by: Eduardo Apolinario <[email protected]> * Handle nested Annotated Signed-off-by: Eduardo Apolinario <[email protected]> * Leave TODO re: strucutured dataset type castability Signed-off-by: Eduardo Apolinario <[email protected]> * Remove mention to flyteidl@union_type in doc-requirements Signed-off-by: Eduardo Apolinario <[email protected]> * Linting Signed-off-by: Eduardo Apolinario <[email protected]> * Use tempfile.mkdtemp to create a temporary directory for local data persistence. Signed-off-by: Eduardo Apolinario <[email protected]> * Revert "Use tempfile.mkdtemp to create a temporary directory for local data persistence." This reverts commit 861b157. Signed-off-by: eduardo apolinario <[email protected]> * Force temporary file to not be deleted in test Signed-off-by: eduardo apolinario <[email protected]> * Regenerate papermill dev-requirements Signed-off-by: eduardo apolinario <[email protected]> * Remove duplicate code Signed-off-by: Eduardo Apolinario <[email protected]> * Put a lower bound on the pip version installed in CI Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Kevin Su <[email protected]> Co-authored-by: Emirhan Karagül <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Co-authored-by: Kevin Su <[email protected]> Co-authored-by: Yee Hing Tong <[email protected]> Co-authored-by: Ketan Umare <[email protected]> Co-authored-by: Lisa <[email protected]> Co-authored-by: Niels Bantilan <[email protected]> Co-authored-by: Stef Nelson-Lindall <[email protected]> Co-authored-by: Snyk bot <[email protected]> Co-authored-by: Dan Rammer <[email protected]> Co-authored-by: bstadlbauer <[email protected]> Co-authored-by: Bernhard Stadlbauer <[email protected]> Co-authored-by: Bernhard Stadlbauer <[email protected]> Co-authored-by: Samhita Alla <[email protected]> Co-authored-by: Yuvraj <[email protected]> Co-authored-by: Zach Palchick <[email protected]> Co-authored-by: Snyk bot <[email protected]> Co-authored-by: Kenny Workman <[email protected]> Co-authored-by: ggydush-fn <[email protected]> Co-authored-by: SmritiSatyanV <[email protected]> Co-authored-by: Matthew Griffin <[email protected]> Co-authored-by: eduardo apolinario <[email protected]> Signed-off-by: Mike Zhong <[email protected]>
- Loading branch information
25 people
authored and
committed
Apr 11, 2022
1 parent
084dbff
commit daf5c86