[infra] Python 1.6.2 #1109
Merged
[infra] Python 1.6.2 #1109
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* [infra] fix dataset_config.yaml folder path (#1067) * feat(infra) merge master * [infra] conform Metadata to new metadata changes (#1093) * [dados-bot] br_ms_vacinacao_covid19 (2022-01-23) (#1086) Co-authored-by: terminal_name <github_email> * [dados] br_bd_diretorios_brasil.etnia_indigena (#1087) * Sobe diretorio etnia_indigena * Update table_config.yaml * Update table_config.yaml * feat: conform Metadata's schema to new one * fix: conform yaml generation to new schema * fix: delete test_dataset folder Co-authored-by: Lucas Moreira <[email protected]> Co-authored-by: Gustavo Aires Tiago <[email protected]> Co-authored-by: Ricardo Dahis <[email protected]> Co-authored-by: Lucas Moreira <[email protected]> Co-authored-by: Gustavo Aires Tiago <[email protected]>
* feat(infra): option to make dataset public * feat(infra): fix None data * fix(infra): roll back * fix(infra): fix retry in storage upload * fix(infra): add option to dataset data location * feat(infra): make staging dataset not public * feat(infra): make staging dataset not public
…1130) * [infra] add function to wrap bd_dataset_search endpoint * Update download.py * [infra] modify list_datasets function to consume CKAN endpoint * [infra] fix list_dataset function to include limit and remove order_by * [infra] change function list_dataset_tables to use CKAN endpoint * [infra] apply PEP8 to list_dataset_tables and respective tests * add get_dataset_description, get_table_description, get_table_columns * [infra] fix dataset_config.yaml folder path (#1067) * feat(infra) merge master * fix files organization to match master * remove download.py * remove test_download * Delete test_download.py * remove test files * remove test_download.py * remove test_download.py * remove test_download.py * remove test_download.py * add tests metadata * remove test_download.py * remove unused imports * [infra] add _safe_fetch and get_table_size functions Co-authored-by: lucascr91 <[email protected]>
lucascr91
approved these changes
Mar 4, 2022
* adiciona suporte a Avro e Parquet para upload * Adds test for source formats * [infra] update tests for avro, parquet, and csv upload Co-authored-by: Gabriel Gazola Milan <[email protected]> Co-authored-by: Isadora Bugarin <[email protected] > Co-authored-by: lucascr91 <[email protected]>
* Creating dataclass config * Success messages - create and update (table.py) using loguru * feat: improve log level control * refa: move logger config to Base.__init__ * Improving log level control * Adjusting log level control function in base.py * Fixing repeated 'DELETE' messages everytime Table is replaced. * Importing 'dataclass' from 'dataclasses' to make config work. * Fixing repeated 'UPDATE' messages inside other functions. * Defining a new script message format. * Definng standard log messages for 'dataset.py' functions * Definng standard log messages for 'storage.py' functions * Definng standard log messages for 'table.py' functions * Definng standard log messages for 'metadata.py' functions * Adds standard configuration to billing_project_id in download.py * Configuring billing_project_id in download.py * Configuring config_path in base.py Co-authored-by: Guilherme Salustiano <[email protected]> Co-authored-by: Isadora Bugarin <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
TODO
Curent release
pip install basedosdados==1.6.2a11Changelog
dataset_is_public[infra] option to make dataset public #1020*_stagingagora sãoprivados por default[infra] option to make dataset public #1020locationdefine localização da criação dataset [infra] option to make dataset public #1020observation_level,short_description, etc), matamos campos (e.g.entity,time_unit, etc). [infra] conform Metadata to new metadata changes #1093dataset_idnão ser automaticamente criado emdataset_config.yaml. [infra] conform Metadata to new metadata changes #1093columns_config_urlviroucolumns_config_url_or_pathe agora suporta carregamento de tabela de arquitetura local