Merge apache 20200122a

CHANGELOG.md

@@ -18,6 +18,29 @@ under the License.
 -->
 ## Change Log
 
+### 0.35.2 (2020/01/03 16:42 +00:00)
+- [#8918](https://github.com/apache/incubator-superset/pull/8918) [database] [log] Fix, Limit the amount of info on response (#8918) (@dpgaspar)
+- [#8759](https://github.com/apache/incubator-superset/pull/8759) Bump viz plugins for bug bash (#8759) (@etr2460)
+- [#8729](https://github.com/apache/incubator-superset/pull/8729) fix: don't show filter popover on explore view load (#8729) (@mistercrunch)
+- [#8718](https://github.com/apache/incubator-superset/pull/8718) Migrate filter_immune_slice_fields (#8718) (@betodealmeida)
+- [#8714](https://github.com/apache/incubator-superset/pull/8714) [fix] Druid IS NULL/IS NOT NULL filters (#8714) (@john-bodley)
+- [#8678](https://github.com/apache/incubator-superset/pull/8678) fix: default missing values to zero on area chart (#8678) (@villebro)
+- [#8573](https://github.com/apache/incubator-superset/pull/8573) [SECURITY] bump packages with security vulnerabilities (#8573) (@nytai)
+- [#8583](https://github.com/apache/incubator-superset/pull/8583) Bump pyarrow to 0.15.1 due to CVE (#8583) (@robdiciuccio)
+- [#8581](https://github.com/apache/incubator-superset/pull/8581) Fix error when templateParams is undefined (#8581) (@betodealmeida)
+- [#8575](https://github.com/apache/incubator-superset/pull/8575) Math.max(...array) considered harmful (#8575) (@betodealmeida)
+- [#8550](https://github.com/apache/incubator-superset/pull/8550) [Dashboard Import] Log error on dashboard import failure (#8550) (@etr2460)
+- [#8427](https://github.com/apache/incubator-superset/pull/8427) Default page: remove title attribute to fall back on appbuilder.app_name (#8427) (@Zanoellia)
+- [#8495](https://github.com/apache/incubator-superset/pull/8495) [datasource editor] Only one click target for edit action (#8495) (@graceguo-supercat)
+- [#8465](https://github.com/apache/incubator-superset/pull/8465) Update UPDATING.md (#8465) (@john-bodley)
+- [#8471](https://github.com/apache/incubator-superset/pull/8471) [Datasource Editor] A few small UI changes in modal to prevent accidental edits (#8471) (@graceguo-supercat)
+- [#8439](https://github.com/apache/incubator-superset/pull/8439) [setup] Fix, download_url (#8439) (@dpgaspar)
+- [#8449](https://github.com/apache/incubator-superset/pull/8449) Split up tests/db_engine_test.py (#8449) (@willbarrett)
+- [#8438](https://github.com/apache/incubator-superset/pull/8438) introduce a space in command line option (#8438) (@christophlingg)
+- [#8444](https://github.com/apache/incubator-superset/pull/8444) explain the need to enable async queries (#8444) (@christophlingg)
+- [#8442](https://github.com/apache/incubator-superset/pull/8442) spelling fix (#8442) (@austinpray)
+- [#8419](https://github.com/apache/incubator-superset/pull/8419) docs: add a note to RELEASING about Slack Channel (#8419) (@mistercrunch)
+
 ### 0.35.1 (2019/11/15 13:47 +00:00)
 - [#8457](https://github.com/apache/incubator-superset/pull/8457) [fix] Improve csv upload functionality (#8457) (@villebro)
 - [#8566](https://github.com/apache/incubator-superset/pull/8566) [druid] Fix, pydruid forced dependency (#8566) (@dpgaspar)

README.md

@@ -189,9 +189,11 @@ the world know they are using Superset. Join our growing community!
  1. [Now](https://www.now.vn/)
  1. [Ona](https://ona.io)
  1. [PeopleDoc](https://www.people-doc.com)
+ 1. [Popoko VM Games Studio](https://popoko.live)
  1. [Pronto Tools](http://www.prontotools.io)
- 1. [QPID Health](http://www.qpidhealth.com/    )
+ 1. [QPID Health](http://www.qpidhealth.com/)
  1. [Qunar](https://www.qunar.com/)
+ 1. [Reward Gateway](https://www.rewardgateway.com)
  1. [Safaricom](https://www.safaricom.co.ke/)
  1. [Scoot](https://scoot.co/)
  1. [ScopeAI](https://www.getscopeai.com)

UPDATING.md

@@ -22,8 +22,12 @@ This file documents any backwards-incompatible changes in Superset and
 assists people when migrating to a new version.
 
 ## Next
+* [8901](https://github.com/apache/incubator-superset/pull/8901): The datasource's update
+timestamp has been added to the query object's cache key to ensure updates to
+datasources are always reflected in associated query results. As a consequence all
+previously cached results will be invalidated when updating to the next version.
 
-* [8732](https://github.com/apache/incubator-superset/pull/8732): Swagger user interface is now enabled by default. 
+* [8732](https://github.com/apache/incubator-superset/pull/8732): Swagger user interface is now enabled by default.
 A new permission `show on SwaggerView` is created by `superset init` and given to the `Admin` Role. To disable the UI,
 set `FAB_API_SWAGGER_UI = False` on config.
 
@@ -90,17 +94,17 @@ which adds missing non-nullable fields to the `datasources` table. Depending on
 the integrity of the data, manual intervention may be required.
 
 * [5452](https://github.com/apache/incubator-superset/pull/5452): a change
-which adds missing non-nullable fields and uniqueness constraints (which may be 
-case insensitive depending on your database configuration) to the `columns`and 
-`table_columns` tables. Depending on the integrity of the data, manual 
+which adds missing non-nullable fields and uniqueness constraints (which may be
+case insensitive depending on your database configuration) to the `columns`and
+`table_columns` tables. Depending on the integrity of the data, manual
 intervention may be required.
 * `fabmanager` command line is deprecated since Flask-AppBuilder 2.0.0, use
 the new `flask fab <command>` integrated with *Flask cli*.
 * `SUPERSET_UPDATE_PERMS` environment variable was replaced by
 `FAB_UPDATE_PERMS` config boolean key. To disable automatic
 creation of permissions set `FAB_UPDATE_PERMS = False` on config.
 * [5453](https://github.com/apache/incubator-superset/pull/5453): a change
-which adds missing non-nullable fields and uniqueness constraints (which may be 
+which adds missing non-nullable fields and uniqueness constraints (which may be
 case insensitive depending on your database configuration) to the metrics
 and sql_metrics tables. Depending on the integrity of the data, manual
 intervention may be required.

docker-compose.yml

@@ -19,7 +19,7 @@ x-superset-build: &superset-build
   dockerfile: Dockerfile
   target: dev
 x-superset-depends-on: &superset-depends-on
-  - postgres
+  - db
   - redis
 x-superset-volumes: &superset-volumes
   # /app/pythonpath_docker will be appended to the PYTHONPATH in the final container
@@ -38,14 +38,14 @@ services:
     volumes:
       - redis:/data
 
-  postgres:
+  db:
     env_file: docker/.env
     image: postgres:10
     restart: unless-stopped
     ports:
       - "127.0.0.1:5432:5432"
     volumes:
-      - postgres:/var/lib/postgresql/data
+      - db_home:/var/lib/postgresql/data
 
   superset:
     build: *superset-build
@@ -82,7 +82,7 @@ services:
 volumes:
   superset_home:
     external: false
-  postgres:
+  db_home:
     external: false
   redis:
     external: false

docker/.env

@@ -15,16 +15,30 @@
 # limitations under the License.
 #
 COMPOSE_PROJECT_NAME=superset
+
+# database configurations (do not modify)
+DATABASE_DB=superset
+DATABASE_HOST=db
+DATABASE_PASSWORD=superset
+DATABASE_USER=superset
+
+# database engine specific environment variables
+# change the below if you prefers another database engine
+DATABASE_PORT=5432
+DATABASE_DIALECT=postgresql
 POSTGRES_DB=superset
-POSTGRES_HOST=postgres
-POSTGRES_PASSWORD=superset
-POSTGRES_PORT=5432
 POSTGRES_USER=superset
+POSTGRES_PASSWORD=superset
+#MYSQL_DATABASE=superset
+#MYSQL_USER=superset
+#MYSQL_PASSWORD=superset
+#MYSQL_RANDOM_ROOT_PASSWORD=yes
+
 # Add the mapped in /app/pythonpath_docker which allows devs to override stuff
 PYTHONPATH=/app/pythonpath:/app/pythonpath_docker
 REDIS_HOST=redis
 REDIS_PORT=6379
 
 FLASK_ENV=development
 SUPERSET_ENV=development
-SUPERSET_LOAD_EXAMPLES=yes
+SUPERSET_LOAD_EXAMPLES=yes

docker/docker-init.sh

@@ -34,7 +34,7 @@ EOF
 }
 
 # Create an admin user
-echo_step "1" "Starting" "Setting up admin user"
+echo_step "1" "Starting" "Setting up admin user ( admin / admin )"
 superset fab create-admin \
               --username admin \
               --firstname Superset \

docker/pythonpath_dev/superset_config.py

@@ -45,19 +45,21 @@ def get_env_variable(var_name, default=None):
             raise EnvironmentError(error_msg)
 
 
-POSTGRES_USER = get_env_variable("POSTGRES_USER")
-POSTGRES_PASSWORD = get_env_variable("POSTGRES_PASSWORD")
-POSTGRES_HOST = get_env_variable("POSTGRES_HOST")
-POSTGRES_PORT = get_env_variable("POSTGRES_PORT")
-POSTGRES_DB = get_env_variable("POSTGRES_DB")
+DATABASE_DIALECT = get_env_variable("DATABASE_DIALECT")
+DATABASE_USER = get_env_variable("DATABASE_USER")
+DATABASE_PASSWORD = get_env_variable("DATABASE_PASSWORD")
+DATABASE_HOST = get_env_variable("DATABASE_HOST")
+DATABASE_PORT = get_env_variable("DATABASE_PORT")
+DATABASE_DB = get_env_variable("DATABASE_DB")
 
 # The SQLAlchemy connection string.
-SQLALCHEMY_DATABASE_URI = "postgresql://%s:%s@%s:%s/%s" % (
-    POSTGRES_USER,
-    POSTGRES_PASSWORD,
-    POSTGRES_HOST,
-    POSTGRES_PORT,
-    POSTGRES_DB,
+SQLALCHEMY_DATABASE_URI = "%s://%s:%s@%s:%s/%s" % (
+    DATABASE_DIALECT,
+    DATABASE_USER,
+    DATABASE_PASSWORD,
+    DATABASE_HOST,
+    DATABASE_PORT,
+    DATABASE_DB,
 )
 
 REDIS_HOST = get_env_variable("REDIS_HOST")

docs/index.rst

@@ -109,6 +109,7 @@ The following RDBMS are currently supported:
 - `Apache Spark SQL <https://spark.apache.org/sql/>`_
 - `BigQuery <https://cloud.google.com/bigquery/>`_
 - `ClickHouse <https://clickhouse.yandex/>`_
+- `Dremio <https://dremio.com/>`_
 - `Elasticsearch <https://www.elastic.co/products/elasticsearch/>`_
 - `Exasol <https://www.exasol.com/>`_
 - `Google Sheets <https://www.google.com/sheets/about/>`_

docs/installation.rst

@@ -372,6 +372,8 @@ Here's a list of some of the recommended packages.
 +------------------+---------------------------------------+-------------------------------------------------+
 | ClickHouse       | ``pip install sqlalchemy-clickhouse`` |                                                 |
 +------------------+---------------------------------------+-------------------------------------------------+
+| Dremio           | ``pip install sqlalchemy_dremio``     | ``dremio://user:pwd@host:31010/``               |
++------------------+---------------------------------------+-------------------------------------------------+
 | Elasticsearch    | ``pip install elasticsearch-dbapi``   | ``elasticsearch+http://``                       |
 +------------------+---------------------------------------+-------------------------------------------------+
 | Exasol           | ``pip install sqlalchemy-exasol``     | ``exa+pyodbc://``                               |
@@ -400,7 +402,8 @@ Here's a list of some of the recommended packages.
 |                  | sqlalchemy-vertica-python``           |                                                 |
 +------------------+---------------------------------------+-------------------------------------------------+
 | Hana             | ``pip install hdbcli sqlalchemy-hana``|  ``hana://``                                    |
-|                  | or ``pip install superset[hana]``     |                                                 |
+|                  | or                                    |                                                 |
+|                  | ``pip install apache-superset[hana]`` |                                                 |
 +------------------+---------------------------------------+-------------------------------------------------+
 
 
@@ -724,6 +727,15 @@ Druid
 Note that you can run the ``superset refresh_druid`` command to refresh the
 metadata from your Druid cluster(s)
 
+Dremio
+------
+
+Install the following dependencies to connect to Dremio:
+
+* Dremio SQLAlchemy: ``pip install sqlalchemy_dremio``
+* Dremio's ODBC driver: https://www.dremio.com/drivers/
+
+Example SQLAlchemy URI: ``dremio://dremio:dremio123@localhost:31010/dremio``
 
 Presto
 ------
@@ -761,8 +773,9 @@ CORS
 
 The extra CORS Dependency must be installed:
 
-    superset[cors]
+.. code-block:: text
 
+    pip install apache-superset[cors]
 
 The following keys in `superset_config.py` can be specified to configure CORS:
 

requirements.txt

@@ -53,7 +53,7 @@ marshmallow-sqlalchemy==0.17.0  # via flask-appbuilder
 marshmallow==2.19.5       # via flask-appbuilder, marshmallow-enum, marshmallow-sqlalchemy
 msgpack==0.6.1
 numpy==1.17.0             # via pandas, pyarrow
-pandas==0.24.2
+pandas==0.25.3
 parsedatetime==2.4
 pathlib2==2.3.4
 polyline==1.4.0

setup.cfg

@@ -45,7 +45,7 @@ combine_as_imports = true
 include_trailing_comma = true
 line_length = 88
 known_first_party = superset
-known_third_party =alembic,backoff,bleach,celery,click,colorama,contextlib2,croniter,dateutil,flask,flask_appbuilder,flask_babel,flask_caching,flask_compress,flask_login,flask_migrate,flask_sqlalchemy,flask_talisman,flask_testing,flask_wtf,geohash,geopy,humanize,isodate,jinja2,markdown,marshmallow,msgpack,numpy,pandas,parsedatetime,pathlib2,polyline,prison,psycopg2,pyarrow,pyhive,pytz,retry,selenium,setuptools,simplejson,sphinx_rtd_theme,sqlalchemy,sqlalchemy_utils,sqlparse,werkzeug,wtforms,wtforms_json,yaml
+known_third_party =alembic,backoff,bleach,celery,click,colorama,contextlib2,croniter,dateutil,flask,flask_appbuilder,flask_babel,flask_caching,flask_compress,flask_login,flask_migrate,flask_sqlalchemy,flask_talisman,flask_testing,flask_wtf,geohash,geopy,humanize,isodate,jinja2,markdown,markupsafe,marshmallow,msgpack,numpy,pandas,parsedatetime,pathlib2,polyline,prison,pyarrow,pyhive,pytz,retry,selenium,setuptools,simplejson,sphinx_rtd_theme,sqlalchemy,sqlalchemy_utils,sqlparse,werkzeug,wtforms,wtforms_json,yaml
 multi_line_output = 3
 order_by_type = false
 

setup.py

@@ -90,7 +90,7 @@ def get_git_sha():
         "isodate",
         "markdown>=3.0",
         "msgpack>=0.6.1, <0.7.0",
-        "pandas>=0.24.2, <0.25.0",
+        "pandas>=0.25.3, <1.0",
         "parsedatetime",
         "pathlib2",
         "polyline",
@@ -118,6 +118,7 @@ def get_git_sha():
         "elasticsearch": ["elasticsearch-dbapi>=0.1.0, <0.2.0"],
         "druid": ["pydruid==0.5.7", "requests==2.22.0"],
         "hana": ["hdbcli==2.4.162", "sqlalchemy_hana==0.4.0"],
+        "dremio": ["sqlalchemy_dremio>=0.5.0dev0"],
     },
     python_requires="~=3.6",
     author="Apache Software Foundation",