Skip to content

Commit

Permalink
feat(sqla): add time grain and time column to jinja params (apache#16680
Browse files Browse the repository at this point in the history
)
  • Loading branch information
villebro authored Sep 13, 2021
1 parent 301a7d1 commit ca7f456
Show file tree
Hide file tree
Showing 2 changed files with 36 additions and 16 deletions.
38 changes: 28 additions & 10 deletions docs/src/pages/docs/installation/sql_templating.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,33 @@ version: 1

### Jinja Templates

SQL Lab supports [Jinja templating](https://jinja.palletsprojects.com/en/2.11.x/) in queries. You'll
need to to overload the default Jinja context in your environment by defining the
JINJA_CONTEXT_ADDONS in your superset configuration (`superset_config.py`). Objects referenced in
this dictionary are made available for users to use in their SQL code.
SQL Lab and Explore supports [Jinja templating](https://jinja.palletsprojects.com/en/2.11.x/) in queries.
To enable templating, the `ENABLE_TEMPLATE_PROCESSING` feature flag needs to be enabled in
`superset_config.py`. When templating is enabled, python code can be embedded in virtual datasets and
in Custom SQL in the filter and metric controls in Explore. By default, the following variables are
made available in the Jinja context:

- `columns`: columns available in the dataset
- `filter`: filters applied in the query
- `from_dttm`: start `datetime` value from the selected time range (`None` if undefined)
- `to_dttm`: end `datetime` value from the selected time range (`None` if undefined)
- `groupby`: columns which to group by in the query
- `metrics`: aggregate expressions in the query
- `row_limit`: row limit of the query
- `row_offset`: row offset of the query
- `time_column`: temporal column of the query (`None` if undefined)
- `time_grain`: selected time grain (`None` if undefined)

For example, to add a time range to a virtual dataset, you can write the following:

```sql
SELECT * from tbl where dttm_col > '{{ from_dttm }}' and dttm_col < '{{ to_dttm }}'
```

To add custom functionality to the Jinja context, you need to to to overload the default Jinja
context in your environment by defining the `JINJA_CONTEXT_ADDONS` in your superset configuration
(`superset_config.py`). Objects referenced in this dictionary are made available for users to use
where the Jinja context is made available.

```python
JINJA_CONTEXT_ADDONS = {
Expand Down Expand Up @@ -174,7 +197,7 @@ You can retrieve the value for a specific filter as a list using `{{ filter_valu

This is useful if:
- you want to use a filter component to filter a query where the name of filter component column doesn't match the one in the select statement
- you want to have the ability for filter inside the main query for speed purposes
- you want to have the ability for filter inside the main query for performance purposes

Here's a concrete example:

Expand All @@ -186,11 +209,6 @@ WHERE
GROUP BY action
```

You can use thisfeature to reference the start & end datetimes from a time filter using:

- `{{ from_dttm }}`: start datetime value
- `{{ to_dttm }}`: end datetime value

**Filters for a Specific Column**

The `{{ get_filters() }}` macro returns the filters applied to a given column. In addition to
Expand Down
14 changes: 8 additions & 6 deletions superset/connectors/sqla/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -953,12 +953,20 @@ def get_sqla_query( # pylint: disable=too-many-arguments,too-many-locals,too-ma
apply_fetch_values_predicate: bool = False,
) -> SqlaQuery:
"""Querying any sqla table from this common interface"""
if granularity not in self.dttm_cols and granularity is not None:
granularity = self.main_dttm_col

extras = extras or {}
time_grain = extras.get("time_grain_sqla")

template_kwargs = {
"from_dttm": from_dttm.isoformat() if from_dttm else None,
"groupby": groupby,
"metrics": metrics,
"row_limit": row_limit,
"row_offset": row_offset,
"time_column": granularity,
"time_grain": time_grain,
"to_dttm": to_dttm.isoformat() if to_dttm else None,
"filter": filter,
"columns": [col.column_name for col in self.columns],
Expand All @@ -972,14 +980,9 @@ def get_sqla_query( # pylint: disable=too-many-arguments,too-many-locals,too-ma
db_engine_spec = self.db_engine_spec
prequeries: List[str] = []
orderby = orderby or []
extras = extras or {}
need_groupby = bool(metrics is not None or groupby)
metrics = metrics or []

# For backward compatibility
if granularity not in self.dttm_cols and granularity is not None:
granularity = self.main_dttm_col

# Database spec supports join-free timeslot grouping
time_groupby_inline = db_engine_spec.time_groupby_inline

Expand Down Expand Up @@ -1058,7 +1061,6 @@ def get_sqla_query( # pylint: disable=too-many-arguments,too-many-locals,too-ma
# filter out the pseudo column __timestamp from columns
columns = columns or []
columns = [col for col in columns if col != utils.DTTM_ALIAS]
time_grain = extras.get("time_grain_sqla")
dttm_col = columns_by_name.get(granularity) if granularity else None

if need_groupby:
Expand Down

0 comments on commit ca7f456

Please sign in to comment.