Adds a new macro to allow getting filter values easily

[mjsilva]

We've added a new macro filter_values to make it easy to grab the values of a filter box when writing dynamic filter dependent queries.

Usage:

SELECT action,
count(*) as times
FROM logs
WHERE action in ( {{ "'" + "','".join(filter_values('action_type')) + "'" }} )
GROUP BY 1

Returns:

SELECT action,
count(*) as times
FROM logs
WHERE action in ( "dashboard", "log" )
GROUP BY 1

When filter box is like this:

Fixes #1996

[codecov-io]

Codecov Report

Merging #5547 into master will increase coverage by 0.13%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #5547      +/-   ##
==========================================
+ Coverage   63.12%   63.26%   +0.13%     
==========================================
  Files         349      351       +2     
  Lines       22167    22273     +106     
  Branches     2462     2470       +8     
==========================================
+ Hits        13992    14090      +98     
- Misses       8161     8168       +7     
- Partials       14       15       +1

Impacted Files	Coverage Δ
superset/jinja_context.py	77.77% <100%> (+4.44%)	⬆️
superset/db_engine_specs.py	54.19% <0%> (-1.21%)	⬇️
...uperset/assets/src/dashboard/components/Header.jsx	58.51% <0%> (-0.8%)	⬇️
superset/views/base.py	66.89% <0%> (-0.68%)	⬇️
superset/models/core.py	86.57% <0%> (-0.48%)	⬇️
superset/utils.py	88.88% <0%> (-0.31%)	⬇️
...src/dashboard/components/HeaderActionsDropdown.jsx	81.63% <0%> (-0.19%)	⬇️
superset/views/core.py	73.98% <0%> (-0.11%)	⬇️
superset/data/__init__.py	100% <0%> (ø)	⬆️
superset/assets/src/explore/controls.jsx	46.96% <0%> (ø)	⬆️
... and 10 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9d95c4c...ff5673b. Read the comment docs.

[mjsilva]

Ready for review. Thanks!

[ksaagariconic]

This is amazing and exactly what we wanted to fix our reports for a 700 people company - thank you so much @mjsilva! It also addresses this issue

[ankoh]

Thanks!
FYI, this is also an effective workaround for issue #5755 .
If I'm not mistaken that's the best way atm to configure SQL Lab queries via the URL parameters of the dashboard.
(I spent only 1 day on this issue so feel free to correct me if I'm mistaken)

Just add the following snippet to your superset_config.py to use Manuels code via the jinja context addons until this is merged.
You can then configure the charts via the preselect_filters.

SQL:

...
WHERE foo = {{ filter_values('bar', 'default_bar')[0] }}
...

URL:

.../superset/dashboard/baz/?standalone=true&preselect_filters={"CHART-ID":{"bar":["30"]}}

superset_config.py:

# -----------------------------------------------------------------------------------
# Read filter values in jinja templates
# -----------------------------------------------------------------------------------
import json
from flask import request

def filter_values(column, default=None):
    """ Gets a values for a particular filter as a list
     This is useful if:
        - you want to use a filter box to filter a query where the name of filter box
          column doesn't match the one in the select statement
        - you want to have the ability for filter inside the main query for speed purposes
     This searches for "filters" and "extra_filters" in form_data for a match
     Usage example:
        * SELECT action, count(*) as times
        FROM logs
        WHERE action in ( {{ "'" + "','".join(filter_values('action_type')) + "'" )
        GROUP BY 1
     :param column: column/filter name to lookup
    :type column: str
    :param default: default value to return if there's no matching columns
    :type default: str
    :return: returns a list of filter values
    :rtype: list
    """
    form_data = json.loads(request.form.get('form_data', '{}'))
    return_val = []
    for filter_type in ['filters', 'extra_filters']:
        if filter_type not in form_data:
            continue
        for f in form_data[filter_type]:
            if f['col'] == column:
                for v in f['val']:
                    return_val.append(v)
    if return_val:
        return return_val
    if default:
        return [default]
    else:
        return []

JINJA_CONTEXT_ADDONS = {
    "filter_values": filter_values
}

[Geetha-LMI]

SELECT action,
count(*) as times
FROM logs
WHERE action in ( {{ "'" + "','".join(filter_values('action_type')) + "'" }} )
GROUP BY 1
It is not giving any result set in sql lab. can you explain where to write

[Geetha-LMI]

@mjsilva how can i pass the since and until dates to the query from dashboard.

[zoltantalas]

filter_values is broken in the newest superset version 0.30.
(using the same sql like in 0.28.1, but we dont have sqllab view checkbox any more)

'filter_values' is undefined
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/superset/viz.py", line 329, in get_df_payload
df = self.get_df(query_obj)
File "/usr/local/lib/python3.5/dist-packages/superset/viz.py", line 142, in get_df
self.results = self.datasource.query(query_obj)
File "/usr/local/lib/python3.5/dist-packages/superset/connectors/sqla/models.py", line 683, in query
sql = self.get_query_str(query_obj)
File "/usr/local/lib/python3.5/dist-packages/superset/connectors/sqla/models.py", line 410, in get_query_str
qry = self.get_sqla_query(**query_obj)
File "/usr/local/lib/python3.5/dist-packages/superset/connectors/sqla/models.py", line 538, in get_sqla_query
tbl = self.get_from_clause(template_processor, db_engine_spec)
File "/usr/local/lib/python3.5/dist-packages/superset/connectors/sqla/models.py", line 434, in get_from_clause
from_sql = template_processor.process_template(from_sql)
File "/usr/local/lib/python3.5/dist-packages/superset/jinja_context.py", line 109, in process_template
return template.render(kwargs)
File "/usr/local/lib/python3.5/dist-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/local/lib/python3.5/dist-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.5/dist-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "", line 32, in top-level template code
File "/usr/local/lib/python3.5/dist-packages/jinja2/sandbox.py", line 425, in call
if not __self.is_safe_callable(__obj):
File "/usr/local/lib/python3.5/dist-packages/jinja2/sandbox.py", line 338, in is_safe_callable
return not (getattr(obj, 'unsafe_callable', False) or
jinja2.exceptions.UndefinedError: 'filter_values' is undefined

[avinasht1]

How do I access the date filter values using this macro? I'm looking to access the "since" and "until" in my date filter.

I tried "__from" and "__to" but to no luck.
{{ "'" + "','".join(filter_values('__from')) + "'" }}

[fishfree]

Is it possible to get the current username as the parameter?

[ksaagariconic]

@fishfree - Yes you can do that like this, thanks! Just an example in both predicate and select clauses.

SELECT '{{current_username()}}' as user
where '{{current_username()}}' = 'someone'

[saurabh1-singh]

This is useful if:
- you want to use a filter box to filter a query where the name of filter box
column doesn't match the one in the select statement
- you want to have the ability for filter inside the main query for speed purposes

This does not work as mentioned in the PR, if the column in filter box does not match the one in the other visualization select statement, it throws error because it tries to wrap the query and apply the filter with the column name of the filter box.

Steps to reproduce:
SELECT col_1 FROM xyz where col_2 >= {{ something }} and col_2 < {{ something else }}

After that I name the column in Filter Box "something" and "something else" but I get error "something" not present in expr_query,
say, I pass value 123 in "something" column and 321 in "something else"

The query generated is somewhat like this
SELECT col_1 FROM xyz where col_2 >= 123 and col_2 < 321
and "something"=123 and "something else" = 321

What do i do to make the last line disappear from the final query output

[dominiczy]

@saurabh1-singh did you ever figure this one out? Having the same problem at our company

[dharamgajera]

How do I access the date filter values using this macro? I'm looking to access the "since" and "until" in my date filter.

I tried "__from" and "__to" but to no luck.
{{ "'" + "','".join(filter_values('__from')) + "'" }}

Did you find any solution for this ?
I'm also looking for same.

[ebuildy]

SELECT action,
count(*) as times
FROM logs
WHERE action in ( {{ "'" + "','".join(filter_values('action_type')) + "'" }} )
GROUP BY 1
It is not giving any result set in sql lab. can you explain where to write

I guess this was a dummy example, in a real life, wrap it between condition:

{% if filter_values('Product type') | length %}
WHERE shop_order_detail.product_type in ( {{ "'" + "','".join(filter_values('Product type')) + "'" }} )
{% endif %}

[qlpanalytics]

@avinasht1 were you ever able to resolve the date filter values asked about here?
#5547 (comment)