An error occurred when trying to visualize a group of 4-million-row data

[lanyusea]

I got a group of data about 4million stored in mysql, I'm trying to load it in superset and do the virtualization.

It works well when there is only thousands data but after I put all data in the mysql, the superset failed to do the job with error An error occurred in the chart page but no further information, nor in the log.

Expected results

Superset can fetch the data though it reports problems

Actual results

1. It says An error occurred with no result.
2. a large temp db is created in my mysql

Screenshots

and the log file has nothing useful:

How to reproduce the bug

1. Add 4million rows of data
2. do the virtualization

Environment

(please complete the following information):

• superset version: 0.37.2
• python version: 3.6.9
• node.js version: didn't install

superset is installed from Scratch, not docker

Checklist

Make sure to follow these steps before submitting your issue - thank you!

• I have checked the superset logs for python stacktraces and included it here as text if there are any.
• I have reproduced the issue with at least the latest released version of superset.
• I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context

I have searched and find few discussion about the data scale. the official document only mentions issue about the loading speed while in #4588 people also talks about the speed. Seems nobody meets a fail issue as me.

also, to make it runable, I have extended the timeout to 3000s and several other limit settings.

The problem always happen after querying about 600s, so I'm thinking if there is still some timeout in the connection but I didn't find any other in the superset config.py.

So I'm wondering how could I know what the exact error is, and how to I solve it.

or is there any suggestion I can bypass this issue.

Thanks!

[lanyusea]

I just found the superset will resend the query request if the result doesn't go back in around 120s. So if the mysql failed to finish the query in 120s, it will run into a terrible condition until all resources used up.

anyone know how could I modify this retry behavior?
didn't find any related settings in the config file.

[robdiciuccio]

@lanyusea can you elaborate on the MySQL retry logic you mentioned?

I see that you increased the SUPERSET_WEBSERVER_TIMEOUT value in superset_config.py. How are you running Superset? Gunicorn? Do you have nginx or another web server proxying requests to the app? These each have their own timeout settings as well.

[lanyusea]

@robdiciuccio by gunicorn, I managed to increase the timeout in gunicorn thread setting, and it works.

but it still cannot handle a long connection when the query costs more than 20 minutes. currently I bypass this issue by speeding up the query.