Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds index definitions to the obs_raw table #24

Merged
merged 1 commit into from
May 31, 2019
Merged

Conversation

jameshiebert
Copy link
Contributor

obs_raw is generally a big table. Usually on the order of hundreds of millions of rows. Searches on this table require a little assistance so they don't end up doing a sequential scan on the whole table (and take minutes to complete).

When we build the BC Provincial Climate Data Set using raw SQL, we put a set of indexes on every column of the table. Those definitions didn't get ported to pycds when we wrote it, so here they are now. Could someone (@rod-glover or @corviday ) take a look at this and see if it makes sense?

@jameshiebert
Copy link
Contributor Author

PS This came up, because we have now created a couple new databases from pycds and they didn't automatically get the indexes (and are now slow as can be).

Copy link
Contributor

@rod-glover rod-glover left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apart from a minor question, LGTM!

@@ -185,6 +185,13 @@ class Obs(Base):
name='time_place_variable_unique'),
)

# Indexes
mod_time_idx = Index('mod_time_idx', 'mod_time')
obs_raw_comp_idx = Index('obs_raw_comp_idx', 'obs_time', 'vars_id',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe use a more explanatory name, say 'obs_raw_time_vars_history_idx'?

@jameshiebert jameshiebert merged commit c5dd8af into master May 31, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants