Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Harmonize prior and posterior predictive plots in MMM module #1170

Merged
merged 22 commits into from
Nov 7, 2024

Conversation

juanitorduz
Copy link
Collaborator

@juanitorduz juanitorduz commented Nov 5, 2024

Closes #1144

I simply extend the posterior predictive functionality to the prior predictive case without copy-paste but rather abstracting the plotting function by "group".


📚 Documentation preview 📚: https://pymc-marketing--1170.org.readthedocs.build/en/1170/

@juanitorduz juanitorduz self-assigned this Nov 5, 2024
@juanitorduz juanitorduz added the MMM label Nov 5, 2024
@juanitorduz juanitorduz marked this pull request as draft November 5, 2024 13:32
@github-actions github-actions bot added enhancement New feature or request good first issue Good for newcomers . Doesn't require extensive knowledge of the repo and package plots labels Nov 5, 2024
@github-actions github-actions bot added the tests label Nov 5, 2024
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@github-actions github-actions bot added the docs Improvements or additions to documentation label Nov 5, 2024
@juanitorduz juanitorduz marked this pull request as ready for review November 5, 2024 14:13
Copy link

codecov bot commented Nov 5, 2024

Codecov Report

Attention: Patch coverage is 95.55556% with 2 lines in your changes missing coverage. Please review.

Project coverage is 95.59%. Comparing base (59a35cb) to head (0c6139a).
Report is 124 commits behind head on main.

Files with missing lines Patch % Lines
pymc_marketing/mmm/base.py 95.55% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1170      +/-   ##
==========================================
- Coverage   95.61%   95.59%   -0.03%     
==========================================
  Files          39       39              
  Lines        4063     4059       -4     
==========================================
- Hits         3885     3880       -5     
- Misses        178      179       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@juanitorduz
Copy link
Collaborator Author

I'll revert the error message change that is making a test fail.

@juanitorduz
Copy link
Collaborator Author

juanitorduz commented Nov 6, 2024

Can the end-to-end MM example also be updated to use this when conducting the prior predictive check?

Added in 53acf08 @cluhmann

@cluhmann
Copy link
Contributor

cluhmann commented Nov 6, 2024

Can I ask why there is an "observed" series plotted in the sample_prior_predictive() plot and why it's approximately zero (this is in the end to end MMM notebook)?

https://pymc-marketing--1170.org.readthedocs.build/en/1170/_images/2f0680c00d41aaca16ccd466cb2d48ba317aeef9f046c9da07ecaa81bfe225aa.png

@cluhmann
Copy link
Contributor

cluhmann commented Nov 6, 2024

(this is in the end to end MMM notebook)?

There is also an "observed" series in the MMM example notebook prior predictive plot.

@juanitorduz
Copy link
Collaborator Author

Can I ask why there is an "observed" series plotted in the sample_prior_predictive() plot and why it's approximately zero (this is in the end to end MMM notebook)?

https://pymc-marketing--1170.org.readthedocs.build/en/1170/_images/2f0680c00d41aaca16ccd466cb2d48ba317aeef9f046c9da07ecaa81bfe225aa.png

@cluhmann this is the observed series we (I?) usually compare with when I am doing the prior-predictive checks.

@cluhmann
Copy link
Contributor

cluhmann commented Nov 6, 2024

Ah. So the observed series looks like it's zero because it's variability is an order of magnitude smaller than the prior predictive plot?

@juanitorduz
Copy link
Collaborator Author

Exactly! I'll probably make the prior stricter.

@cluhmann
Copy link
Contributor

cluhmann commented Nov 6, 2024

I should have realized that. I need more coffee.

@juanitorduz
Copy link
Collaborator Author

@cluhmann can we merge this one so that I can tackle #1176 (git with. notebooks is a pain ;) )

@cluhmann
Copy link
Contributor

cluhmann commented Nov 6, 2024

I am rushing to get my pymc-marketing talk done (which is why I am running into these little documentation issues), so I can't give this a thorough review right now. But I probably could tomorrow (or someone like @wd60622 could). Sorry 😬

@juanitorduz
Copy link
Collaborator Author

all good! I will start solving in another branch the other issues (good luck in the talk ;) and thanks for all the feedback)

@wd60622
Copy link
Contributor

wd60622 commented Nov 6, 2024

I will give review today when I can

@@ -45,7 +45,7 @@
},
Copy link
Contributor

@wd60622 wd60622 Nov 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we change the verbiage here as well. The use of "reasonable" is a bit strange

Also, would we want to comment on the fact that the prior predictive includes negative values... That might element to highlight


Reply via ReviewNB

Copy link

review-notebook-app bot commented Nov 6, 2024

View / edit / reply to this conversation on ReviewNB

wd60622 commented on 2024-11-06T20:35:15Z
----------------------------------------------------------------

Can we switch this out with Prior class. Also, why the change in prior for the other notebook and not here


Copy link

review-notebook-app bot commented Nov 6, 2024

View / edit / reply to this conversation on ReviewNB

wd60622 commented on 2024-11-06T20:35:16Z
----------------------------------------------------------------

Same idea here for conclusion. At least highlight that idea of negative density


Copy link
Contributor

@wd60622 wd60622 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some commentary on the verbiage around the stuff mainly. Do you think the Likelihood distribution is something that we would like to change? Maybe first here in the notebook then in the defaults

WDYT?

@juanitorduz
Copy link
Collaborator Author

Just some commentary on the verbiage around the stuff mainly. Do you think the Likelihood distribution is something that we would like to change? Maybe first here in the notebook then in the defaults

WDYT?

Yes, let's look this into next iterations.

@juanitorduz juanitorduz requested a review from wd60622 November 6, 2024 21:30
@juanitorduz
Copy link
Collaborator Author

@wd60622 I addressed your comments :)

@wd60622
Copy link
Contributor

wd60622 commented Nov 6, 2024

@wd60622 I addressed your comments :)

Let's ship it!

@wd60622
Copy link
Contributor

wd60622 commented Nov 7, 2024

Failing because of this: #1148

@juanitorduz juanitorduz merged commit 5c08f7f into main Nov 7, 2024
11 of 13 checks passed
@juanitorduz juanitorduz deleted the unify-prior-posterior-plots-mmm branch November 7, 2024 08:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Improvements or additions to documentation enhancement New feature or request good first issue Good for newcomers . Doesn't require extensive knowledge of the repo and package MMM plots tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

plot_prior_predictive to accept original_scale parameter
3 participants