Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Stan + jmpost examples #251

Merged
merged 2 commits into from
Jan 31, 2024
Merged

Add Stan + jmpost examples #251

merged 2 commits into from
Jan 31, 2024

Conversation

gowerc
Copy link
Collaborator

@gowerc gowerc commented Jan 30, 2024

Closes #250

Attempts to address @mercifr1's request to have a working example showing how jmpost code compares to raw Stan code. Note that jmpost has significantly worse performance due to the fact that we calculate the survival function by approximating the integral of the hazard function (as opposed to the raw Stan code here that just uses the closed form of the survival function as this is an OS only model.)

Copy link
Collaborator

@danielinteractive danielinteractive left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @gowerc , looks good, however I wonder if we could directly have this as a vignette in the package itself? As the design folder won't be visible for users, and is just for us as developers?

@gowerc
Copy link
Collaborator Author

gowerc commented Jan 30, 2024

Well thats annoying, despite this PR having 0 code changes on the packages code base the automatic roxygen pipeline has decided to re-write all of the help files due to an updated version of the roxygen package 😢

@gowerc
Copy link
Collaborator Author

gowerc commented Jan 30, 2024

@danielinteractive - At the moment I was thinking this would just be for us (tbh I'm only including it here as an easy way of sharing with @mercifr1 as opposed to sending an email). I didn't feel vignettes for Stan tutorials was particularly useful for our packages end users as my thinking was that our main purpose was to be to abstract away the raw Stan code?

Copy link
Contributor

badge

Code Coverage Summary

Filename                       Stmts    Miss  Cover    Missing
---------------------------  -------  ------  -------  ---------------------------
R/brier_score.R                  166       0  100.00%
R/DataJoint.R                     76       2  97.37%   264, 270
R/DataLongitudinal.R             119       1  99.16%   245
R/DataSubject.R                   69       1  98.55%   124
R/DataSurvival.R                  74       1  98.65%   146
R/defaults.R                      13       7  46.15%   27, 45-84, 111
R/generics.R                      16       1  93.75%   69
R/JointModel.R                    91       8  91.21%   113-125, 204, 210, 216
R/JointModelSamples.R             54       0  100.00%
R/Link.R                          18       0  100.00%
R/LinkGSF.R                       69      13  81.16%   116-128
R/LinkNone.R                       5       1  80.00%   36
R/LinkRandomSlope.R               11       0  100.00%
R/LongitudinalGSF.R               24       0  100.00%
R/LongitudinalModel.R             17       0  100.00%
R/LongitudinalQuantities.R        85       8  90.59%   100-107
R/LongitudinalRandomSlope.R       22       0  100.00%
R/Parameter.R                     14       0  100.00%
R/ParameterList.R                 37       1  97.30%   175
R/Prior.R                        117       0  100.00%
R/Quantities.R                   105       0  100.00%
R/simulations_gsf.R               43       0  100.00%
R/simulations_os.R                11       5  54.55%   35-39
R/simulations_rs.R                21       0  100.00%
R/simulations.R                   99       1  98.99%   122
R/StanModel.R                     14       0  100.00%
R/StanModule.R                   175       6  96.57%   197-198, 240, 251, 386, 414
R/SurvivalExponential.R           10       0  100.00%
R/SurvivalLoglogistic.R           11       0  100.00%
R/SurvivalModel.R                 19       0  100.00%
R/SurvivalQuantities.R           151       6  96.03%   173-178
R/SurvivalWeibullPH.R             11       0  100.00%
R/utilities.R                    145       1  99.31%   13
R/zzz.R                           11      11  0.00%    4-29
TOTAL                           1923      74  96.15%

Diff against main

Filename      Stmts    Miss  Cover
----------  -------  ------  --------
TOTAL             0       0  +100.00%

Results for commit: 82ba6ce

Minimum allowed coverage is 80%

♻️ This comment has been updated with latest results

Copy link
Contributor

Unit Test Performance Difference

Test Suite $Status$ Time on main $±Time$ $±Tests$ $±Skipped$ $±Failures$ $±Errors$
JointModelSamples 💔 $5.47$ $+25.23$ $0$ $0$ $0$ $0$
LongitudinalGSF 💔 $37.86$ $+21.35$ $0$ $0$ $0$ $0$
LongitudinalQuantiles 💔 $13.94$ $+28.01$ $0$ $0$ $0$ $0$
SurvivalExponential 💔 $6.29$ $+22.43$ $0$ $0$ $0$ $0$
brierScore 💔 $5.16$ $+23.43$ $0$ $0$ $0$ $0$
extract_quantities 💔 $0.36$ $+15.00$ $0$ $0$ $0$ $0$
misc_models 💔 $0.83$ $+21.56$ $0$ $0$ $0$ $0$
model_random_slope_2chain 💔 $9.06$ $+26.67$ $0$ $0$ $0$ $0$
stan_functions 💔 $1.27$ $+45.67$ $0$ $0$ $0$ $0$
Additional test case details
Test Suite $Status$ Time on main $±Time$ Test Case
JointModelSamples 💔 $5.47$ $+25.23$ smoke_test_for_JointModelSamples
LongitudinalGSF 💔 $37.61$ $+21.36$ LongitudinalGSF_works_as_expected_with_a_single_study
LongitudinalQuantiles 💔 $8.15$ $+28.15$ Test_that_LongitudinalQuantities_works_as_expected
SurvivalExponential 💔 $2.84$ $+22.43$ SurvivalExponential_can_recover_true_parameter_no_covariates_
brierScore 💔 $5.02$ $+23.44$ brierScore_SurvivalQuantities_returns_same_results_as_survreg
extract_quantities 💔 $0.36$ $+15.00$ extract_quantities_works_as_expected
misc_models 💔 $0.83$ $+21.56$ Longitudinal_Model_doesn_t_print_sampler_rejection_messages
model_random_slope_2chain 💔 $9.06$ $+26.67$ Can_recover_known_distribution_parameters_from_random_slope_model_when_using_multiple_chains
stan_functions 💔 $0.46$ $+15.73$ GSF_Identity_Link_Function_works_as_expected
stan_functions 💔 $0.38$ $+14.86$ GSF_SLD_function_works_as_expected
stan_functions 💔 $0.44$ $+15.08$ Normal_Log_Density_functions_work_as_expected

Results for commit 422156f

♻️ This comment has been updated with latest results.

@danielinteractive
Copy link
Collaborator

I think it could be helpful for Stan users to see the analogies. But ok can keep in design for now

@gowerc gowerc merged commit 61b431b into main Jan 31, 2024
@gowerc gowerc deleted the stan-examples branch January 31, 2024 14:26
Copy link
Collaborator

@mercifr1 mercifr1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much!


stan_data <- list(
n = nrow(dat),
x = rnorm(50, 5, 2),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together.
Sorry if I am not using the 'review' feature correcting here - I am using it for the 1st time
May I ask what is 'x' and why is it a vector of length 50?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question....

I think this was just some old code that I forgot to delete, as far as I can see x isn't used anywhere in the stan program so it is contributing nothing.

Will clean up in a future update as this PR has already been merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add examples comparing the UI/UX of JMpost to Stan
3 participants