-
Notifications
You must be signed in to change notification settings - Fork 18
Standard Errors for Performance and Risk
The goal of this project will be to enhance to quality of performance and risk analyses in PerformanceAnalytics (PA) and PortfolioAnalytics by adding both specific and generalized wrappers to provide multiple estimators of the standard error of risk and performance measures (metrics) contained in the target packages.
Analytics functions in PerformanceAnalytics and PortfolioAnalytics are written to provide estimates of several hundred different performance and risk metrics.
For most of these metric estimators, there is no measure of the standard error(SE) in the estimate.
Except for very simple estimators such as the sample mean and sample volatility and (typically) assuming normality of returns, in many cases no exact formulas exist. See for example:
[http://web.eecs.umich.edu/~fessler/papers/lists/files/tr/stderr.pdf] (http://web.eecs.umich.edu/~fessler/papers/lists/files/tr/stderr.pdf)
Most risk and performance measure estimators (rsk^hat) are nonlinear functions of returns and the returns are non-normally distributed. In this case a common way of obtaining finite sample approximate standard errors under for the case of i.i.d. returns is to derive an asymptotic variance formula var(rsk^hat) and corresponding asymptotic standard deviation SD(rsk^hat) = sqrt(var(rsk^hat)) and compute the finite sample standard error as
sd_n(rsk^hat) = sd(rsk^hat)/sqrt(sum(!is.na(x)))
Complete treatment of how the above can be done for the case of expected shortfall (ES), including verification of good finite-sample size standard error accuracy, can be found in Martin and Zhang (2016)
An alternative to using asymptotic variance formulas under the i.i.d. assumption for returns is to use bootstrap methods. These are often a useful complement to to the asymptotic variance approach
It is important to keep in mind that returns sometimes have serial correlation that renders the above finite sample standard error approaches invalid. What is needed is good approximations of standard errors of risk and performance measure estimators when there is serial correlation as well as non-normality. The econometrics literature discusses some methods for dealing with this, the best-known of which is the Newey-West method.
David Ardia and co-authors have been working on a package for finite sample standard error estimates in time series measures which they refer to as "numerical standard errors" (nse's) including the presence of serial correlation; see package here:
https://r-forge.r-project.org/projects/nse/
and an extensive simulation study in the working paper here:
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2741587
The primary goals for this project should be to: (1) utilize the 'nse' functionality where possible, and (2) extend it by adding asymptotic variance based finite sample standard errors for both i.i.d. and serially correlated returns where possible, e.g., for value-at-risk, expected shortfall, Sharpe ration, Sortino ration, STARR ratio, etc.
Also, it would be value to extend the the Ardia et al. discussion of which nse methods work best to risk and performance standard error estimators provided in (2).
The general plan should be to provide a standard error wrapper for PerformanceAnalytics metrics that will cover multiple methodologies:
- sqrt(n) and bootstrap method for i.i.d. non-normal returns
- sqrt(n) and bootstrap method for serially correlated non-normal returns
- evaluate different assumptions for serial correlation (both from sample autocorrelation and from autocorrelation in resampled inputs)
- utilizing different estimates of error (e.g. Student's-t or skewed Student's-t)
We should also plan to provide visualization for these metrics, where possible, using plot.xts or appropriate base graphics.
In addition to wrappers for PerformanceAnalytics, generic functions not dependent on the metrics in PerformanceAnalytics should be developed as extensions to the 'nse' package.
The student in this project will develop a list of specific functions identified from this outline in the packages discussed above that will be modified/re-written to integrate new functionality. New code will be consistent and backwards compatible with interfaces of other functions in its respective package.
In addition, the student will review and extend as necessary the complete documentation for each function.
Students should plan to create a vignette that goes beyond the functional documentation to show how the functions might be used, including some extended examples. Building interactive examples using Shiny would also be very interesting.
- ReturnAnalytics on R-Forge
- xts on Github
- quantmod on Github
- nse
- Nonparametric Versus Parametric Expected Shortfall. 2016. Martin and Zhang
Applicants should have:
- Prior experience using or developing in R, and comfort with tools such as svn, git, Roxygen2, LaTeX (for equations in particular), and familiarity with base graphics.
- Those with demonstrable experience with finance-related or graphics related R packages will be preferred.
- A background in computer science or engineering with graduate training in finance would likely be ideal.
- Discuss the list of proposed functionality for inclusion in quantmod, PerformanceAnalytics, and PortfolioAnalytics.
- Develop a specific plan with a timeline for development, documentation (consider adding a vignette as well), and testing the functionality.
- Provide a complete code example of a function with documentation that could be used in the PerformanceAnalytics or similar package (exceptional examples will be considered for inclusion in the package and their author given full credit for the contribution). The example does not need to pertain to the subject of this project, but should demonstrate your coding ability and familiarity with the tools used in development.
- Clearly identify any other commitments or possible conflicts for your time during the coming summer.
Brian Peterson ([@](mailto:brian {at} braverock {dot} com)) is one of the primary authors of these packages, and has previously mentored GSoC projects.
Peter Carl ([@](mailto:peter {at} braverock {dot} com)) is one of the primary authors of these packages, and has previously mentored GSoC projects.
Douglas Martin ([@](mailto:doug {at} amath {dot} washington {dot} edu)) is Director of the Computational Finance and Risk Management program at the University of Washington, and has extensive experience in evaluating robustness of analytical methods.
Ross Bennett ([@](mailto:rossbennett34 {at} gmail {dot} com)) is a contributor to these packages, and has previously participated in GSoC as a student.