Support truncated distribution.

[yebai]

The current HMC sampler does not work with truncated distribution. For example, the following example would throw an error

negbindata = [0, 1, 4, 0, 2, 2, 5, 0, 1]

@model negbinmodel(y) begin
  local α, β
  α ~ Truncated(Cauchy(0,10), 0, +Inf)
  β ~ Truncated(Cauchy(0,10), 0, +Inf)
  for i = 1:length(y)
    y[i] ~ NegativeBinomial(α, β)  # α > 0, 0 < β < 1
  end
  return(α, β)
end

@sample(negbinmodel(negbindata), HMC(1000, 0.02, 1))

ERROR MESSAGE:

MethodError: no method matching pdf(::Distributions.Truncated{Distributions.Cauchy{Float64},Distributions.Continuous}, ::ForwardDiff.Dual{0,Float64})
Closest candidates are:
  pdf(::Distributions.Distribution{Distributions.Univariate,Distributions.Continuous}, ::Real) at /Users/yebai/.julia/v0.5/Distributions/src/univariates.jl:102
  pdf(!Matched::Distributions.Kolmogorov, ::Real) at /Users/yebai/.julia/v0.5/Distributions/src/univariate/continuous/kolmogorov.jl:74
  pdf(!Matched::Distributions.Arcsine{T<:Real}, ::Real) at /Users/yebai/.julia/v0.5/Distributions/src/univariate/continuous/arcsine.jl:74
  ...
 in logpdf(::Distributions.Truncated{Distributions.Cauchy{Float64},Distributions.Continuous}, ::ForwardDiff.Dual{0,Float64}, ::Bool) at transform.jl:39
 in assume(::Turing.HMCSampler{Turing.HMC}, ::Distributions.Truncated{Distributions.Cauchy{Float64},Distributions.Continuous}, ::Turing.Var, ::Turing.VarInfo) at hmc.jl:154
 in macro expansion at compiler.jl:10 [inlined]
 in negbinmodel(::Dict{Any,Any}, ::Turing.VarInfo, ::Turing.HMCSampler{Turing.HMC}) at compiler.jl:21
 in step(::#negbinmodel, ::Dict{Any,Any}, ::Turing.HMCSampler{Turing.HMC}, ::Turing.VarInfo, ::Bool) at hmc.jl:59
 in run(::Function, ::Dict{Any,Any}, ::Turing.HMCSampler{Turing.HMC}) at hmc.jl:109
 in sample(::Function, ::Dict{Any,Any}, ::Turing.HMC) at hmc.jl:184
 in macro expansion; at compiler.jl:295 [inlined]
 in anonymous at <missing>:?
 in include_string(::String, ::String) at loading.jl:441
 in include_string(::String, ::String, ::Int64) at eval.jl:28
 in include_string(::Module, ::String, ::String, ::Int64, ::Vararg{Int64,N}) at eval.jl:32
 in (::Atom.##53#56{String,Int64,String})() at eval.jl:40
 in withpath(::Atom.##53#56{String,Int64,String}, ::String) at utils.jl:30
 in withpath(::Function, ::String) at eval.jl:46
 in macro expansion at eval.jl:57 [inlined]
 in (::Atom.##52#55{Dict{String,Any}})() at task.jl:60

[xukai92]

I've looked into Distributions.jl. This error is due to the fact that helper functions for Truncated still are not defined with generic types (e.g. they use pdf(d::Truncated, x::Float64) = ... for pdf (https://github.com/JuliaStats/Distributions.jl/blob/master/src/truncate.jl). I will overload them in Turing and submit a PR soon.

[xukai92]

Submitted a PR for Distributions.jl JuliaStats/Distributions.jl#586

This example below (same as above but changed β to the right support range) is passed on my local Distribution.jl fork (https://github.com/xukai92/Distributions.jl/tree/generic-support-to-truncated-dist).

using Distributions, Turing

negbindata = [0, 1, 4, 0, 2, 2, 5, 0, 1]

@model negbinmodel(y) begin
  local α, β
  α ~ Truncated(Cauchy(0,10), 0, +Inf)
  β ~ Truncated(Cauchy(0,10), 0, 1.0)
  for i = 1:length(y)
    y[i] ~ NegativeBinomial(α, β)  # α > 0, 0 < β < 1
  end
  return(α, β)
end

@sample(negbinmodel(negbindata), HMC(1000, 0.02, 1))

[yebai]

@xukai92
Is there anything that we can do to speed up the merging by Distributions package developers?

[xukai92]

Let me chase them on the PR

[xukai92]

Closed because PR is merged.