Independent MH
Last updated
Last updated
Robert and Casella (2013) presents the following algorithm:
And we can implement this algorithm with the Julia code:
The likelihood is
Note that
where
is the Euler's Constant.
We can use MLE to estimate the coefficient in the logistical model (see my post for the derivation), and the following Julia code can help us fit the model quickly.
Wrote the following Julia code to implement the independent MH algorithm,
The saddlepoint can be used to approximate the tail area of a distribution. We have the approximation
I implemented the independent MH algorithm via the following code to produce random variables from the saddlepoint distribution.
I can successfully reproduce the results.
Interval
1 - pchisq(x, 6, 9*2)
IMH
0.1000076
0.1037
0.05000061
0.0516
0.01000057
0.0114
Then use these cutpoints to estimate the probability by using samples produced from independent MH algorithm.
Interval
1 - pchisq(x*100, 6*100, 9*2*100)
IMH
0.10
0.1045
0.05
0.0516
0.01
0.0093
If there exists a constant such that
the algorithm produces a uniformly ergodic chain (Theorem), and the expected acceptance probability associated with the algorithm is at least when the chain is stationary, and in that sense, the IMH is more efficient than the Accept-Reject algorithm.
Let's illustrate this algorithm with . We have introduced how to sample from Gamma distribution via Accept-Reject algorithm in Special Distributions, and it is straightforward to get the Gamma Metropolis-Hastings based on the ratio of ,
To sample , and estimate .
We observe according to the model
and let and put a flat prior on , i.e.,
Choose the data-dependent value that makes , where is the MLE of , so .
The estimates of the parameters are and .
If is the cumulant generating function, solving the saddlepoint equation yields the saddlepoint. For noncentral chi squared random variable , the moment generating function is
where is the number of degrees of freedom and is the noncentrality parameter, and its saddlepoint is
where is the sample from the saddlepoint distribution. Using a Taylor series approximation,
so the instrumental density can be chosen as , where
If ,
For , note that if , then , then we can use the following R code to figure out the cutpoints.