Bayesian Model Choice
Green's algorithm
Fixed Dimension Reassessment
RJMCMC for change points
The log-likelihood function is
H Move
the acceptance probability is
Note that
and
so
Then we have
P Move
The acceptance probability is
since
Birth Move
and define the perturbation to be such that
The prior ratio, becomes
the proposal ration becomes
and the Jacobian is
Death Move
The acceptance probability for the corresponding death step has the same form with the appropriate change of labelling of the variables, and the ratio terms inverted.
Results
The histogram of number of change points is
References
p ( y ∣ x ) = ∑ i = 1 n log { x ( y i ) } − ∫ 0 L x ( t ) d t = ∑ j = 1 k m j log h j − ∑ j = 0 k h j ( s j + 1 − s j ) \begin{aligned}
p(y\mid x) &= \sum_{i=1}^n\log\{x(y_i)\}-\int_0^Lx(t)dt\\
&= \sum_{j=1}^km_j\log h_j-\sum_{j=0}^kh_j(s_{j+1}-s_j)
\end{aligned} p ( y ∣ x ) = i = 1 ∑ n log { x ( y i )} − ∫ 0 L x ( t ) d t = j = 1 ∑ k m j log h j − j = 0 ∑ k h j ( s j + 1 − s j ) where m j = # { y i ∈ [ s j , s j + 1 ) } m_j=\#\{y_i\in[s_j,s_{j+1})\} m j = # { y i ∈ [ s j , s j + 1 )}
choose one of h 0 , h 1 , … , h k h_0,h_1,\ldots,h_k h 0 , h 1 , … , h k h j h_j h j
propose a change to h j ′ h_j' h j ′ log ( h j ′ / h j ) ∼ U [ − 1 2 , 1 2 ] \log(h_j'/h_j)\sim U[-\frac 12, \frac 12] log ( h j ′ / h j ) ∼ U [ − 2 1 , 2 1 ]
α ( h j , h j ′ ) = p ( h j ′ ∣ y ) p ( h j ∣ y ) × J ( h j ∣ h j ′ ) J ( h j ′ ∣ h j ) = p ( y ∣ h j ′ ) p ( y ∣ h j ) × π ( h j ′ ) π ( h j ) × J ( h j ∣ h j ′ ) J ( h j ′ ∣ h j ) . \begin{aligned}
\alpha(h_j, h_j') &=\frac{p(h_j'\mid y)}{p(h_j\mid y)}\times \frac{J(h_j\mid h_j')}{J(h_j'\mid h_j)}\\
&= \frac{p(y\mid h_j')}{p(y\mid h_j)}\times\frac{\pi(h_j')}{\pi(h_j)}\times\frac{J(h_j\mid h_j')}{J(h_j'\mid h_j)}\,.
\end{aligned} α ( h j , h j ′ ) = p ( h j ∣ y ) p ( h j ′ ∣ y ) × J ( h j ′ ∣ h j ) J ( h j ∣ h j ′ ) = p ( y ∣ h j ) p ( y ∣ h j ′ ) × π ( h j ) π ( h j ′ ) × J ( h j ′ ∣ h j ) J ( h j ∣ h j ′ ) . log h j ′ h j = u , \log\frac{h_j'}{h_j}= u \,, log h j h j ′ = u , it follows that the CDF of h j ′ h_j' h j ′
F ( x ) = P ( h j ′ ≤ x ) = P ( e u h ≤ x ) = P ( u ≤ log ( x / h ) ) = log ( x / h ) + 1 / 2 F(x) = P(h_j'\le x) = P(e^uh\le x) = P(u\le \log(x/h)) = \log(x/h) + 1/2 F ( x ) = P ( h j ′ ≤ x ) = P ( e u h ≤ x ) = P ( u ≤ log ( x / h )) = log ( x / h ) + 1/2 f ( x ) = F ′ ( x ) = 1 x , f(x) = F'(x) = \frac{1}{x}\,, f ( x ) = F ′ ( x ) = x 1 , J ( h j ′ ∣ h j ) = 1 h j ′ . J(h_j'\mid h_j) = \frac{1}{h_j'}\,. J ( h j ′ ∣ h j ) = h j ′ 1 . α ( h j , h j ′ ) = p ( y ∣ h j ′ ) p ( y ∣ h j ) × ( h j ′ ) α exp ( − β h j ′ ) h j α exp ( − β h j ) = likelihood ratio × ( h j ′ / h j ) α exp { − β ( h j ′ − h j ) } \begin{aligned}
\alpha(h_j,h_j') &= \frac{p(y\mid h_j')}{p(y\mid h_j)}\times \frac{(h_j')^{\alpha}\exp(-\beta h_j')}{h_j^\alpha\exp(-\beta h_j)}\\
&=\text{likelihood ratio}\times (h_j'/h_j)^\alpha\exp\{-\beta(h_j'-h_j)\}
\end{aligned} α ( h j , h j ′ ) = p ( y ∣ h j ) p ( y ∣ h j ′ ) × h j α exp ( − β h j ) ( h j ′ ) α exp ( − β h j ′ ) = likelihood ratio × ( h j ′ / h j ) α exp { − β ( h j ′ − h j )} Draw one of s 1 , s 2 , … , s k s_1,s_2,\ldots,s_k s 1 , s 2 , … , s k s j s_j s j
Propose s j ′ ∼ U [ s j − 1 , s j + 1 ] s_j'\sim U[s_{j-1}, s_{j+1}] s j ′ ∼ U [ s j − 1 , s j + 1 ]
α ( s j , s j ′ ) = p ( y ∣ s j ′ ) p ( y ∣ s j ) × π ( s j ′ ) π ( s j ) × J ( s j ∣ s j ′ ) J ( s j ′ ∣ s j ) = likelihood ratio × ( s j + 1 − s j ′ ) ( s j ′ − s j − 1 ) ( s j + 1 − s j ) ( s j − s j − 1 ) \begin{aligned}
\alpha(s_j,s_j') &=\frac{p(y\mid s_j')}{p(y\mid s_j)}\times \frac{\pi(s_j')}{\pi(s_j)}\times \frac{J(s_j\mid s_j')}{J(s_j'\mid s_j)}\\
&=\text{likelihood ratio}\times \frac{(s_{j+1}-s_j')(s_j'-s_{j-1})}{(s_{j+1}-s_j)(s_j-s_{j-1})}
\end{aligned} α ( s j , s j ′ ) = p ( y ∣ s j ) p ( y ∣ s j ′ ) × π ( s j ) π ( s j ′ ) × J ( s j ′ ∣ s j ) J ( s j ∣ s j ′ ) = likelihood ratio × ( s j + 1 − s j ) ( s j − s j − 1 ) ( s j + 1 − s j ′ ) ( s j ′ − s j − 1 ) π ( s 1 , s 2 , … , s k ) = ( 2 k + 1 ) ! L 2 k + 1 ∏ j = 0 k + 1 ( s j + 1 − s j ) \pi(s_1,s_2,\ldots,s_k)=\frac{(2k+1)!}{L^{2k+1}}\prod_{j=0}^{k+1}(s_{j+1}-s_j) π ( s 1 , s 2 , … , s k ) = L 2 k + 1 ( 2 k + 1 )! j = 0 ∏ k + 1 ( s j + 1 − s j ) Choose a position s ∗ s^* s ∗ [ 0 , L ] [0,L] [ 0 , L ] ( s j , s j + 1 ) (s_j,s_{j+1}) ( s j , s j + 1 )
Propose new heights h j ′ , h j + 1 ′ h_j', h_{j+1}' h j ′ , h j + 1 ′ ( s j , s ∗ ) (s_j,s^*) ( s j , s ∗ ) ( s ∗ , s j + 1 ) (s^*,s_{j+1}) ( s ∗ , s j + 1 )
( s ∗ − s j ) log ( h j ′ ) + ( s j + 1 − s ∗ ) log ( h j + 1 ′ ) = ( s j + 1 − s j ) log ( h j ) (s^*-s_j)\log(h_j') + (s_{j+1}-s^*)\log(h_{j+1}')=(s_{j+1}-s_j)\log(h_j) ( s ∗ − s j ) log ( h j ′ ) + ( s j + 1 − s ∗ ) log ( h j + 1 ′ ) = ( s j + 1 − s j ) log ( h j ) h j + 1 ′ h j ′ = 1 − u u \frac{h_{j+1}'}{h_j'}=\frac{1-u}{u} h j ′ h j + 1 ′ = u 1 − u with u u u [ 0 , 1 ] [0,1] [ 0 , 1 ]
p ( k + 1 ) p ( k ) 2 ( k + 1 ) ( 2 k + 3 ) L 2 ( s ∗ − s j ) ∗ ( s j + 1 − s ∗ ) s j + 1 − s j × β α Γ ( α ) ( h j ′ h j + 1 ′ h j ) α − 1 exp { − β ( h j ′ + h j + 1 ′ − h ) } \frac{p(k+1)}{p(k)}\frac{2(k+1)(2k+3)}{L^2}\frac{(s^*-s_j)*(s_{j+1}-s^*)}{s_{j+1}-s_j}\times \frac{\beta^\alpha}{\Gamma(\alpha)}\Big(\frac{h_j'h_{j+1}'}{h_j}\Big)^{\alpha-1}\exp\{-\beta(h_j'+h_{j+1}'-h)\} p ( k ) p ( k + 1 ) L 2 2 ( k + 1 ) ( 2 k + 3 ) s j + 1 − s j ( s ∗ − s j ) ∗ ( s j + 1 − s ∗ ) × Γ ( α ) β α ( h j h j ′ h j + 1 ′ ) α − 1 exp { − β ( h j ′ + h j + 1 ′ − h )} d k + 1 L b k ( k + 1 ) \frac{d_{k+1}L}{b_k(k+1)} b k ( k + 1 ) d k + 1 L ( h j ′ + h j + 1 ′ ) 2 h j \frac{(h_j'+h_{j+1}')^2}{h_j} h j ( h j ′ + h j + 1 ′ ) 2 If s j + 1 s_{j+1} s j + 1 ( s j ′ , s j + 1 ′ ) = ( s j , s j + 2 ) (s_j',s_{j+1}')=(s_j,s_{j+2}) ( s j ′ , s j + 1 ′ ) = ( s j , s j + 2 ) h j ′ h_j' h j ′
( s j + 1 − s j ) log ( h j ) + ( s j + 2 − s j + 1 ) log ( h j + 1 ) = ( s j + 1 ′ − s j ′ ) log ( h j ′ ) (s_{j+1}-s_j)\log(h_j) + (s_{j+2}-s_{j+1})\log(h_{j+1}) = (s_{j+1}'-s_j')\log(h_j') ( s j + 1 − s j ) log ( h j ) + ( s j + 2 − s j + 1 ) log ( h j + 1 ) = ( s j + 1 ′ − s j ′ ) log ( h j ′ ) And we can get the density plot of position (or height) conditional on the number of change points k k k k = 1 , 2 , 3 k=1,2,3 k = 1 , 2 , 3
