Question

我正在尝试在Python中实现Metropolis算法（Metropolis-Hastings算法的简单版本）。

这是我的实现方式：

def Metropolis_Gaussian(p, z0, sigma, n_samples=100, burn_in=0, m=1):
    """
    Metropolis Algorithm using a Gaussian proposal distribution.
    p: distribution that we want to sample from (can be unnormalized)
    z0: Initial sample
    sigma: standard deviation of the proposal normal distribution.
    n_samples: number of final samples that we want to obtain.
    burn_in: number of initial samples to discard.
    m: this number is used to take every mth sample at the end
    """
    # List of samples, check feasibility of first sample and set z to first sample
    sample_list = [z0]
    _ = p(z0) 
    z = z0
    # set a counter of samples for burn-in
    n_sampled = 0

    while len(sample_list[::m]) < n_samples:
        # Sample a candidate from Normal(mu, sigma),  draw a uniform sample, find acceptance probability
        cand = np.random.normal(loc=z, scale=sigma)
        u = np.random.rand()
        try:
            prob = min(1, p(cand) / p(z))
        except (OverflowError, ValueError) as error:
            continue
        n_sampled += 1

        if prob > u:
            z = cand  # accept and make candidate the new sample

        # do not add burn-in samples
        if n_sampled > burn_in:
            sample_list.append(z)

    # Finally want to take every Mth sample in order to achieve independence
return np.array(sample_list)[::m]

当我尝试将算法应用于指数函数时，它花费的时间很少。但是，当我在 t-distribution 上尝试时，考虑到它并没有进行那么多的计算，这需要花一些时间。这是复制我的代码的方法：

t_samples = Metropolis_Gaussian(pdf_t, 3, 1, 1000, 1000, m=100)
plt.hist(t_samples, density=True, bins=15, label='histogram of samples')
x = np.linspace(min(t_samples), max(t_samples), 100)
plt.plot(x, pdf_t(x), label='t pdf')
plt.xlim(min(t_samples), max(t_samples))
plt.title("Sampling t distribution via Metropolis")
plt.xlabel(r'$x$')
plt.ylabel(r'$y$')
plt.legend()

此代码需要相当长的时间才能运行，我不确定为什么。在我的 Metropolis_Gaussian 代码中，我试图通过

不将重复的样本添加到列表中
不记录老化样品

函数pdf_t的定义如下

from scipy.stats import t
def pdf_t(x, df=10):
    return t.pdf(x, df=df)

Answer 1

我回答了similar question previously。我在这里提到的许多内容（不计算每次迭代的当前可能性，预先计算随机创新等）都可以在这里使用。

实施方面的其他改进将不会使用列表来存储样本。相反，您应该为样本预先分配内存，并将其存储为数组。像samples = np.zeros(n_samples)这样的方法比每次迭代都附加到列表上要有效。

您已经提到过尝试通过不记录老化样本来提高效率。这是一个好主意。您也可以通过仅记录第m个样本来进行细化操作，因为无论如何您都使用np.array(sample_list)[::m]将它们丢弃在return语句中。您可以通过以下方式来做到这一点：

   # do not add burn-in samples
    if n_sampled > burn_in:
        sample_list.append(z)

到

    # Only keep iterations after burn-in and for every m-th iteration
    if n_sampled > burn_in and n_sampled % m == 0:
        samples[(n_sampled - burn_in) // m] = z

还值得注意的是，您不需要计算min(1, p(cand) / p(z))，而只需要计算p(cand) / p(z)就可以摆脱困境。我意识到形式上min是必要的（以确保概率限制在0和1之间）。但是，在计算上，我们不需要最小值，因为如果p(cand) / p(z) > 1则p(cand) / p(z)总是大于u 。

将所有这些放在一起，并预先计算出随机创新，接受概率u，并且只计算您真正想出的可能性：

def my_Metropolis_Gaussian(p, z0, sigma, n_samples=100, burn_in=0, m=1):
    # Pre-allocate memory for samples (much more efficient than using append)
    samples = np.zeros(n_samples)

    # Store initial value
    samples[0] = z0
    z = z0
    # Compute the current likelihood
    l_cur = p(z)

    # Counter
    iter = 0
    # Total number of iterations to make to achieve desired number of samples
    iters = (n_samples * m) + burn_in

    # Sample outside the for loop
    innov = np.random.normal(loc=0, scale=sigma, size=iters)
    u = np.random.rand(iters)

    while iter < iters:
        # Random walk innovation on z
        cand = z + innov[iter]

        # Compute candidate likelihood
        l_cand = p(cand)

        # Accept or reject candidate
        if l_cand / l_cur > u[iter]:
            z = cand
            l_cur = l_cand

        # Only keep iterations after burn-in and for every m-th iteration
        if iter > burn_in and iter % m == 0:
            samples[(iter - burn_in) // m] = z

        iter += 1

    return samples

如果我们看一下性能，会发现该实现比原始实现快了 2倍，这对于一些小的更改也不错。

In [1]: %timeit Metropolis_Gaussian(pdf_t, 3, 1, n_samples=100, burn_in=100, m=10)
205 ms ± 2.16 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [2]: %timeit my_Metropolis_Gaussian(pdf_t, 3, 1, n_samples=100, burn_in=100, m=10)
102 ms ± 1.12 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

为什么我的Metropolis算法（mcmc）的python实现这么慢？

1 个答案: