Question

如果我们有一个分层模型，该模型将来自不同站点的数据作为模型中的不同组，我们如何预测新组（以前从未见过的新站点）？例如使用以下逻辑回归模型：

from pymc3 import Model, sample, Normal, HalfCauchy,Bernoulli
import theano.tensor as tt
with Model() as varying_slope:

    mu_beta = Normal('mu_beta', mu=0., sd=1e5)
    sigma_beta = HalfCauchy('sigma_beta', 5)
    a = Normal('a', mu=0., sd=1e5)
    betas = Normal('b',mu=mu_beta,sd=sigma_beta,shape=(n_features,n_site))
    y_hat = a + tt.dot(X_shared,betas[:,site_shared])
    y_like = Bernoulli('y_like', logit_p=y_hat, observed=train_y)

在拟合此模型之后，我们可以使用以下方法从特定位置预测新数据（即后验预测的样本）：

site_to_predict = 1
samples = 500

x = tt.matrix('X',dtype='float64')
new_site = tt.vector('new_site',dtype='int32')
n_samples = tt.iscalar('n_samples')
x.tag.test_value = np.empty(shape=(1,X.shape[1]))
new_site.tag.test_value = np.empty(shape=(1,1))

_sample_proba =  approx.sample_node(varying_slope.y_like.distribution.p,
                               size=n_samples,
                               more_replacements={X_shared: x,site_shared:new_site})

sample_proba = theano.function([x,new_site,n_samples], _sample_proba)

pred_test = sample_proba(test_X.reshape(1,-1),np.array(site_to_predict).reshape(-1),samples)

但是如果我们有一个新的看不见的位置，从后验预测分布中采样的正确方法是什么？

Answer 1

如果某人偶然遇到这个问题或另一个类似的问题，我只是从pymc discourse thread复制我的答案。

首先，请注意所使用的居中分层参数化1，它在拟合时可能会导致分歧和困难。

话虽如此，您的模型或多或少看起来像是GLM，具有跨要素和站点共享的先验随机变量mu_beta和sigma_beta。一旦获得这两个的后验分布，您的预测应类似于

y_hat = a + dot(X_shared, Normal(mu=mu_beta, sigma=sigma_beta))
y_like = Bernoulli('y_like', logit_p=y_hat)

所以，我们将力争做到这一点。

我们总是建议从样本后验预测检查中脱颖而出的方法是使用theano.shared。我将使用一种受功能API启发的不同方法，该API是pymc4的核心设计思想。在pymc3和pymc4的框架之间，我不会涉及很多差异，但是我开始更多使用的一件事是工厂函数来获取Model实例。我没有尝试使用theano.shared定义模型内部的内容，而只是使用新数据创建一个新模型并从中提取后验预测样本。我刚刚在这里发布了有关此内容的信息。

这个想法是用训练数据创建模型并从中采样以进行跟踪。然后，您必须从跟踪中提取与看不见的站点共享的分层部分：mu_beta，sigma_beta和a。最后，使用测试站点的新数据创建一个新模型，并使用包含mu_beta，sigma_beta和一部分训练轨迹的词典列表从后验预测中进行采样。这是一个独立的示例

import numpy as np
import pymc3 as pm
from theano import tensor as tt
from matplotlib import pyplot as plt


def model_factory(X, y, site_shared, n_site, n_features=None):
    if n_features is None:
        n_features = X.shape[-1]
    with pm.Model() as model:
        mu_beta = pm.Normal('mu_beta', mu=0., sd=1)
        sigma_beta = pm.HalfCauchy('sigma_beta', 5)
        a = pm.Normal('a', mu=0., sd=1)
        b = pm.Normal('b', mu=0, sd=1, shape=(n_features, n_site))
        betas = mu_beta + sigma_beta * b
        y_hat = a + tt.dot(X, betas[:, site_shared])
        pm.Bernoulli('y_like', logit_p=y_hat, observed=y)
    return model


# First I generate some training X data
n_features = 10
ntrain_site = 5
ntrain_obs = 100
ntest_site = 1
ntest_obs = 1
train_X = np.random.randn(ntrain_obs, n_features)
train_site_shared = np.random.randint(ntrain_site, size=ntrain_obs)
new_site_X = np.random.randn(ntest_obs, n_features)
test_site_shared = np.zeros(ntest_obs, dtype=np.int32)
# Now I generate the training and test y data with a sample from the prior
with model_factory(X=train_X,
                   y=np.empty(ntrain_obs, dtype=np.int32),
                   site_shared=train_site_shared,
                   n_site=ntrain_site) as train_y_generator:
    train_Y = pm.sample_prior_predictive(1, vars=['y_like'])['y_like'][0]
with model_factory(X=new_site_X,
                   y=np.empty(ntest_obs, dtype=np.int32),
                   site_shared=test_site_shared,
                   n_site=ntest_site) as test_y_generator:
    new_site_Y = pm.sample_prior_predictive(1, vars=['y_like'])['y_like'][0]

# The previous part is just to get some toy data to fit
# Now comes the important parts. First training
with model_factory(X=train_X,
                   y=train_Y,
                   site_shared=train_site_shared,
                   n_site=ntrain_site) as train_model:
    train_trace = pm.sample()

# Second comes the hold out data posterior predictive
with model_factory(X=new_site_X,
                   y=new_site_Y,
                   site_shared=test_site_shared,
                   n_site=ntrain_site) as test_model:
    # We first have to extract the learnt global effect from the train_trace
    df = pm.trace_to_dataframe(train_trace,
                               varnames=['mu_beta', 'sigma_beta', 'a'],
                               include_transformed=True)
    # We have to supply the samples kwarg because it cannot be inferred if the
    # input trace is not a MultiTrace instance
    ppc = pm.sample_posterior_predictive(trace=df.to_dict('records'),
                                         samples=len(df))
    plt.figure()
    plt.hist(ppc['y_like'], 30)
    plt.axvline(new_site_Y, linestyle='--', color='r')

我得到的后验预测如下：

当然，我不知道具体将哪种数据作为您的X_shared，site_shared或train_y放置，因此我只是在代码的开头组成了一些废话玩具数据，您应该将其替换为实际数据

我们如何在PyMC3的层次模型中预测新的看不见的群体？

1 个答案: