Question

我正在尝试在MXNet中构建和部署一个简单的神经网络，并使用mxnet-model-server将其部署在服务器上。

最大的问题是部署模型-上传.mar文件后，模型服务器崩溃，但是我不知道可能是什么问题。

我使用以下代码创建了一个自定义（但非常简单）的神经网络进行测试：

from __future__ import print_function
import numpy as np
import mxnet as mx
from mxnet import nd, autograd, gluon

data_ctx = mx.cpu()
model_ctx = mx.cpu()

# fix the seed
np.random.seed(42)
mx.random.seed(42)

num_examples = 1000

X = mx.random.uniform(shape=(num_examples, 49))
y = mx.random.uniform(shape=(num_examples, 1))
dataset_train = mx.gluon.data.dataset.ArrayDataset(X, y)

dataset_test = dataset_train

data_loader_train = mx.gluon.data.DataLoader(dataset_train, batch_size=25)
data_loader_test = mx.gluon.data.DataLoader(dataset_test, batch_size=25)

num_outputs = 2
net = gluon.nn.HybridSequential()
net.hybridize()
with net.name_scope():
    net.add(gluon.nn.Dense(49, activation="relu"))
    net.add(gluon.nn.Dense(64, activation="relu"))
    net.add(gluon.nn.Dense(num_outputs))

net.collect_params().initialize(mx.init.Normal(sigma=.1), ctx=model_ctx)
softmax_cross_entropy = gluon.loss.SoftmaxCrossEntropyLoss()
trainer = gluon.Trainer(net.collect_params(), 'sgd', {'learning_rate': .01})

epochs = 1
smoothing_constant = .01

for e in range(epochs):
    cumulative_loss = 0
    for i, (data, label) in enumerate(data_loader_train):
        data = data.as_in_context(model_ctx).reshape((-1, 49))
        label = label.as_in_context(model_ctx)
        with autograd.record():
            output = net(data)
            loss = softmax_cross_entropy(output, label)
        loss.backward()
        trainer.step(data.shape[0])
        cumulative_loss += nd.sum(loss).asscalar()

以下，使用以下命令导出模型：

net.export("model_files/my_project")

结果是一个.json和.params文件。

我创建了一个signature.json

{
  "inputs": [
    {
      "data_name": "data",
      "data_shape": [
        1,
        49
      ]
    }
  ]
}

模型处理程序与mxnet教程中的相同：

# Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.
# Licensed under the Apache License, Version 2.0 (the "License").
# You may not use this file except in compliance with the License.
# A copy of the License is located at
#     http://www.apache.org/licenses/LICENSE-2.0
# or in the "license" file accompanying this file. This file is distributed
# on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
# express or implied. See the License for the specific language governing
# permissions and limitations under the License.

"""
ModelHandler defines a base model handler.
"""
import logging
import time


class ModelHandler(object):
    """
    A base Model handler implementation.
    """

    def __init__(self):
        self.error = None
        self._context = None
        self._batch_size = 0
        self.initialized = False

    def initialize(self, context):
        """
        Initialize model. This will be called during model loading time

        :param context: Initial context contains model server system properties.
        :return:
        """
        self._context = context
        self._batch_size = context.system_properties["batch_size"]
        self.initialized = True

    def preprocess(self, batch):
        """
        Transform raw input into model input data.

        :param batch: list of raw requests, should match batch size
        :return: list of preprocessed model input data
        """
        assert self._batch_size == len(batch), "Invalid input batch size: {}".format(len(batch))
        return None

    def inference(self, model_input):
        """
        Internal inference methods

        :param model_input: transformed model input data
        :return: list of inference output in NDArray
        """
        return None

    def postprocess(self, inference_output):
        """
        Return predict result in batch.

        :param inference_output: list of inference output
        :return: list of predict results
        """
        return ["OK"] * self._batch_size

    def handle(self, data, context):
        """
        Custom service entry point function.

        :param data: list of objects, raw input from request
        :param context: model server context
        :return: list of outputs to be send back to client
        """
        self.error = None  # reset earlier errors

        try:
            preprocess_start = time.time()
            data = self.preprocess(data)
            inference_start = time.time()
            data = self.inference(data)
            postprocess_start = time.time()
            data = self.postprocess(data)
            end_time = time.time()

            metrics = context.metrics
            metrics.add_time("PreprocessTime", round((inference_start - preprocess_start) * 1000, 2))
            metrics.add_time("InferenceTime", round((postprocess_start - inference_start) * 1000, 2))
            metrics.add_time("PostprocessTime", round((end_time - postprocess_start) * 1000, 2))

            return data
        except Exception as e:
            logging.error(e, exc_info=True)
            request_processor = context.request_processor
            request_processor.report_status(500, "Unknown inference error")
            return [str(e)] * self._batch_size

接下来，我使用以下命令创建了.mar文件：

model-archiver --model-name my_project --model-path my_project --handler ssd_service:handle

在服务器上启动模型：

mxnet-model-server --start --model_store my_project --models ssd=my_project.mar

我确实遵循了每个教程： https://github.com/awslabs/mxnet-model-server

但是，服务器崩溃了。工作人员死亡，后端工作人员死亡，工作人员断开连接，负载模型失败：ssd，错误：工作人员死亡

我绝对不知道该怎么办，所以如果您帮助我，我将非常高兴！

最佳

Answer 1

我尝试了您的代码，并且可以在笔记本电脑上正常工作。如果我运行：curl -X POST http://127.0.0.1:8080/predictions/ssd -F "data=[0 1 2 3 4]"，我得到：OK%

我只能猜测为什么它在您的计算机上不起作用：

请注意，model-store参数应使用-而不是_进行编写，就像您的示例一样。我运行mxnet-model-server的命令如下：mxnet-model-server --start --model-store ./ --models ssd=my_project.mar
您使用哪个版本的mxnet-model-server？最新的是1.0.2，但是我已经安装了1.0.1，所以也许您想降级并尝试一下：pip install mxnet-model-server==1.0.1。
与MXNet版本相同的问题。就我而言，我使用每晚通过pip install mxnet --pre获得的版本。我看到您的模型非常基础，因此它不应该过分依赖...不过，以防万一，请安装1.4.0（当前版本）。

不确定，但希望能对您有所帮助。

如何在MXNet中从A-Z部署简单的神经网络

1 个答案: