在AWS中使用Mesos进行Strom

Question

我正在AWS上运行风暴群集。但我希望风暴群集能够在需要时自动扩展。我发现mesos就是这样的。但我对Mesos及其在AWS上的部署知之甚少。

AWS上的mesos可以通过启动新实例并在不需要时关闭它们来自动增加拓扑任务的并行性吗？如果可以，我们如何配置相同的mesos。

Answer 1

Mesos does not directly handle autoscaling itself, but allows frameworks running on top of it to receive new resource offers and react to them by launching new task instances. I haven't used it personally, but you could try the Storm-Mesos framework for running Storm on Mesos: https://github.com/mesos/storm

Once you have Storm running on Mesos, ready to launch new instances as resources come available, you're ready to autoscale within the existing cluster's capacity. You'll probably want to take advantage of Amazon's Auto-Scaling Groups (ASGs) to scale up the number of Mesos nodes based on your need. As the ASG scales up more Mesos nodes, the resources from those nodes will be automatically offered to the Storm-Mesos framework, which can launch more Storm instances.

Answer 2

是的，你正朝着正确的方向前进。不过，我建议使用Marathon而不是低级别的Mesos API。

例如参见GitHub repo obaidsalikeen/storm-marathon，在完整性和文档丰富度方面做得特别好。