Question

我在数据中心有很多主机。主机只能通过HTTP负载均衡器进行网络访问，我有IP地址。每个主机都有相同的restful服务，它提供我所追求的主机名和其他主机特定信息。我有一个节点应用程序在另一个位置运行，它将向负载均衡器提交HTTP GET请求。这些请求将由负载均衡器为其分配请求的主机提供服务。我的问题是如何为并发，受限制的HTTP GET请求编写节点。我是javascript的新手，并试图了解使用异步，promises或本机回调的最佳实践，或者需要哪些组合。

我的要求：

从节点到负载均衡器的请求具有可在代码中配置的最大并发请求限制。
继续发出http请求，直到我收到来自可配置数量的主机的响应，每个主机都会响应主机名和其他信息。
可配置的最大总HTTP请求限制。

Answer 1

我不知道这是否会对你有所帮助，因为它需要学习一个新的图书馆。

无论如何......您可以使用RxJS做您想做的事情jsbin随时提出您可能遇到的任何问题。

import Rx from 'rx';

var concurrency = 2;
var maxRequests = 100;
var maxDistinct = 3;


// mock of your actual request function
// instead of setTimout you should make an actual request and onNext the response of that
var hostIndex = 0;
function executeRequest(requestConfig) {
  return Rx.Observable.create(observable => {
    setTimeout(function() {
      console.log('start request ', hostIndex);
      observable.onNext({request: requestConfig, response: 42, hostname: hostIndex});
      hostIndex += 1;
      observable.onCompleted();
    }, 1000);
  });
}

var endObservable = new Rx.Subject();
var requests = Rx.Observable.from( ['url1', 'url2', 'url3', 'url4', 'url5', 'url6'] );
var responses = requests
      .take(maxRequests)   // only take in this many urls (you could do that by limiting the size of the array, did I misunderstand you here?)
      .map(executeRequest)
      .merge(concurrency)  // limit concurrently running observables
      .takeUntil(endObservable) // stop the observable and thereby any remaining requests once endObservable fires
      .publish(); // makes multiple subscriptions use the same underlying source - needed so requests are not executed multiple times.

// your actual responses
responses.subscribe(value => console.log(value),
                    error => console.log(error),
                    ended => console.log('completed'));

// controls when the stream ends due to maxDistinct hosts being reached
responses
  .scan((state, currentResponse) => {
    // keep list of unique hosts that responded
    state.hosts[currentResponse.hostname] = true;
    return state;
  }, { hosts: {} })
  .subscribe(state => {
    var distinctHosts = Object.keys(state.hosts).length;
    if(distinctHosts >= maxDistinct) endObservable.onNext(); //if maxDistinct hosts has been reached emit an event on the endObservable, which terminates any remaining requests via .takeUntil(endObservable)
  });

responses.connect();

节点并发http请求限制

1 个答案: