聚合资源请求&向每个订户发送响应

时间:2016-04-29 10:17:13

标签: aggregate rx-java observable dispatch

我对RxJava相当陌生并且正在努力解决一个对我来说很常见的用例:

从应用程序的不同部分收集多个请求,聚合它们,进行单个资源调用并将结果分发给每个订阅者。

我尝试了一种很多的不同方法,使用主题,可连接的observables,延迟的observables ......到目前为止,没有人做过这个伎俩。

我对这种方法非常乐观,但事实证明它与其他方法一样失败:

    //(...)
    static HashMap<String, String> requests = new HashMap<>();
    //(...)

    @Test
    public void myTest() throws InterruptedException {
        TestScheduler scheduler = new TestScheduler();
        Observable<String> interval = Observable.interval(10, TimeUnit.MILLISECONDS, scheduler)
                .doOnSubscribe(() -> System.out.println("new subscriber!"))
                .doOnUnsubscribe(() -> System.out.println("unsubscribed"))
                .filter(l -> !requests.isEmpty())
                .doOnNext(aLong -> System.out.println(requests.size() + " requests to send"))
                .flatMap(aLong -> {
                    System.out.println("requests " + requests);
                    return Observable.from(requests.keySet()).take(10).distinct().toList();
                })
                .doOnNext(strings -> System.out.println("calling aggregate for " + strings + " (from " + requests + ")"))
                .flatMap(Observable::from)
                .doOnNext(s -> {
                    System.out.println("----");
                    System.out.println("removing " + s);
                    requests.remove(s);
                })
                .doOnNext(s -> System.out.println("remaining " + requests));

        TestSubscriber<String> ts1 = new TestSubscriber<>();
        TestSubscriber<String> ts2 = new TestSubscriber<>();
        TestSubscriber<String> ts3 = new TestSubscriber<>();
        TestSubscriber<String> ts4 = new TestSubscriber<>();

        Observable<String> defer = buildObservable(interval, "1");
        defer.subscribe(ts1);
        Observable<String> defer2 = buildObservable(interval, "2");
        defer2.subscribe(ts2);
        Observable<String> defer3 = buildObservable(interval, "3");
        defer3.subscribe(ts3);
        scheduler.advanceTimeBy(200, TimeUnit.MILLISECONDS);
        Observable<String> defer4 = buildObservable(interval, "4");
        defer4.subscribe(ts4);

        scheduler.advanceTimeBy(100, TimeUnit.MILLISECONDS);
        ts1.awaitTerminalEvent(1, TimeUnit.SECONDS);
        ts2.awaitTerminalEvent(1, TimeUnit.SECONDS);
        ts3.awaitTerminalEvent(1, TimeUnit.SECONDS);
        ts4.awaitTerminalEvent(1, TimeUnit.SECONDS);

        ts1.assertValue("1");
        ts2.assertValue("2"); //fails (test stops here)
        ts3.assertValue("3"); //fails
        ts4.assertValue("4"); //fails


    }

    public Observable<String> buildObservable(Observable<String> interval, String key) {

        return  Observable.defer(() -> {
                            System.out.printf("creating observable for key " + key);
                            return Observable.create(subscriber -> {
                                requests.put(key, "xxx");
                                interval.doOnNext(s -> System.out.println("filtering : key/val  " + key + "/" + s))
                                        .filter(s1 -> s1.equals(key))
                                        .doOnError(subscriber::onError)
                                        .subscribe(s -> {
                                            System.out.println("intern " + s);
                                            subscriber.onNext(s);
                                            subscriber.onCompleted();
                                            subscriber.unsubscribe();
                                        });
                            });
                        }
                )
                ;
    }

输出:

creating observable for key 1new subscriber!
creating observable for key 2new subscriber!
creating observable for key 3new subscriber!
3 requests to send
requests {3=xxx, 2=xxx, 1=xxx}
calling aggregate for [3, 2, 1] (from {3=xxx, 2=xxx, 1=xxx})
----
removing 3
remaining {2=xxx, 1=xxx}
filtering : key/val  1/3
----
removing 2
remaining {1=xxx}
filtering : key/val  1/2
----
removing 1
remaining {}
filtering : key/val  1/1
intern 1
creating observable for key 4new subscriber!
1 requests to send
requests {4=xxx}
calling aggregate for [4] (from {4=xxx})
----
removing 4
remaining {}
filtering : key/val  1/4

第二次断言测试失败(ts2未接收“2”) 结果伪聚合按预期工作,但不会将值分派给相应的订阅者(只有第一个订阅者接收它)

知道为什么吗?

另外,我觉得我在这里错过了明显的东西。如果你想到一个更好的方法,我非常愿意听到它。

编辑:添加一些关于我想要实现的内容。

我有一个REST API通过多个端点公开数据(例如user / {userid})。此API还可以聚合请求(例如,user / user1&amp; user / user2)并在一个http请求中获取相应的数据,而不是两个。

我的目标是能够在给定的时间范围内(比如10毫秒)自动聚合我的应用程序的不同部分发出的请求,其中包含最大批量大小(比如10),发出聚合http请求,然后发送结果到相应的订阅者。

这样的事情:

// NOTE: those calls can be fired from anywhere in the app, and randomly combined. The timing and order is completely unpredictable

//ts : 0ms
api.call(userProfileRequest1).subscribe(this::show); 
api.call(userProfileRequest2).subscribe(this::show);

// - &GT;在10ms之后,应该用这两个呼叫发起一个单独的http聚合请求,映射响应项目&amp;将它们发送给相应的订阅者(将显示正确的用户个人资料)

//ts : 20ms
api.call(userProfileRequest3).subscribe(this::show); 
api.call(userProfileRequest4).subscribe(this::show);
api.call(userProfileRequest5).subscribe(this::show); 
api.call(userProfileRequest6).subscribe(this::show);
api.call(userProfileRequest7).subscribe(this::show); 
api.call(userProfileRequest8).subscribe(this::show);
api.call(userProfileRequest9).subscribe(this::show); 
api.call(userProfileRequest10).subscribe(this::show);
api.call(userProfileRequest11).subscribe(this::show); 
api.call(userProfileRequest12).subscribe(this::show);

//--> should fire a single http aggregate request RIGHT AWAY (we hit the max batch size) with the 10 items, map the response items & send them to the corresponding subscribers (that will show the right user profile)   

我写的测试代码(只有字符串)并粘贴在这个问题的顶部,这是我最终实现的概念证明。

1 个答案:

答案 0 :(得分:1)

您的Observable构造不正确

 public Observable<String> buildObservable(Observable<String> interval, String key) {

    return interval.doOnSubscribe(() -> System.out.printf("creating observable for key " + key))
                   .doOnSubscribe(() -> requests.put(key, "xxx"))
                   .doOnNext(s -> System.out.println("filtering : key/val  " + key + "/" + s))
                   .filter(s1 -> s1.equals(key));
        }

subsribe中的subscriber:它设计错误。

我不明白你想要达到什么目标,但我认为我的代码应该与你的代码非常接近。

请注意,对于所有副作用,我使用doMethods(例如doOnNextdoOnSubscribe)来显示我明确表示我想要产生副作用。

我直接返回defer来替换您的interval来电:因为您希望在interval来电中的自定义可观察版本中发出所有defer个事件,并返回interval可观察性更好。

请注意,您过滤了interval Observable:

Observable<String> interval = Observable.interval(10, TimeUnit.MILLISECONDS, scheduler)
            .filter(l -> !requests.isEmpty()).
            // ... 

因此,只要您将某些内容放入requests地图中,interval就会停止发送。

我不了解您希望通过请求地图实现的目标,但请注意您可能希望避免副作用,更新此地图显然是副作用。

有关评论的更新

您可能希望使用buffer运算符来聚合请求,然后以批量方式执行请求:

    PublishSubject<String> subject = PublishSubject.create();


    TestScheduler scheduler = new TestScheduler();

    Observable<Pair> broker = subject.buffer(100, TimeUnit.MILLISECONDS, 10, scheduler)
                                     .flatMapIterable(list -> list) // you can bulk calls here
                                     .flatMap(id -> Observable.fromCallable(() -> api.call(id)).map(response -> Pair.of(id, response)));

    TestSubscriber<Object> ts1 = new TestSubscriber<>();
    TestSubscriber<Object> ts2 = new TestSubscriber<>();
    TestSubscriber<Object> ts3 = new TestSubscriber<>();
    TestSubscriber<Object> ts4 = new TestSubscriber<>();

    broker.filter(pair -> pair.id.equals("1")).take(1).map(pair -> pair.response).subscribe(ts1);
    broker.filter(pair -> pair.id.equals("2")).take(1).map(pair -> pair.response).subscribe(ts2);
    broker.filter(pair -> pair.id.equals("3")).take(1).map(pair -> pair.response).subscribe(ts3);
    broker.filter(pair -> pair.id.equals("4")).take(1).map(pair -> pair.response).subscribe(ts4);

    subject.onNext("1");
    subject.onNext("2");
    subject.onNext("3");

    scheduler.advanceTimeBy(1, TimeUnit.SECONDS);

    ts1.assertValue("resp1");
    ts2.assertValue("resp2");
    ts3.assertValue("resp3");
    ts4.assertNotCompleted();

    subject.onNext("4");
    scheduler.advanceTimeBy(1, TimeUnit.SECONDS);
    ts4.assertValue("resp4");
    ts4.assertCompleted();

如果要执行网络请求collapsin,您可能需要检查Hystrix:https://github.com/Netflix/Hystrix