流向Kafka到Elasticsearch整合

时间:2016-04-23 04:11:28

标签: elasticsearch apache-kafka flume

我将一些数据从Flume传递给Kafka。我的Flafka配置文件如下所示:

public static void main(String[] args) throws InterruptedException {
    System.setProperty("webdriver.chrome.driver", "C:\\Users\\rajnish\\Desktop\\chromedriver\\chromedriver_win32\\chromedriver.exe");
    WebDriver driver = new ChromeDriver();
    driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
    driver.manage().window().maximize();
    driver.get("https://slashdot.org/");

    WebElement pollHeader = driver.findElement(By.id("poll-title"));
    String HeadinIs = pollHeader.findElement(By.tagName("h2")).getText();
    System.out.println(HeadinIs);

    // getting the vote counter count first 
    String votecountBefore = driver.findElement(By.xpath("//*[@id='pollBooth']/footer/span/strong")).getText();
    System.out.println(votecountBefore);
    // calling polling 
    List<WebElement> allPollItems = driver.findElements(By.xpath("//*[@id='pollBooth']/label"));
    for(int i=0;i<allPollItems.size();i++){
        System.out.println("POll options : "+ allPollItems.get(i).getText());
        // select one of the option 
        if(allPollItems.get(i).getText().equals("Android")){
            allPollItems.get(i).findElement(By.tagName("input")).click();
        }
    }
    // now submit poll
    driver.findElement(By.xpath("//*[@class='poll-controls']/button")).click();

    // getting the vote counter count after
    String votecounterAfter = driver.findElement(By.className("totalVotes")).getText();
    System.out.println(votecounterAfter);
    String [] myVote = votecounterAfter.split(" ");
    if(!myVote[0].equals(votecountBefore)){
        System.out.println("Counter Incremented");
    }else{
        System.out.println("Counter not incremented");
    }

我使用kafka river plugin连接了Kafka和Elasticsearch。 FLume源正在向Kafka Consumer发送数据,而Elasticsearch期待来自Kafka Producer的数据。

有没有办法可以将数据从Flume推送到Kafka Producer,而不是直接转到Kafka Consumer,这样Elasticsearch可以读取数据?

有什么建议吗?感谢。

0 个答案:

没有答案