使用java通过Apache Ignite访问IGFS文件

时间:2016-10-04 07:48:55

标签: java apache hadoop ignite

我是Apache Ignite的新手,我需要通过java访问igfs文件。 直到现在我尝试的是:

import java.io.InputStream;
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteFileSystem;
import org.apache.ignite.Ignition;
import org.apache.ignite.igfs.IgfsPath;

public class Ignite_Spark {

    public static void main(String[] args) throws Exception {
        Ignite ignite = Ignition.start("/usr/local/hadoop/apache-ignite-hadoop-1.6.0-bin/config/default-config.xml");
        Ignite ignites = Ignition.ignite();

        IgniteFileSystem fs = ignites.fileSystem("igfs");

        IgfsPath workDir = new IgfsPath("igfs://localhost:8020/ignite/fs");
        fs.mkdirs(workDir);


        IgfsPath filePath = new IgfsPath(workDir, "ttypes.csv");

        try (InputStream in = fs.open(filePath)) {
            in.read();
        }
    }

}

但是当我运行这个iam时会遇到异常,例如:

  

引起:class org.apache.ignite.IgniteCheckedException:失败   实例化Spring XML应用程序上下文(确保使用所有类   在Spring配置中出现在CLASSPATH)   [springUrl =文件:/usr/local/hadoop/ignite/config/default-config.xml]

2 个答案:

答案 0 :(得分:0)

    Data inserting and retriving from igfs file system

    My csc File(test.csv)
    ////////////////////////////////
    123,111,wqw,131,11221,1212
    123,111,wqw,131,11221,1212
    eke,lelel,elelel,lelel
    w2,2,,2,2,,2,2,2,
    /////////////////////////////////


    Here configuration file is "D:/apache-ignite-fabric-1.5.0-b1-bin/examples/config/filesystem/example-igfs.xml"



    java code:
    //////////////////////////////////////////////////////////////////////////////////////start////////////////////////////////////////
    package org.apache.ignite.igfs;
    //package org.apache.ignite.igfs;
    import java.io.FileReader;

    import java.io.BufferedReader;
    import java.io.IOException;
    import java.io.InputStream;
    import java.io.InputStreamReader;
    import java.io.OutputStream;
    import org.apache.ignite.Ignite;
    import org.apache.ignite.Ignition;
    import org.apache.ignite.igfs.IgfsPath;

    public class test1 {



    public static void main(String[] args) throws IOException {
           // Here config file example-igfs.xml contanis igfs configuration
           Ignite ignite = Ignition.start("D:/apache-ignite-fabric-1.5.0-b1-bin/examples/config/filesystem/example-igfs.xml");
           org.apache.ignite.IgniteFileSystem fs = ignite.fileSystem("igfs");


           //creating a directory in igfs
           IgfsPath dir = new IgfsPath("/workDir");


           //creating a file 
           IgfsPath file = new IgfsPath(dir, "myFile1");

           //deleting files and directory to avoid file overwrite
           fs.delete(dir,true);
           fs.mkdirs(dir);


           //reading data from desktop("C:\\Users\\rb059\\Desktop\\test.csv") and writing into igfs file("MyFile1")

           try (OutputStream out = fs.create(file, true)) {
               String csvFile = "C:\\Users\\rb059\\Desktop\\test.csv";
               String line = "";
               String cvsSplitBy = ",";
               BufferedReader br = new BufferedReader(new FileReader(csvFile));

                    while ((line = br.readLine()) != null) {


                                    String s = line + "\n";

                                    out.write(s.getBytes());
                                    System.out.println("inserting data from" + csvFile + "    to   " + file +"  inserted record is " + s);

                    }


                }

           //getting data from ignite file 
           try (InputStream in = fs.open(file)) {

                    System.out.println("this is data retriving from" + file);
                    String line = "";
                    BufferedReader br = new BufferedReader(new InputStreamReader(in));

                           while ((line = br.readLine()) != null) {
                               System.out.println(line);
                           }


                }


       }



    }
    ///////////////////////////////////////////////java code end/////////////////////////////////////////




    output:

    [11:34:19]   /  _/ ___/ |/ /  _/_  __/ __/ 
    [11:34:19]  _/ // (7 7    // /  / / / _/   
    [11:34:19] /___/\___/_/|_/___/ /_/ /___/  
    [11:34:19] 
    [11:34:19] ver. 1.5.0-b1#20151202-sha1:062d440c
    [11:34:19] 2015 Copyright(C) Apache Software Foundation
    [11:34:19] 
    [11:34:19] Ignite documentation: http://ignite.apache.org
    [11:34:19] 
    [11:34:19] Quiet mode.
    [11:34:19]   ^-- Logging to file 'D:\apache-ignite-fabric-1.5.0-b1-bin\work\log\ignite-b474b4d7.0.log'
    [11:34:19]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false or "-v" to ignite.{sh|bat}
    [11:34:19] 
    [11:34:19] OS: Windows 7 6.1 amd64
    [11:34:19] VM information: Java(TM) SE Runtime Environment 1.8.0_111-b14 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 25.111-b14
    [11:34:19] Initial heap size is 128MB (should be no less than 512MB, use -Xms512m -Xmx512m).
    [11:34:21] Configured plugins:
    [11:34:21]   ^-- None
    [11:34:21] 
    [11:34:21] Security status [authentication=off, tls/ssl=off]
    [11:34:26] Attempting to start more nodes than physical RAM available on current host (this can cause significant slowdown)
    [11:34:30] Performance suggestions for grid  (fix if possible)
    [11:34:30] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true
    [11:34:30]   ^-- Disable peer class loading (set 'peerClassLoadingEnabled' to false)
    [11:34:30]   ^-- Use default binary marshaller (do not set 'marshaller' explicitly)
    [11:34:30] 
    [11:34:30] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat}
    [11:34:30] 
    [11:34:30] Ignite node started OK (id=b474b4d7)
    [11:34:30] Topology snapshot [ver=18, servers=18, clients=0, CPUs=4, heap=31.0GB]
    inserting data fromC:\Users\rb059\Desktop\test.csv    to   /workDir/myFile1  inserted record is 123,111,wqw,131,11221,1212

    inserting data fromC:\Users\rb059\Desktop\test.csv    to   /workDir/myFile1  inserted record is 123,111,wqw,131,11221,1212

    inserting data fromC:\Users\rb059\Desktop\test.csv    to   /workDir/myFile1  inserted record is 123,111,wqw,131,11221,1212

    inserting data fromC:\Users\rb059\Desktop\test.csv    to   /workDir/myFile1  inserted record is 123,111,wqw,131,11221,1212

    inserting data fromC:\Users\rb059\Desktop\test.csv    to   /workDir/myFile1  inserted record is 123,111,wqw,131,11221,1212

    inserting data fromC:\Users\rb059\Desktop\test.csv    to   /workDir/myFile1  inserted record is 123,111,wqw,131,11221,1212

    inserting data fromC:\Users\rb059\Desktop\test.csv    to   /workDir/myFile1  inserted record is eke,lelel,elelel,lelel

    inserting data fromC:\Users\rb059\Desktop\test.csv    to   /workDir/myFile1  inserted record is w2,2,,2,2,,2,2,2,

    this is data retriving from/workDir/myFile1
    123,111,wqw,131,11221,1212
    123,111,wqw,131,11221,1212
    123,111,wqw,131,11221,1212
    123,111,wqw,131,11221,1212
    123,111,wqw,131,11221,1212
    123,111,wqw,131,11221,1212
    eke,lelel,elelel,lelel
    w2,2,,2,2,,2,2,2,
    [11:34:31] New version is available at ignite.apache.org: 1.7.0

答案 1 :(得分:0)

我相信您可能需要包含ignite-spring mvn依赖项,以便使用spring .xml文件启动点燃。