使用csvJDBC和mapreduce

时间:2016-09-01 14:26:11

标签: java hadoop mapreduce csvjdbc

我想使用带有MapReduce的csvJDBC在CSV文件上执行选择查询。

我在Map-Only功能中使用以下代码,但输出文件为空:

         public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {

        // Load the driver.
            try {
                Class.forName("org.relique.jdbc.csv.CsvDriver");
                // Create a connection. The first command line parameter is
                // the directory containing the .csv files.
               String csv1 = csv.substring(0, csv.indexOf("file.csv"));
                Connection conn = DriverManager.getConnection("jdbc:relique:csv:" + csv1);

                // Create a Statement object to execute the query with.
                // A Statement is not thread-safe.
                Statement stmt = conn.createStatement();


                ResultSet results = stmt.executeQuery(input);

                // Dump out the results to a CSV file with the same format
                // using CsvJdbc helper function

                while (results.next())
                  {

                      out.set(results.getString("id")); 
                      context.write( out, new IntWritable(1));
                  }

                // Clean up
                conn.close();
            } catch (ClassNotFoundException | SQLException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }

0 个答案:

没有答案