Question

我有一些HBase表有数百万行但只有几列。我想提取每个表的列名并将其存储在单独的文件中。做这个的最好方式是什么？感谢。

Answer 1

这应该保存本地（不是hdfs）上Hbase_table_columns.txt文件中的列名：

echo "scan 'table_name'" | $HBASE_HOME/bin/hbase shell | awk -F'=' '{print $2}' | awk -F ':' '{print $1}' > Hbase_table_columns.txt

这应该在控制台上保存列名：

echo "scan 'table_name'" | $HBASE_HOME/bin/hbase shell | awk -F'=' '{print $2}' | awk -F ':' '{print $1}'

这应该保存Hbase_table_columns.txt文件中的列名，并在控制台上打印：

echo "scan 'table_name'" | $HBASE_HOME/bin/hbase shell | awk -F'=' '{print $2}' | awk -F ':' '{print $1}' |tee Hbase_table_columns.txt

这应保存/打印column family:column name：

echo "scan 'table_name'" | $HBASE_HOME/bin/hbase shell | awk -F'=' '{print $2}'|tee Hbase_table_columns.txt

Answer 2

我提供了由package mytest; import com.usertest.; import java.io.IOException; import java.util.Date; import java.util.HashSet; import java.util.List; import java.util.Set; public class ListHbaseTablesAndColumns { public static void main(String[] args) { try { HbaseMetaData hbaseMetaData =new HbaseMetaData(); for(String hbaseTable:hbaseMetaData .getTableNames(".yourtables.*")){ for (String column : hbaseMetaData .getColumns(hbaseTable, 10000)) { System.out.println(hbaseTable + "," + column); } } } catch (IOException e) { e.printStackTrace(); } } }类公开的java Hbase客户端API，如下所示......

客户端就像

package com.usertest;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.*;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.PageFilter;

import java.io.IOException;
import java.util.*;
import java.util.regex.Pattern;

public class HbaseMetaData {
    private HBaseAdmin hBaseAdmin;
    private Configuration hBaseConfiguration;

    public HbaseMetaData () throws IOException {
        this.hBaseConfiguration = HBaseConfiguration.create();
        this.hBaseAdmin = new HBaseAdmin(hBaseConfiguration);
    }
/** get all Table names **/
    public List<String> getTableNames(String regex) throws IOException {
        Pattern pattern=Pattern.compile(regex);
        List<String> tableList = new ArrayList<String>();
        TableName[] tableNames=hBaseAdmin.listTableNames();
        for (TableName tableName:tableNames){
            if(pattern.matcher(tableName.toString()).find()){
                tableList.add(tableName.toString());
            }
        }
        return tableList;
    }
/** Get all columns **/
    public Set<String> getColumns(String hbaseTable) throws IOException {
        return getColumns(hbaseTable, 10000);
    }
/** get all columns from the table **/
    public Set<String> getColumns(String hbaseTable, int limitScan) throws IOException {
        Set<String> columnList = new TreeSet<String>();
        HTable hTable=new HTable(hBaseConfiguration, hbaseTable);
        Scan scan=new Scan();
        scan.setFilter(new PageFilter(limitScan));
        ResultScanner results = hTable.getScanner(scan);
        for(Result result:results){
            for(KeyValue keyValue:result.list()){
                columnList.add(
                        new String(keyValue.getFamily()) + ":" +
                                new String(keyValue.getQualifier())
                );
            }
        }
        return columnList;
    }
}

使用以下类来获取HbaseMetaData ..

{{1}}

Answer 3

以下内容将有助于获取有关特定键的列

scan 'namespace:tablename',{FILTER=>'KeyOnlyFilter()'}

Hbase - 如何在表中获取列名？

3 个答案: