Spark JDBC SQLException

时间:2016-01-27 15:13:47

标签: java jdbc apache-spark

我一直在获得SQLException,但我怀疑这不是问题所在。 表是:

<FrameLayout
    android:id="@+id/frame_test"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:background="?android:selectableItemBackground"
    android:clickable="true">

    <TextView
        android:id="@+id/test"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:background="@drawable/rectangle_empty_white"
        android:padding="8dp"
        android:text="@string/test"
        android:textAllCaps="true"
        android:textColor="@color/white"
        android:textSize="14sp" />
</FrameLayout>

插入陈述:

create table person (first varchar(30) DEFAULT NULL, last varchar(30)      DEFAULT NULL, gender char(1) DEFAULT NULL, age tinyint(4) DEFAULT NULL);

Spark代码:

insert into person values('Barack','Obama','M',54);
insert into person values('Hillary','Clinton','f',34);

错误:

public static void main(String[] args) {
        SparkConf conf = new SparkConf().setAppName("Stackoverflow")
                            .setMaster("local[4]");
        JavaSparkContext sc = new JavaSparkContext(conf);
        SQLContext sqlContext = new SQLContext(sc);
        Map<String, String> options = new HashMap<>();
        options.put("url", "jdbc:mariadb://localhost:3306/persondb");
        options.put("user", "user");
        options.put("password", "password333");
        options.put("dbtable", "(select * from person where gender = 'M') as someone");

        DataFrame jdbcDF = sqlContext.read().format("jdbc"). options(options).load();
        jdbcDF.show();

我尝试更改表格stmt(@jmj):

Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID 0, localhost): java.sql.SQLException: Out of range value for column 'age' : value age is not in Integer range

然后它适用于某些查询,但主要是它给出了:

create table person (first varchar(30) DEFAULT NULL, last varchar(30)      DEFAULT NULL, gender char(1) DEFAULT NULL, age int DEFAULT NULL);

1 个答案:

答案 0 :(得分:2)

问题的根源是使用 TINYINT(4)来存储年龄。

INT 来自 TINYINT(4)更改类型。

要了解检查此post的原因。

希望这有助于。