在Scala中过滤空值

Question

我有一个hive表，在某些列中有空值和空值。我表中的空值表示为\N。 我试图在Scala中选择列a不为null的行。 我尝试过使用

一个。 if (a.equals("\\N"))

湾if (!Option(a).getOrElse("").isEmpty)

℃。使用length(trim(a))

这些选项都不适用于我的代码，过滤器不起作用或排除所有行。

有没有人有任何建议？

Answer 1

According to https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Transform, Hive's \N is a string, that is "\N", however in Java and hence in Scala that's represented as "\N" which can be matched using {scala.Predef.String = java.lang.String}.matches("\\N") as in the following test:

"dependencies": {
    "jquery": ">=1.7"
  },

If that does not work for you please provide information about how you are accessing the hive table and the hive version so I can attempt to replicate an error.