为什么我得到＆＃34;分区值：[空行]＆＃34;读取文件时记录消息？

Question

我使用Spark SQL读取csv，我也收到很多这样的消息：

/* ---------- The EgComponent class ---------- */
;(function () {
  /* Create a new object from the prototype of HTMLInputElement. */
  var inputProto = Object.create(HTMLInputElement.prototype);

  /* Define some basic methods for the prototype of the component */
  Object.defineProperties(inputProto, {
    /* The function that clears a component instance. */
    onClear: {
      value: function() {
        this.value = "";
        this.style.position = "static";
        this.placeholder = "New Text";
      }
    },

    /* The function that sets the position of a component instance. */
    setPos: {
      value: function(x, y) {
        this.style.top = x + "px";
        this.style.left = y + "px";
        this.style.position = "relative";
      }
    },

    /* The function that sets the dimensions of a component instance. */
    setDimensions: {
      value: function(width, height) {
        this.style.width = width + "px";
        this.style.height = height + "px";
      }
    },

    /* The function that returns if the value of a component instance is uppercase. */
    caps: {
      value: function(input) {
        alert(/[A-Z]/.test(this.value) ? "Valid" : "Not Valid");
      }
    },

    /* The function that returns if the value of a component instance is lowercase. */
    lowerCaps: {
      value: function(input) {
        alert(/[a-z]/.test(this.value) ? "Valid" : "Not Valid");
      }
    },
  });

  /* Register the EgComponent in the browser. */
  window.EgComponent = document.registerElement("eg-input", {
    prototype: inputProto,
    extends: "input"
  });
})();



/* ---------- Instantiating an EgComponent ---------- */

/* Create a new instance of EgComponent. */
var egcomp = new EgComponent();

/* Set the 'placeholder' property. */
egcomp.placeholder = 20;

/* Set the 'pattern' property. */
egcomp.pattern = /[a-z]/;

/* Insert the component into the body of the document. */
document.body.appendChild(egcomp);

/* Log the component to the console. */
console.log(egcomp);

为什么说这是空行？分区真的是空的吗？

Answer 1

文件和从文件读取数据的Spark分区都不为空。

由于两件事，日志消息可能会有些混乱：

• 消息中的 partition 一词是指 Hive样式的分区，即，可以具有多个值的命名分区列。这样的分区可以是inferred from your directory structure，例如对于/path/to/partition/a=1/b=hello/c=3.14，它们分别是a，b和c，其值分别为1，hello和3.14。如果外部表已分区，它们也可以come from the Hive Metastore。
• 记录的分区值是wrapped in an InternalRow，不在集合中。

在您的情况下，目录结构是平坦的或不包含分区名称（例如/path/to/partition/1/hello/3.14），因此没有Hive样式的分区，因此您在消息中看到[empty row]