所以我已经得到了这个示例代码，其中我有一个Dataset[Event]我希望根据动态计算的泛型类型的键进行分组。
import org.apache.spark.sql.{ Dataset, KeyValueGroupedDataset }
case class Event(id: Int, name: String)
trait Key
case class NameKey(name: String) extends Key
abstract class EventProc[K <: Key] {
def key(e: Event): K
def group(ds: Dataset[Event]): KeyValueGroupedDataset[K, Event] = {
import ds.sparkSession.implicits._
ds.groupByKey { e => key(e) }
}
}
class NameEventProc extends EventProc[NameKey] {
def key(e: Event): NameKey = NameKey(e.name)
}
我的想法是，我应该可以在new NameEventProc().group(ds)扩展EventProc的不同类上进行<console>:26: error: Unable to find encoder for type stored in a Dataset. Primitive types (Int, String, etc) and Product types (case classes) are supported by importing spark.implicits._ Support for serializing other types will be added in future releases.
ds.groupByKey { e => key(e) }
^
或类似操作。但是代码甚至没有编译并且因错误而失败。
K从我收集的内容来看，Spark无法理解/**
*@NApiVersion 2.x
*/
require(['N/search'], function(search) {
function loadSearchAndAddFilter() {
var searchObj = search.load({
id: 'customsearch_ca_export_detail_search__38'
});
var filters = searchObj.filters;
var filter = search.createFilter({
// create new filter here
});
filters.push(filter);
searchObj.filters = filters;
searchObj.run().each(function(result) {
// access search results here
return true;
});
}
loadSearchAndAddFilter();
});
的类型，因此无法使用适当的编码器。但我不知道如何解决这个问题。