我有下表:
+----------+----------+--------------------+--------------------+--------------------+--------------------+--------------------+
| _created| _updated| name| description| indication| name| patents_patent|
+----------+----------+--------------------+--------------------+--------------------+--------------------+--------------------+
|2005-06-13|2016-08-17| Lepirudin|Lepirudin is iden...|For the treatment...| Lepirudin|1|
|2005-06-13|2017-04-27| Cetuximab|Cetuximab is an e...|Cetuximab, used i...| Cetuximab|1|
|2005-06-13|2017-06-14| Dornase alfa|Dornase alfa is a...|Used as adjunct t...| Dornase alfa|1|
|2005-06-13|2017-03-10| Etanercept|Dimeric fusion pr...|Etanercept is ind...| Etanercept|1|
|2005-06-13|2017-07-06| Bivalirudin|Bivalirudin is a ...|For treatment of ...| Bivalirudin|1|
|2005-06-13|2017-07-05| Leuprolide|Leuprolide belong...|For treatment of ...| Leuprolide|1|
|2005-06-13|2017-06-16|Peginterferon alf...|Peginterferon alf...|Peginterferon alf...|Peginterferon alf...|1|
|
理想情况下,我需要派生2个表:
table_one我将过滤掉patent_patent不为NULL的表格,并将专利专利中的字符串替换为1:
+----------+----------+--------------------+--------------------+--------------------+--------------------+--------------------+
| _created| _updated| name| description| indication| name| patents_patent|
+----------+----------+--------------------+--------------------+--------------------+--------------------+------------------
|2005-06-13|2016-08-17| Denileukin diftitox|A recombinant DNA...|For treatment of ...| Denileukin diftitox| 0|
|2005-06-13|2017-06-08| Alteplase|Human tissue plas...|For management of...| Alteplase| 0|
|2005-06-13|2016-12-08| Sermorelin|Sermorelin acetat...|For the treatment...| Sermorelin| 0|
|2005-06-13|2016-08-17| Interferon alfa-n1|Purified, natural...|For treatment of ...| Interferon alfa-n1| 0|
table_two =过滤掉个表,其中patents_patent为null并将null替换为0
from pyspark.sql.functions import col, expr, when
data = table.where(col("patents_patent").isNull())
data = table.filter("patents_patent is not NULL")
我试过了:
我试过了:
root
|-- _created: string (nullable = true)
|-- _updated: string (nullable = true)
|-- name: string (nullable = true)
|-- description: string (nullable = true)
|-- indication: string (nullable = true)
|-- patents_patent: string (nullable = true)
结果错误或空:!
@media (min-width: 1920px) and (max-width: 2560px) {
//insert styles here...
}
感谢您的帮助!
答案 0 :(得分:0)
table_one我将过滤掉patent_patent不为NULL的表格,并将专利专利中的字符串替换为1:
对于第一种情况,你应该做
data_not_null = table.filter((table['patents_patent'] != "NULL")).withColumn('patents_patent', f.lit("1"))
table_two =过滤掉个表,其中patents_patent为null并将null替换为0
对于第二个,您应该执行以下操作
data_null = table.where(f.col("patents_patent").isNull() | (table['patents_patent'] == "NULL")).withColumn('patents_patent', f.lit("0"))
我将导入为
from pyspark.sql import functions as f
当然f.col("patents_patent")
和table['patents_patent']
表示相同