Question

我有一个要投射的spark DataFrame中的列['col1','col2','col3']的列表。我尝试了以下操作，但似乎无法正常工作。我确实尝试过将数据从DECIMAL（11,3）转换为DECIMAL（3,2）和INT

create table database_nm.test_kou
(
YEAR_MNTH_NBR        INT,
PARTN_ID             TINYINT,
EMP_NAME             STRING,
FST_AMT              DECIMAL(11,3),
SND_AMT              DECIMAL(11,3),
FST_PCT              DECIMAL(11,3),
SND_PCT              DECIMAL(11,3),
TRD_AMT              DECIMAL(11,3),
TRD_PCT              DECIMAL(11,3),
FTH_AMT              DECIMAL(11,3)
);

INSERT INTO database_nm.test_kou VALUES 
    (201812,1,'Kou',11.11,12.12,13.13,14.14,15.15,16.16,17.17);

INSERT INTO database_nm.test_kou VALUES
    (201812,1,'Cha',21.21,22.22,23.23,24.24,25.25,26.26,27.27);

INSERT INTO database_nm.test_kou VALUES 
    (201812,1,'Cha',21.21,22.22,23.23,24.24,25.25,100.00,27.27);

我的pyspark代码：

col_unfmt=sql("select * from database_nm.test_kou")
colWithAmt = [column for column in col_unfmt.columns if '_amt' in column]
colWithPct = [column for column in col_unfmt.columns if '_pct' in column]
colWithRemainings = [
    column for column in col_unfmt.columns if column not in colWithPct+colWithAmt
]

selectAmtColsExpr = [lit(y).cast("int").alias(y) for y in colWithAmt]
selectPctColsExpr = [(lit(z)/lit(100)).cast("decimal(3,2)").alias(z) for z in colWithPct]
selectRemainingColsExpr = colWithRemainings

col_unfmt.select(selectRemainingColsExpr+selectPctColsExpr+selectAmtColsExpr).show()
+-------------+--------+--------+-------+-------+-------+-------+-------+-------+-------+
|year_mnth_nbr|partn_id|emp_name|fst_pct|snd_pct|trd_pct|fst_amt|snd_amt|trd_amt|fth_amt|
+-------------+--------+--------+-------+-------+-------+-------+-------+-------+-------+
|       201812|       1|     Kou|   null|   null|   null|   null|   null|   null|   null|
|       201812|       1|     Cha|   null|   null|   null|   null|   null|   null|   null|
+-------------+--------+--------+-------+-------+-------+-------+-------+-------+-------+

Answer 1

正如Pault上面建议的那样，使用col而不是lit看起来可以解决问题。

@Override
public void onSignalStrengthsChanged(SignalStrength signalStrength) {
    super.onSignalStrengthsChanged(signalStrength);

    if (ContextCompat.checkSelfPermission(MainActivity.mContext, android.Manifest.permission.ACCESS_COARSE_LOCATION) != PackageManager.PERMISSION_GRANTED) {
        ActivityCompat.requestPermissions(MainActivity.mActivity, new String[]{Manifest.permission.ACCESS_COARSE_LOCATION}, MainActivity.MY_REQUEST_CODE);
    } else {
        GetLteInfo.listCellInfo = GetLteInfo.tManager.getAllCellInfo();
        System.out.println("access ok");
    }


    if (listCellInfo != null) {
        for (CellInfo a_Info : listCellInfo) {
            if (CellInfoLte.class.isInstance(a_Info)) {
                CellInfoLte cellInfoLte = (CellInfoLte) a_Info;
                CIL = cellInfoLte.getCellIdentity();
                CSS = cellInfoLte.getCellSignalStrength();
                if (a_Info.isRegistered()) {
                    CellIdentity = CIL.getCi();
                    MobileCountryCode = CIL.getMcc();
                    MobileNetworkCode = CIL.getMnc();
                    CellId = CIL.getPci();
                    TrackingAreaCode = CIL.getTac();
                    AsuLevel = CSS.getAsuLevel();
                    SignalStrengthdBm = CSS.getDbm();
                    if(Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
                        CQI = CSS.getCqi();
                        SNR = CSS.getRssnr();
                        RSRP = CSS.getRsrp();
                        RSRQ = CSS.getRsrq();
                    }
                    pushToFirebase(counter, generatedKey);
                    counter++;
                }
            }
        }
    }
}

数据类型转换Spark数据框架列-pyspark

1 个答案: