tsv上的“ BCP复制失败”(没有更多错误消息信息)

时间:2018-08-20 19:35:13

标签: sql-server bcp

尝试在Linux CentOS7上使用BCP实用程序将tsv文件复制到远程MSSQL Server DB(在Windows Server 2012计算机中),得到提示错误消息

  

开始复制...

     

BCP复制失败

有问题的BCP命令是

%config InlineBackend.figure_format = 'retina'

版本是

/opt/mssql-tools/bin/bcp "$TABLE" in $DATAFILES/part-m-00000 \
        -S -D $SERVER_DSN \
        -U $USER \
        -P $PASSWORD \
        -d $DB \
        -c \
        -t "\t" \
        -e /home/me/projects/bcp-experiments/examples/recent-err-rows.log

目标表具有如下所示的架构(在这里无法真正想到格式化数据的好方法,这是一个非常宽的表)

[me@mapr001 examples]$ bcp -v
BCP - Bulk Copy Program for Microsoft SQL Server.
Copyright (C) Microsoft Corporation. All Rights Reserved.
Version: 17.2.0000.1

和tsv文件中的模拟样本行看起来像

Column_name Type    Computed    Length  Prec    Scale   Nullable    TrimTrailingBlanks  FixedLenNullInSource    Collation
TDL_ID  numeric no  9   18      0       yes (n/a)   (n/a)   NULL
DETAIL_TYPE smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
TYPE    smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
POST_DATE   datetime    no  8                   yes (n/a)   (n/a)   NULL
ORIG_POST_DATE  datetime    no  8                   yes (n/a)   (n/a)   NULL
ORIG_SERVICE_DATE   datetime    no  8                   yes (n/a)   (n/a)   NULL
PERIOD  varchar no  6                   yes no  yes SQL_Latin1_General_CP1_CI_AS
TX_ID   numeric no  9   18      0       yes (n/a)   (n/a)   NULL
TRAN_TYPE   smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
ALLOWED_AMOUNT  numeric no  9   12      2       yes (n/a)   (n/a)   NULL
CHARGE_SLIP_NUMBER  varchar no  15                  yes no  yes SQL_Latin1_General_CP1_CI_AS
TYPE_OF_SERVICE smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
MATCH_TRX_ID    numeric no  9   18      0       yes (n/a)   (n/a)   NULL
MATCH_TX_TYPE   smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
MATCH_PROC_ID   numeric no  9   18      0       yes (n/a)   (n/a)   NULL
MATCH_PROV_ID   varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
MATCH_LOC_ID    numeric no  9   18      0       yes (n/a)   (n/a)   NULL
ACCOUNT_ID  numeric no  9   18      0       yes (n/a)   (n/a)   NULL
PAT_ID  varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
AMOUNT  numeric no  9   12      2       yes (n/a)   (n/a)   NULL
PATIENT_AMOUNT  numeric no  9   12      2       yes (n/a)   (n/a)   NULL
INSURANCE_AMOUNT    numeric no  9   12      2       yes (n/a)   (n/a)   NULL
RELATIVE_VALUE_UNIT numeric no  9   12      2       yes (n/a)   (n/a)   NULL
CUR_CVG_ID  numeric no  9   18      0       yes (n/a)   (n/a)   NULL
CUR_PLAN_ID numeric no  9   18      0       yes (n/a)   (n/a)   NULL
CUR_PAYOR_ID    numeric no  9   18      0       yes (n/a)   (n/a)   NULL
CUR_FIN_CLASS   varchar no  66                  yes no  yes SQL_Latin1_General_CP1_CI_AS
PERFORMING_PROV_ID  varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
BILLING_PROVIDER_ID varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
ORIGINAL_CVG_ID numeric no  9   18      0       yes (n/a)   (n/a)   NULL
ORIGINAL_PLAN_ID    numeric no  9   18      0       yes (n/a)   (n/a)   NULL
ORIGINAL_PAYOR_ID   numeric no  9   18      0       yes (n/a)   (n/a)   NULL
ORIGINAL_FIN_CLASS  varchar no  66                  yes no  yes SQL_Latin1_General_CP1_CI_AS
PROC_ID numeric no  9   18      0       yes (n/a)   (n/a)   NULL
PROCEDURE_QUANTITY  numeric no  5   8       3       yes (n/a)   (n/a)   NULL
CPT_CODE    varchar no  20                  yes no  yes SQL_Latin1_General_CP1_CI_AS
MODIFIER_ONE    varchar no  254                 yes no  yes SQL_Latin1_General_CP1_CI_AS
MODIFIER_TWO    varchar no  254                 yes no  yes SQL_Latin1_General_CP1_CI_AS
MODIFIER_THREE  varchar no  254                 yes no  yes SQL_Latin1_General_CP1_CI_AS
MODIFIER_FOUR   varchar no  254                 yes no  yes SQL_Latin1_General_CP1_CI_AS
DX_ONE_ID   numeric no  9   18      0       yes (n/a)   (n/a)   NULL
DX_TWO_ID   numeric no  9   18      0       yes (n/a)   (n/a)   NULL
DX_THREE_ID numeric no  9   18      0       yes (n/a)   (n/a)   NULL
DX_FOUR_ID  numeric no  9   18      0       yes (n/a)   (n/a)   NULL
DX_FIVE_ID  numeric no  9   18      0       yes (n/a)   (n/a)   NULL
DX_SIX_ID   numeric no  9   18      0       yes (n/a)   (n/a)   NULL
SERV_AREA_ID    numeric no  9   18      0       yes (n/a)   (n/a)   NULL
LOC_ID  numeric no  9   18      0       yes (n/a)   (n/a)   NULL
DEPT_ID numeric no  9   18      0       yes (n/a)   (n/a)   NULL
POS_ID  numeric no  9   18      0       yes (n/a)   (n/a)   NULL
CUSTOMER_ITEM_ONE   varchar no  20                  yes no  yes SQL_Latin1_General_CP1_CI_AS
CUSTOMER_ITEM_TWO   varchar no  20                  yes no  yes SQL_Latin1_General_CP1_CI_AS
CUSTOMER_ITEM_THREE varchar no  20                  yes no  yes SQL_Latin1_General_CP1_CI_AS
CUSTOMER_ITEM_FOUR  varchar no  20                  yes no  yes SQL_Latin1_General_CP1_CI_AS
CUSTOMER_ITEM_FIVE  varchar no  20                  yes no  yes SQL_Latin1_General_CP1_CI_AS
BILL_CLAIM_AMOUNT   numeric no  9   12      2       yes (n/a)   (n/a)   NULL
INVOICE_NUMBER  varchar no  20                  yes no  yes SQL_Latin1_General_CP1_CI_AS
ABF_FORM_ID varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
CLM_CLAIM_ID    numeric no  9   18      0       yes (n/a)   (n/a)   NULL
BILL_HOLD_AMOUNT    numeric no  9   12      2       yes (n/a)   (n/a)   NULL
PAT_AGING_DAYS  int no  4   10      0       yes (n/a)   (n/a)   NULL
INS_AGING_DAYS  int no  4   10      0       yes (n/a)   (n/a)   NULL
ACTION_CVG_ID   numeric no  9   18      0       yes (n/a)   (n/a)   NULL
ACTION_PLAN_ID  numeric no  9   18      0       yes (n/a)   (n/a)   NULL
ACTION_PAYOR_ID numeric no  9   18      0       yes (n/a)   (n/a)   NULL
ACTION_FIN_CLASS    varchar no  66                  yes no  yes SQL_Latin1_General_CP1_CI_AS
DEBIT_GL_NUM    varchar no  20                  yes no  yes SQL_Latin1_General_CP1_CI_AS
CREDIT_GL_NUM   varchar no  20                  yes no  yes SQL_Latin1_General_CP1_CI_AS
REASON_CODE_ID  varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
USER_ID varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
COPAY_INDICATOR smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
TX_NUM  smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
ORIG_PRICE  numeric no  9   12      2       yes (n/a)   (n/a)   NULL
PRICE_CONTRACT_ID   numeric no  9   18      0       yes (n/a)   (n/a)   NULL
CONTRACT_DISCOUNT   numeric no  9   12      2       yes (n/a)   (n/a)   NULL
INT_PAT_ID  varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
ORIG_AMT    numeric no  9   12      2       yes (n/a)   (n/a)   NULL
PAT_TYPE_C  smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
PRIM_CARE_PROV  varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
REFERRAL_ID numeric no  9   18      0       yes (n/a)   (n/a)   NULL
REFERRAL_SOURCE_ID  varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
PRIM_LOCATION   numeric no  9   18      0       yes (n/a)   (n/a)   NULL
RVU_WORK    numeric no  9   12      2       yes (n/a)   (n/a)   NULL
RVU_OVERHEAD    numeric no  9   12      2       yes (n/a)   (n/a)   NULL
RVU_MALPRACTICE numeric no  9   12      2       yes (n/a)   (n/a)   NULL
POSTING_BATCH_NUM   varchar no  12                  yes no  yes SQL_Latin1_General_CP1_CI_AS
VOID_USER_ID    varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
R_NEW_CHG_TX_ID numeric no  9   18      0       yes (n/a)   (n/a)   NULL
R_ORIG_CHG_TX_ID    numeric no  9   18      0       yes (n/a)   (n/a)   NULL
R_ORIG_POST_DATE    datetime    no  8                   yes (n/a)   (n/a)   NULL
R_ORIG_AMOUNT   numeric no  9   12      2       yes (n/a)   (n/a)   NULL
R_ORIG_PAT_AMOUNT   numeric no  9   12      2       yes (n/a)   (n/a)   NULL
R_ORIG_FIN_CLASS    varchar no  66                  yes no  yes SQL_Latin1_General_CP1_CI_AS
R_ORIG_PAYOR_ID numeric no  9   18      0       yes (n/a)   (n/a)   NULL
R_ORIG_PLAN_ID  numeric no  9   18      0       yes (n/a)   (n/a)   NULL
R_ORIG_CVG_ID   numeric no  9   18      0       yes (n/a)   (n/a)   NULL
REFERENCE_NUMBER    varchar no  255                 yes no  yes SQL_Latin1_General_CP1_CI_AS
PAYMENT_SOURCE_C    smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
TX_COMMENT  varchar no  255                 yes no  yes SQL_Latin1_General_CP1_CI_AS
ANESTHESIA_TYPE_C   varchar no  66                  yes no  yes SQL_Latin1_General_CP1_CI_AS
EMERGENCY_STATUS_C  smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
PHYSICAL_STATUS_C   varchar no  66                  yes no  yes SQL_Latin1_General_CP1_CI_AS
CONCURRENCY_CALC    numeric no  5   6       2       yes (n/a)   (n/a)   NULL
CONCURRENCY_OVER    numeric no  5   6       2       yes (n/a)   (n/a)   NULL
BASE_UNITS  numeric no  5   6       2       yes (n/a)   (n/a)   NULL
TIMED_UNITS numeric no  5   6       2       yes (n/a)   (n/a)   NULL
PHY_STATUS_UNITS    numeric no  5   6       2       yes (n/a)   (n/a)   NULL
EMERGENCY_UNITS numeric no  5   6       2       yes (n/a)   (n/a)   NULL
AGE_UNITS   numeric no  5   6       2       yes (n/a)   (n/a)   NULL
MED_SUP_MOD_OVER_C  varchar no  1                   yes no  yes SQL_Latin1_General_CP1_CI_AS
ANESTHESIA_CERT varchar no  254                 yes no  yes SQL_Latin1_General_CP1_CI_AS
TECH_CHARGE_FLAG    varchar no  1                   yes no  yes SQL_Latin1_General_CP1_CI_AS
NIA_OUTCOME_CODE_C  smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
PROC_MED_NEC_FLAG   varchar no  1                   yes no  yes SQL_Latin1_General_CP1_CI_AS
CRD_CHARGE_SLIP_NO  varchar no  15                  yes no  yes SQL_Latin1_General_CP1_CI_AS
VISIT_NUMBER    varchar no  50                  yes no  yes SQL_Latin1_General_CP1_CI_AS
CRNA_CHARGE_ID  numeric no  9   18      0       yes (n/a)   (n/a)   NULL
ORIG_ANES_CHG_ID    numeric no  9   18      0       yes (n/a)   (n/a)   NULL
CONTESTED_FLAG_YN   varchar no  1                   yes no  yes SQL_Latin1_General_CP1_CI_AS
PAT_ENC_CSN_ID  numeric no  9   18      0       yes (n/a)   (n/a)   NULL
ACTION_USER_ID  varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
TDL_EXTRACT_DATE    datetime    no  8                   yes (n/a)   (n/a)   NULL
INSURANCE_ID    varchar no  17                  yes no  yes SQL_Latin1_General_CP1_CI_AS
SPECIMEN_ID varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
TEST_ID varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
LAB_ID  varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
PANEL_ID    varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
PROV_SPECIALTY_C    varchar no  66                  yes no  yes SQL_Latin1_General_CP1_CI_AS
DIST_USER_ID    varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
UNDIST_USER_ID  varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
PROC_ALT_IDFR   varchar no  50                  yes no  yes SQL_Latin1_General_CP1_CI_AS
SERV_PROV_SUFFIX    varchar no  30                  yes no  yes SQL_Latin1_General_CP1_CI_AS
SERV_PROV_ATTR_1_C  smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
SERV_PROV_ATTR_2_C  smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
SERV_PROV_ATTR_3_C  smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
BILL_PROV_SUFFIX    varchar no  30                  yes no  yes SQL_Latin1_General_CP1_CI_AS
BILL_PROV_ATTR_1_C  smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
BILL_PROV_ATTR_2_C  smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
BILL_PROV_ATTR_3_C  smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
VERIFIED    varchar no  1                   yes no  yes SQL_Latin1_General_CP1_CI_AS
RVU_PROC_UNITS  numeric no  9   12      2       yes (n/a)   (n/a)   NULL
CASE_STATE_C    smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
CASE_RATE_ID    varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
CASE_TX_STATE_C smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
MATCH_RFD_TX_ID numeric no  9   18      0       yes (n/a)   (n/a)   NULL
MATCH_RFD_TX_AMT    numeric no  9   12      2       yes (n/a)   (n/a)   NULL
CUR_ACTN_PAYOR_ID   numeric no  9   18      0       yes (n/a)   (n/a)   NULL
ORIG_PST_BATCH_NUM  varchar no  12                  yes no  yes SQL_Latin1_General_CP1_CI_AS
ORIG_REF_NUM    varchar no  255                 yes no  yes SQL_Latin1_General_CP1_CI_AS
ORIG_PAY_SOURCE_C   smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
TX_DESC_OVERRIDE    varchar no  255                 yes no  yes SQL_Latin1_General_CP1_CI_AS
TRANSFERRED_TX_ID   numeric no  9   18      0       yes (n/a)   (n/a)   NULL
IS_TRANFERED_YN varchar no  1                   yes no  yes SQL_Latin1_General_CP1_CI_AS
CM_ORIG_DPLY_ID varchar no  25                  yes no  yes SQL_Latin1_General_CP1_CI_AS
CM_ORIG_ETR_CID numeric no  9   18      0       yes (n/a)   (n/a)   NULL
CM_ORIG_ACCT_LINE   int no  4   10      0       yes (n/a)   (n/a)   NULL
CM_HOME_LINK    numeric no  9   18      0       yes (n/a)   (n/a)   NULL
HOMED_INDICATOR_YN  varchar no  1                   yes no  yes SQL_Latin1_General_CP1_CI_AS
CM_PHY_OWNER_ID varchar no  25                  yes no  yes SQL_Latin1_General_CP1_CI_AS
CM_LOG_OWNER_ID varchar no  25                  yes no  yes SQL_Latin1_General_CP1_CI_AS
CM_ORIG_ACCT_ID numeric no  9   18      0       yes (n/a)   (n/a)   NULL
CM_HOMED_INSTANT    datetime    no  8                   yes (n/a)   (n/a)   NULL
CH_EDIT_SEQUENCE    int no  4   10      0       yes (n/a)   (n/a)   NULL
CH_EDIT_CONTEXTS    varchar no  254                 yes no  yes SQL_Latin1_General_CP1_CI_AS
ACTION_AMOUNT   numeric no  9   18      2       yes (n/a)   (n/a)   NULL
HOM_CLARITY_FLG_YN  varchar no  254                 yes no  yes SQL_Latin1_General_CP1_CI_AS
CSH_ID  numeric no  9   18      0       yes (n/a)   (n/a)   NULL
ORIG_HTR_ID numeric no  9   18      0       yes (n/a)   (n/a)   NULL
WORKSTATION_ID  varchar no  18                  yes no  yes SQL_Latin1_General_CP1_CI_AS
POS_SESSID  varchar no  100                 yes no  yes SQL_Latin1_General_CP1_CI_AS
POS_TXID    varchar no  100                 yes no  yes SQL_Latin1_General_CP1_CI_AS
VOID_POS_SESSID varchar no  90                  yes no  yes SQL_Latin1_General_CP1_CI_AS
VOID_POS_TXID   varchar no  90                  yes no  yes SQL_Latin1_General_CP1_CI_AS
POS_TX_LINE int no  4   10      0       yes (n/a)   (n/a)   NULL
VOID_POS_TX_LINE    int no  4   10      0       yes (n/a)   (n/a)   NULL
HSP_ACCOUNT_ID  numeric no  9   18      0       yes (n/a)   (n/a)   NULL
MATCH_PAYOR_ID  numeric no  9   18      0       yes (n/a)   (n/a)   NULL
ACTION_MATCH_LINE   smallint    no  2   5       0       yes (n/a)   (n/a)   NULL
DYNAMIC_ALLOWED_AMT numeric no  9   12      2       yes (n/a)   (n/a)   NULL
ACTIVE_AR_AMOUNT    numeric no  9   18      2       yes (n/a)   (n/a)   NULL
EXTERNAL_AR_AMOUNT  numeric no  9   18      2       yes (n/a)   (n/a)   NULL
BAD_DEBT_AR_AMOUNT  numeric no  9   18      2       yes (n/a)   (n/a)   NULL
BILL_AREA_ID    numeric no  9   18      0       yes (n/a)   (n/a)   NULL
FIN_DIV_ID  numeric no  9   18      0       yes (n/a)   (n/a)   NULL
FIN_SUBDIV_ID   numeric no  9   18      0       yes (n/a)   (n/a)   NULL
etl_date    datetime    no  8                   yes (n/a)   (n/a)   NULL

尝试使用不同表和数据文件起作用的命令,此时错误消息太短,无法给我更多调试信息(从未使用过) BCP实用工具之前)。如果有人有使用此工具的经验,将不胜感激任何调试建议或如何解决此问题。

1 个答案:

答案 0 :(得分:3)

进一步调试后,发现使用BCP将tsv文件复制到MSSQL DB时,tsv中的文字'null'值将作为字符串导入,并且(我认为)这是导致错误的原因,因为目标表中的某些字段期望数值等(在使用BCP之前,正在使用处理此隐藏方法的进程)。将命令更改为表单

    echo "Replacing null literal values with empty chars"
    NULL_WITH_TAB="null\t"
    TAB="\t"
    sed -i -e "s/$NULL_WITH_TAB/$TAB/g" $filename

    /opt/mssql-tools/bin/bcp "$TABLE" in "$filename" \
        $TO_SERVER_ODBCDSN \
        -U $USER \
        -P $PASSWORD \
        -d $DB \
        $RECOMMEDED_IMPORT_MODE \
        -t "\t" \
        -e /home/me/projects/bcp-experiments/examples/recent-err-row.log

解决了该问题。如果有人知道将来如何从BCP强制发出更多详细的错误消息,或者还有哪些其他类型的问题可以引发这种模糊的错误,请告诉我。