尝试在Linux CentOS7上使用BCP实用程序将tsv文件复制到远程MSSQL Server DB(在Windows Server 2012计算机中),得到提示错误消息
开始复制...
BCP复制失败
有问题的BCP命令是
%config InlineBackend.figure_format = 'retina'
版本是
/opt/mssql-tools/bin/bcp "$TABLE" in $DATAFILES/part-m-00000 \
-S -D $SERVER_DSN \
-U $USER \
-P $PASSWORD \
-d $DB \
-c \
-t "\t" \
-e /home/me/projects/bcp-experiments/examples/recent-err-rows.log
目标表具有如下所示的架构(在这里无法真正想到格式化数据的好方法,这是一个非常宽的表)
[me@mapr001 examples]$ bcp -v
BCP - Bulk Copy Program for Microsoft SQL Server.
Copyright (C) Microsoft Corporation. All Rights Reserved.
Version: 17.2.0000.1
和tsv文件中的模拟样本行看起来像
Column_name Type Computed Length Prec Scale Nullable TrimTrailingBlanks FixedLenNullInSource Collation
TDL_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
DETAIL_TYPE smallint no 2 5 0 yes (n/a) (n/a) NULL
TYPE smallint no 2 5 0 yes (n/a) (n/a) NULL
POST_DATE datetime no 8 yes (n/a) (n/a) NULL
ORIG_POST_DATE datetime no 8 yes (n/a) (n/a) NULL
ORIG_SERVICE_DATE datetime no 8 yes (n/a) (n/a) NULL
PERIOD varchar no 6 yes no yes SQL_Latin1_General_CP1_CI_AS
TX_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
TRAN_TYPE smallint no 2 5 0 yes (n/a) (n/a) NULL
ALLOWED_AMOUNT numeric no 9 12 2 yes (n/a) (n/a) NULL
CHARGE_SLIP_NUMBER varchar no 15 yes no yes SQL_Latin1_General_CP1_CI_AS
TYPE_OF_SERVICE smallint no 2 5 0 yes (n/a) (n/a) NULL
MATCH_TRX_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
MATCH_TX_TYPE smallint no 2 5 0 yes (n/a) (n/a) NULL
MATCH_PROC_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
MATCH_PROV_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
MATCH_LOC_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ACCOUNT_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
PAT_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
AMOUNT numeric no 9 12 2 yes (n/a) (n/a) NULL
PATIENT_AMOUNT numeric no 9 12 2 yes (n/a) (n/a) NULL
INSURANCE_AMOUNT numeric no 9 12 2 yes (n/a) (n/a) NULL
RELATIVE_VALUE_UNIT numeric no 9 12 2 yes (n/a) (n/a) NULL
CUR_CVG_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
CUR_PLAN_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
CUR_PAYOR_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
CUR_FIN_CLASS varchar no 66 yes no yes SQL_Latin1_General_CP1_CI_AS
PERFORMING_PROV_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
BILLING_PROVIDER_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
ORIGINAL_CVG_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ORIGINAL_PLAN_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ORIGINAL_PAYOR_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ORIGINAL_FIN_CLASS varchar no 66 yes no yes SQL_Latin1_General_CP1_CI_AS
PROC_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
PROCEDURE_QUANTITY numeric no 5 8 3 yes (n/a) (n/a) NULL
CPT_CODE varchar no 20 yes no yes SQL_Latin1_General_CP1_CI_AS
MODIFIER_ONE varchar no 254 yes no yes SQL_Latin1_General_CP1_CI_AS
MODIFIER_TWO varchar no 254 yes no yes SQL_Latin1_General_CP1_CI_AS
MODIFIER_THREE varchar no 254 yes no yes SQL_Latin1_General_CP1_CI_AS
MODIFIER_FOUR varchar no 254 yes no yes SQL_Latin1_General_CP1_CI_AS
DX_ONE_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
DX_TWO_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
DX_THREE_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
DX_FOUR_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
DX_FIVE_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
DX_SIX_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
SERV_AREA_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
LOC_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
DEPT_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
POS_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
CUSTOMER_ITEM_ONE varchar no 20 yes no yes SQL_Latin1_General_CP1_CI_AS
CUSTOMER_ITEM_TWO varchar no 20 yes no yes SQL_Latin1_General_CP1_CI_AS
CUSTOMER_ITEM_THREE varchar no 20 yes no yes SQL_Latin1_General_CP1_CI_AS
CUSTOMER_ITEM_FOUR varchar no 20 yes no yes SQL_Latin1_General_CP1_CI_AS
CUSTOMER_ITEM_FIVE varchar no 20 yes no yes SQL_Latin1_General_CP1_CI_AS
BILL_CLAIM_AMOUNT numeric no 9 12 2 yes (n/a) (n/a) NULL
INVOICE_NUMBER varchar no 20 yes no yes SQL_Latin1_General_CP1_CI_AS
ABF_FORM_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
CLM_CLAIM_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
BILL_HOLD_AMOUNT numeric no 9 12 2 yes (n/a) (n/a) NULL
PAT_AGING_DAYS int no 4 10 0 yes (n/a) (n/a) NULL
INS_AGING_DAYS int no 4 10 0 yes (n/a) (n/a) NULL
ACTION_CVG_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ACTION_PLAN_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ACTION_PAYOR_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ACTION_FIN_CLASS varchar no 66 yes no yes SQL_Latin1_General_CP1_CI_AS
DEBIT_GL_NUM varchar no 20 yes no yes SQL_Latin1_General_CP1_CI_AS
CREDIT_GL_NUM varchar no 20 yes no yes SQL_Latin1_General_CP1_CI_AS
REASON_CODE_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
USER_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
COPAY_INDICATOR smallint no 2 5 0 yes (n/a) (n/a) NULL
TX_NUM smallint no 2 5 0 yes (n/a) (n/a) NULL
ORIG_PRICE numeric no 9 12 2 yes (n/a) (n/a) NULL
PRICE_CONTRACT_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
CONTRACT_DISCOUNT numeric no 9 12 2 yes (n/a) (n/a) NULL
INT_PAT_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
ORIG_AMT numeric no 9 12 2 yes (n/a) (n/a) NULL
PAT_TYPE_C smallint no 2 5 0 yes (n/a) (n/a) NULL
PRIM_CARE_PROV varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
REFERRAL_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
REFERRAL_SOURCE_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
PRIM_LOCATION numeric no 9 18 0 yes (n/a) (n/a) NULL
RVU_WORK numeric no 9 12 2 yes (n/a) (n/a) NULL
RVU_OVERHEAD numeric no 9 12 2 yes (n/a) (n/a) NULL
RVU_MALPRACTICE numeric no 9 12 2 yes (n/a) (n/a) NULL
POSTING_BATCH_NUM varchar no 12 yes no yes SQL_Latin1_General_CP1_CI_AS
VOID_USER_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
R_NEW_CHG_TX_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
R_ORIG_CHG_TX_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
R_ORIG_POST_DATE datetime no 8 yes (n/a) (n/a) NULL
R_ORIG_AMOUNT numeric no 9 12 2 yes (n/a) (n/a) NULL
R_ORIG_PAT_AMOUNT numeric no 9 12 2 yes (n/a) (n/a) NULL
R_ORIG_FIN_CLASS varchar no 66 yes no yes SQL_Latin1_General_CP1_CI_AS
R_ORIG_PAYOR_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
R_ORIG_PLAN_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
R_ORIG_CVG_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
REFERENCE_NUMBER varchar no 255 yes no yes SQL_Latin1_General_CP1_CI_AS
PAYMENT_SOURCE_C smallint no 2 5 0 yes (n/a) (n/a) NULL
TX_COMMENT varchar no 255 yes no yes SQL_Latin1_General_CP1_CI_AS
ANESTHESIA_TYPE_C varchar no 66 yes no yes SQL_Latin1_General_CP1_CI_AS
EMERGENCY_STATUS_C smallint no 2 5 0 yes (n/a) (n/a) NULL
PHYSICAL_STATUS_C varchar no 66 yes no yes SQL_Latin1_General_CP1_CI_AS
CONCURRENCY_CALC numeric no 5 6 2 yes (n/a) (n/a) NULL
CONCURRENCY_OVER numeric no 5 6 2 yes (n/a) (n/a) NULL
BASE_UNITS numeric no 5 6 2 yes (n/a) (n/a) NULL
TIMED_UNITS numeric no 5 6 2 yes (n/a) (n/a) NULL
PHY_STATUS_UNITS numeric no 5 6 2 yes (n/a) (n/a) NULL
EMERGENCY_UNITS numeric no 5 6 2 yes (n/a) (n/a) NULL
AGE_UNITS numeric no 5 6 2 yes (n/a) (n/a) NULL
MED_SUP_MOD_OVER_C varchar no 1 yes no yes SQL_Latin1_General_CP1_CI_AS
ANESTHESIA_CERT varchar no 254 yes no yes SQL_Latin1_General_CP1_CI_AS
TECH_CHARGE_FLAG varchar no 1 yes no yes SQL_Latin1_General_CP1_CI_AS
NIA_OUTCOME_CODE_C smallint no 2 5 0 yes (n/a) (n/a) NULL
PROC_MED_NEC_FLAG varchar no 1 yes no yes SQL_Latin1_General_CP1_CI_AS
CRD_CHARGE_SLIP_NO varchar no 15 yes no yes SQL_Latin1_General_CP1_CI_AS
VISIT_NUMBER varchar no 50 yes no yes SQL_Latin1_General_CP1_CI_AS
CRNA_CHARGE_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ORIG_ANES_CHG_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
CONTESTED_FLAG_YN varchar no 1 yes no yes SQL_Latin1_General_CP1_CI_AS
PAT_ENC_CSN_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ACTION_USER_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
TDL_EXTRACT_DATE datetime no 8 yes (n/a) (n/a) NULL
INSURANCE_ID varchar no 17 yes no yes SQL_Latin1_General_CP1_CI_AS
SPECIMEN_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
TEST_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
LAB_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
PANEL_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
PROV_SPECIALTY_C varchar no 66 yes no yes SQL_Latin1_General_CP1_CI_AS
DIST_USER_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
UNDIST_USER_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
PROC_ALT_IDFR varchar no 50 yes no yes SQL_Latin1_General_CP1_CI_AS
SERV_PROV_SUFFIX varchar no 30 yes no yes SQL_Latin1_General_CP1_CI_AS
SERV_PROV_ATTR_1_C smallint no 2 5 0 yes (n/a) (n/a) NULL
SERV_PROV_ATTR_2_C smallint no 2 5 0 yes (n/a) (n/a) NULL
SERV_PROV_ATTR_3_C smallint no 2 5 0 yes (n/a) (n/a) NULL
BILL_PROV_SUFFIX varchar no 30 yes no yes SQL_Latin1_General_CP1_CI_AS
BILL_PROV_ATTR_1_C smallint no 2 5 0 yes (n/a) (n/a) NULL
BILL_PROV_ATTR_2_C smallint no 2 5 0 yes (n/a) (n/a) NULL
BILL_PROV_ATTR_3_C smallint no 2 5 0 yes (n/a) (n/a) NULL
VERIFIED varchar no 1 yes no yes SQL_Latin1_General_CP1_CI_AS
RVU_PROC_UNITS numeric no 9 12 2 yes (n/a) (n/a) NULL
CASE_STATE_C smallint no 2 5 0 yes (n/a) (n/a) NULL
CASE_RATE_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
CASE_TX_STATE_C smallint no 2 5 0 yes (n/a) (n/a) NULL
MATCH_RFD_TX_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
MATCH_RFD_TX_AMT numeric no 9 12 2 yes (n/a) (n/a) NULL
CUR_ACTN_PAYOR_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ORIG_PST_BATCH_NUM varchar no 12 yes no yes SQL_Latin1_General_CP1_CI_AS
ORIG_REF_NUM varchar no 255 yes no yes SQL_Latin1_General_CP1_CI_AS
ORIG_PAY_SOURCE_C smallint no 2 5 0 yes (n/a) (n/a) NULL
TX_DESC_OVERRIDE varchar no 255 yes no yes SQL_Latin1_General_CP1_CI_AS
TRANSFERRED_TX_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
IS_TRANFERED_YN varchar no 1 yes no yes SQL_Latin1_General_CP1_CI_AS
CM_ORIG_DPLY_ID varchar no 25 yes no yes SQL_Latin1_General_CP1_CI_AS
CM_ORIG_ETR_CID numeric no 9 18 0 yes (n/a) (n/a) NULL
CM_ORIG_ACCT_LINE int no 4 10 0 yes (n/a) (n/a) NULL
CM_HOME_LINK numeric no 9 18 0 yes (n/a) (n/a) NULL
HOMED_INDICATOR_YN varchar no 1 yes no yes SQL_Latin1_General_CP1_CI_AS
CM_PHY_OWNER_ID varchar no 25 yes no yes SQL_Latin1_General_CP1_CI_AS
CM_LOG_OWNER_ID varchar no 25 yes no yes SQL_Latin1_General_CP1_CI_AS
CM_ORIG_ACCT_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
CM_HOMED_INSTANT datetime no 8 yes (n/a) (n/a) NULL
CH_EDIT_SEQUENCE int no 4 10 0 yes (n/a) (n/a) NULL
CH_EDIT_CONTEXTS varchar no 254 yes no yes SQL_Latin1_General_CP1_CI_AS
ACTION_AMOUNT numeric no 9 18 2 yes (n/a) (n/a) NULL
HOM_CLARITY_FLG_YN varchar no 254 yes no yes SQL_Latin1_General_CP1_CI_AS
CSH_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ORIG_HTR_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
WORKSTATION_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
POS_SESSID varchar no 100 yes no yes SQL_Latin1_General_CP1_CI_AS
POS_TXID varchar no 100 yes no yes SQL_Latin1_General_CP1_CI_AS
VOID_POS_SESSID varchar no 90 yes no yes SQL_Latin1_General_CP1_CI_AS
VOID_POS_TXID varchar no 90 yes no yes SQL_Latin1_General_CP1_CI_AS
POS_TX_LINE int no 4 10 0 yes (n/a) (n/a) NULL
VOID_POS_TX_LINE int no 4 10 0 yes (n/a) (n/a) NULL
HSP_ACCOUNT_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
MATCH_PAYOR_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ACTION_MATCH_LINE smallint no 2 5 0 yes (n/a) (n/a) NULL
DYNAMIC_ALLOWED_AMT numeric no 9 12 2 yes (n/a) (n/a) NULL
ACTIVE_AR_AMOUNT numeric no 9 18 2 yes (n/a) (n/a) NULL
EXTERNAL_AR_AMOUNT numeric no 9 18 2 yes (n/a) (n/a) NULL
BAD_DEBT_AR_AMOUNT numeric no 9 18 2 yes (n/a) (n/a) NULL
BILL_AREA_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
FIN_DIV_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
FIN_SUBDIV_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
etl_date datetime no 8 yes (n/a) (n/a) NULL
尝试使用不同表和数据文件起作用的命令,此时错误消息太短,无法给我更多调试信息(从未使用过) BCP实用工具之前)。如果有人有使用此工具的经验,将不胜感激任何调试建议或如何解决此问题。
答案 0 :(得分:3)
进一步调试后,发现使用BCP将tsv文件复制到MSSQL DB时,tsv中的文字'null'值将作为字符串导入,并且(我认为)这是导致错误的原因,因为目标表中的某些字段期望数值等(在使用BCP之前,正在使用处理此隐藏方法的进程)。将命令更改为表单
echo "Replacing null literal values with empty chars"
NULL_WITH_TAB="null\t"
TAB="\t"
sed -i -e "s/$NULL_WITH_TAB/$TAB/g" $filename
/opt/mssql-tools/bin/bcp "$TABLE" in "$filename" \
$TO_SERVER_ODBCDSN \
-U $USER \
-P $PASSWORD \
-d $DB \
$RECOMMEDED_IMPORT_MODE \
-t "\t" \
-e /home/me/projects/bcp-experiments/examples/recent-err-row.log
解决了该问题。如果有人知道将来如何从BCP强制发出更多详细的错误消息,或者还有哪些其他类型的问题可以引发这种模糊的错误,请告诉我。