问题
Trying to use BCP utility on linux CentOS7 to copy tsv files into remote MSSQL Server DB (in a Windows Server 2012 machine), getting the curt error message that
Starting copy...
BCP copy in failed
The BCP command in question is
/opt/mssql-tools/bin/bcp "$TABLE" in $DATAFILES/part-m-00000 \
-S -D $SERVER_DSN \
-U $USER \
-P $PASSWORD \
-d $DB \
-c \
-t "\t" \
-e /home/me/projects/bcp-experiments/examples/recent-err-rows.log
and the version is
[me@mapr001 examples]$ bcp -v
BCP - Bulk Copy Program for Microsoft SQL Server.
Copyright (C) Microsoft Corporation. All Rights Reserved.
Version: 17.2.0000.1
The destination table has the schema shown below (couldn't really think of a great way to format the data here, it's a very wide table)
Column_name Type Computed Length Prec Scale Nullable TrimTrailingBlanks FixedLenNullInSource Collation
TDL_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
DETAIL_TYPE smallint no 2 5 0 yes (n/a) (n/a) NULL
TYPE smallint no 2 5 0 yes (n/a) (n/a) NULL
POST_DATE datetime no 8 yes (n/a) (n/a) NULL
ORIG_POST_DATE datetime no 8 yes (n/a) (n/a) NULL
ORIG_SERVICE_DATE datetime no 8 yes (n/a) (n/a) NULL
PERIOD varchar no 6 yes no yes SQL_Latin1_General_CP1_CI_AS
TX_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
TRAN_TYPE smallint no 2 5 0 yes (n/a) (n/a) NULL
ALLOWED_AMOUNT numeric no 9 12 2 yes (n/a) (n/a) NULL
CHARGE_SLIP_NUMBER varchar no 15 yes no yes SQL_Latin1_General_CP1_CI_AS
TYPE_OF_SERVICE smallint no 2 5 0 yes (n/a) (n/a) NULL
MATCH_TRX_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
MATCH_TX_TYPE smallint no 2 5 0 yes (n/a) (n/a) NULL
MATCH_PROC_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
MATCH_PROV_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
MATCH_LOC_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ACCOUNT_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
PAT_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
AMOUNT numeric no 9 12 2 yes (n/a) (n/a) NULL
PATIENT_AMOUNT numeric no 9 12 2 yes (n/a) (n/a) NULL
INSURANCE_AMOUNT numeric no 9 12 2 yes (n/a) (n/a) NULL
RELATIVE_VALUE_UNIT numeric no 9 12 2 yes (n/a) (n/a) NULL
CUR_CVG_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
CUR_PLAN_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
CUR_PAYOR_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
CUR_FIN_CLASS varchar no 66 yes no yes SQL_Latin1_General_CP1_CI_AS
PERFORMING_PROV_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
BILLING_PROVIDER_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
ORIGINAL_CVG_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ORIGINAL_PLAN_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ORIGINAL_PAYOR_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ORIGINAL_FIN_CLASS varchar no 66 yes no yes SQL_Latin1_General_CP1_CI_AS
PROC_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
PROCEDURE_QUANTITY numeric no 5 8 3 yes (n/a) (n/a) NULL
CPT_CODE varchar no 20 yes no yes SQL_Latin1_General_CP1_CI_AS
MODIFIER_ONE varchar no 254 yes no yes SQL_Latin1_General_CP1_CI_AS
MODIFIER_TWO varchar no 254 yes no yes SQL_Latin1_General_CP1_CI_AS
MODIFIER_THREE varchar no 254 yes no yes SQL_Latin1_General_CP1_CI_AS
MODIFIER_FOUR varchar no 254 yes no yes SQL_Latin1_General_CP1_CI_AS
DX_ONE_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
DX_TWO_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
DX_THREE_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
DX_FOUR_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
DX_FIVE_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
DX_SIX_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
SERV_AREA_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
LOC_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
DEPT_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
POS_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
CUSTOMER_ITEM_ONE varchar no 20 yes no yes SQL_Latin1_General_CP1_CI_AS
CUSTOMER_ITEM_TWO varchar no 20 yes no yes SQL_Latin1_General_CP1_CI_AS
CUSTOMER_ITEM_THREE varchar no 20 yes no yes SQL_Latin1_General_CP1_CI_AS
CUSTOMER_ITEM_FOUR varchar no 20 yes no yes SQL_Latin1_General_CP1_CI_AS
CUSTOMER_ITEM_FIVE varchar no 20 yes no yes SQL_Latin1_General_CP1_CI_AS
BILL_CLAIM_AMOUNT numeric no 9 12 2 yes (n/a) (n/a) NULL
INVOICE_NUMBER varchar no 20 yes no yes SQL_Latin1_General_CP1_CI_AS
ABF_FORM_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
CLM_CLAIM_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
BILL_HOLD_AMOUNT numeric no 9 12 2 yes (n/a) (n/a) NULL
PAT_AGING_DAYS int no 4 10 0 yes (n/a) (n/a) NULL
INS_AGING_DAYS int no 4 10 0 yes (n/a) (n/a) NULL
ACTION_CVG_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ACTION_PLAN_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ACTION_PAYOR_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ACTION_FIN_CLASS varchar no 66 yes no yes SQL_Latin1_General_CP1_CI_AS
DEBIT_GL_NUM varchar no 20 yes no yes SQL_Latin1_General_CP1_CI_AS
CREDIT_GL_NUM varchar no 20 yes no yes SQL_Latin1_General_CP1_CI_AS
REASON_CODE_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
USER_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
COPAY_INDICATOR smallint no 2 5 0 yes (n/a) (n/a) NULL
TX_NUM smallint no 2 5 0 yes (n/a) (n/a) NULL
ORIG_PRICE numeric no 9 12 2 yes (n/a) (n/a) NULL
PRICE_CONTRACT_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
CONTRACT_DISCOUNT numeric no 9 12 2 yes (n/a) (n/a) NULL
INT_PAT_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
ORIG_AMT numeric no 9 12 2 yes (n/a) (n/a) NULL
PAT_TYPE_C smallint no 2 5 0 yes (n/a) (n/a) NULL
PRIM_CARE_PROV varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
REFERRAL_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
REFERRAL_SOURCE_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
PRIM_LOCATION numeric no 9 18 0 yes (n/a) (n/a) NULL
RVU_WORK numeric no 9 12 2 yes (n/a) (n/a) NULL
RVU_OVERHEAD numeric no 9 12 2 yes (n/a) (n/a) NULL
RVU_MALPRACTICE numeric no 9 12 2 yes (n/a) (n/a) NULL
POSTING_BATCH_NUM varchar no 12 yes no yes SQL_Latin1_General_CP1_CI_AS
VOID_USER_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
R_NEW_CHG_TX_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
R_ORIG_CHG_TX_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
R_ORIG_POST_DATE datetime no 8 yes (n/a) (n/a) NULL
R_ORIG_AMOUNT numeric no 9 12 2 yes (n/a) (n/a) NULL
R_ORIG_PAT_AMOUNT numeric no 9 12 2 yes (n/a) (n/a) NULL
R_ORIG_FIN_CLASS varchar no 66 yes no yes SQL_Latin1_General_CP1_CI_AS
R_ORIG_PAYOR_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
R_ORIG_PLAN_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
R_ORIG_CVG_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
REFERENCE_NUMBER varchar no 255 yes no yes SQL_Latin1_General_CP1_CI_AS
PAYMENT_SOURCE_C smallint no 2 5 0 yes (n/a) (n/a) NULL
TX_COMMENT varchar no 255 yes no yes SQL_Latin1_General_CP1_CI_AS
ANESTHESIA_TYPE_C varchar no 66 yes no yes SQL_Latin1_General_CP1_CI_AS
EMERGENCY_STATUS_C smallint no 2 5 0 yes (n/a) (n/a) NULL
PHYSICAL_STATUS_C varchar no 66 yes no yes SQL_Latin1_General_CP1_CI_AS
CONCURRENCY_CALC numeric no 5 6 2 yes (n/a) (n/a) NULL
CONCURRENCY_OVER numeric no 5 6 2 yes (n/a) (n/a) NULL
BASE_UNITS numeric no 5 6 2 yes (n/a) (n/a) NULL
TIMED_UNITS numeric no 5 6 2 yes (n/a) (n/a) NULL
PHY_STATUS_UNITS numeric no 5 6 2 yes (n/a) (n/a) NULL
EMERGENCY_UNITS numeric no 5 6 2 yes (n/a) (n/a) NULL
AGE_UNITS numeric no 5 6 2 yes (n/a) (n/a) NULL
MED_SUP_MOD_OVER_C varchar no 1 yes no yes SQL_Latin1_General_CP1_CI_AS
ANESTHESIA_CERT varchar no 254 yes no yes SQL_Latin1_General_CP1_CI_AS
TECH_CHARGE_FLAG varchar no 1 yes no yes SQL_Latin1_General_CP1_CI_AS
NIA_OUTCOME_CODE_C smallint no 2 5 0 yes (n/a) (n/a) NULL
PROC_MED_NEC_FLAG varchar no 1 yes no yes SQL_Latin1_General_CP1_CI_AS
CRD_CHARGE_SLIP_NO varchar no 15 yes no yes SQL_Latin1_General_CP1_CI_AS
VISIT_NUMBER varchar no 50 yes no yes SQL_Latin1_General_CP1_CI_AS
CRNA_CHARGE_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ORIG_ANES_CHG_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
CONTESTED_FLAG_YN varchar no 1 yes no yes SQL_Latin1_General_CP1_CI_AS
PAT_ENC_CSN_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ACTION_USER_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
TDL_EXTRACT_DATE datetime no 8 yes (n/a) (n/a) NULL
INSURANCE_ID varchar no 17 yes no yes SQL_Latin1_General_CP1_CI_AS
SPECIMEN_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
TEST_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
LAB_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
PANEL_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
PROV_SPECIALTY_C varchar no 66 yes no yes SQL_Latin1_General_CP1_CI_AS
DIST_USER_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
UNDIST_USER_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
PROC_ALT_IDFR varchar no 50 yes no yes SQL_Latin1_General_CP1_CI_AS
SERV_PROV_SUFFIX varchar no 30 yes no yes SQL_Latin1_General_CP1_CI_AS
SERV_PROV_ATTR_1_C smallint no 2 5 0 yes (n/a) (n/a) NULL
SERV_PROV_ATTR_2_C smallint no 2 5 0 yes (n/a) (n/a) NULL
SERV_PROV_ATTR_3_C smallint no 2 5 0 yes (n/a) (n/a) NULL
BILL_PROV_SUFFIX varchar no 30 yes no yes SQL_Latin1_General_CP1_CI_AS
BILL_PROV_ATTR_1_C smallint no 2 5 0 yes (n/a) (n/a) NULL
BILL_PROV_ATTR_2_C smallint no 2 5 0 yes (n/a) (n/a) NULL
BILL_PROV_ATTR_3_C smallint no 2 5 0 yes (n/a) (n/a) NULL
VERIFIED varchar no 1 yes no yes SQL_Latin1_General_CP1_CI_AS
RVU_PROC_UNITS numeric no 9 12 2 yes (n/a) (n/a) NULL
CASE_STATE_C smallint no 2 5 0 yes (n/a) (n/a) NULL
CASE_RATE_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
CASE_TX_STATE_C smallint no 2 5 0 yes (n/a) (n/a) NULL
MATCH_RFD_TX_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
MATCH_RFD_TX_AMT numeric no 9 12 2 yes (n/a) (n/a) NULL
CUR_ACTN_PAYOR_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ORIG_PST_BATCH_NUM varchar no 12 yes no yes SQL_Latin1_General_CP1_CI_AS
ORIG_REF_NUM varchar no 255 yes no yes SQL_Latin1_General_CP1_CI_AS
ORIG_PAY_SOURCE_C smallint no 2 5 0 yes (n/a) (n/a) NULL
TX_DESC_OVERRIDE varchar no 255 yes no yes SQL_Latin1_General_CP1_CI_AS
TRANSFERRED_TX_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
IS_TRANFERED_YN varchar no 1 yes no yes SQL_Latin1_General_CP1_CI_AS
CM_ORIG_DPLY_ID varchar no 25 yes no yes SQL_Latin1_General_CP1_CI_AS
CM_ORIG_ETR_CID numeric no 9 18 0 yes (n/a) (n/a) NULL
CM_ORIG_ACCT_LINE int no 4 10 0 yes (n/a) (n/a) NULL
CM_HOME_LINK numeric no 9 18 0 yes (n/a) (n/a) NULL
HOMED_INDICATOR_YN varchar no 1 yes no yes SQL_Latin1_General_CP1_CI_AS
CM_PHY_OWNER_ID varchar no 25 yes no yes SQL_Latin1_General_CP1_CI_AS
CM_LOG_OWNER_ID varchar no 25 yes no yes SQL_Latin1_General_CP1_CI_AS
CM_ORIG_ACCT_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
CM_HOMED_INSTANT datetime no 8 yes (n/a) (n/a) NULL
CH_EDIT_SEQUENCE int no 4 10 0 yes (n/a) (n/a) NULL
CH_EDIT_CONTEXTS varchar no 254 yes no yes SQL_Latin1_General_CP1_CI_AS
ACTION_AMOUNT numeric no 9 18 2 yes (n/a) (n/a) NULL
HOM_CLARITY_FLG_YN varchar no 254 yes no yes SQL_Latin1_General_CP1_CI_AS
CSH_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ORIG_HTR_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
WORKSTATION_ID varchar no 18 yes no yes SQL_Latin1_General_CP1_CI_AS
POS_SESSID varchar no 100 yes no yes SQL_Latin1_General_CP1_CI_AS
POS_TXID varchar no 100 yes no yes SQL_Latin1_General_CP1_CI_AS
VOID_POS_SESSID varchar no 90 yes no yes SQL_Latin1_General_CP1_CI_AS
VOID_POS_TXID varchar no 90 yes no yes SQL_Latin1_General_CP1_CI_AS
POS_TX_LINE int no 4 10 0 yes (n/a) (n/a) NULL
VOID_POS_TX_LINE int no 4 10 0 yes (n/a) (n/a) NULL
HSP_ACCOUNT_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
MATCH_PAYOR_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
ACTION_MATCH_LINE smallint no 2 5 0 yes (n/a) (n/a) NULL
DYNAMIC_ALLOWED_AMT numeric no 9 12 2 yes (n/a) (n/a) NULL
ACTIVE_AR_AMOUNT numeric no 9 18 2 yes (n/a) (n/a) NULL
EXTERNAL_AR_AMOUNT numeric no 9 18 2 yes (n/a) (n/a) NULL
BAD_DEBT_AR_AMOUNT numeric no 9 18 2 yes (n/a) (n/a) NULL
BILL_AREA_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
FIN_DIV_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
FIN_SUBDIV_ID numeric no 9 18 0 yes (n/a) (n/a) NULL
etl_date datetime no 8 yes (n/a) (n/a) NULL
and a mock sample row from the tsv file looks like
19133829999997 99 99 2017-01-06 00:00:00.0 2010-07-09 00:00:00.0 2010-07-06 00:00:00.0 299907 19999920 1 null -1249999 null 32899994 9 50999 null 99 21600999999 32131999999.18 199.98 9 null null null null 9 109999 109999 null null null 9 329999 9 76999 99 null null null 209999 null null null null null 99 219999 2199999999 2999 null null null null null null null null null null null null null null null 9 null null null XXXXXXXX null 9 999.1 null null B2639999 199.1 null null null 109999 99 1.99 null null 2479999 null null null null null null null null null null null null null null null null null null null null null null null N null N null N null 9 null null N 24999999 null 2017-01-07 00:00:00.0 null null null null null null null 21ZX9 null null null null null null null null null null 1.99 null null null null null null 2499999 null null null null N null null null null N null null null null null null null Y null null null null null null null null null null null null null 3 null 199.99 null null null 2018-08-17 15:48:43.0
Trying this command with a different table and data-file does work and at this point the error message is too brief to give me more to debug on (never used BCP utility before). If anyone has experience using this tool, any debugging advice or how to solve this problem would be appreciated.
回答1:
After further debugging, found that when using BCP to copy the tsv files into the MSSQL DB, the literal 'null' values in the tsv where being imported in as strings and (I assume that) that is what was causing the error, since some of the fields in the destination table were expecting numerical values, etc (before using BCP, was using a process that handled this some more hidden way). Changing the command to the form
echo "Replacing null literal values with empty chars"
NULL_WITH_TAB="null\t"
TAB="\t"
sed -i -e "s/$NULL_WITH_TAB/$TAB/g" $filename
/opt/mssql-tools/bin/bcp "$TABLE" in "$filename" \
$TO_SERVER_ODBCDSN \
-U $USER \
-P $PASSWORD \
-d $DB \
$RECOMMEDED_IMPORT_MODE \
-t "\t" \
-e /home/me/projects/bcp-experiments/examples/recent-err-row.log
fixed the problem. If anyone knows how to force more verbose error messages from BCP for the future or what other kinds of problems can raise this same kind of vague error, please let me know.
来源：https://stackoverflow.com/questions/51937432/bcp-copy-in-failed-on-tsv-no-further-error-message-info