Building SQL update statements using a Bash script

Question

For a recent project I need to loop through rows of a csv (ignore.csv) with 3 columns:

acctnum, errcode, date ← in that order…

(Column naming doesn\

Answer 1

As Barmar pointed-out:

You're overwriting the file every time you echo inside the loop.

You can do it this way with including space into the field separators of the CSV.

With some important caution:

• Shell cannot reliably parse CSV fields witch may include quotes, escaping of quotes. You should consider using a command able to parse CSV correctly and return values as arguments or null delimited fields like csvtool.
• If your SQL database engine knows about PREPARE statements, it would be safer to PREPARE the SQL UPDATE request, and feed it with arguments. You'd prepare the request before the loop, and then feed the arguments and execute the prepared statement in the loop.

#!/usr/bin/env bash

while IFS=', ' read -r acctnum errcode date; do

  cat < sqlfile.txt

Parsing and converting your exclude.csv into SQL queries using csvtool:

excludeCSV2sql:

#!/usr/bin/env bash

# This script uses csvtool to parse exclude.csv CSV data from stdin
# and output SQL queries to stdout

# Convert arguments from csvtool call, into an SQL query
to_sql ()
{
  # Double single-quote for SQL string values if any
  local -- \
    acct_num="${1//\'/\'\'}" \
    err_code="${2//\'/\'\'}" \
    date="${3//\'/\'\'}"

  cat <

Testing:

Make the above script executable:

chmod +x excludeCSV2sql

Create a sample test_exclude.csv:

cat >test_exclude.csv <

Run the test:

./excludeCSV2sql test.sql

Check the result:

test.sql

UPDATE ODS.PERF_ACCT_ERR_DTL SET REC_ACTV_IND='T'
  WHERE BUS_DT='baz' AND ETL_ERR_CD='bar' AND ACCT_KEY
  IN (
    SELECT ACCT_KEY FROM ODS.ACCT_PORTFOLIO
      WHERE ACCT_SRCH_NBR='foo'
  );
UPDATE ODS.PERF_ACCT_ERR_DTL SET REC_ACTV_IND='T'
  WHERE BUS_DT='string' AND ETL_ERR_CD='it''s using a single quote' AND ACCT_KEY
  IN (
    SELECT ACCT_KEY FROM ODS.ACCT_PORTFOLIO
      WHERE ACCT_SRCH_NBR='here'
  );
UPDATE ODS.PERF_ACCT_ERR_DTL SET REC_ACTV_IND='T'
  WHERE BUS_DT='in it' AND ETL_ERR_CD='has a double-quoted string
with a newline' AND ACCT_KEY
  IN (
    SELECT ACCT_KEY FROM ODS.ACCT_PORTFOLIO
      WHERE ACCT_SRCH_NBR='this'
  );
UPDATE ODS.PERF_ACCT_ERR_DTL SET REC_ACTV_IND='T'
  WHERE BUS_DT='20200524' AND ETL_ERR_CD='R4242, has comma' AND ACCT_KEY
  IN (
    SELECT ACCT_KEY FROM ODS.ACCT_PORTFOLIO
      WHERE ACCT_SRCH_NBR='10100000012'
  );
UPDATE ODS.PERF_ACCT_ERR_DTL SET REC_ACTV_IND='T'
  WHERE BUS_DT='20200525' AND ETL_ERR_CD='R1337' AND ACCT_KEY
  IN (
    SELECT ACCT_KEY FROM ODS.ACCT_PORTFOLIO
      WHERE ACCT_SRCH_NBR='10100000042'
  );