IFS separate a string like “Hello”,“World”,“this”,“is, a boring”, “line”

Question

I'm trying to parse a .csv file and I have some problems with IFS. The file contains lines like this:

"Hello","World","this","is, a boring","line"

The columns are separated with a comma, so I tried to explode the line with this code:

IFS=, read -r -a tempArr <<< "$line"

But I get this output:

"Hello"
"World"
"this"
"is
a boring"
"line"

I understand why, so I tried some other commands but I don't get my expected output.

IFS=\",\"
IFS=\",
IFS=',\"'
IFS=,\"

Every time the third element is seperated in 2 parts. How can I use IFS to seperate the string to 5 parts like this?

"Hello"
"World" 
"this" 
"is, a boring" 
"line"

Answer 1

give this a try:

sed 's/","/"\n"/g' <<<"${line}"

sed has a search and replace command s which is using regex to search pattern.

The regex replaces , in "," with new line char.

As a consequence each element is on a separate line.

Answer 2

You may wish to use the gawk with FPAT to define what makes a valid string -

Input :

"hello","world","this,is"

Script :

gawk -n 'BEGIN{FS=",";OFS="\n";FPAT="([^,]+)|(\"[^\"]+\")"}{$1=$1;print $0}' somefile.csv

Output :

"hello"
"world"
"this,is"

Answer 3

bashlib provides a csvline function. Assuming you've installed it somewhere in your PATH:

line='"Hello","World","this","is, a boring","line"'

source bashlib
csvline <<<"$line"
printf '%s\n' "${CSVLINE[@]}"

...output from the above being:

Hello
World
this
is, a boring
line

---

To quote the implementation (which is copyright lhunath, the below text being taken from this specific revision of the relevant git repo):

#  _______________________________________________________________________
# |__ csvline ____________________________________________________________|
#
#       csvline [-d delimiter] [-D line-delimiter]
#
# Parse a CSV record from standard input, storing the fields in the CSVLINE array.
#
# By default, a single line of input is read and parsed into comma-delimited fields.
# Fields can optionally contain double-quoted data, including field delimiters.
#
# A different field delimiter can be specified using -d.  You can use -D
# to change the definition of a "record" (eg. to support NULL-delimited records).
#
csvline() {
    CSVLINE=()
    local line field quoted=0 delimiter=, lineDelimiter=$'\n' c
    local OPTIND=1 arg
    while getopts :d: arg; do
        case $arg in
            d) delimiter=$OPTARG ;;
        esac
    done

    IFS= read -d "$lineDelimiter" -r line || return
    while IFS= read -rn1 c; do
        case $c in
            \")
                (( quoted = !quoted ))
                continue ;;
            $delimiter)
                if (( ! quoted )); then
                    CSVLINE+=( "$field" ) field=
                    continue
                fi ;;
        esac
        field+=$c
    done <<< "$line"
    [[ $field ]] && CSVLINE+=( "$field" ) ||:
} # _____________________________________________________________________