Converting a GenBank-like multiline record into a new file format (fasta format)

问题

I have a file with the following architecture:

 source          1..3566367
                 /organism="Laccaria bicolor S238N-H82"
                 /mol_type="genomic DNA"
                 /strain="S238N-H82"
                 /db_xref="taxon:486041"
 gene            <143..>783
                 /locus_tag="LACBIDRAFT_300552"
                 /db_xref="GeneID:6069681"
 mRNA            join(<143..224,274..309,357..470,524..>783)
                 /locus_tag="LACBIDRAFT_300552"
                 /product="helix-turn-helix transcription factor, AraC
                 type"
                 /note="Has EST support"
                 /transcript_id="XM_001873113.1"
                 /db_xref="GeneID:6069681"
 CDS             join(143..224,274..309,357..470,524..783)
                 /locus_tag="LACBIDRAFT_300552"
                 /note="Helix-turn-helix transcription factor; AraC type"
                 /codon_start=1
                 /product="helix-turn-helix transcription factor, AraC
                 type"
                 /protein_id="XP_001873148.1"
                 /db_xref="GeneID:6069681"
                 /translation="MHAKIFLTILFASAVSVYASPQLEERQILSGVVSAITSAGGAVA
                 SGVTSVAGQVTSVAGSIGGDITSEAGQVFQTVTSIGGKAVTIVTSVGGDAITLATSGA
                 GVATSKFGSVYTVATAAAASEASAATGKSSAALPIHGLQSSLIVGLVTVVGSALLGAA
                 ITL"
 gap             1104..3435
                 /estimated_length=2332
 gene            complement(<3738..>4636)
                 /locus_tag="LACBIDRAFT_242762"
                 /db_xref="GeneID:6069481"
 mRNA            complement(join(<3738..3852,3910..4045,4101..4244,
                 4296..4356,4409..4491,4540..4565,4620..>4636))
                 /locus_tag="LACBIDRAFT_242762"
                 /product="predicted protein"
                 /transcript_id="XM_001873722.1"
                 /db_xref="GeneID:6069481"
 CDS             complement(join(3738..3852,3910..4045,4101..4244,
                 4296..4356,4409..4491,4540..4565,4620..>4636))
                 /locus_tag="LACBIDRAFT_242762"
                 /note="Ribulose-5-phosphate 3-epimerase. RPE2. Fourth step
                 of pentose phosphate pathway; Ribulose-5-phosphate
                 3-epimerase. RPE2. TRuncated gene. No EST evidence"
                 /codon_start=1
                 /product="hypothetical protein"
                 /protein_id="XP_001873757.1"
                 /db_xref="InterPro:IPR000056"
                 /db_xref="GeneID:6069481"
                 /translation="LDVMDGHFVPNITMGAPILSCVHKGVPGIFMDCHMMVAKPEQWV
                 DDIADAGGSLYCFHIEATSDPVSLINTIHKRNMKAGVAISPDTPSTAITDEIANAADM
                 LLVMTVYPGRGGQKFIERCVPKVAELRARFPEKDIEVDGGVGPNTIGICADAGCNVIV
                 AGTAIFGSENPMEVIQRLKDTVNAAQAQSGAKY"
 gap             4637..5256
                 /estimated_length=620
 gap             7701..7750
                 /estimated_length=50

Within this file, the records I am interested are the ones that start with the word CDS:

 CDS             join(143..224,274..309,357..470,524..783)
                 /locus_tag="LACBIDRAFT_300552"
                 /note="Helix-turn-helix transcription factor; AraC type"
                 /codon_start=1
                 /product="helix-turn-helix transcription factor, AraC
                 type"
                 /protein_id="XP_001873148.1"
                 /db_xref="GeneID:6069681"
                 /translation="MHAKIFLTILFASAVSVYASPQLEERQILSGVVSAITSAGGAVA
                 SGVTSVAGQVTSVAGSIGGDITSEAGQVFQTVTSIGGKAVTIVTSVGGDAITLATSGA
                 GVATSKFGSVYTVATAAAASEASAATGKSSAALPIHGLQSSLIVGLVTVVGSALLGAA
                 ITL"
 CDS             complement(join(3738..3852,3910..4045,4101..4244,
                 4296..4356,4409..4491,4540..4565,4620..>4636))
                 /locus_tag="LACBIDRAFT_242762"
                 /note="Ribulose-5-phosphate 3-epimerase. RPE2. Fourth step
                 of pentose phosphate pathway; Ribulose-5-phosphate
                 3-epimerase. RPE2. TRuncated gene. No EST evidence"
                 /codon_start=1
                 /product="hypothetical protein"
                 /protein_id="XP_001873757.1"
                 /db_xref="InterPro:IPR000056"
                 /db_xref="GeneID:6069481"
                 /translation="LDVMDGHFVPNITMGAPILSCVHKGVPGIFMDCHMMVAKPEQWV
                 DDIADAGGSLYCFHIEATSDPVSLINTIHKRNMKAGVAISPDTPSTAITDEIANAADM
                 LLVMTVYPGRGGQKFIERCVPKVAELRARFPEKDIEVDGGVGPNTIGICADAGCNVIV
                 AGTAIFGSENPMEVIQRLKDTVNAAQAQSGAKY"

From them, I would like to convert the information on these records into the following format:

>XP_001873148.1 GeneID:6069681 LACBIDRAFT_300552 helix-turn-helix transcription factor, AraC
MHAKIFLTILFASAVSVYASPQLEERQILSGVVSAITSAGGAVA
SGVTSVAGQVTSVAGSIGGDITSEAGQVFQTVTSIGGKAVTIVT
SVGGDAITLATSGAGVATSKFGSVYTVATAAAASEASAATGKSS
AALPIHGLQSSLIVGLVTVVGSALLGAAITL
>XP_001873757.1 GeneID:6069481 LACBIDRAFT_242762 hypothetical protein
LDVMDGHFVPNITMGAPILSCVHKGVPGIFMDCHMMVAKPEQWV
DDIADAGGSLYCFHIEATSDPVSLINTIHKRNMKAGVAISPDTP
STAITDEIANAADMLLVMTVYPGRGGQKFIERCVPKVAELRARF
PEKDIEVDGGVGPNTIGICADAGCNVIVAGTAIFGSENPMEVIQ
RLKDTVNAAQAQSGAKY

The records of this new file should start with the ">" symbol and contain the following information extracted from:

XP_001873148.1 From: /protein_id="XP_001873148.1"

GeneID:6069681 From: /db_xref="GeneID:6069681"

LACBIDRAFT_300552 From: /locus_tag="LACBIDRAFT_300552"

helix-turn-helix transcription factor, AraC From: /product="helix-turn-helix transcription factor, AraC

Finally, the sequence:

MHAKIFLTILFASAVSVYASPQLEERQILSGVVSAITSAGGAVA
SGVTSVAGQVTSVAGSIGGDITSEAGQVFQTVTSIGGKAVTIVT
SVGGDAITLATSGAGVATSKFGSVYTVATAAAASEASAATGKSS
AALPIHGLQSSLIVGLVTVVGSALLGAAITL

Can we 'folded' to any number (usually 80, 44 in this example)

I would be very grateful if anyone could point me in the right direction to get this done using AWK, not Perl or Python. The Python/Perl tools I used to perform this task are increasingly becoming harder to maintain/compile. I hope/believe that good/old AWK has the potential to perform these tasks.

回答1:

Whenever you have tag-value pairs in your data I find it best to first build an array to contain that mapping (f[]) below and then you can just access the values by their tags/names.

Using GNU awk for the 3rd arg to match() and then taking advantage of the \s/\S gawk extensions:

$ cat tst.awk
BEGIN {
    wid=(wid ? wid : 44)
}
/^ ?\S/ {
    prt()
    key = $1
    sub(/\s*\S+/,"")
}
{
    gsub(/^\s+|\s+$/,"")
    rec = (rec == "" ? "" : rec " ") $0
}
END { prt() }

function prt(   f, tag, val) {
    if ( key == "CDS" ) {
        while ( match(rec,/\/([^=]+)=(\S+|"[^"]+")/,a) ) {
            tag = a[1]
            val = a[2]
            gsub(/^"|"$/,"",val)
            f[tag] = val
            rec = substr(rec,RSTART+RLENGTH)
        }

        gsub(/\s+/,"",f["translation"])
        gsub(".{"wid"}","&"RS,f["translation"])
        sub(RS"$","",f["translation"])

        print ">" f["protein_id"], f["db_xref"], f["locus_tag"], f["product"]
        print f["translation"]
    }
    rec = ""
}

.

$ awk -f tst.awk file
>XP_001873148.1 GeneID:6069681 LACBIDRAFT_300552 helix-turn-helix transcription factor, AraC type
MHAKIFLTILFASAVSVYASPQLEERQILSGVVSAITSAGGAVA
SGVTSVAGQVTSVAGSIGGDITSEAGQVFQTVTSIGGKAVTIVT
SVGGDAITLATSGAGVATSKFGSVYTVATAAAASEASAATGKSS
AALPIHGLQSSLIVGLVTVVGSALLGAAITL
>XP_001873757.1 GeneID:6069481 LACBIDRAFT_242762 hypothetical protein
LDVMDGHFVPNITMGAPILSCVHKGVPGIFMDCHMMVAKPEQWV
DDIADAGGSLYCFHIEATSDPVSLINTIHKRNMKAGVAISPDTP
STAITDEIANAADMLLVMTVYPGRGGQKFIERCVPKVAELRARF
PEKDIEVDGGVGPNTIGICADAGCNVIVAGTAIFGSENPMEVIQ
RLKDTVNAAQAQSGAKY

If you want to use a different width for the translation field then you can either change the code or specify it on the command line:

$ awk -v wid=80 -f tst.awk file
>XP_001873148.1 GeneID:6069681 LACBIDRAFT_300552 helix-turn-helix transcription factor, AraC type
MHAKIFLTILFASAVSVYASPQLEERQILSGVVSAITSAGGAVASGVTSVAGQVTSVAGSIGGDITSEAGQVFQTVTSIG
GKAVTIVTSVGGDAITLATSGAGVATSKFGSVYTVATAAAASEASAATGKSSAALPIHGLQSSLIVGLVTVVGSALLGAA
ITL
>XP_001873757.1 GeneID:6069481 LACBIDRAFT_242762 hypothetical protein
LDVMDGHFVPNITMGAPILSCVHKGVPGIFMDCHMMVAKPEQWVDDIADAGGSLYCFHIEATSDPVSLINTIHKRNMKAG
VAISPDTPSTAITDEIANAADMLLVMTVYPGRGGQKFIERCVPKVAELRARFPEKDIEVDGGVGPNTIGICADAGCNVIV
AGTAIFGSENPMEVIQRLKDTVNAAQAQSGAKY

来源：https://stackoverflow.com/questions/63652927/converting-a-genbank-like-multiline-record-into-a-new-file-format-fasta-format