modify a datastream on the fly

Question

I need to hijack and modify a datastream. The stream consists of fixed-width commands. Each command is a new line, and the documentation says that each command starts and ends

Answer 1

Yes, the iPocket does raw pass-through of the data. The options are typically for packetization of the incoming data which won't actually affect what you're trying to do here. (I wrote the firmware for that device.)

However, none of the command-line answers here will work because they are line-based and your data is not. That is to say, there are to linefeeds in the data stream that sed and perl use as "packet" boundaries.

I don't believe there is any way you're going to be able to accomplish this from the command-line. You're going to have to write a simple program that reads an incoming TCP stream, looks for the STX/ETX framing, replaces the characters as you wish, and then writes the data out the other side. (Don't forget pass-through the back direction, too.)

Python is probably the simplest way to do this.

Answer 2

Instead of the sed command, you can use this:

perl -pe 's/(,\w+)/" " x length $1/ge'

The e option on the substitution means that the right side of the s/// is evaluated as a Perl expression. In this case, the expression returns the correct number of spaces, based on the length of the captured match.

Answer 3

Here's a sed version:

sed 'h;s/[^,]*,\([^ ]*\) ETX/\1/;s/./ /g;x;s/,.*/,/;G;s/\n//;s/$/ ETX/'

I'd be willing to bet that not only is the command fixed width, but that the fields are also fixed width. If that's the case then something like this would probably work:

sed 's/\(.\{22\}\).\{9\}\(.\{4\}\)/\1         \2/'

or

sed -r 's/(.{22}).{9}(.{4})/\1         \2/'

or

sed -r 's/STX (.{18}).{9} ETX/STX \1          ETX/'

Answer 4

In the Perl, with added check that the string starts with NAM and replacing really the ,FIRSTNAME:

nc -kl 1100 | \
perl -pe '/^STX NAM / && do { s/(,FIRSTNAME)/" " x length("$1")/ge }' | \
nc <...>

In Perl s///e as expected does substitution, but evaluates the replacement string as an expression. Operator x makes a new string, duplicating string on the left number of times given on the right. Non-matching strings are obviously not modified.

Answer 5

This should do it, its a one liner

perl -pe '/^STX\s\w+\s\d+#\s\w+,(\w+)\sETX$/;$len=length($1);s/$1/" " x $len/e'

I tested using.

echo "STX NAM 100# LASTNAME,FIRSTNAME ETX" | perl -pe '/^STX\s\w+\s\d+#\s\w+,(\w+)\sETX$/;$len=length($1);s/$1/" " x $len/e'

and it returns

STX NAM 100# LASTNAME,          ETX

To ensure the strings are the same length i tested.

echo "STX NAM 100# LASTNAME,FIRSTNAME ETX" | perl -pe '/^STX\s\w+\s\d+#\s\w+,(\w+)\sETX$/;$len=length($1);s/$1/" " x $len/e'| perl -pe 'print length($_);'

Gives 36.

echo "STX NAM 100# LASTNAME,FIRSTNAME ETX"|perl -pe 'print length($_);'

Gives 36.