Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
2.4k views
in Technique[技术] by (71.8m points)

linux - grep with continuous pipe does not work

(maybe it is the "tcpflow" problem)

I write a script to monitoring http traffic, and I install tcpflow, then grep

it works (and you should make a http request, for example curl www.163.com)

sudo tcpflow -p -c -i eth0 port 80 2>/dev/null | grep '^Host: '

it outputs like this (continuously)

Host: config.getsync.com
Host: i.stack.imgur.com
Host: www.gravatar.com
Host: www.gravatar.com

but I can't continue to use pipe

does not work (nothing output)

sudo tcpflow -p -c -i eth0 port 80 2>/dev/null | grep '^Host: ' | cut -b 7- 

does not work (nothing output)

sudo tcpflow -p -c -i eth0 port 80 2>/dev/null | grep '^Host: ' | grep H

When I replace sudo tcpflow with cat foo.txt, it works:

cat foo.txt | grep '^Host: ' | grep H

so what's wrong with pipe or grep or tcpflow ?


update:

This is my final script: https://github.com/zhengkai/config/blob/master/script/monitor_outgoing_http.sh

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

To grep a continuous stream use --line-buffered option:

sudo tcpflow -p -c -i eth0 port 80 2> /dev/null | grep --line-buffered '^Host'

--line-buffered

Use line buffering on output. This can cause a performance penalty.


Some reflections about buffered outputting(stdbuf tool is also mentioned):

Pipes, how do data flow in a pipeline?


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...