0

Sooo, I am trying to log my ethminer hashrate, however I am having some troubles to chain the output of a egrep / grep into another one, I even tried with different commands (sed/cat...) none seems to actually get any sort of output, I can't even redirect stdout to a file! The weird part; when I don't try to chain or redirect anything, it prints what it is supposed to on the terminal!

after about 2h of research, I'm still on the same error (probably mine, but I can't really figure it out) and even after copying some stuff i found, I'm still in the same place...

here is the command i would like to filter:

/home/USER/ethminer -U -S eu1.ethermine.org:4444 -O 0x*****************.********** --cuda-parallel-hash 4

output this (it outputs about a bit more than a line/sec, and this needs to run basically forever):

  ℹ  23:36:26|CUDA0     set work; seed: #4be89018, target:  #0000000112e0
  ℹ  23:36:26|CUDA0     set work; seed: #4be89018, target:  #0000000112e0
  m  23:36:28|ethminer  Speed  20.01 Mh/s    gpu/0 20.01  [A0+0:R0+0:F0] Time: 00:00
  m  23:36:30|ethminer  Speed  22.13 Mh/s    gpu/0 22.13  [A0+0:R0+0:F0] Time: 00:00

The thing which interest me is the part after Speed, though there are hidden characters, here is what a chained

  cat -e 

displays:

^[[32m  m  ^[[35m23:38:10^[[0m^[[30m|^[[34methminer^[[0m  Speed ^[[1;36m 23.09^[[0m Mh/s    gpu/0 ^[[36m23.09^[[0m  [A0+0:R0+0:F0] Time: 00:00^[[0m$

note the "Speed ^[[1;36m 23.09", I am using the 'm' as a reference to find the hashrate in the expression with m [[:digit:]]+\.[[:digit:]]{2} (btw you'll see that I need to redirect the stderr into the stdout, this is because ethminer seems to output the hashrate in the stderr channel... kinda weird...) This give me:

/home/USER/ethminer -U -S eu1.ethermine.org:4444 -O 0x*****************.********** --cuda-parallel-hash 4 2>&1 | cat -e | egrep -o --color=never "m [[:digit:]]+\.[[:digit:]]{2}" | egrep -o [[:digit:]]+\.[[:digit:]]{2}$

this output me what it is supposed to do:

m 0.00
m 15.74
m 19.41

So up to now, It's all OK... but after this one... everything is getting weird... I havn't managed to get any output... not even on my terminal from any chained command, I tried many things, from playing with redirection of stderr and stdout about everywhere, tried cat: no output, played a bit with sed: no luck, with grep: not any more chance than with sed... Idealy I would like something like this:

/home/USER/ethminer -U -S eu1.ethermine.org:4444 -O 0x*****************.********** --cuda-parallel-hash 4 2>&1 | cat -e | egrep -o --color=never "m [[:digit:]]+\.[[:digit:]]{2}" | egrep -o [[:digit:]]+\.[[:digit:]]{2}$ >> /home/USER/mining.log

Which would output:

22.26
22.47

in the mining.log file...

Help!

EDIT: ethminer is coming from here : https://github.com/ethereum-mining/ethminer this is the version 0.12.0

UDATE: The answer of egmont ended up working, if I understand correctly this is because of the way the data sent through pipes is buffered with ethminer... here is the command I ended up using:

stdbuf -oL ethminer [arguments] 2>&1 | stdbuf -oL cat -e | stdbuf -oL egrep -o "m [0-9]+\.[0-9]{2}" | stdbuf -oL egrep -o "[0-9]+\.[0-9]{2}" >> mining.log
9
  • 2
    I'd forget about cat -e and just use something like grep -Po 'Speed\s+\K[0-9.]+' Nov 27, 2017 at 22:58
  • sadly it doesn't want to output anything from this :/
    – Celivalg
    Nov 27, 2017 at 23:05
  • I tried a few more expressions, which should match, but I havn't got much luck
    – Celivalg
    Nov 27, 2017 at 23:13
  • Hmm - perhaps the issue is that ethminer isn't smart enough to turn off color codes when it's piped? Nov 27, 2017 at 23:26
  • @dessert I havn't managed to make awk work with this, however grep Speed | sed ... returned this : 22.32 Mh/s gpu/0 22.32 [A0+0:R0+0:F0] Time: 00:01 so I might be able to make it work
    – Celivalg
    Nov 27, 2017 at 23:27

2 Answers 2

1

Most command line programs that provide colored terminal output are smart enough to recognize when their output is not going to a terminal, and suppress the color codes. It seems that ethminer is not.

Your solution was to convert the non-printing ANSI codes into plain text using cat -e, and then parse that. An alternative (and possibly more robust) workaround might be to strip the color codes - one way to do that is using the perl Term::ANSIColor module e.g.

/home/USER/ethminer -U -S eu1.ethermine.org:4444 -O 0x*****************.********** --cuda-parallel-hash 4 2>&1 | 
  perl -MTerm::ANSIColor=colorstrip -lne 'print colorstrip $_' | grep ...

However, if we're going to do that, we may as well use perl in place of grep to do the matching. There are many ways to do that - here's one:

  • split the line into a whitespace separated array @F
  • call colorstrip on @F and save the results in @a
  • find the index of array element matching Speed, then look up and print the next element

ex.

/home/USER/ethminer -U -S eu1.ethermine.org:4444 -O 0x*****************.********** --cuda-parallel-hash 4 2>&1 | 
  perl -MTerm::ANSIColor=colorstrip -alne '@a = colorstrip @F; print map { $a[$_+1] } grep { $a[$_] eq "Speed" } 0..$#a'
0

The default behavior, when output is written to a terminal, is to be line buffered, however, when directed to a file or to another process using a pipe, it is buffered in maybe 4kB or 8kB chunks of data. That is, your ethminer needs to print this much data for the first chunk to actually get written out and become available to the next phase in your timeline.

Use stdbuf to enable line buffered mode even if the output goes to another process, e.g. stdbuf -oL ethminer [parameters] | cat-or-grep-or-whatever.

For the same reason, if you have a longer pipeline, probably all phases but the last need to be changed to line buffered, something like: stdbuf -oL ethminer | stdbuf -oL cat -e | stdbuf -oL egrep pattern1 | egrep pattern2.

Colors are irrelevant to this story, although as pointed out in a different answer, ethminer should detect if its standard output isn't connected to a terminal and shouldn't emit color codes in that case.

2
  • Thank you soo much! I had to add stdbuf -oL even to the last egrep because otherwise it wasn't able to redirect the output to a file...
    – Celivalg
    Nov 28, 2017 at 17:40
  • Sure I wasn't clear there, it's not needed for the last step if it goes to a terminal (because that's the default behavior then), but is still needed if it goes to a file and you expect each line to appear there as soon as it can.
    – egmont
    Nov 28, 2017 at 19:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .