20

I have a file, that consists of a repeating sequence of three lines, that I want to merge together. Put in other words, I'd like to replace every but third \n into space. E.g. I'd like the transform input

 href="file:///home/adam/MyDocs/some_file.pdf"
 visited="2013-06-02T20:40:06Z"
 exec="'firefox %u'"
 href="file:///home/adam/Desktop/FreeRDP-WebConnect-1.0.0.167-Setup.exe"
 visited="2013-06-03T08:50:37Z"
 exec="'firefox %u'"
 href="file:///home/adam/Friends/contact.txt"
 visited="2013-06-03T16:01:16Z"
 exec="'gedit %u'"
 href="file:///home/adam/Pictures/Screenshot%20from%202013-06-03%2019:10:36.png"
 visited="2013-06-03T17:10:36Z"
 exec="'eog %u'"

into 

 href="file:///home/adam/MyDocs/some_file.pdf" visited="2013-06-02T20:40:06Z" exec="'firefox %u'"
 href="file:///home/adam/Desktop/FreeRDP-WebConnect-1.0.0.167-Setup.exe" visited="2013-06-03T08:50:37Z" exec="'firefox %u'"
 href="file:///home/adam/Friends/contact.txt" visited="2013-06-03T16:01:16Z" exec="'gedit %u'"
 href="file:///home/adam/Pictures/Screenshot%20from%202013-06-03%2019:10:36.png" visited="2013-06-03T17:10:36Z" exec="'eog %u'"

Unfortunately the file is rather long, so I'd prefer not to load the whole file into memory and not to write to result back into file - just print the concatenated lines into the standard output so I can pipe it further.

I know that potentially sed might just work for it, but after I had given it a honest try, I am still at square one; the learning curve is just too steep for me. :-(

I did a rough benchmarking and I found out, that the sed variant is almost twice as fast.

time awk '{ printf "%s", $0; if (NR % 3 == 0) print ""; else printf " " }' out.txt >/dev/null

real    0m1.893s
user    0m1.860s
sys     0m0.028s

and

time cat out.txt | sed 'N;N;s/\n/ /g' > /dev/null

real    0m1.360s
user    0m1.264s
sys    0m0.236s

It is interesting: why does sed require more kernel time than awk?

The out.txt is 200MB long and the processor is Intel(R) Core(TM) i7-3610QM CPU @ 2.30GHz on Linux-Mint 14 with kernel 3.8.13-030813-generic.

I need this in my effort to parse the recently-used.xbel, the recently opened files list in the Cinnamon

If you came here for this specific problem, this line should help you:

xpath -q  -e "//bookmark[*]/@href | //bookmark[*]/@visited | //bookmark[*]/info/metadata/bookmark:applications[1]/bookmark:application[1]/@exec"  recently-used.xbel | sed 's/href="\(.*\)"/"\1"/;N;s/visited="\(.*\)"/\1/;N;s/exec="\(.*\)"/"\1"/;s/\n/ /g' | xargs -n3 whatever-script-you-write

2 Answers 2

Reset to default
29

how about this:

 sed 'N;N;s/\n/ /g' file
2
  • 1
    what does that "N" represent? ...a line?
    – aiman
    Commented May 18, 2018 at 14:05
  • 1
    You can see N adds a new line to the pattern space. You can find more information in the gnu sed manual
    – hugoShaka
    Commented Jun 8, 2018 at 12:01
17

You can use awk to do this pretty easily:

awk '{ printf "%s", $0; if (NR % 3 == 0) print ""; else printf " " }' file

The basic idea is "print each line folowed by a space, unless it's every third line, in which case print a newline".

4
  • 3
    +1 Shorter version: awk '{ printf "%s ", $0 } !(NR%3) { print "" }' inputFile
    – jaypal singh
    Commented Jun 3, 2013 at 22:14
  • Yeah, I had that. But I didn't know if the extra space mattered, so I erred on the side of caution.
    – Carl Norum
    Commented Jun 3, 2013 at 22:21
  • 6
    @Jaypal keep golfing, how about: awk 'NR%3{printf "%s ",$0;next}1' file
    – Kent
    Commented Jun 3, 2013 at 22:49
  • 1
    The sed command is almost twice faster; When testing I've got 0.188s user time and 0.012s system for the awk variant and 0.128s user and 0.30 system time for the sed variant.
    – Adam Ryczkowski
    Commented Jun 4, 2013 at 8:04

Not the answer you're looking for? Browse other questions tagged or ask your own question.