587
  • What is the difference between awk and sed ?
  • What kind of application are best use cases for sed and awk tools ?
3

3 Answers 3

661

sed is a stream editor. It works with streams of characters on a per-line basis. It has a primitive programming language that includes goto-style loops and simple conditionals (in addition to pattern matching and address matching). There are essentially only two "variables": pattern space and hold space. Readability of scripts can be difficult. Mathematical operations are extraordinarily awkward at best.

There are various versions of sed with different levels of support for command line options and language features.

awk is oriented toward delimited fields on a per-line basis. It has much more robust programming constructs including if/else, while, do/while and for (C-style and array iteration). There is complete support for variables and single-dimension associative arrays plus (IMO) kludgey multi-dimension arrays. Mathematical operations resemble those in C. It has printf and functions. The "K" in "AWK" stands for "Kernighan" as in "Kernighan and Ritchie" of the book "C Programming Language" fame (not to forget Aho and Weinberger). One could conceivably write a detector of academic plagiarism using awk.

GNU awk (gawk) has numerous extensions, including true multidimensional arrays in the latest version. There are other variations of awk including mawk and nawk.

Both programs use regular expressions for selecting and processing text.

I would tend to use sed where there are patterns in the text. For example, you could replace all the negative numbers in some text that are in the form "minus-sign followed by a sequence of digits" (e.g. "-231.45") with the "accountant's brackets" form (e.g. "(231.45)") using this (which has room for improvement):

sed 's/-\([0-9.]\+\)/(\1)/g' inputfile

I would use awk when the text looks more like rows and columns or, as awk refers to them "records" and "fields". If I was going to do a similar operation as above, but only on the third field in a simple comma delimited file I might do something like:

awk -F, 'BEGIN {OFS = ","} {gsub("-([0-9.]+)", "(" substr($3, 2) ")", $3); print}' inputfile

Of course those are just very simple examples that don't illustrate the full range of capabilities that each has to offer.

12
  • 11
    To see some examples of pushing the boundaries of sed: sed.sourceforge.net/#scripts Commented Oct 27, 2009 at 21:58
  • 3
    @DennisWilliamson - Am I at a disadvantage if I only learn awk ? Is awk far more commonly used than sed ?
    – Steam
    Commented Aug 20, 2013 at 18:24
  • 8
    @blasto: My recommendation is to learn both but with more emphasis on awk. Lots of the regular expression stuff applies to both (and other tools and languages). Use sed for simpler stuff and try to avoid the complex things. It's really cool that you can do loops and branches in sed, but the resulting command lines are complex and hard to read. The answer to your question really depends on what you're doing. Commented Aug 20, 2013 at 19:52
  • @DennisWilliamson - What would be most useful to an ETL developer ? ETL or Extract Transform and Load is a data-warehousing term. Put crudely, the job involves EXTRACTION of data from different disparate sources (such as DB's, excel files, csv files etc), TRANSFORMATION of the same and then LOADING into a datawarehouse (DW) for analysis, finding patterns in data, or just historical records. eg. End use of a DW - Algorithms applied to a DW of a grocery store which has data from the past 10 years might reveal that people who tend to buy apples also buy oranges or something similar.
    – Steam
    Commented Aug 20, 2013 at 19:56
  • 1
    Just an FYI for anyone stumbling across this on a mac, try "sed -E 's/-([0-9]+.[0-9]*)/(\1)/g'" for the first sed example Commented May 8, 2018 at 21:05
159

1) What is the difference between awk and sed ?

Both are tools that transform text. BUT awk can do more things besides just manipulating text. Its a programming language by itself with most of the things you learn in programming, like arrays, loops, if/else flow control etc You can "program" in sed as well, but you won't want to maintain the code written in it.

2) What kind of application are best use cases for sed and awk tools ?

Conclusion: Use sed for very simple text parsing. Anything beyond that, awk is better. In fact, you can ditch sed altogether and just use awk. Since their functions overlap and awk can do more, just use awk. You will reduce your learning curve as well.

4
  • 13
    I find sed is much easier to learn though, so you need to account for that. As you learn to master awk, it might be useful to quickly learn sed to be able to use it faster for things you might not know how to do in awk yet.
    – Didier A.
    Commented May 7, 2016 at 8:03
  • 39
    Don't ditch sed, 's/search/replace' is way easier to type than awk's syntax and is what you need most of the time.
    – sjas
    Commented Jun 4, 2017 at 12:10
  • 2
    Better to use a mixture. For instance, if you are looking for a pattern in a huge file, use grep to find the line number, and then sed to edit the line. Otherwise, sed is going to be much slower to process the entire file.
    – Konchog
    Commented Oct 30, 2018 at 11:27
  • 3
    If you're a craftsman, you learn how to use all your tools and you'll know which to use when. If you're just fixing stuff, a hammer and a screwdriver might just be all you need.
    – pbarney
    Commented Jan 12, 2021 at 17:20
81

Both tools are meant to work with text and there are tasks both tools can be used for.

For me the rule to separate them is: Use sed to automate tasks you would do otherwise in a text editor manually. That's why it is called stream editor. (You can use the same commands to edit text in vim). Use awk if you want to analyze text, meaning counting fields, calculate totals, extract and reorganize structures etc.

Also you should not forget about grep. Use grep if you only want to search/extract something in a text (file)

Not the answer you're looking for? Browse other questions tagged or ask your own question.