- What is the difference between awk and sed ?
- What kind of application are best use cases for sed and awk tools ?
-
4A related QA on unix.stackexchange: Is there a basic tutorial for grep, awk and sed?– DaniCommented Oct 27, 2009 at 16:40
-
pediaa.com/difference-between-sed-and-awk/…– Hamza BelmelloukiCommented May 20, 2019 at 1:48
-
Check out this question and its answers, stackoverflow.com/questions/46734615/…. I believe you would get a feeling from it.– BruceCommented Mar 15, 2022 at 10:43
3 Answers
sed
is a stream editor. It works with streams of characters on a per-line basis. It has a primitive programming language that includes goto-style loops and simple conditionals (in addition to pattern matching and address matching). There are essentially only two "variables": pattern space and hold space. Readability of scripts can be difficult. Mathematical operations are extraordinarily awkward at best.
There are various versions of sed
with different levels of support for command line options and language features.
awk
is oriented toward delimited fields on a per-line basis. It has much more robust programming constructs including if
/else
, while
, do
/while
and for
(C-style and array iteration). There is complete support for variables and single-dimension associative arrays plus (IMO) kludgey multi-dimension arrays. Mathematical operations resemble those in C. It has printf
and functions. The "K" in "AWK" stands for "Kernighan" as in "Kernighan and Ritchie" of the book "C Programming Language" fame (not to forget Aho and Weinberger). One could conceivably write a detector of academic plagiarism using awk
.
GNU awk
(gawk
) has numerous extensions, including true multidimensional arrays in the latest version. There are other variations of awk
including mawk
and nawk
.
Both programs use regular expressions for selecting and processing text.
I would tend to use sed
where there are patterns in the text. For example, you could replace all the negative numbers in some text that are in the form "minus-sign followed by a sequence of digits" (e.g. "-231.45") with the "accountant's brackets" form (e.g. "(231.45)") using this (which has room for improvement):
sed 's/-\([0-9.]\+\)/(\1)/g' inputfile
I would use awk
when the text looks more like rows and columns or, as awk
refers to them "records" and "fields". If I was going to do a similar operation as above, but only on the third field in a simple comma delimited file I might do something like:
awk -F, 'BEGIN {OFS = ","} {gsub("-([0-9.]+)", "(" substr($3, 2) ")", $3); print}' inputfile
Of course those are just very simple examples that don't illustrate the full range of capabilities that each has to offer.
-
11To see some examples of pushing the boundaries of
sed
: sed.sourceforge.net/#scripts Commented Oct 27, 2009 at 21:58 -
3@DennisWilliamson - Am I at a disadvantage if I only learn awk ? Is awk far more commonly used than sed ?– SteamCommented Aug 20, 2013 at 18:24
-
8@blasto: My recommendation is to learn both but with more emphasis on awk. Lots of the regular expression stuff applies to both (and other tools and languages). Use sed for simpler stuff and try to avoid the complex things. It's really cool that you can do loops and branches in sed, but the resulting command lines are complex and hard to read. The answer to your question really depends on what you're doing. Commented Aug 20, 2013 at 19:52
-
@DennisWilliamson - What would be most useful to an ETL developer ? ETL or Extract Transform and Load is a data-warehousing term. Put crudely, the job involves EXTRACTION of data from different disparate sources (such as DB's, excel files, csv files etc), TRANSFORMATION of the same and then LOADING into a datawarehouse (DW) for analysis, finding patterns in data, or just historical records. eg. End use of a DW - Algorithms applied to a DW of a grocery store which has data from the past 10 years might reveal that people who tend to buy apples also buy oranges or something similar.– SteamCommented Aug 20, 2013 at 19:56
-
1Just an FYI for anyone stumbling across this on a mac, try "sed -E 's/-([0-9]+.[0-9]*)/(\1)/g'" for the first sed example Commented May 8, 2018 at 21:05
1) What is the difference between awk and sed ?
Both are tools that transform text. BUT awk can do more things besides just manipulating text. Its a programming language by itself with most of the things you learn in programming, like arrays, loops, if/else flow control etc You can "program" in sed as well, but you won't want to maintain the code written in it.
2) What kind of application are best use cases for sed and awk tools ?
Conclusion: Use sed for very simple text parsing. Anything beyond that, awk is better. In fact, you can ditch sed altogether and just use awk. Since their functions overlap and awk can do more, just use awk. You will reduce your learning curve as well.
-
13I find sed is much easier to learn though, so you need to account for that. As you learn to master awk, it might be useful to quickly learn sed to be able to use it faster for things you might not know how to do in awk yet. Commented May 7, 2016 at 8:03
-
39Don't ditch
sed
,'s/search/replace'
is way easier to type thanawk
's syntax and is what you need most of the time.– sjasCommented Jun 4, 2017 at 12:10 -
2Better to use a mixture. For instance, if you are looking for a pattern in a huge file, use grep to find the line number, and then sed to edit the line. Otherwise, sed is going to be much slower to process the entire file.– KonchogCommented Oct 30, 2018 at 11:27
-
3If you're a craftsman, you learn how to use all your tools and you'll know which to use when. If you're just fixing stuff, a hammer and a screwdriver might just be all you need.– pbarneyCommented Jan 12, 2021 at 17:20
Both tools are meant to work with text and there are tasks both tools can be used for.
For me the rule to separate them is: Use sed
to automate tasks you would do otherwise in a text editor manually. That's why it is called stream editor. (You can use the same commands to edit text in vim). Use awk
if you want to analyze text, meaning counting fields, calculate totals, extract and reorganize structures etc.
Also you should not forget about grep
. Use grep
if you only want to search/extract something in a text (file)