Here are some Linux command line tricks I use on occasion for search and replace. This presumes the use of GNU utilities, so commands and results will vary on different platforms.
Some cheat sheets: Sed | Awk | Perl | Bash
See also the sed/awk and grep pages.
Replace A with B in all files containing “searchpattern” in the current directory and subdirectories:
perl -p -i -e 's/A/B/g' `grep -ril "searchpattern" *`
Replace A with B in all files of type .html containing “searchpattern” in the current directory and subdirectories:
perl -p -i -e 's/A/B/g' `grep -ril --include="*.html" "searchpattern" *`
Same, using find and sed:
find -name '*' -print -exec sed -i 's/A/B/g' {} ;
Replace A with B in all files in current directory only:
sed -i 's/A/B/g' *
Delete lines matching “searchpattern” in a specified file, output new file to disk:
sed '/searchpattern/d' file.txt > newfile.txt
Delete the first 20 lines of a file, output new file to disk:
sed '1,20d' file.txt > newfile.txt
Delete duplicate, consecutive lines from a file (first line of duplicates is kept, the rest are deleted), output new file to disk:
sed '$!N; /^(.*)n1$/!P; D' file.txt > newfile.txt
Delete duplicate, nonconsecutive lines from a file, output new file to disk:
sed -n 'G; s/n/&&/; /^([ -~]*n).*n1/d; s/n//; h; P' file.txt > newfile.txt
Delete all lines in a file except duplicates, output new file to disk:
sed '$!N; s/^(.*)n1$/1/; t; D' file.txt > newfile.txt
Number each line of a file (right-aligned), output new file to disk:
sed = file.txt | sed 'N; s/^/ /; s/ *(.{6,})n/1 /' > newfile.txt
Double-space a file, output new file to disk:
sed G file.txt > newfile.txt
Double-space a file that already has blank lines in it, output new file to disk:
sed '/^$/d;G' file.txt > newfile.txt
Insert a blank line above/below/above and below every line matching “searchpattern”, output new file to disk:
sed '/searchpattern/{x;p;x;}' file.txt > newfile.txt
sed '/searchpattern/G' file.txt > newfile.txt
sed '/searchpattern/{x;p;x;G;}' file.txt > newfile.txt
Remove most HTML tags, output new file to disk:
sed -e :a -e 's/<[^>]*>//g;/</N;//ba' file.txt > newfile.txt
