How to search and replace a multi-line string in a file

Question: I have a text file in which I want to change multiple lines of text to something else, but without using a text editor. Is there a way to find and replace a multi-line string pattern from the Linux command line?

Suppose you have a text file that looks like the following.

Beginning of a text file.

This is a cat.
This is a dog.
This is a guinea pig.
This is a gecko.
This is a hamster.
This is a parrot.
This is a rabbit.
This is a ferret.
This is a turtle.

You want to replace the highlighted multiple lines with the following one-line, but without using a text editor.

Beginning of a text file.

This is a cat.
This is a dog.
These were removed.
This is a parrot.
This is a rabbit.
This is a ferret.
This is a turtle.

Search and Replace a Multi-line String Pattern

When it comes to finding and replacing a multi-line string pattern, Perl's regular expression based pattern matching comes in handy.

The following one-liner can get the job done.

$ perl -i -0pe 's/<multi-line-string-pattern>/<replacement-string>/' input.txt

A couple of notes in the above expression:

  • The "-i" option tells Perl to perform in-place editing, meaning that Perl reads the input file, makes substitutions, and writes the results back to the same input file. If you want to dry-run the changes, you can omit this option, in which case the results will be shown to stdout without modifying the input file.
  • The "-0" option turns Perl into "file slurp" mode, where Perl reads the entire input file in one shot (intead of line by line). This enables multi-line search and replace.
  • The "-pe" option allows you to run Perl code (pattern matching and replacement in this case) and display output from the command line.

In this particular example, the following command will replace a multi-line string as required, and display the result (without applying the change to the input file).

$ perl -0pe 's/This is a guinea pig.\nThis is a gecko.\nThis is a hamster.\n/These were removed.\n/' input.txt

Once confirming that the result is correct, you can apply the change to the input file by adding "-i" option:

$ perl -i -0pe 's/This is a guinea pig.\nThis is a gecko.\nThis is a hamster.\n/These were removed.\n/' input.txt

Here are some variations:

To find a multi-line string pattern in a case-insensitive fashion:

$ perl -i -0pe 's/<multi-line-string-pattern>/<replacement-string>/i' input.txt

To remove a multi-line string pattern:

$ perl -i -0pe 's/<multi-line-string-pattern>//' input.txt

Search and Replace a Multi-line String in Multiple Files

If you want to find and replace a multi-line string in more than one files, you can use a combination of find and xargs as follows.

$ find ~/doc -name "*.txt" | xargs perl -i -0pe 's/<multi-line-string-pattern>/<replacement-string>/'

This one-liner searches for all text files in ~/doc folder, and apply multi-line string replacement in each file.

Download this article as ad-free PDF (made possible by your kind donation): 
Download PDF

Subscribe to Ask Xmodulo

Do you want to receive Linux related questions & answers published at Ask Xmodulo? Enter your email address below, and we will deliver our Linux Q&A straight to your email box, for free. Delivery powered by Google Feedburner.


Support Xmodulo

Did you find this tutorial helpful? Then please be generous and support Xmodulo!

3 thoughts on “How to search and replace a multi-line string in a file

  1. What if the line you want to be replaced, or removed, is not the same each line but has some of the same content?
    For example, I have an SMTP log that I need multiple lines removed but the lines are not completely the same. The lines start with a date, time, destination IP address and other items; basically some unique information per line. However there is a consistent string in each line the can be compared, like so:

    2015-10-31 18:35:33 'unique IP address' OutboundConnectionCommand SMTPSVC1 SMTP1 - 25 MAIL - FROM:+SIZE=137851 0 0 4 0 78 SMTP - - - -

    The only item consistent in each of the lines that I want to remove is the string 'OutboundConnectionCommand'.
    Can this be done? Can Perl remove those multiple lines with that string in the line?
    I tried this from your example in this post but it did not work:

    perl -i -0pe 's/OutboundConnectionCommand\n/0\n/' smtp_log.txt

    • Your example is not "multi-line string pattern". Rather it's line-by-line find and replace.

      The following will do:
      perl -i -pe 's/^.*OutboundConnectionCommand.*\n$//' smtp_log.txt

      Basically you look for any line that contains "OutboundConnectionCommand" anywhere in the middle, and remove the entire line. In Perl, "^" and "$" match the beginning and end of a line, respectively.

Leave a comment

Your email address will not be published. Required fields are marked *