Posts in category: grep

While troubleshooting any issue,  log analysis is the most important step.  Mostly the log files capture enormous amount of information, and reading those becomes a difficult and time consuming task. In our daily debugging we need to analyze logs files of various products.

 Analyzing the logs to isolate and resolving the issues, can be complex and requires special debugging skills which are gained through experience or by god’s grace .  During debugging we might need to extract some data from the log files, or we need to play with a log file which can not be done by just reading through the log file line by line , there is need for special commands to reduce the overall efforts and provide the specific information we seek.

There are many commands in linux which are used by debuggers like grep, awk, sed, wc, taskset, ps, sort, uniq, cut, xargs etc .

In this blog we will see some examples of practical usage of <strong>grep</strong>,  useful in real world debugging&nbsp; in Linux . The examples which we will see in this blog are super basic but very useful in real life which a beginner should read to enhance the debugging skills

Grep (global search for regular expression and print out)</strong> is a linux command searches a file for a given pattern, and displays the lines which match the pattern.  The pattern is also referred to as regular expression.

Let’s Go to the Practical Part.

Lets say we have a file “”file1.log””, which has following lines.

root@localhost playground]# cat file1.log
hello
i am sahil
i am software engineer
Sahil is a software engineer
sahil is a software engineer

Search the lines which contains some particular word

root@localhost playground]# grep 'sahil' file1.log
i am sahil
sahil is a software engineer

Search number of lines matched for a particular word in a file

grep -c 'sahil' file1.log
2

Another way :

grep 'sahil' file1.log | wc -l
2

Search all the lines which contains some word (case insensitive)

root@localhost playground]# grep -i 'sahil' file1.log
i am sahil
Sahil is a software engineer
sahil is a software engineer

Search the lines in which either of two words are present in a file

root@localhost playground]# grep 'sahil|software' file1.log
i am sahil
i am software engineer
Sahil is a software engineer
sahil is a software engineer

Search lines in which two words are present

root@localhost playground]# grep 'sahil' file1.log | grep 'software'
sahil is a software engineer

Search lines excluding some word

root@localhost playground]# grep -v 'sahil' file1.log
hello
i am software engineer
Sahil is a software engineer

Exclude words case insensitively

root@localhost playground]# grep -iv 'sahil' file1.log
hello
i am software engineer

Search the lines that start with a string

root@localhost playground]# grep '^sahil' file1.log
sahil is a software engineer

Search the lines that end with a string

grep 'engineer$' file1.log
i am software engineer
Sahil is a software engineer
sahil is a software engineer

Getting n number of lines after each match

root@localhost playground]# grep 'hello' file1.log
hello

root@localhost playground]# grep -A 1 'hello' file1.log
hello
i am sahil

root@localhost playground]# grep -A 2 'hello' file1.log
hello
i am sahil
i am software engineer

Getting n number of lines before each match

root@localhost playground]# grep 'i am sahil' file1.log
i am sahil

root@localhost playground]# grep -B 1 'i am sahil' file1.log
hello
i am sahil

root@localhost playground]# grep -B 2 'i am sahil' file1.log
hello
i am sahil

in the second case only one line is printed as it is the only line before our pattern

Get n lines after and m lines before every match

root@localhost playground]# grep -A 2 -B 1 'i am sahil' file1.log
hello
i am sahil
i am software engineer
Sahil is a software engineer

Get some word in more than one file in current directory

For this purpose we will assume we also have a second file “”file2.log”” in the same directory

root@localhost playground]# cat file2.log
hello
i am sahil
i am tech blogger
Sahil is a tech blogger
sahil is a tech blogger

Grep can be used to search in more than one file or within a directory

root@localhost playground]# grep 'sahil' file1.log file2.log
file1.log:i am sahil
file1.log:sahil is a software engineer
file2.log:i am sahil
file2.log:sahil is a tech blogger

Grep some word in all files in current directory

root@localhost playground]# grep 'sahil' *
file1.log:i am sahil
file1.log:sahil is a software engineer
file2.log:i am sahil
file2.log:sahil is a tech blogger

Check how many lines matched in each file

root@localhost playground]# grep -c 'sahil' *
file1.log:2
file2.log:2
file.log:0

Note : the above output signifies, we have a third file in the directory “”file.log””, but it has no lines that have a word “”sahil””

Grep using regular expression

Regular expressions are patterns used to match character combinations in strings

Suppose the content of files are as follows

root@localhost playground]# cat file3.log
time taken by api is 1211 ms
time taken by api is 2000 ms
time taken by api is 3000 ms
time taken by api is 4000 ms
time taken by api is 50000 ms
time taken by api is 123 ms
time taken by api is 213 ms
time taken by api is 456 ms
time taken by api is 1000 ms

Now suppose we want to grep all the lines in which time taken by any api is more than 1 second or more than 1000 ms , it means it should have minimum 4 digit number, grep command for this will be as follows

root@localhost playground]# grep -P '[0-9]{4} ms' file3.log
time taken by api is 1211 ms
time taken by api is 2000 ms
time taken by api is 3000 ms
time taken by api is 4000 ms
time taken by api is 50000 ms
time taken by api is 1000 ms

If we want to get 5 digit number

root@localhost playground]# grep -P '[0-9]{5} ms' file3.log
time taken by api is 50000 ms

Recursively search in a directory and sub directories

root@localhost playground]# grep -R 'sahil' .
./dir1/file.log:i am sahil
./dir1/file.log:sahil is a software engineer
./file1.log:i am sahil
./file1.log:sahil is a software engineer
./file2.log:i am sahil
./file2.log:sahil is a tech blogger

All above are basic use cases of grep . One can mix all the command options of grep to achieve the complex use cases and one can also mix different grep commands using pipe operator to achieve complex use cases

In future blogs we will explain some complex use cases and example how to achieve that using linux commands which can ease logs debugging.

Stay Tuned . . .

See Original Posts at hello-worlds

and at medium

and at dzone

..

Author

Sahil Aggarwal
Senior Software Engineer 3/Tech Lead at Ameyo

I am a Tech Professional working in IT industry from 10 years . I have worked on many technologies like java, databases, linux, netrowking, security and lot more . I generally believe in taking outside-in approach to any new product you are looking for . This behaviour helped me lot in debugging issues of new products or even issues of any other system throughout by journey. I like to write tech blogs and I have my personal blog sites also, I write on medium and dzone also . I am very adaptive working between Individual Contributor and as a Leader/Manager. Apart from technology, I like eating fast and junk food . I also like to listen and write shyari.