While troubleshooting any issue, log analysis is the most important step. Mostly the log files capture enormous amount of information, and reading those becomes a difficult and time consuming task. In our daily debugging we need to analyze logs files of various products.
Analyzing the logs to isolate and resolving the issues, can be complex and requires special debugging skills which are gained through experience or by god’s grace . During debugging we might need to extract some data from the log files, or we need to play with a log file which can not be done by just reading through the log file line by line , there is need for special commands to reduce the overall efforts and provide the specific information we seek.
There are many commands in linux which are used by debuggers like grep, awk, sed, wc, taskset, ps, sort, uniq, cut, xargs etc .
In this blog we will see some examples of practical usage of <strong>grep</strong>, useful in real world debugging in Linux . The examples which we will see in this blog are super basic but very useful in real life which a beginner should read to enhance the debugging skills
Grep (global search for regular expression and print out)</strong> is a linux command searches a file for a given pattern, and displays the lines which match the pattern. The pattern is also referred to as regular expression.
Let’s Go to the Practical Part.
Lets say we have a file “”file1.log””, which has following lines.
root@localhost playground]# cat file1.log
hello
i am sahil
i am software engineer
Sahil is a software engineer
sahil is a software engineerSearch the lines which contains some particular word
root@localhost playground]# grep 'sahil' file1.log
i am sahil
sahil is a software engineerSearch number of lines matched for a particular word in a file
grep -c 'sahil' file1.log
2Another way :
grep 'sahil' file1.log | wc -l
2Search all the lines which contains some word (case insensitive)
root@localhost playground]# grep -i 'sahil' file1.log
i am sahil
Sahil is a software engineer
sahil is a software engineerSearch the lines in which either of two words are present in a file
root@localhost playground]# grep 'sahil|software' file1.log
i am sahil
i am software engineer
Sahil is a software engineer
sahil is a software engineerSearch lines in which two words are present
root@localhost playground]# grep 'sahil' file1.log | grep 'software'
sahil is a software engineerSearch lines excluding some word
root@localhost playground]# grep -v 'sahil' file1.log
hello
i am software engineer
Sahil is a software engineerExclude words case insensitively
root@localhost playground]# grep -iv 'sahil' file1.log
hello
i am software engineerSearch the lines that start with a string
root@localhost playground]# grep '^sahil' file1.log
sahil is a software engineerSearch the lines that end with a string
grep 'engineer$' file1.log
i am software engineer
Sahil is a software engineer
sahil is a software engineerGetting n number of lines after each match
root@localhost playground]# grep 'hello' file1.log
hello
root@localhost playground]# grep -A 1 'hello' file1.log
hello
i am sahil
root@localhost playground]# grep -A 2 'hello' file1.log
hello
i am sahil
i am software engineerGetting n number of lines before each match
root@localhost playground]# grep 'i am sahil' file1.log
i am sahil
root@localhost playground]# grep -B 1 'i am sahil' file1.log
hello
i am sahil
root@localhost playground]# grep -B 2 'i am sahil' file1.log
hello
i am sahilin the second case only one line is printed as it is the only line before our pattern
Get n lines after and m lines before every match
root@localhost playground]# grep -A 2 -B 1 'i am sahil' file1.log
hello
i am sahil
i am software engineer
Sahil is a software engineerGet some word in more than one file in current directory
For this purpose we will assume we also have a second file “”file2.log”” in the same directory
root@localhost playground]# cat file2.log
hello
i am sahil
i am tech blogger
Sahil is a tech blogger
sahil is a tech bloggerGrep can be used to search in more than one file or within a directory
root@localhost playground]# grep 'sahil' file1.log file2.log
file1.log:i am sahil
file1.log:sahil is a software engineer
file2.log:i am sahil
file2.log:sahil is a tech bloggerGrep some word in all files in current directory
root@localhost playground]# grep 'sahil' *
file1.log:i am sahil
file1.log:sahil is a software engineer
file2.log:i am sahil
file2.log:sahil is a tech bloggerCheck how many lines matched in each file
root@localhost playground]# grep -c 'sahil' *
file1.log:2
file2.log:2
file.log:0Note : the above output signifies, we have a third file in the directory “”file.log””, but it has no lines that have a word “”sahil””
Grep using regular expression
Regular expressions are patterns used to match character combinations in strings
Suppose the content of files are as follows
root@localhost playground]# cat file3.log
time taken by api is 1211 ms
time taken by api is 2000 ms
time taken by api is 3000 ms
time taken by api is 4000 ms
time taken by api is 50000 ms
time taken by api is 123 ms
time taken by api is 213 ms
time taken by api is 456 ms
time taken by api is 1000 msNow suppose we want to grep all the lines in which time taken by any api is more than 1 second or more than 1000 ms , it means it should have minimum 4 digit number, grep command for this will be as follows
root@localhost playground]# grep -P '[0-9]{4} ms' file3.log
time taken by api is 1211 ms
time taken by api is 2000 ms
time taken by api is 3000 ms
time taken by api is 4000 ms
time taken by api is 50000 ms
time taken by api is 1000 msIf we want to get 5 digit number
root@localhost playground]# grep -P '[0-9]{5} ms' file3.log
time taken by api is 50000 msRecursively search in a directory and sub directories
root@localhost playground]# grep -R 'sahil' .
./dir1/file.log:i am sahil
./dir1/file.log:sahil is a software engineer
./file1.log:i am sahil
./file1.log:sahil is a software engineer
./file2.log:i am sahil
./file2.log:sahil is a tech bloggerAll above are basic use cases of grep . One can mix all the command options of grep to achieve the complex use cases and one can also mix different grep commands using pipe operator to achieve complex use cases
In future blogs we will explain some complex use cases and example how to achieve that using linux commands which can ease logs debugging.
Stay Tuned . . .
See Original Posts at hello-worlds
and at medium
and at dzone
..
Author
Sahil Aggarwal
Senior Software Engineer 3/Tech Lead at Ameyo
I am a Tech Professional working in IT industry from 10 years . I have worked on many technologies like java, databases, linux, netrowking, security and lot more . I generally believe in taking outside-in approach to any new product you are looking for . This behaviour helped me lot in debugging issues of new products or even issues of any other system throughout by journey. I like to write tech blogs and I have my personal blog sites also, I write on medium and dzone also . I am very adaptive working between Individual Contributor and as a Leader/Manager. Apart from technology, I like eating fast and junk food . I also like to listen and write shyari.