While troubleshooting any issue, log analysis is the most important step. Mostly the log files capture enormous amount of information, and reading those becomes a difficult and time consuming task. In our daily debugging we need to analyze logs files of various products.
Analyzing the logs to isolate and resolving the issues, can be complex and requires special debugging skills which are gained through experience or by god’s grace . During debugging we might need to extract some data from the log files, or we need to play with a log file which can not be done by just reading through the log file line by line , there is need for special commands to reduce the overall efforts and provide the specific information we seek.
There are many commands in linux which are used by debuggers like grep, awk, sed, wc, taskset, ps, sort, uniq, cut, xargs etc .
In this blog we will see some examples of practical usage of <strong>grep</strong>, useful in real world debugging in Linux . The examples which we will see in this blog are super basic but very useful in real life which a beginner should read to enhance the debugging skills
Grep (global search for regular expression and print out)</strong> is a linux command searches a file for a given pattern, and displays the lines which match the pattern. The pattern is also referred to as regular expression.
Let’s Go to the Practical Part.
Lets say we have a file “”file1.log””, which has following lines.
root@localhost playground]# cat file1.log
hello
i am sahil
i am software engineer
Sahil is a software engineer
sahil is a software engineer
Search the lines which contains some particular word
root@localhost playground]# grep 'sahil' file1.log
i am sahil
sahil is a software engineer
Search number of lines matched for a particular word in a file
grep -c 'sahil' file1.log
2
Another way :
grep 'sahil' file1.log | wc -l
2
Search all the lines which contains some word (case insensitive)
root@localhost playground]# grep -i 'sahil' file1.log
i am sahil
Sahil is a software engineer
sahil is a software engineer
Search the lines in which either of two words are present in a file
root@localhost playground]# grep 'sahil|software' file1.log
i am sahil
i am software engineer
Sahil is a software engineer
sahil is a software engineer
Search lines in which two words are present
root@localhost playground]# grep 'sahil' file1.log | grep 'software'
sahil is a software engineer
Search lines excluding some word
root@localhost playground]# grep -v 'sahil' file1.log
hello
i am software engineer
Sahil is a software engineer
Exclude words case insensitively
root@localhost playground]# grep -iv 'sahil' file1.log
hello
i am software engineer
Search the lines that start with a string
root@localhost playground]# grep '^sahil' file1.log
sahil is a software engineer
Search the lines that end with a string
grep 'engineer$' file1.log
i am software engineer
Sahil is a software engineer
sahil is a software engineer
Getting n number of lines after each match
root@localhost playground]# grep 'hello' file1.log
hello
root@localhost playground]# grep -A 1 'hello' file1.log
hello
i am sahil
root@localhost playground]# grep -A 2 'hello' file1.log
hello
i am sahil
i am software engineer
Getting n number of lines before each match
root@localhost playground]# grep 'i am sahil' file1.log
i am sahil
root@localhost playground]# grep -B 1 'i am sahil' file1.log
hello
i am sahil
root@localhost playground]# grep -B 2 'i am sahil' file1.log
hello
i am sahil
in the second case only one line is printed as it is the only line before our pattern
Get n lines after and m lines before every match
root@localhost playground]# grep -A 2 -B 1 'i am sahil' file1.log
hello
i am sahil
i am software engineer
Sahil is a software engineer
Get some word in more than one file in current directory
For this purpose we will assume we also have a second file “”file2.log”” in the same directory
root@localhost playground]# cat file2.log
hello
i am sahil
i am tech blogger
Sahil is a tech blogger
sahil is a tech blogger
Grep can be used to search in more than one file or within a directory
root@localhost playground]# grep 'sahil' file1.log file2.log
file1.log:i am sahil
file1.log:sahil is a software engineer
file2.log:i am sahil
file2.log:sahil is a tech blogger
Grep some word in all files in current directory
root@localhost playground]# grep 'sahil' *
file1.log:i am sahil
file1.log:sahil is a software engineer
file2.log:i am sahil
file2.log:sahil is a tech blogger
Check how many lines matched in each file
root@localhost playground]# grep -c 'sahil' *
file1.log:2
file2.log:2
file.log:0
Note : the above output signifies, we have a third file in the directory “”file.log””, but it has no lines that have a word “”sahil””
Grep using regular expression
Regular expressions are patterns used to match character combinations in strings
Suppose the content of files are as follows
root@localhost playground]# cat file3.log
time taken by api is 1211 ms
time taken by api is 2000 ms
time taken by api is 3000 ms
time taken by api is 4000 ms
time taken by api is 50000 ms
time taken by api is 123 ms
time taken by api is 213 ms
time taken by api is 456 ms
time taken by api is 1000 ms
Now suppose we want to grep all the lines in which time taken by any api is more than 1 second or more than 1000 ms , it means it should have minimum 4 digit number, grep command for this will be as follows
root@localhost playground]# grep -P '[0-9]{4} ms' file3.log
time taken by api is 1211 ms
time taken by api is 2000 ms
time taken by api is 3000 ms
time taken by api is 4000 ms
time taken by api is 50000 ms
time taken by api is 1000 ms
If we want to get 5 digit number
root@localhost playground]# grep -P '[0-9]{5} ms' file3.log
time taken by api is 50000 ms
Recursively search in a directory and sub directories
root@localhost playground]# grep -R 'sahil' .
./dir1/file.log:i am sahil
./dir1/file.log:sahil is a software engineer
./file1.log:i am sahil
./file1.log:sahil is a software engineer
./file2.log:i am sahil
./file2.log:sahil is a tech blogger
All above are basic use cases of grep . One can mix all the command options of grep to achieve the complex use cases and one can also mix different grep commands using pipe operator to achieve complex use cases
In future blogs we will explain some complex use cases and example how to achieve that using linux commands which can ease logs debugging.
Stay Tuned . . .
See Original Posts at hello-worlds
and at medium
and at dzone
..
Author
Sahil Aggarwal
Senior Software Engineer 3/Tech Lead at Ameyo
I am a Tech Professional working in IT industry from 10 years . I have worked on many technologies like java, databases, linux, netrowking, security and lot more . I generally believe in taking outside-in approach to any new product you are looking for . This behaviour helped me lot in debugging issues of new products or even issues of any other system throughout by journey. I like to write tech blogs and I have my personal blog sites also, I write on medium and dzone also . I am very adaptive working between Individual Contributor and as a Leader/Manager. Apart from technology, I like eating fast and junk food . I also like to listen and write shyari.
