Awk is one of those command-line tools that is sadly underused. To help with that, I'm going to blog about any useful application of awk I come across.
Quite often at work I need to analyse Java thread dumps that come at the end of an enormous log file residing on a remote server somewhere. Grabbing X lines at the end of a file is one of the things awk is great at.
My first solution to this problem was to use grep with line numbers turned on to find the line number of the start of the thread dump:
$ grep -n 'Full thread dump' /var/log/java/extranet.atlassian.com/sysout.log 103720:Full thread dump Java HotSpot(TM) Server VM (1.5.0_11-b03 mixed mode):
With the line number, you use awk to print out everything from line 103720 onwards:
$ awk 'NR >= 103720 { print }' /var/log/java/extranet.atlassian.com/sysout.log | head
Full thread dump Java HotSpot(TM) Server VM (1.5.0_11-b03 mixed mode):
"TP-Processor60" daemon prio=1 tid=0x600abc30 nid=0x3d42 runnable [0x5814b000..0x5814bda0]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
- locked <0x97687c58> (a java.io.BufferedInputStream)
at org.apache.jk.common.ChannelSocket.read(ChannelSocket.java:626)
But an even better solution is possible with awk variables. You can skip the grep step entirely. Here we use an awk variable, p, as a flag to turn on printing once we reach the first thread dump line:
$ awk '/Full thread dump/ { p = 1 }; p { print }' /var/log/java/extranet.atlassian.com/sysout.log | head
Full thread dump Java HotSpot(TM) Server VM (1.5.0_11-b03 mixed mode):
"TP-Processor60" daemon prio=1 tid=0x600abc30 nid=0x3d42 runnable [0x5814b000..0x5814bda0]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
- locked <0x97687c58> (a java.io.BufferedInputStream)
at org.apache.jk.common.ChannelSocket.read(ChannelSocket.java:626)
