Unix - Effective use of Tail / Find / Grep / Sed / Awk with Application logs For IBM websphere Application server

Unix - Effective use of Tail / Find / Grep / Sed / Awk with Application logs For IBM websphere Application server

To see only lines containing an error / exception in last 5000 lines.

tail -5000 /opt/WebSphere/AppServer/profiles/application/logs/SystemOut.log | grep -i "FileNotFoundException"

To see lines containing any of the multiple errors / exceptions in running logs -

tail -f /opt/WebSphere/AppServer/profiles/application/logs/SystemOut.log | egrep "(WSWS3713E|WSWS3734W|WSVR0605W|javax.net.ssl.SSLHandshakeException|ThreadMonitor)"

To see Error Snipets in running logs.

tail -f /opt/WebSphere/AppServer/profiles/application/logs/SystemOut.log | sed -n '/ERROR/,/EST/p'

To get all error / exception snippets in another file.

sed -n '/ERROR/,/EST/p' /opt/WebSphere/AppServer/profiles/application/logs/SystemOut.log* >> newFile.txt

To find occurences of a particular error in last n days.

find /opt/WebSphere/AppServer/profiles/application/logs/ -iname "SystemOut*" -mtime -7 -exec zgrep "FileNotFoundException" {} \; >> logAnalysis.txt

To count number of error / exception occurences in a log file.

sed -n '/ERROR/,/EST/p' /opt/WebSphere/AppServer/profiles/application/logs/SystemOut.log | grep "LogicBlockSetupException" | wc -l

To report the file size of all files bigger than 2 mb and older than 30 days.

find . -type f -size +4096 -atime +30 -exec \du -sk '{}' \;

Unix Commands for Junior Programmers

Q.How to check if the last command was successful in Unix?
Correct Answer.echo $?

Q.Which command is used to see running processes ?
Correct Answer.ps

Q.Command to print the top n lines of a file.
Correct Answer.head

Q.Command to print the last n lines of a file.
Correct Answer.tail

Q.Which command tells you how many lines, words, and characters there are in a file ?
Correct Answer.wc

Q.Command to know the directory where you are in currently ?
Correct Answer.pwd

Q.Command to display the disk usage ?
Correct Answer.du
Q.Commands used to compare files in unix
Correct Answer.Diff and Cmp

Q.Command to change file permissions ?
Correct Answer.chmod

Q.Command to search text in file ?
Correct Answer.all of above i.e. sed, awk, grep

e.g.
1)
>cat file.txt
unix is great os. unix is opensource. unix is free os.
learn operating system.
unixlinux which one you choose.

>sed 's/unix/linux/' file.txt
linux is great os. unix is opensource. unix is free os.
learn operating system.
linuxlinux which one you choose.

2)
Awk is one of the most powerful tools in Unix used for processing the rows and columns in a file. Awk has built in 
string functions and associative arrays. Awk supports most of the operators, conditional blocks, and loops available in C language. 

One of the good things is that you can convert Awk scripts into Perl scripts using a2p utility. 

The basic syntax of AWK:

awk 'BEGIN {start_action} {action} END {stop_action}' filename.

3)
Grep is the frequently used command in Unix (or Linux). Most of us use grep just for finding the words in a file. The power of grep comes with using its options and regular expressions. You can analyze large sets of log files with the help of grep command.

Grep stands for Global search for Regular Expressions and Print.

The basic syntax of grep command is

grep [options] pattern [list of files]

Let see some practical examples on grep command.

1. Running the last executed grep command

This saves a lot of time if you are executing the same command again and again.

!grep

This displays the last executed grep command and also prints the result set of the command on the terminal.

2. Search for a string in a file

This is the basic usage of grep command. It searches for the given string in the specified file.

grep "Error" logfile.txt

This searches for the string "Error" in the log file and prints all the lines that has the word "Error".

3. Searching for a string in multiple files.

grep "string" file1 file2
grep "string" file_pattern

This is also the basic usage of the grep command. You can manually specify the list of files you want to search or you can specify a file pattern (use regular expressions) to search for.

4. Case insensitive search

The -i option enables to search for a string case insensitively in the give file. It matches the words like "UNIX", "Unix", "unix".

grep -i "UNix" file.txt

5. Specifying the search string as a regular expression pattern.

grep "^[0-9].*" file.txt

This will search for the lines which starts with a number. Regular expressions is huge topic and I am not covering it here. This example is just for providing the usage of regular expressions.

6. Checking for the whole words in a file.

By default, grep matches the given string/pattern even if it found as a substring in a file. The -w option to grep makes it match only the whole words.

grep -w "world" file.txt

7. Displaying the lines before the match.

Some times, if you are searching for an error in a log file; it is always good to know the lines around the error lines to know the cause of the error.

grep -B 2 "Error" file.txt

This will prints the matched lines along with the two lines before the matched lines.

8. Displaying the lines after the match.

grep -A 3 "Error" file.txt

This will display the matched lines along with the three lines after the matched lines.

9. Displaying the lines around the match

grep -C 5 "Error" file.txt

This will display the matched lines and also five lines before and after the matched lines.

10. Searching for a sting in all files recursively

You can search for a string in all the files under the current directory and sub-directories with the help -r option.

grep -r "string" *

11. Inverting the pattern match

You can display the lines that are not matched with the specified search sting pattern using the -v option.

grep -v "string" file.txt

12. Displaying the non-empty lines

You can remove the blank lines using the grep command.

grep -v "^$" file.txt

13. Displaying the count of number of matches.

We can find the number of lines that matches the given string/pattern

grep -c "sting" file.txt

14. Display the file names that matches the pattern.

We can just display the files that contains the given string/pattern.

grep -l "string" *

15. Display the file names that do not contain the pattern.

We can display the files which do not contain the matched string/pattern.

grep -L "string" *

16. Displaying only the matched pattern.

By default, grep displays the entire line which has the matched string. We can make the grep to display only the matched string by using the -o option.

grep -o "string" file.txt

17. Displaying the line numbers.

We can make the grep command to display the position of the line which contains the matched string in a file using the -n option

grep -n "string" file.txt

18. Displaying the position of the matched string in the line

The -b option allows the grep command to display the character position of the matched string in a file.

grep -o -b "string" file.txt

19. Matching the lines that start with a string

The ^ regular expression pattern specifies the start of a line. This can be used in grep to match the lines which start with the given string or pattern.

grep "^start" file.txt

20. Matching the lines that end with a string

The $ regular expression pattern specifies the end of a line. This can be used in grep to match the lines which end with the given string or pattern.

grep "end$" file.txt

#############################################################################

Q.Which command is used to get the list of all files in a directory?
Correct Answer.ls

Q.Which command is used to find which operating system your system is running on in UNIX ?
Correct Answer.uname
Q.How to run a process in the background ?
Correct Answer.using &

Q.Command to bring the background process to foreground.
Correct Answer.fg
Q.Which command tells you about the unix command hostory ?
Correct Answer.history

Q.Command to know if the remote host is alive or not ?
Correct Answer.

ping and telnet

Q.Command to display the disk usage ?
Correct Answer.du
Q.Which of the following command is used for Displaying memory usage ?
Correct Answer.vmstat

Q.Command to display the contents page wise ?
Correct Answer.more
Q.Command to replace characters in a string or file ?
Correct Answer.sed

Do let me know for queries :)

Java 7 javadoc css -You could change it as per your requirements 

/* Javadoc style sheet */
/*
Overall document style
*/
body {
    background-color:#ffffff;
    color:#353833;
    font-family:Arial, Helvetica, sans-serif;
    font-size:76%;
    margin:0;
}
a:link, a:visited {
    text-decoration:none;
    color:#4c6b87;
}
a:hover, a:focus {
    text-decoration:none;
    color:#bb7a2a;
}
a:active {
    text-decoration:none;
    color:#4c6b87;
}
a[name] {
    color:#353833;
}
a[name]:hover {
    text-decoration:none;
    color:#353833;
}
pre {
    font-size:1.3em;
}
h1 {
    font-size:1.8em;
}
h2 {
    font-size:1.5em;
}
h3 {
    font-size:1.4em;
}
h4 {
    font-size:1.3em;
}
h5 {
    font-size:1.2em;
}
h6 {
    font-size:1.1em;
}
ul {
    list-style-type:disc;
}
code, tt {
    font-size:1.2em;
}
dt code {
    font-size:1.2em;
}
table tr td dt code {
    font-size:1.2em;
    vertical-align:top;
}
sup {
    font-size:.6em;
}
/*
Document title and Copyright styles
*/
.clear {
    clear:both;
    height:0px;
    overflow:hidden;
}
.aboutLanguage {
    float:right;
    padding:0px 21px;
    font-size:.8em;
    z-index:200;
    margin-top:-7px;
}
.legalCopy {
    margin-left:.5em;
}
.bar a, .bar a:link, .bar a:visited, .bar a:active {
    color:#FFFFFF;
    text-decoration:none;
}
.bar a:hover, .bar a:focus {
    color:#bb7a2a;
}
.tab {
    background-color:#0066FF;
    background-image:url(resources/titlebar.gif);
    background-position:left top;
    background-repeat:no-repeat;
    color:#ffffff;
    padding:8px;
    width:5em;
    font-weight:bold;
}
/*
Navigation bar styles
*/
.bar {
    background-image:url(resources/background.gif);
    background-repeat:repeat-x;
    color:#FFFFFF;
    padding:.8em .5em .4em .8em;
    height:auto;/*height:1.8em;*/
    font-size:1em;
    margin:0;
}
.topNav {
    background-image:url(resources/background.gif);
    background-repeat:repeat-x;
    color:#FFFFFF;
    float:left;
    padding:0;
    width:100%;
    clear:right;
    height:2.8em;
    padding-top:10px;
    overflow:hidden;
}
.bottomNav {
    margin-top:10px;
    background-image:url(resources/background.gif);
    background-repeat:repeat-x;
    color:#FFFFFF;
    float:left;
    padding:0;
    width:100%;
    clear:right;
    height:2.8em;
    padding-top:10px;
    overflow:hidden;
}
.subNav {
    background-color:#dee3e9;
    border-bottom:1px solid #9eadc0;
    float:left;
    width:100%;
    overflow:hidden;
}
.subNav div {
    clear:left;
    float:left;
    padding:0 0 5px 6px;
}
ul.navList, ul.subNavList {
    float:left;
    margin:0 25px 0 0;
    padding:0;
}
ul.navList li{
    list-style:none;
    float:left;
    padding:3px 6px;
}
ul.subNavList li{
    list-style:none;
    float:left;
    font-size:90%;
}
.topNav a:link, .topNav a:active, .topNav a:visited, .bottomNav a:link, .bottomNav a:active, .bottomNav a:visited {
    color:#FFFFFF;
    text-decoration:none;
}
.topNav a:hover, .bottomNav a:hover {
    text-decoration:none;
    color:#bb7a2a;
}
.navBarCell1Rev {
    background-image:url(resources/tab.gif);
    background-color:#a88834;
    color:#FFFFFF;
    margin: auto 5px;
    border:1px solid #c9aa44;
}
/*
Page header and footer styles
*/
.header, .footer {
    clear:both;
    margin:0 20px;
    padding:5px 0 0 0;
}
.indexHeader {
    margin:10px;
    position:relative;
}
.indexHeader h1 {
    font-size:1.3em;
}
.title {
    color:#2c4557;
    margin:10px 0;
}
.subTitle {
    margin:5px 0 0 0;
}
.header ul {
    margin:0 0 25px 0;
    padding:0;
}
.footer ul {
    margin:20px 0 5px 0;
}
.header ul li, .footer ul li {
    list-style:none;
    font-size:1.2em;
}
/*
Heading styles
*/
div.details ul.blockList ul.blockList ul.blockList li.blockList h4, div.details ul.blockList ul.blockList ul.blockListLast li.blockList h4 {
    background-color:#dee3e9;
    border-top:1px solid #9eadc0;
    border-bottom:1px solid #9eadc0;
    margin:0 0 6px -8px;
    padding:2px 5px;
}
ul.blockList ul.blockList ul.blockList li.blockList h3 {
    background-color:#dee3e9;
    border-top:1px solid #9eadc0;
    border-bottom:1px solid #9eadc0;
    margin:0 0 6px -8px;
    padding:2px 5px;
}
ul.blockList ul.blockList li.blockList h3 {
    padding:0;
    margin:15px 0;
}
ul.blockList li.blockList h2 {
    padding:0px 0 20px 0;
}
/*
Page layout container styles
*/
.contentContainer, .sourceContainer, .classUseContainer, .serializedFormContainer, .constantValuesContainer {
    clear:both;
    padding:10px 20px;
    position:relative;
}
.indexContainer {
    margin:10px;
    position:relative;
    font-size:1.0em;
}
.indexContainer h2 {
    font-size:1.1em;
    padding:0 0 3px 0;
}
.indexContainer ul {
    margin:0;
    padding:0;
}
.indexContainer ul li {
    list-style:none;
}
.contentContainer .description dl dt, .contentContainer .details dl dt, .serializedFormContainer dl dt {
    font-size:1.1em;
    font-weight:bold;
    margin:10px 0 0 0;
    color:#4E4E4E;
}
.contentContainer .description dl dd, .contentContainer .details dl dd, .serializedFormContainer dl dd {
    margin:10px 0 10px 20px;
}
.serializedFormContainer dl.nameValue dt {
    margin-left:1px;
    font-size:1.1em;
    display:inline;
    font-weight:bold;
}
.serializedFormContainer dl.nameValue dd {
    margin:0 0 0 1px;
    font-size:1.1em;
    display:inline;
}
/*
List styles
*/
ul.horizontal li {
    display:inline;
    font-size:0.9em;
}
ul.inheritance {
    margin:0;
    padding:0;
}
ul.inheritance li {
    display:inline;
    list-style:none;
}
ul.inheritance li ul.inheritance {
    margin-left:15px;
    padding-left:15px;
    padding-top:1px;
}
ul.blockList, ul.blockListLast {
    margin:10px 0 10px 0;
    padding:0;
}
ul.blockList li.blockList, ul.blockListLast li.blockList {
    list-style:none;
    margin-bottom:25px;
}
ul.blockList ul.blockList li.blockList, ul.blockList ul.blockListLast li.blockList {
    padding:0px 20px 5px 10px;
    border:1px solid #9eadc0;
    background-color:#f9f9f9;
}
ul.blockList ul.blockList ul.blockList li.blockList, ul.blockList ul.blockList ul.blockListLast li.blockList {
    padding:0 0 5px 8px;
    background-color:#ffffff;
    border:1px solid #9eadc0;
    border-top:none;
}
ul.blockList ul.blockList ul.blockList ul.blockList li.blockList {
    margin-left:0;
    padding-left:0;
    padding-bottom:15px;
    border:none;
    border-bottom:1px solid #9eadc0;
}
ul.blockList ul.blockList ul.blockList ul.blockList li.blockListLast {
    list-style:none;
    border-bottom:none;
    padding-bottom:0;
}
table tr td dl, table tr td dl dt, table tr td dl dd {
    margin-top:0;
    margin-bottom:1px;
}
/*
Table styles
*/
.contentContainer table, .classUseContainer table, .constantValuesContainer table {
    border-bottom:1px solid #9eadc0;
    width:100%;
}
.contentContainer ul li table, .classUseContainer ul li table, .constantValuesContainer ul li table {
    width:100%;
}
.contentContainer .description table, .contentContainer .details table {
    border-bottom:none;
}
.contentContainer ul li table th.colOne, .contentContainer ul li table th.colFirst, .contentContainer ul li table th.colLast, .classUseContainer ul li table th, .constantValuesContainer ul li table th, .contentContainer ul li table td.colOne, .contentContainer ul li table td.colFirst, .contentContainer ul li table td.colLast, .classUseContainer ul li table td, .constantValuesContainer ul li table td{
    vertical-align:top;
    padding-right:20px;
}
.contentContainer ul li table th.colLast, .classUseContainer ul li table th.colLast,.constantValuesContainer ul li table th.colLast,
.contentContainer ul li table td.colLast, .classUseContainer ul li table td.colLast,.constantValuesContainer ul li table td.colLast,
.contentContainer ul li table th.colOne, .classUseContainer ul li table th.colOne,
.contentContainer ul li table td.colOne, .classUseContainer ul li table td.colOne {
    padding-right:3px;
}
.overviewSummary caption, .packageSummary caption, .contentContainer ul.blockList li.blockList caption, .summary caption, .classUseContainer caption, .constantValuesContainer caption {
    position:relative;
    text-align:left;
    background-repeat:no-repeat;
    color:#FFFFFF;
    font-weight:bold;
    clear:none;
    overflow:hidden;
    padding:0px;
    margin:0px;
}
caption a:link, caption a:hover, caption a:active, caption a:visited {
    color:#FFFFFF;
}
.overviewSummary caption span, .packageSummary caption span, .contentContainer ul.blockList li.blockList caption span, .summary caption span, .classUseContainer caption span, .constantValuesContainer caption span {
    white-space:nowrap;
    padding-top:8px;
    padding-left:8px;
    display:block;
    float:left;
    background-image:url(resources/titlebar.gif);
    height:18px;
}
.overviewSummary .tabEnd, .packageSummary .tabEnd, .contentContainer ul.blockList li.blockList .tabEnd, .summary .tabEnd, .classUseContainer .tabEnd, .constantValuesContainer .tabEnd {
    width:10px;
    background-image:url(resources/titlebar_end.gif);
    background-repeat:no-repeat;
    background-position:top right;
    position:relative;
    float:left;
}
ul.blockList ul.blockList li.blockList table {
    margin:0 0 12px 0px;
    width:100%;
}
.tableSubHeadingColor {
    background-color: #EEEEFF;
}
.altColor {
    background-color:#eeeeef;
}
.rowColor {
    background-color:#ffffff;
}
.overviewSummary td, .packageSummary td, .contentContainer ul.blockList li.blockList td, .summary td, .classUseContainer td, .constantValuesContainer td {
    text-align:left;
    padding:3px 3px 3px 7px;
}
th.colFirst, th.colLast, th.colOne, .constantValuesContainer th {
    background:#dee3e9;
    border-top:1px solid #9eadc0;
    border-bottom:1px solid #9eadc0;
    text-align:left;
    padding:3px 3px 3px 7px;
}
td.colOne a:link, td.colOne a:active, td.colOne a:visited, td.colOne a:hover, td.colFirst a:link, td.colFirst a:active, td.colFirst a:visited, td.colFirst a:hover, td.colLast a:link, td.colLast a:active, td.colLast a:visited, td.colLast a:hover, .constantValuesContainer td a:link, .constantValuesContainer td a:active, .constantValuesContainer td a:visited, .constantValuesContainer td a:hover {
    font-weight:bold;
}
td.colFirst, th.colFirst {
    border-left:1px solid #9eadc0;
    white-space:nowrap;
}
td.colLast, th.colLast {
    border-right:1px solid #9eadc0;
}
td.colOne, th.colOne {
    border-right:1px solid #9eadc0;
    border-left:1px solid #9eadc0;
}
table.overviewSummary  {
    padding:0px;
    margin-left:0px;
}
table.overviewSummary td.colFirst, table.overviewSummary th.colFirst,
table.overviewSummary td.colOne, table.overviewSummary th.colOne {
    width:25%;
    vertical-align:middle;
}
table.packageSummary td.colFirst, table.overviewSummary th.colFirst {
    width:25%;
    vertical-align:middle;
}
/*
Content styles
*/
.description pre {
    margin-top:0;
}
.deprecatedContent {
    margin:0;
    padding:10px 0;
}
.docSummary {
    padding:0;
}
/*
Formatting effect styles
*/
.sourceLineNo {
    color:green;
    padding:0 30px 0 0;
}
h1.hidden {
    visibility:hidden;
    overflow:hidden;
    font-size:.9em;
}
.block {
    display:block;
    margin:3px 0 0 0;
}
.strong {
    font-weight:bold;
}

Direct path -
http://docs.oracle.com/javase/7/docs/api/stylesheet.css

RESTful Web Services With JSON response

With Jersey + Jackson + Jersey Java Client

Jersey uses Jackson to convert object to / form JSON. In this tutorial, we show you how to convert a “TTInformation” object into JSON format, and return it back to user.

1) Dependencies

To make Jersey support JSON mapping, use “jersey-json_version.jar” 

To use Jersey client APIs, declares “jersey-client_version.jar”

Please look at selected ones.

2) Integrate JSON with Jersey

In web.xml, declares “com.sun.jersey.api.json.POJOMappingFeature” as “init-param” in Jersey mapped servlet. It will make Jersey support JSON/object mapping.

<init-param>

<param-name>com.sun.jersey.api.json.POJOMappingFeature

<param-value>true

</init-param>

File : web.xml - full example.

3) POJO

A simple “TTInformation” object, later Jersey will convert it into JSON format.

4) JAX-RS with Jersey - GET and POST service

Annotate the method with @Produces(MediaType.APPLICATION_JSON). Jersey will use Jackson to handle the JSON conversion automatically.

5) Want to see OutPut     .... :)

6) Standalone Jersey GET Client

Jersey client to send a “GET” request and print out the returned json data.

Output

7) Standalone Jersey POST Client

Jersey client to send a “POST” request, with json data and print out the returned output.

Output

For source code -> sac10nikam@gmail.com