nixCraft Linux / UNIX / Shell Scripting Forum

nixCraft

Linux / UNIX Tech Support Forum

awk script to search number and strings

This is a discussion on awk script to search number and strings within the Shell scripting forums, part of the Development/Scripting category; Hello All, I am new to awk I have a logfile name "app.log" with the following contnets All i have ...


Register free or login to your account to remove all advertisements.

Go Back   nixCraft Linux / UNIX / Shell Scripting Forum > Development/Scripting > Shell scripting

Linux answers from nixCraft.


Shell scripting You can discuss the shell scripting, request shell scripts and scripting techniques

Reply

 

Thread Tools Display Modes
  #1 (permalink)  
Old 1st February 2010, 09:59 PM
Junior Member
 
Join Date: Jan 2010
OS: Debian
Posts: 6
Thanks: 0
Thanked 1 Time in 1 Post
Rep Power: 0
george mal is on a distinguished road
Post awk script to search number and strings

Hello All,

I am new to awk

I have a logfile name "app.log" with the following contnets
All i have to do is check if the first field is time (14:3....) if so remove it and print the rest
-------------------------------------------------------------------------------------------------------------
14:32:07.331 ERROR set User Value fail, xx Field is not exist:IMEI
14:32:10.607 ERROR Cannot cancel cache update - current state (2) is not UPDATE_IN_PROGRESS
java.lang.IllegalStateException: Cannot cancel cache update - current state (2) is not UPDATE_IN_PROGRESS

xx_application error id=2030 Other errors

14:35:07.331 ERROR set User Value fail, xx Field is not exist:IMEI
14:35:10.607 ERROR Cannot cancel cache update - current state (2) is not UPDATE_IN_PROGRESS
-------------------------------------------------------------------------------------------------------------


I have written an awk script named "log_filter.awk" as given below
-------------------------------------------------------------------------------------------------------------
BEGIN {
if ( $1 ~ /[a-z]/ ) {print $0
} else {
print $2, $3, $4, $5, $6, $7, $8
}
}
-------------------------------------------------------------------------------------------------------------

Now when i run the command
awk -f log_filter.awk app.log

I am not getting any output
All i have to do is check if the first field is time (14:3....) if so remove it and print the rest

Also If i remove the BEGIN Statement i get the below given output
Should the awk script be always be enclosed within "BEGIN {}" Statement ?????
-------------------------------------------------------------------------------------------------------------
awk: log_filter.awk:4: WB
awk: log_filter.awk:4: ^ syntax error
-------------------------------------------------------------------------------------------------------------

Any Help would be appreciated
Regards
George
Reply With Quote
  #2 (permalink)  
Old 1st February 2010, 10:27 PM
jaysunn's Avatar
Contributors
 
Join Date: Apr 2009
Location: 41.332032,-73.089775
OS: RHEL - OSX
Scripting language: bourne again shell
Posts: 767
Thanks: 108
Thanked 104 Times in 95 Posts
Rep Power: 14
jaysunn has much to be proud ofjaysunn has much to be proud ofjaysunn has much to be proud ofjaysunn has much to be proud ofjaysunn has much to be proud ofjaysunn has much to be proud ofjaysunn has much to be proud ofjaysunn has much to be proud ofjaysunn has much to be proud of
Default

Does it need to be awk?

If not why not try this:

Code:
jason@kalturavideo1:~/test$ cat infile 
14:32:07.331 ERROR set User Value fail, xx Field is not exist:IMEI
14:32:10.607 ERROR Cannot cancel cache update - current state (2) is not UPDATE_IN_PROGRESS
java.lang.IllegalStateException: Cannot cancel cache update - current state (2) is not UPDATE_IN_PROGRESS

xx_application error id=2030 Other errors

14:35:07.331 ERROR set User Value fail, xx Field is not exist:IMEI
14:35:10.607 ERROR Cannot cancel cache update - current state (2) is not UPDATE_IN_PROGRESS
Code:
jason@kalturavideo1:~/test$ grep -v "14:3" infile 
java.lang.IllegalStateException: Cannot cancel cache update - current state (2) is not UPDATE_IN_PROGRESS

xx_application error id=2030 Other errors

Not sure if this is what you are after. And if you need it be be piped to a file you can use:

Code:
jason@kalturavideo1:~/test$ grep -v "14:3" infile > outfile

HTH,

jaysunn
__________________
Have a look at what I have been working on
http://www.shellasaurus.com
Reply With Quote
  #3 (permalink)  
Old 2nd February 2010, 07:28 AM
Senior Member
 
Join Date: Sep 2006
Posts: 110
Thanks: 0
Thanked 32 Times in 28 Posts
Rep Power: 7
ghostdog74 is a jewel in the roughghostdog74 is a jewel in the roughghostdog74 is a jewel in the roughghostdog74 is a jewel in the rough
Default

@OP, $1 would not have a value if you put it in BEGIN{} block.

Code:
awk '/^14:3/' app.log

Last edited by ghostdog74; 2nd February 2010 at 07:38 AM.
Reply With Quote
  #4 (permalink)  
Old 2nd February 2010, 07:29 AM
Senior Member
 
Join Date: Sep 2006
Posts: 110
Thanks: 0
Thanked 32 Times in 28 Posts
Rep Power: 7
ghostdog74 is a jewel in the roughghostdog74 is a jewel in the roughghostdog74 is a jewel in the roughghostdog74 is a jewel in the rough
Default

Quote:
Originally Posted by jaysunn View Post

Code:
jason@kalturavideo1:~/test$ grep -v "14:3" infile > outfile
you might want to give a ^ to signify the start of the line.
Reply With Quote
  #5 (permalink)  
Old 2nd February 2010, 06:40 PM
Junior Member
 
Join Date: Jan 2010
OS: Debian
Posts: 6
Thanks: 0
Thanked 1 Time in 1 Post
Rep Power: 0
george mal is on a distinguished road
Default

Hi jaysunn,

Thanks for the reply, But as you can see from your result
It has completely omitted the line with "14:3"
All I want to do is cut the first field (only time part), if it has numeric value
I want to see the entire log file as it is, I just want to remove the time field (if present)

Last edited by george mal; 2nd February 2010 at 10:09 PM.
Reply With Quote
  #6 (permalink)  
Old 2nd February 2010, 06:49 PM
Junior Member
 
Join Date: Jan 2010
OS: Debian
Posts: 6
Thanks: 0
Thanked 1 Time in 1 Post
Rep Power: 0
george mal is on a distinguished road
Default

Quote:
Originally Posted by ghostdog74 View Post
@OP, $1 would not have a value if you put it in BEGIN{} block.

Code:
awk '/^14:3/' app.log
------------------------------
Thanks for the help
but as you can see from my query above
if i do not put the Begin quote, it gives me a syntax error as shown below
--------------------------------------------
awk: log_filter.awk:4: WB
awk: log_filter.awk:4: ^ syntax error
--------------------------------------------
Reply With Quote
  #7 (permalink)  
Old 2nd February 2010, 06:55 PM
jaysunn's Avatar
Contributors
 
Join Date: Apr 2009
Location: 41.332032,-73.089775
OS: RHEL - OSX
Scripting language: bourne again shell
Posts: 767
Thanks: 108
Thanked 104 Times in 95 Posts
Rep Power: 14
jaysunn has much to be proud ofjaysunn has much to be proud ofjaysunn has much to be proud ofjaysunn has much to be proud ofjaysunn has much to be proud ofjaysunn has much to be proud ofjaysunn has much to be proud ofjaysunn has much to be proud ofjaysunn has much to be proud of
Default

Maybe this:

Code:
[root@radio5 jason]# awk ' /14:3*/  {print $2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13}' app.log 
ERROR set User Value fail, xx Field is not exist:IMEI  
ERROR Cannot cancel cache update - current state (2) is not UPDATE_IN_PROGRESS
ERROR set User Value fail, xx Field is not exist:IMEI  
ERROR Cannot cancel cache update - current state (2) is not UPDATE_IN_PROGRESS
I am sure there a much neater ways of achieving this. Hopefully this is what you want.

Jaysunn
__________________
Have a look at what I have been working on
http://www.shellasaurus.com
Reply With Quote
  #8 (permalink)  
Old 2nd February 2010, 07:26 PM
Junior Member
 
Join Date: Jan 2010
OS: Debian
Posts: 6
Thanks: 0
Thanked 1 Time in 1 Post
Rep Power: 0
george mal is on a distinguished road
Default

Quote:
Originally Posted by jaysunn View Post
Maybe this:

Code:
[root@radio5 jason]# awk ' /14:3*/  {print $2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13}' app.log 
ERROR set User Value fail, xx Field is not exist:IMEI  
ERROR Cannot cancel cache update - current state (2) is not UPDATE_IN_PROGRESS
ERROR set User Value fail, xx Field is not exist:IMEI  
ERROR Cannot cancel cache update - current state (2) is not UPDATE_IN_PROGRESS
I am sure there a much neater ways of achieving this. Hopefully this is what you want.

Jaysunn

Thanks
I had come up with a similar thing
awk -F " " '$1 ~ /^[1-9]/ { print $2 $3 $4 $5 $6 $7 $8 $9 $10 $11 $12 $13 }' test_1.log

However the problem with this is that, here it omits the line that do not start with the time-field
As you can see even for your command it has omitted the below given line
--------------------------
java.lang.IllegalStateException: Cannot cancel cache update - current state (2) is not UPDATE_IN_PROGRESS
xx_application error id=2030 Other errors
--------------------------

I want all line to be printed, just want to remove the time field (which is the first field) (if present)
This is exactly the reason why i had to write a script with an "if " clause in it.

Last edited by george mal; 2nd February 2010 at 07:58 PM.
Reply With Quote
  #9 (permalink)  
Old 10th February 2010, 04:30 PM
Junior Member
 
Join Date: Jan 2010
OS: Debian
Posts: 6
Thanks: 0
Thanked 1 Time in 1 Post
Rep Power: 0
george mal is on a distinguished road
Thumbs up

Dear All

I finally figured out the problem
for some reason you just need to add an extra "{" & "}" in the awk script
As of now the updated script is as given below

--------------------------------------------------------------------
{
if ($1 ~/[a-z]/)
{
print $0
}
else
{
print $2 " " $3 " " $4 " " $5 " " $6 " " $7 " " $8 " " $9 " " $10 " " $11 " " $12 " " $13 " " $14
}
}
--------------------------------------------------------------------

Now simply call this 'script" and "file-name" on which operation is to be performed with "awk" command and you will find the desired output
The Second print statement will remove the un-necessary "Time-Stamp" field

Last edited by george mal; 10th February 2010 at 04:33 PM.
Reply With Quote
The Following User Says Thank You to george mal For This Useful Post:
jaysunn (10th February 2010)
Reply

Tags
awk, awk script, awk syntax error


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads

Thread Thread Starter Forum Replies Last Post
Need shell script to search directories and delete based on criteria newtoscripting Shell scripting 1 7th January 2010 06:26 AM
Script to search and delete mails then log this etoshx Shell scripting 1 8th October 2009 10:17 PM
BIND named.conf search zone and comment it out script asim.mcp Shell scripting 2 15th June 2009 12:39 PM
Shell Script To change strings / text in a text file jaysunn Shell scripting 1 8th May 2009 05:58 PM
script to find repeted number/alphabet asim.mcp Shell scripting 2 9th April 2009 01:42 PM


All times are GMT +5.5. The time now is 08:46 PM.


Powered by vBulletin® Version 3.8.6 - Copyright ©2000 - 2010, Jelsoft Enterprises Ltd.
Search Engine Optimization by vBSEO 3.3.2
©2005-2010 nixCraft. All rights reserved

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40