Page 1 of 2 1 2 LastLast
Results 1 to 10 of 18

Thread: Splitting one file into few others with column in the name

  1. #1
    Junior Member
    Join Date
    Jan 2013
    Posts
    9
    Thanks
    2
    Thanked 0 Times in 0 Posts
    Rep Power
    0

    Default Splitting one file into few others with column in the name

    Hello everybody,

    I faced an issue to solve.

    Let's say that I have a file like this:

    Big_file.txt

    1|2|3|4|message1
    2|3|4|5|message1
    3|4|1|4|message2
    3|4|2|2|message2
    3|3|1|2|message2
    4|1|0|2|message3
    4|1|2|6|message1

    I want to write a shell script that will split this file into few smallers like this:

    File 1:

    Name: File_message1.txt
    1|2|3|4|message1
    2|3|4|5|message1
    4|1|2|6|message1

    File 2:

    Name: File_message2.txt
    3|4|1|4|message2
    3|4|2|2|message2
    3|3|1|2|message2


    File 3:

    Name File_message3.txt
    4|1|0|2|message3

    Note: The column with messageX textis dynamic - always will be different text there.
    Anyone knows how to solve it?
    Thank you.
    Nodex

  2. #2
    Senior Member Rahul.Patil's Avatar
    Join Date
    Feb 2012
    Location
    Mumbai india
    Posts
    486
    Thanks
    10
    Thanked 50 Times in 47 Posts
    Rep Power
    8

    Default

    based which calculation, you want to split file ? by line , any specific word or what ?
    Rahul Patil <http://www.linuxian.com>

  3. #3
    Junior Member
    Join Date
    Jan 2013
    Posts
    9
    Thanks
    2
    Thanked 0 Times in 0 Posts
    Rep Power
    0

    Default

    Quote Originally Posted by Rahul.Patil View Post
    based which calculation, you want to split file ? by line , any specific word or what ?
    I want to split file by last column, let's assume it's always 5th column.

    So.. If last column = 'message1' then add this record to file named: "File_message1.txt"
    If in last column will apear i.e. 40 different values then create 40 different files.

    In other words - I want to move all the records from base file to smallers just looking at 5th column.

    I hope this help..

    Thank you.

  4. #4
    Senior Member
    Join Date
    Jun 2007
    Location
    Hyderabad, AP, India
    Posts
    806
    Thanks
    44
    Thanked 55 Times in 48 Posts
    Rep Power
    13

    Default

    Code:
    awk 'BEGIN {FS="|";OFS="|"}{if ($NF=="message1") print $0 > "file1.txt"; else if ($NF=="message2") print $0 > "file2.txt";else print $0 > "file3.txt";}' abc.txt
    where abc.txt is your file and file1.txt, file2.txt and file3.txt are corsponding files with your required data..
    Thanks,
    Surendra Kumar Anne
    Linux: Fast, friendly, flexible and .... free!
    Support Open source.
    http://www.linuxnix.com

  5. #5
    Junior Member
    Join Date
    Jan 2013
    Posts
    9
    Thanks
    2
    Thanked 0 Times in 0 Posts
    Rep Power
    0

    Default

    Quote Originally Posted by kumarat9pm View Post
    Code:
    awk 'BEGIN {FS="|";OFS="|"}{if ($NF=="message1") print $0 > "file1.txt"; else if ($NF=="message2") print $0 > "file2.txt";else print $0 > "file3.txt";}' abc.txt
    where abc.txt is your file and file1.txt, file2.txt and file3.txt are corsponding files with your required data..
    Thank you kumarat but it will works only with this file.
    This file will be different each time. messageX is just an example. I can't put "message1" static into this script. And I expect more than 3 different values.

    I guess we need to use a variable that will read value from 5th column and put it into file name (together with moving record there). Then next line should be read and do the same and over and over again till the EOF.

  6. #6
    Senior Member Rahul.Patil's Avatar
    Join Date
    Feb 2012
    Location
    Mumbai india
    Posts
    486
    Thanks
    10
    Thanked 50 Times in 47 Posts
    Rep Power
    8

    Default

    Hi,

    PHP Code:
    #!/bin/bash
    Input_File="/tmp/input"
    OutPut_Dir="/tmp/test"


    [ -"{OutPut_Dir}" ] || mkdir "${OutPut_Dir}"
    k=0
    oldline
    ="$(head -1 "${Input_File}" | awk -F"|" '{print $5}')"
            
    while read getlines
    do
            
    Check_last="$(echo "${getlines}" | awk -F"|" '{print $NF}' )"

            
    if [ "${Check_last}== "${oldline}]; then
                
                
    echo "${getlines}>> $OutPut_Dir/file$k
                
            
    else
                
                ((
    k++))
                echo 
    "${getlines}>> $OutPut_Dir/file$k
            
            fi
            
            oldline
    ="${Check_last}"
            
    done < <(sort  --field-separator="|" -k5  "${Input_File}"
    Rahul Patil <http://www.linuxian.com>

  7. The Following User Says Thank You to Rahul.Patil For This Useful Post:

    Nodex (12th January 2013)

  8. #7
    Junior Member
    Join Date
    Jan 2013
    Posts
    9
    Thanks
    2
    Thanked 0 Times in 0 Posts
    Rep Power
    0

    Default

    Quote Originally Posted by Rahul.Patil View Post
    Hi,

    PHP Code:
    #!/bin/bash
    Input_File="/tmp/input"
    OutPut_Dir="/tmp/test"


    [ -"{OutPut_Dir}" ] || mkdir "${OutPut_Dir}"
    k=0
    oldline
    ="$(head -1 "${Input_File}" | awk -F"|" '{print $5}')"
            
    while read getlines
    do
            
    Check_last="$(echo "${getlines}" | awk -F"|" '{print $NF}' )"

            
    if [ "${Check_last}== "${oldline}]; then
                
                
    echo "${getlines}>> $OutPut_Dir/file$k
                
            
    else
                
                ((
    k++))
                echo 
    "${getlines}>> $OutPut_Dir/file$k
            
            fi
            
            oldline
    ="${Check_last}"
            
    done < <(sort  --field-separator="|" -k5  "${Input_File}"
    Hi

    Can you translate it into shell script? )

    Nodex.

  9. #8
    Senior Member Rahul.Patil's Avatar
    Join Date
    Feb 2012
    Location
    Mumbai india
    Posts
    486
    Thanks
    10
    Thanked 50 Times in 47 Posts
    Rep Power
    8

    Default

    just run it , it's bash script.
    Rahul Patil <http://www.linuxian.com>

  10. #9
    Junior Member
    Join Date
    Jan 2013
    Posts
    9
    Thanks
    2
    Thanked 0 Times in 0 Posts
    Rep Power
    0

    Default

    Quote Originally Posted by Rahul.Patil View Post
    just run it , it's bash script.
    I'm sorry. You're right. Give me 5 mins.

  11. #10
    Junior Member
    Join Date
    Jan 2013
    Posts
    9
    Thanks
    2
    Thanked 0 Times in 0 Posts
    Rep Power
    0

    Default

    Quote Originally Posted by Rahul.Patil View Post
    just run it , it's bash script.
    Hmm.. I receive an error like this:

    sort: unrecognized option: -
    sort: unrecognized option: e
    sort: unrecognized option: l
    sort: unrecognized option: -
    sort: unrecognized option: s
    sort: unrecognized option: e
    sort: unrecognized option: p
    sort: unrecognized option: a
    sort: unrecognized option: a
    Syntax: sort [-Abcdfimnru] [-T Catalog]...

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. convert a column to row output?
    By mr_manny in forum Shell scripting
    Replies: 7
    Last Post: 21st March 2011, 11:50 AM
  2. To remove a particular column from a file
    By devi_vi in forum Shell scripting
    Replies: 7
    Last Post: 6th September 2010, 12:00 PM
  3. Taking an input and splitting it
    By cpickering in forum Shell scripting
    Replies: 3
    Last Post: 2nd March 2010, 10:48 PM
  4. [Solved] Line and column count in awk
    By raj in forum Shell scripting
    Replies: 2
    Last Post: 9th January 2010, 10:16 AM
  5. Splitting strings on complex criteria
    By DragonMaster in forum Coding in General
    Replies: 1
    Last Post: 5th November 2008, 08:47 PM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •