Thread: Splitting one file into few others with column in the name

Hello everybody,

I faced an issue to solve.

Let's say that I have a file like this:

Big_file.txt

1|2|3|4|message1
2|3|4|5|message1
3|4|1|4|message2
3|4|2|2|message2
3|3|1|2|message2
4|1|0|2|message3
4|1|2|6|message1

I want to write a shell script that will split this file into few smallers like this:

File 1:

Name: File_message1.txt
1|2|3|4|message1
2|3|4|5|message1
4|1|2|6|message1

File 2:

Name: File_message2.txt
3|4|1|4|message2
3|4|2|2|message2
3|3|1|2|message2


File 3:

Name File_message3.txt
4|1|0|2|message3

Note: The column with messageX textis dynamic - always will be different text there.
Anyone knows how to solve it?
Thank you.
Nodex

based which calculation, you want to split file ? by line , any specific word or what ?

Originally Posted by Rahul.Patil

based which calculation, you want to split file ? by line , any specific word or what ?

I want to split file by last column, let's assume it's always 5th column.

So.. If last column = 'message1' then add this record to file named: "File_message1.txt"
If in last column will apear i.e. 40 different values then create 40 different files.

In other words - I want to move all the records from base file to smallers just looking at 5th column.

I hope this help..

Thank you.

Code:

awk 'BEGIN {FS="|";OFS="|"}{if ($NF=="message1") print $0 > "file1.txt"; else if ($NF=="message2") print $0 > "file2.txt";else print $0 > "file3.txt";}' abc.txt

where abc.txt is your file and file1.txt, file2.txt and file3.txt are corsponding files with your required data..

Originally Posted by kumarat9pm

Code:

awk 'BEGIN {FS="|";OFS="|"}{if ($NF=="message1") print $0 > "file1.txt"; else if ($NF=="message2") print $0 > "file2.txt";else print $0 > "file3.txt";}' abc.txt

where abc.txt is your file and file1.txt, file2.txt and file3.txt are corsponding files with your required data..

Thank you kumarat but it will works only with this file.
This file will be different each time. messageX is just an example. I can't put "message1" static into this script. And I expect more than 3 different values.

I guess we need to use a variable that will read value from 5th column and put it into file name (together with moving record there). Then next line should be read and do the same and over and over again till the EOF.

Hi,

PHP Code:

#!/bin/bash
Input_File="/tmp/input"
OutPut_Dir="/tmp/test"


[ -d "{OutPut_Dir}" ] || mkdir "${OutPut_Dir}"
k=0
oldline="$(head -1 "${Input_File}" | awk -F"|" '{print $5}')"
        
while read getlines
do
        
Check_last="$(echo "${getlines}" | awk -F"|" '{print $NF}' )"

        if [ "${Check_last}" == "${oldline}" ]; then
            
            echo "${getlines}" >> $OutPut_Dir/file$k
            
        else
            
            ((k++))
            echo "${getlines}" >> $OutPut_Dir/file$k
        
        fi
        
        oldline="${Check_last}"
        
done < <(sort  --field-separator="|" -k5  "${Input_File}") 


Originally Posted by Rahul.Patil

Hi,

PHP Code:

#!/bin/bash
Input_File="/tmp/input"
OutPut_Dir="/tmp/test"


[ -d "{OutPut_Dir}" ] || mkdir "${OutPut_Dir}"
k=0
oldline="$(head -1 "${Input_File}" | awk -F"|" '{print $5}')"
        
while read getlines
do
        
Check_last="$(echo "${getlines}" | awk -F"|" '{print $NF}' )"

        if [ "${Check_last}" == "${oldline}" ]; then
            
            echo "${getlines}" >> $OutPut_Dir/file$k
            
        else
            
            ((k++))
            echo "${getlines}" >> $OutPut_Dir/file$k
        
        fi
        
        oldline="${Check_last}"
        
done < <(sort  --field-separator="|" -k5  "${Input_File}") 


Hi

Can you translate it into shell script? )

Nodex.

just run it , it's bash script.

Originally Posted by Rahul.Patil

just run it , it's bash script.

I'm sorry. You're right. Give me 5 mins.

Originally Posted by Rahul.Patil

just run it , it's bash script.

Hmm.. I receive an error like this:

sort: unrecognized option: -
sort: unrecognized option: e
sort: unrecognized option: l
sort: unrecognized option: -
sort: unrecognized option: s
sort: unrecognized option: e
sort: unrecognized option: p
sort: unrecognized option: a
sort: unrecognized option: a
Syntax: sort [-Abcdfimnru] [-T Catalog]...