nixCraft

I am facing problem while i execute one C program on large input files. The program is running correctly, there is no segmentation fault. But after time (around 56-60 sec:- as shown in 'top' command) the program is getting aborted automatically. When i ran the command 'top', i could see that root executes the process kswapd (pid=9) many times till my a.out gets stopped..

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAN
6245 shinni 24 0 284m 170m 2156 R 73.9 68.1 0:57.60 a.out
9 root 15 0 0 0 0 D 0.7 0.0 0:00 .53 kswapd0

I tried several inputs, but when the time is exceeds the above specified threshold, the program gets aborted automatically.. is there any system varibale i can configure so that i can run and check the programs for any period of time I want ? System becomes very slow when i run the program ? Why system kills the process automatically ? I was running the program , using KDE desktop theme and there were no other forground processes. OS is Fedora core 3. RAM is 256 MB. What alternatives I shall try to solve this ?

Is the program is getting aborted because of insufficient swap memory space, what shall i do ?

Please help.

Regards

Shinni

Well several things
1) Since you are running the KDE and other GUI which itself eats up 256 core ram and how much swap you got? Kernel kills the process if it won’t found much RAM+SWAP in order to avoid the instability

2) How much swap space you got?

3) Did you got any core file created in directory?

4) Can you run program without GUI in run level 3?

5) I hope your program is not buggy? If it is buggy especially using pointers then it could be blocked because of SELinux (Security-Enhanced Linux) FC kernel. It includes Discretionary access control (DAC) and Mandatory access control (MAC). The later closely controls user and process interactions with the system, and can provide protection from broken software or malware So try to disable SELinux termporary and then run the program different ways exist first method is type following at shell prompt as root
and run the program
Second method is disabled it all tougher and reboot the system
1) Open the file /etc/sysconfig/selinux
2) And setup look for SELINUX and set value to disabled
3) Reboot linux
Caution you should always backup the system before playing with SELinux especially if you are a new

See NSA SELinux main website — http://www.nsa.gov/selinux/
SELinux community page — http://selinux.sourceforge.net

Hope this help also if possible paste your code here

My code is for mi ning frequent subtrees from a set of input trees. The basic code , i got from the link http://chasen.org/~taku/software/freqt/and i made a few changes so that it will handle a set of XML files as input. The input file size is around 1 MB only , but there is lot of recursion using each substrig, and standard templete libraries are being used. So in the program, there is no need for memory allocation or deallocation

There 4 C++ files for mining , so pasting in this same window will just confuse the readers. I can not send attachments , it seems ?

I had posted in some other forums also , and i got various suggestions. According to those replies , i tried

1. ulimit -s unlimited , to set more swap space
2. created a 128 mb swap file , usind dd , and /sbin/swapon commands..

But both the abpve steps did not help , In second case , the swap file was being created , but program was not using this swap file.

3. I tried setenforce = 0 , but program is getting aborted.
In /etc/sysconfig/ there is no file with name selinux, so i could not try that.

If any of u think of some solutions, please feed back. Is is possible to post the files somewhere for public viewing?

Regards

Shinni

That means your program is buggy or some other stuff causing it to abort what error code you get as soon as program aborted? Use command echo $? to find out. You can also take the help of gdb to debug the program so that you can find out or put your code somewhere on net so that we can understand it above url is not working...

the link is http://chasen.org/~taku/software/fre...qt-0.22.tar.gz for the basic code. I had added XMl parsing modules etc to the main program, but the problematic portion is due the enumeration part in the above program.

I had posted the qn in another forum also where i uploaded my binaries
http://www.fedoraforum.org/forum/sho...743#post220743
( i hope it is ok t mention another forum name here :-/ )

The program is not buggy(I hope..) , but eats up all the memory while creating all possible substrings from the input data. So when it gets aborted, system says "Aborted" - and there is no other error message.

I tried upto 128 MB swap file only , but I'm not sure if program was using that swap file. It seems only from the defualt swap space program was using.

Default swap size seems to be 64 MB acc. to ulimit -aH command.

Core files are not created because , the default size is set to zeo.) .

I tried not using GUI for running the program , but got the same "Aborted" message in the terminal.

Thank you for the reply Nixcraft.

Okay i did run the code and it worked without problem it created big 300+ MB output file i started it as follows:
If this correct then it worked. Now few notes

My system is P4 having 184 MB ram and 512 MB SWAP space and i'm using
* RHEL 3.0 update 2
* gcc-c++ v3.2.3
During the entier run my system upload gone just 0.28 to 0.58 i stoped program after creating 330MB output file.
Next i tried same on OpenBSD this is the big server P42.4+1 GIG RAM+2GIG SWAP (gcc 2.95.3+ openbsd 3.6) and my account limit is 50MB disk space and with other process limitation still it worked and genrated tons of output on screen finally after 2+ minute later i stoped.

So did i run the program correctly? If so then you may try out program on other compiler/distro In all cased as you said it never Aborted i have to stop it to avoid DoS attack against my disk space

But let me know before i dig up the code

Hi Nixcraft ,

Thanks a lot for running the code and giving feed back.

Did you use the input file that i posted at
http://www.fedoraforum.org/forum/sho...43#post220743?
I had given the shell script to run the binary and the input file here.

were the flags your used for running freqT : -m 50 -L 10 ?
./freqT -m 50 -L 10 < data > output

the original code ( freqT) also stops runnig on this input when given these flags.

In your reply , the command tried is given as (./freqt -w <data > output ).
May be -m and -L flags might have got omitted. In this case, default value of m=1 is taken and program outputs all strings that it can enumerate.

Could u run with the flags i said ? I really am not trying put a DoS attack your system . But it will eb very helpful if i see that it is running correctly on some other configuration.

The code seems to be ok , without any memory leakge since it is using only STL containers. But may be due to recursion, program is eating up too much swap space.
I want to try out a 1GB swap file, but program does not seems to use the temporary swap files that is created.

I'm using fedora core version 3 and gcc version 3.3.3 on P4 processor 256 MB RAM.

Thanks and Regards

Shinni

Okay this time it Aborted but i did not experienced the memory problem at all… is your input file correct? If I run same program on old data file it works without any problem but new data file gets Aborted

I think algorithm does not handles the large input data correctly may be you need to optimize the algo or try to use large buffer may be that help. One more thing I noticed the file data (old one) is in ascii text format and the new data file is in Mathematica, or Pascal, Modula-2 or 3 code text format. May that could be problem but not sure

__________________
Vivek | My personal blog
Linux Evangelist
Play hard stay cool