nixCraft

meowing · #1 (**permalink**) 06-02-2007, 06:12 PM

It seems I can use iconv for this. When I do
# iconv --from-code=ISO-8859-1 --to-code=UTF-8 ./oldfile.htm ./newfile.htm
it gives the right result for my webserver's config.

I have huge folders with htm and txt files to do this for, so I would like to not have to rename them by hand.
Plus I would like the script to check for certain characters in the files, in order to decide if a file should be converted or not.

I found this script somewhere:

Code:

#!/bin/bash

  for i in $* ; do
    echo "Converting $i ..."
    mv -i $i $i.bak
    iconv -f ISO8859-1 -t UTF-8 $i.bak >$i
  done

But it doesn't do anything when I run it. Also, I notice the dash in ISO-8859-1 is missing, is that of importance?

How do I add a character-search to it?

Only if my documents (only *.txt and *.ht*) contain
"é" or "à" or "è" or "ï" or "ë" or "ó"
they should be converted to UTF-8, if not they should be left alone.

I have done this before, but unfortunately lost the script and couldn't find much about it online. For example, how do I determine the character's presence? I have a separate backup of all the files, so a total replace script would do.
Any help or examples very much appreciated.

jerry · #2 (**permalink**) 06-03-2007, 12:00 AM

Correct code..

Code:

#!/bin/bash
mypath="$1"
  for i in "$mypath"
  do
    echo "Converting $i ..."
    mv -i $i $i.bak
    iconv -f ISO8859-1 -t UTF-8 $i.bak >$i
  done

set up permision and covert all html files

Code:

./script *.html

Quote:

"é" or "à" or "è" or "ï" or "ë" or "ó"

you can use egrep command

Code:

egrep "é|à|è|ï|ë|ó" $i
if [ $i -eq 0 ]; then # found
  # do something or call above script
fi

meowing · #3 (**permalink**) 06-03-2007, 03:16 AM

Quote:

Originally Posted by jerry

Correct code..

Code:

#!/bin/bash
mypath="$1"
  for i in "$mypath"
  do
    echo "Converting $i ..."
    mv -i $i $i.bak
    iconv -f ISO8859-1 -t UTF-8 $i.bak >$i
  done

set up permision and convert all html files

Code:

./script *.html

It only converts just 1 file each time I run it. The wildcard (*) does not seem to work. How do I get it to convert ALL files in a directory?

Also, I think egrep does not see the correct characters in the files. I should probably use codes instead of the characters as is.

meowing · #4 (**permalink**) 07-23-2007, 09:51 PM

Your script, for some reason, did not work.
This one does:

Code:

#!/bin/bash
FROM=iso-8859-1
TO=UTF-8
ICONV="iconv -f $FROM -t $TO"
# Convert
find /some/folder/ -type f -name "*" | while read fn; do
cp ${fn} ${fn}.bak
$ICONV < ${fn}.bak > ${fn}
rm ${fn}.bak
done

Where /some/folder/ is the one where all the files are (that need conversion).
Just rename it to something.sh and then run it from command line, that will do the job.
You can change the From and To fields to your liking.

gary_johnson_53 · #5 (**permalink**) 09-20-2007, 02:24 AM

I have files with spaces in the names

Example: à blanc.txt

How do I modify this script to handle this case?

cp: target `blanc.txt.bak' is not a directory
testiconv: line 8: CB2/TermsDictionaryGlosary/ZZAcented/à blanc.txt.bak: No such file or directory