UTF-8 issue ????

[mlibeson]

OK, so I have this PERL script that I have been using for some time now. I used to use in a NT environment with an older version of PERL. I am currently running on Windows 2000 and a newer version of PERL. I even have used the same script to process the same files successfully. All of a sudden in preparing to make changes to the code, I decided to run a test with the current version. I received an error complaining about UTF-8. So I looked into where the error was, forced an integer variable to be floating by using sprintf in a division formula. Worked like a charm. Tried the same code on a different day and received the UTF-8 message again. Now I was not able to get rid of it. Spent an entire weekend changing the code to force int to float where needed. The code worked once again. Next I go to demonstrate the code and the UTF-8 error is back.

Has anyone come across this problem and know how to solve it. It is driving my crazy.


Michael Libeson

 

[tchatzi]

Can you show us the code of your script, and maybe we can help make it simpler , or correct any errors that might occur.

TIMTOWTDI

 

[mlibeson]

There are 2 issues with providing code.

1) There are over 8K lines of code.

2) There are sections that I am not allowed to share since the program is used for auditing web traffic.


I am just hoping that sombody else has had the problem already and discovered a solution.

Michael Libeson

 

[icrf]

how about a copy of the error message? Is it coming from perl? Are you using any extra characters from the extended utf8 set anywhere, in code or input?

________________________________________
Andrew

I work for a gift card company!

 

[mlibeson]

Here are some error messages:

Malformed UTF-8 character (unexpected continuation byte 0x90, with no preceding
start byte) in null operation at script.pl line 5305, <STDIN> line 1.

Malformed UTF-8 character (unexpected non-continuation byte 0x67, immediately af
ter start byte 0xc8) in null operation at process.pl line 5310, <SOURCE> line 10
.
Malformed UTF-8 character (unexpected continuation byte 0x91, with no preceding
start byte) in null operation at process.pl line 5310, <SOURCE> line 28.
Malformed UTF-8 character (unexpected continuation byte 0x8c, with no preceding
start byte) in null operation at process.pl line 5310, <SOURCE> line 28.
Malformed UTF-8 character (unexpected continuation byte 0xa7, with no preceding
start byte) in null operation at process.pl line 5310, <SOURCE> line 28.

Also, I was able to pin point where the problem was coming from. It was not an sprintf problem, but I think it might be a TK problem or action problem. Using the code below and pressing the 's' key when the window appears will allow the program to run. While it is running, press the 's' key again to see if you get the UTF-8 error.

$| = 1;

use Tk;
use Tk::ProgressBar;

$main::tfont = 'courier 8';

     $main::mw = MainWindow->new;
     $main::mw->title("[Test]");
     $main::r2 = $main::mw->Frame;
     $main::r2->pack(-side => 'top');
     $main::r2->Label(-width => '6', -textvariable, \$c, -font => "$main::tfont")->pack(-side => 'right');

     $main::progress2 = $main::r2->ProgressBar(
                       -width => 20,
                       -borderwidth => 2,
	                 -padx => 2,
	                 -pady => 2,
                       -relief => 'sunken',
                       -from => 0,
                       -gap => 0,
                       -to => 100,
                       -blocks => 100,
	                 -colors => [0 => 'blue', 25 => 'green' , 50 => 'yellow', => 75 => 'red'],
                       -value => int($c),
                       -variable => \$c
     )->pack(-side => 'right', -fill => 'x');

     $main::mw->bind('<Control-c>' => sub{print STDOUT "Cancelled program.\n";exit(0);});
     $main::mw->bind('<KeyPress-s>' => [\&mainblock]);

     MainLoop; ##############

sub mainblock {

$a = 2929;
$b = 51404504;
$c = 0.0;

  foreach $i (0 .. $a) {
   $c = sprintf("%6.2f",((sprintf("%6.2f", $i) / sprintf("%6.2f", $b)) * 100.0));
   $main::mw->update();
   print STDOUT "$c\n";
  }

$main::mw->destroy;

}

exit(0);

1;

Here is the output of perl -V:

Summary of my perl5 (revision 5 version 8 subversion 6) configuration:
  Platform:
    osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread
    uname=''
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=de
fine
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cl', ccflags ='-nologo -Gf -W3 -MD -Zi -DNDEBUG -O1 -DWIN32 -D_CONSOLE -
DNO_STRICT -DHAVE_DES_FCRYPT  -DNO_HASH_SEED -DPERL_IMPLICIT_CONTEXT -DPERL_IMPL
ICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX',
    optimize='-MD -Zi -DNDEBUG -O1',
    cppflags='-DWIN32'
    ccversion='', gccversion='', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64', lseeksi
ze=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf  -libpath:"C:
\Perl\lib\CORE"  -machine:x86'
    libpth=\lib
    libs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  comdlg32
.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib uuid.lib ws2_
32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib msvcrt.lib
    perllibs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  comd
lg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib uuid.lib
ws2_32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib msvcrt.lib
    libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib
    gnulibc_version='undef'
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf  -
libpath:"C:\Perl\lib\CORE"  -machine:x86'


Characteristics of this binary (from libperl):
  Compile-time options: MULTIPLICITY USE_ITHREADS USE_LARGE_FILES PERL_IMPLICIT_
CONTEXT PERL_IMPLICIT_SYS
  Locally applied patches:
        ActivePerl Build 811
        21540 Fix backward-compatibility issues in if.pm
        23565 Wrong MANIFEST.SKIP
  Built under MSWin32
  Compiled at Dec 13 2004 09:52:01
  @INC:
    C:/Perl/lib
    C:/Perl/site/lib
    .

Michael Libeson

 

[tchatzi]

That's what i get as an error when i press 's' while the script is running

Tk::Error: Usage $widget->update(...)
MainWindow=HASH(0x2c83cf0) is not a Tk object at foo.pl line 42.
 s
 (command bound to event)
  0.00

TIMTOWTDI

 

[mlibeson]

I am using active build perl 5.8.6.811

TK 804.027

perluserpengo: What versions are you using?

I have been using the update for a long time now. I used it in Perl 5.6 as well, but I do not recall the version of TK.

I do not understand why it accepts the update call for me and not for you. Puzzling???


Michael Libeson

 

[tchatzi]

$Tk::version = '8.4'
$Tk:atchLevel = '8.4'
$Tk::VERSION = '804.027'

My Perl is 5.8.6 and was built using the sources for build 811 provided by ActiveState

TIMTOWTDI

 

[mlibeson]

perluserpengo I am using the binaries from active state with the same versions. The script should work. Are you running Unix or Windows? I am running windows 2000 server.


Michael Libeson