[Bug classpath/22945] New: Classes in gnu.java.nio.charset.iconv should be extendable

[pinskia at gcc dot gnu dot org]

Classes in gnu.java.nio.charset.iconv are declared final, but actually

they should be extendable because in some cases char/byte conversion

with iconv is not final but needs modification.



There are cases where iconv may be correct but the real world is

different.



One of such cases I know of is the case of Shift_JIS.



Iconv maps in case of Shift_JIS:



  byte '\' <--> char 0x00a5 

  byte '~'  <--> char 0x203e



This may be correct in the light of the standard JIS, but in practice

the standard is not respected.  And preferred mapping is:



  byte '\' <--> char '\'

  byte '~'  <--> char '~'



That is, identical conversion for US-ASCII is preferred even if the

charset is Shift_JIS. And this is the behavior of Sun's java.



Should iconv be modified so that it copes with the real-world practice?

Maybe yes, but it may be difficult.



Let iconv respect the standard. But we need something that can be used

in the real world.



So I suggest that classes in gnu.java.nio.charset.iconv should be

extendable so that we can write slightly modified subclasses.



I think I will submit my suggested patch to the mailing list.


------- Additional Comments From from-classpath at savannah dot gnu dot org  
2005-05-18 01:21 -------
Hi Ito, thanks for reporting this!



We really need folks who can test these things. I obviously wouldn't know a 
good japanese mapping from a bad one even if it told me so itself :).



I would prefer one of two solutions here:



1) Fix the iconv library. This isn't really a bug in the classpath code.



or



2) Write a java SJIS converter, and do it correctly. My plan is to have java 
converters for all the charsets Java supports natively. So this will need to be 
done sooner or later. GCJ has their own SJIS converter we could perhaps port or 
use.



Although admittedly, I haven't seen your patch.




------- Additional Comments From from-classpath at savannah dot gnu dot org  
2005-05-19 22:36 -------
Hi Sven, Thank you for taking care of my problem.



> 1) Fix the iconv library. This isn't really a bug in the classpath code.



Actually, there exsists unofficial patches to iconv which makes

iconv handle Japanese characters preferrably.



The problem involving only Japanese users, applying those patches wolud

be the simplest solution.



But I do not like applying those unofficial patches, and I cannot

expect them to become official.



> 2) Write a java SJIS converter, and do it correctly. My plan is to have java

> converters for all the charsets Java supports natively. So this will need to

> be done sooner or later. GCJ has their own SJIS converter we could perhaps

> port or use.



Only SJIS is OK, but having java converters for all the charsets Java

supports would be a very hard work beyond my imagination.




------- Additional Comments From from-classpath at savannah dot gnu dot org  
2005-05-20 08:39 -------
I would not insist that classes gnu.java.nio.charset.iconv should be

expandable,  and I would like this issue to be closed now.



Reason:



At first,  I wanted to modify the results from gnu.java.nio.charset.iconv.

IconvDecoder/IconvEncoder by writing subclasses of them.



But I succeeded in writing a new CharsetDecoder and CharsetEncoder,

which handles Shift_JIS preferrably, by wrapping

gnu.java.nio.charset.iconv classes. They can be as they are.



So gnu.java.nio.charset.iconv package can be free from dirty

issues caused by the difference between the standard and

the reality.



By the way, I am ready to publish my Shift_JIS related programs somehow.


------- Additional Comments From pinskia at gcc dot gnu dot org  2005-09-25 
22:32 -------
Closing as invalid per the submitter in comment #3.

-- 
           Summary: Classes in gnu.java.nio.charset.iconv should be
                    extendable
           Product: classpath
           Version: unspecified
            Status: RESOLVED
          Severity: normal
          Priority: P3
         Component: classpath
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: from-classpath at savannah dot gnu dot org
                CC: bug-classpath at gnu dot org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=22945