
IBM XPG4 I18N API Automated Test Tool 
===================================================


---------------------------------------------------
CONTENTS

  Overview of IBM XPG4 I18N API Automated Test Tool
    - Introduction
    - Pre-requisite skill to run the tool
    - System requirement
    - What the Software Contains
    - What language is supported

  Tool Detail 
    - Automated mode and interactvie mode
    - How to set up and run test tool
    - Where to Find More Information
    - Known Problems with the version1.0
    - History


====================================================
  Overview of IBM XPG4 I18N API Automated Test Tool
====================================================



____________
INTRODUCTION

In response to the present situation that a various kinds of Linux
distributions include glibc with a variety of local patches, thereby,
showing unique behaviour on each distribution, application developers
may want to test actual behaviour of each library including exisiting
or coming new versions. The automated test tool is intended to 
decrease a cost for testing internationalization (I18N) APIs of C 
language and quickly discover the bugs.

The automated test tool allows you to test about 70 international 
APIs defined in X/Open Portability Guide Issue 4 (XPG4) with all
the possible parameters.  XPG4 defines internationalization frame
work and APIs. The existing UNIX implementations follows XPG4.
By using XPG4 APIs, applications can be internationalized, and the
source level portability among different UNIX platforms is guaranteed.


_____________
Pre-requisite Skill to Run the Tool

Working experience of XPG4 I18N API helps a lot in determining/
narrowing down the problems. However, even novices can run the 
test tool once he/she understands what the tools does, and how it
works.  No linguistic ability or an ability to operate Asain character
input method is required when the tool is run in automated mode, 
which is  discussed in the later section.


_____________
System requirement

The tool is supposed to work on all XPG4 implementation such
as UNIX in general, OS/2 and Linux with glibc2.0 or above. 
However, note that if a given implementation have some uniqueness
deviating from XPG4 specification, such as difference of API, 
parameter, or header file name, you may not be able to compile
the test tool without making some changes to the original test tool
sources.  The changes include renaming the header file or remarking
APIs that are not supported.The environment on which the tool was
confirmed to work without any changes are:

         AIX, Linux distributions with glibc2.2


_____________
What the software contains

      -Individual test tool 
      -Batch file to run all the test case at a time
      -Answer file generation utility 
      -Answer files 


________________
What language is supported

This tool is desinged to support any languages. What language is
supported totally depends on what language's answer file is 
available. For now, the following language is supported by default:

      ja_JP
      ja_JP.eucJP
      ja_JP.ujis

However, you can support any additional languages by creating
answer file using answer file generation utility.

           	
====================================================
	Tool Detail 
====================================================

_________________
Automated mode and interactvie mode

The test tool can examine a XPG4 implementation of a given C
library in 2 methods:  Automated and Interactive.

-Introduction To Automated Mode

Automated Mode will be most appreciated by testers who want to
test in foreign languages, say, an American tester who is at a
loss how to test the Japanese.  The test data, which was chosen
by the Japanese character encoding professionals and likely to 
discover bugs,  is predefined in a file called answer file. (It
is called answer file, because the API name, test data and 
corresponding results (answer) are predefined in there.). The
format of answer file is:

[API name] [1st param] [2nd param] [3rd param] [expected result]

For example, the API called nl_langinfo( ), which returns value 
specified in the parameter in a culture dependent fashion, must
return "mercredi", meaning Wednesday in French,  for the 4th day
of the week on the fr_FR locale. The statement in the answer file
that describes that looks like:

nl_langinfo  DAY_4  -  -  mercredi

nl_langinfo( ) takes only one parameter, and the 2nd and 3rd 
parameters are not required. Therefore, 2nd and 3rd parameters
are represented as - (hyphen). Note that you must explicitly 
declare that parameters that are not required by each APIs by 
hyphen. However, most of the time, all you have to do would be
just to mimic the predefined items in the answer files without
consulting API references.

The tool takes on the answer file, performs the testing, and
displays the results on stdout (or you may want to redirect it
to the file.) To illustrate the case of failure, the result 
looks like:

[FAIL] nl_langinfo(DAY_4) sys[Wednesday] != exp[mercredi]

meaning that the testing of API, nl_langinfo( ) with its 
parameter "DAY_4" failed (FAIL), because on French locale,
the return "mercredi" is EXPected, but in reality, the 
SYStem returned "Wednesday". 

==Scalable Answer file

The automated test tool is data driven, meaning that you can
add any test case of your interest in an answer file, and the
test tool invokes the APIs according to it.  In this context, 
the combination of API name, parameters, and expected result 
constitutes a test case.

==Answer file generation utility

The tool comes with an answer file generation utility, which
is intended to facilitate the creation of answer file. It 
would be convenient when you want to create an answer file for
additional languages or referential answer file for a particular
system. For example, if you want to compare the behaviour of 
APIs between AIX and Linux, you generate an answer file on AIX,
and use it on Linux to see the differences of the behaviour.
To run the answer file generation utiltiy:

   $./gen_answer.sh

This will create answer file under /ans directory with it's name
including the locale name.  

-Introduction To Interactive Mode

Unlike automated mode, which tests all the APIs and display
lengthy results and is time consuming (it is typically as long
as several minutes though),  the interactive mode allows to 
test any specific API and parameter only. 

The interactive mode would be convenient when you want to analyze
some particular APIs with some specific parameters in depth to
narrow down the problems. As with the UNIX standard command "cat",
if you do not specify the answer file to the test tool, the tool
works in interactive mode. For example, when test_string (one of
the tools) is started without an answer file, it prompts you to
input the test case in the same format as answer file. ( "-p" 
options shows prompt "input =>". Without the option, the prompt 
does not appear, but the subsequesnt operations are exactly the
same.) 

$./test_string -p
input =>

Then,  you input like the following. (portion shown in bold.)

input =>nl_langinfo  DAY_4  -  -  mercredi

Then pressing  enter gets you a result which looks much the same
way as the automated mode.

[FAIL] nl_langinfo(DAY_4) sys[Wednesday] != exp[mercredi]

If you do not know the answer, or just want to see what is
returned against the specific combination of API and parameter,
just omit the "answer" portion when you type the test case 
againt he prompt.

input =>nl_langinfo  CODESET  -  -

Then the result is shown like the following.

input =>nl_langinfo  CODESET  -  - EUC-JP


________________
How to set up and run test tool

Unter the tar file.

$tar xvfz hogehoge

This will generates the directory tree, each representing each
tool that tests APIs that are similar in nature. 

Under each directory, you will see the directory called /ans, 
which is where the answer file for each test case is stored. 
Also, you can find gen_answer.sh, which is an answer file 
generation utility.

Compile

$cd /hogehoge
$make

(**IMPORTANT**)
set the LANG environment variable for the locale you want to 
test like following.

$export LANG=ja_JP.eucJP

To run all the test case at a time.

$make exec

To run individual test case

$cd lang
$./runtest.sh -v


___________________________
Where to Find More Information

 Please send an email to one of the following developers 
 of this tool:
 
  tadayuki@jp.ibm.com

___________________________
Known Problems with the version1.0

  For now, no bug is reported.

___________________________
History
 
  This tool, or comprising each test case to be 
  exact, had been used by IBM to test XPG4 
  implementaion of C library such as AIX, OS/2, 
  OS/400, OS/390.  The experience in the past
  by the testers involved had been reflected in 
  many part of the tool.

  The basic designing and prototyping of this 
  automated version had been done by Yoshito 
  Kawada. Hiroshi Inokawa improved it including 
  the answer file generation utility and 
  better testing methodology for iconv as well as 
  appropriate consultation on a regular basis. 
  
  Overall coding was done by Tadayuki Yoshida, 
  overhauling the tool with better coding practice.
  The performnace and ease-of-maintenace had been 
  drastically improved by that contribution.
  
   
---------------------------------------------------
* All the company names may be trademarks or registered trademarks.

       
