Subversion Repositories tendra.SVN

<?xml version="1.0" standalone="no"?>

<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"

  "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd">

<!--

  $Id$

-->

<book>

  <bookinfo>

    <title>Validation of TenDRA Capability to Implement a UNIX-like Operating

      System</title>

    <corpauthor>The TenDRA Project</corpauthor>

    <authorgroup>

      <author>

        <firstname>Fran&ccedil;ois</firstname>

        <surname>de Ferri&egrave;re</surname>

        <affiliation>

          <orgname>Open Software Foundation Research Institute</orgname>

        </affiliation>

      </author>

      <author>

        <firstname>Fred</firstname>

        <surname>Roy</surname>

        <affiliation>

          <orgname>Open Software Foundation Research Institute</orgname>

        </affiliation>

      </author>

      <editor>

        <firstname>Jeroen</firstname>

        <surname>Ruigrok van der Werven</surname>

      </editor>

    </authorgroup>

    <abstract>

      <para>This is the final report on work undertaken in relation to the

        UnixWare(R) operating system on the Intel platform, carried out under

        contract for the UK Defence Research Agency.</para>

    </abstract>

  </bookinfo>

  <chapter>

    <title>Objectives and Description</title>

    <sect1>

      <title>Objectives</title>

      <para>A first objective of this project was to perform validation,

        performance and robustness testing of the TenDRA technology to ensure

        its capability to implement and fully bootstrap a UNIX-like operating

        system on a variety of target processor architectures.</para>

      <para>Another objective was to provide an assessment of TenDRA

        technology to express a fully portable operating system

        implementation.</para>

      <para>This report summarizes the work done with respect to these two

        objectives.</para>

    </sect1>

    <sect1>

      <title>General description</title>

      <para>A Unix system can be divided into three parts, which characterize

        the portability level of the code:</para>

      <itemizedlist>

        <listitem>

          <para>The Kernel, which has some parts in assembler, e.g.  for

            context switching, interruptions, locks, or parts of device

            drivers.  Assembly code is used either in separate files or is

            embedded in C programs or header files.</para>

        </listitem>

        <listitem>

          <para>The Libraries, some of which may contain assembly code.  The

            crt0 library and code for system calls are examples.</para>

        </listitem>

        <listitem>

          <para>The Commands, in which use of assembly code is very unusual.

            They share a non-explicit API with the libraries against which

            they are built.</para>

        </listitem>

      </itemizedlist>

      <para>As for other software OSF already ported to ANDF, the port of the

        Unix system has been done in three successive steps:</para>

      <itemizedlist>

        <listitem>

          <para>The NAT-NAT step, which consists in rebuilding the system with

            the native compilation chain, to ensure that the system can be

            regenerated from the set of sources.</para>

        </listitem>

        <listitem>

          <para>The DRA-NAT step, for which the TenDRA technology is employed

            as a replacement of the native compilation chain to build the

            system, using the native system header files, as for a classical

            compilation chain. This part involves dealing with assembler

            issues as well as discrepancies between the native and the TenDRA

            code generators. Note that Unix systems are known to be compiler

            dependent.</para>

        </listitem>

        <listitem>

          <para>The DRA-DRA step, which consists in using the TenDRA

            technology as a portability tool. The API shared by the commands

            and libraries has to be defined, and used to produce architecture

            independent ANDF code for these parts. This code is then installed

            and validated on the selected machines. Note that the kernel part

            of the Unix system has not been included in this task since it is

            essentially not portable.</para>

        </listitem>

      </itemizedlist>

    </sect1>

    <sect1>

      <title>Choice of a UNIX system and update of the objectives</title>

      <para>We have used the source for the UnixWare system as the base for

        performing the port of a Unix source code. We originally planned to

        conduct the experiment on two different platforms: an Intel/386

        platform and a Sun/Sparc one. The NAT-NAT, DRA-NAT and part of the

        DRA-DRA steps have been achieved on the Intel/386 platform. Then, in

        the light of the experience we gained from this work, we decided to

        re-focus the project on a more complete DRA-DRA experiment, with a

        different Unix system.  This report only covers that part of the work

        done up to this change.</para>

    </sect1>

    <sect1>

      <title>Choice of the validation tools</title>

      <para>Throughout this experiment we had to validate the various parts of

        the system we built.</para>

      <para>A first level of validation was achieved by using the kernel and

        commands built in the DRA-NAT step for our daily work. In addition, we

        used the VSX4 and AIMIII validation suites to test more thoroughly the

        robustness and performance of the system built in DRA-NAT mode.</para>

    </sect1>

  </chapter>

  <chapter>

    <title>Software Information</title>

    <para><emphasis>Software Category</emphasis></para>

    <para>UnixWare is a desktop Operating System derived from System V release

      4.2.</para>

    <para>It includes support for TCP/IP and Netware networking, X/Window,

      Motif and a Desktop Manager.</para>

    <para><emphasis>UnixWare Version and Release Level</emphasis></para>

    <para>A beta release of UnixWare version 2.0 for the Intel386 family

      (Load q7.1) source code was used.</para>

    <para><emphasis>Authors</emphasis></para>

    <para>The authors of the UnixWare system are the Novell, Inc., with

      portions of code developed by other contributors such as the University

      of Berkeley and MIT.</para>

    <para><emphasis>Source code</emphasis></para>

    <para>The source code delivery for UnixWare 2.0 beta release represents

      approximately 240 Mbytes of code. Most of the system has been recompiled

      during the NAT-NAT phase of this project.</para>

    <para>Included in this code is a graphical interface, based on X11 and

      Motif. However, since this interface contains some code delivered in

      binary format and since this interface is an optional part of a Unix

      system, we did not use this code.</para>

    <table>

      <title>Rough source code distribution</title>

      <tgroup align="center" cols="6" colsep="1" rowsep="1">

        <thead>

          <row>

            <entry>Libraries</entry>

            <entry>Kernel</entry>

            <entry>Commands &amp; tools</entry>

            <entry>Graphics</entry>

            <entry>Misc.</entry>

            <entry>Total</entry>

          </row>

        </thead>

        <tbody>

          <row>

            <entry>15MB</entry>

            <entry>30MB</entry>

            <entry>101MB</entry>

            <entry>85MB</entry>

            <entry>11MB</entry>

            <entry>242MB</entry>

          </row>

        </tbody>

      </tgroup>

    </table>

    <para>In addition, various data files are delivered, for configuration and

      messaging, especially under Command & tools directories. In

      addition, some parts of the system, for example the kernel code for the

      vxfs file system and some device drivers, appear to be delivered in

      binary format only. So, a more accurate view of the volume of source

      code submitted to the TenDRA technology is provided by sizing the C

      language source files only:</para>

    <table>

      <title>C language files</title>

      <tgroup align="center" cols="5" colsep="1" rowsep="1">

        <thead>

          <row>

            <entry>Headers</entry>

            <entry>Libraries</entry>

            <entry>Kernel</entry>

            <entry>Commands &amp; tools</entry>

            <entry>Total</entry>

          </row>

        </thead>

        <tbody>

          <row>

            <entry>19.7MB</entry>

            <entry>11.2MB</entry>

            <entry>19.8MB</entry>

            <entry>39.6MB</entry>

            <entry>90.3MB</entry>

          </row>

        </tbody>

      </tgroup>

    </table>

    <para><emphasis>TenDRA (ANDF) Technology Version Release</emphasis></para>

    <para>The TDF snapshot dated December 94 (svr4_i386 target) has been used;

      it is based on the TDF Specification Issue 3.0.</para>

    <para><emphasis>Validation tools</emphasis></para>

    <itemizedlist>

      <listitem>

        <para>X/OPEN Verification Suite release 4.2.4</para>

      </listitem>

      <listitem>

        <para>AIM Suite III v3.1 (AIM Technology Inc.)</para>

      </listitem>

    </itemizedlist>

    <para><emphasis>OS Platform Environment</emphasis></para>

    <para>Two identical Intel486 PCs running UnixWare 2.0 and more than 1 GB

      of disk space, mostly through NFS servers.</para>

  </chapter>

  <chapter id="project-phase">

    <title>Description of Project phases</title>

    <para>In this section we describe the tasks which have been performed

      under this project on the Intel/386 platform. As already mentioned in

      this report, the work has been split into three main stages, named the

      NAT-NAT, DRA-NAT and DRA-DRA steps. Preliminary to these steps, an

      installation phase was needed to set up the environment and tools for

      this project.</para>

    <para><emphasis>Installation phase:</emphasis></para>

    <para><emphasis>T1.</emphasis> UnixWare installation.</para>

    <itemizedlist>

      <listitem>

        <para>Install the complete binary code of UnixWare.</para>

        <para>This task consisted in running the UnixWare installation

          procedure, from the tape we received, on an Intel/386 machine. When

          this was completed, we installed a second identical machine with the

          same configuration. Because we used a beta version of the system, we

          found a few problems, which were solved with assistance from Novell,

          Inc.</para>

      </listitem>

      <listitem>

        <para>Perform some checks to verify that the installation is

          correct.</para>

        <para>We used these two machines as our normal development

          machines.</para>

      </listitem>

      <listitem>

        <para>Install UnixWare source code and setup various build

          environments.</para>

        <para>A build environment was provided with the source code delivery

          of the system. We made a few modifications to it in order to create

          one build environment for each of the NAT-NAT, DRA-NAT and DRA-DRA

          phases.</para>

      </listitem>

    </itemizedlist>

    <para><emphasis>Delivery</emphasis>: System running.</para>

    <para><emphasis>T2. </emphasis>TenDRA installation.</para>

    <itemizedlist>

      <listitem>

        <para>Install the TenDRA technology for UnixWare.</para>

        <para>We obtained from DRA a complete TenDRA technology delivery for

          the UnixWare system. We just needed to recompile some executables

          since we were not running the same version of the system as

          them.</para>

      </listitem>

    </itemizedlist>

    <para><emphasis>Delivery</emphasis>: TenDRA installed.</para>

    <para><emphasis>T3.</emphasis>Validation suites installation.</para>

    <itemizedlist>

      <listitem>

        <para>Install the VSX4 suite which is used to check the kernel,

        the main libraries and some commands of the system built in DRA-NAT

        phase.</para>

        <para>The VSX4 suite was installed for the NAT-NAT version of the

          system. Next, a new image of the suite for the validation of the

          DRA-NAT system was created.</para>

      </listitem>

      <listitem>

        <para>Install the AIMIII suite which is used to check the robustness

          and performance of the system.</para>

        <para>The AIMIII suite was installed to check the native system and

        then the DRA-NAT one.</para>

      </listitem>

    </itemizedlist>

    <para><emphasis>Delivery</emphasis>: Validation suites installed.</para>

    <para><emphasis>NAT-NAT phase:</emphasis></para>

    <para><emphasis>T4.</emphasis> NAT-NAT build.</para>

    <itemizedlist>

      <listitem>

        <para>Compile the UnixWare build tools and libraries.</para>

        <para>Tools such as make, cc, ld, ..., were built during this step and

          then used to proceed with the compilation of the libraries.</para>

      </listitem>

      <listitem>

        <para>Compile the UnixWare kernel.</para>

        <para>A kernel, along with dynamically loadable modules, has been

          produced.</para>

      </listitem>

      <listitem>

        <para>Compile the UnixWare commands.</para>

        <para>All the UnixWare commands have been built in this step.</para>

      </listitem>

    </itemizedlist>

    <para><emphasis>Delivery</emphasis>: Reference system to which the DRA-NAT

      built system will be compared.</para>

    <para><emphasis>DRA-NAT phase:</emphasis></para>

    <para><emphasis>T5. </emphasis>DRA-NAT build.</para>

    <itemizedlist>

      <listitem>

        <para>Compile the UnixWare build tools &amp; libraries, kernel and

          commands using the TenDRA technology. This may show up some bugs in

          the TenDRA technology, as well as assembly language issues in the

          UnixWare source code.</para>

        <para>The native cc compiler was replaced by a shell script which

          modified some options and called the tcc TenDRA compiler in DRA-NAT

          mode.</para>

      </listitem>

      <listitem>

        <para>Report the build problems to DRA.</para>

        <para>When we found problems with the tcc compiler, we isolated the

          problem in a few lines of C code which were provided to DRA.</para>

      </listitem>

      <listitem>

        <para>Use temporary workarounds to complete the task as much as

          possible.</para>

        <para>When it was possible, we made temporary modifications to the

          source code in order to bypass the problem.</para>

      </listitem>

      <listitem>

        <para>Update the TenDRA technology with the fixes for the build

          problems.</para>

        <para>When fixes were received from DRA, the tenDRA technology was

          updated. Then, the sources where the problem showed up were

          recompiled.</para>

      </listitem>

    </itemizedlist>

    <para><emphasis>Delivery</emphasis>: Details of build problems</para>

    <para>System to be validated.</para>

    <para><emphasis>T6. </emphasis>DRA-NAT validation.</para>

    <itemizedlist>

      <listitem>

        <para>Replace native commands and libraries by those built in DRA-NAT

          mode, and boot the system with a DRA-NAT kernel.</para>

        <para>We progressively replaced native commands and libraries by the

          ones produced with the TenDRA technology in DRA-NAT mode. Similarly,

          we booted the system with a kernel in which we gradually added more

          and more components built in DRA-NAT mode.</para>

      </listitem>

      <listitem>

        <para>Validate the system against the VSX4 validation suite. This will

          exercise some commands, usual libraries and related system calls

          implementation inside the kernel.</para>

        <para>First, we ran the VSX4 validation suite on the native system.

          Then, we ran it on a system built in DRA-NAT mode and compared the

          results against those obtained with the native system.</para>

      </listitem>

      <listitem>

        <para>Validate the system against the AIMIII validation suite.  This

          will check the performances and robustness of the system with

          respect to time-sharing.</para>

        <para>We compared the results obtained when running the suite on a

          native system and a DRA-NAT one.</para>

      </listitem>

    </itemizedlist>

    <para><emphasis>Delivery</emphasis>: Validation report.</para>

    <para><emphasis>DRA-DRA phase:</emphasis></para>

    <para><emphasis>T7. </emphasis>API definition.</para>

    <itemizedlist>

      <listitem>

        <para>Define the non-explicit API used by the commands. Machine

          dependent code issues will be addressed

          specifically.</para>

        <para>We determined the basic standard interfaces upon which the

          interface was built. Then we extended it in order to cover a minimum

          set of commands.</para>

      </listitem>

      <listitem>

        <para>Build the token libraries for the API used by the commands and

          libraries.</para>

        <para>We used the TenDRA technology tools to describe and build the

          interface.</para>

      </listitem>

    </itemizedlist>

    <para><emphasis>T8. </emphasis>DRA-DRA assessment.</para>

    <itemizedlist>

      <listitem>

        <para>Build a selected set of commands.</para>

        <para>We built the commands which were covered by the extended

          interface we implemented.</para>

      </listitem>

      <listitem>

        <para>Report problems to DRA.</para>

      </listitem>

      <listitem>

        <para>Update the TenDRA technology with the fixes.</para>

      </listitem>

      <listitem>

        <para>Complete the compilation of the selected set of

          commands.</para>

      </listitem>

      <listitem>

        <para>Delivery: Assessment report.</para>

      </listitem>

    </itemizedlist>

  </chapter>

  <chapter id="project-environment">

    <title>Project environment</title>

    <sect1>

      <title>Hardware environment</title>

      <para>Two identical Intel/486 PCs running UnixWare 2.0 beta release have

        been used as the development platforms for the various build phases.

        Each of them was equipped with 16MB of RAM, a local 300 MB disk and an

        Ethernet controller.  Another 300MB disk was added to one of the

        machine dedicated to validation.  We also used additional disk space

        through NFS servers to store the UnixWare source delivery, the TenDRA

        compilation system as well as the object files created during the

        three builds. This layout permitted us to run concurrently two

        different builds from any of the two platforms while keeping the

        consistency of project files.</para>

      <para>During the validation process of the DRA-NAT build, one machine

        was dedicated to the validation tests while the other one was

        preserved for development purposes. This was necessary for three

        reasons:</para>

      <itemizedlist>

        <listitem>

          <para>The test machine was more likely to crash or have files

            corrupted.</para>

        </listitem>

        <listitem>

          <para>Some tests from the VSX4 test suite require execution from a

            local disk.</para>

        </listitem>

        <listitem>

          <para>The environment for running the AIMIII benchmark has to be

            kept stable.</para>

        </listitem>

      </itemizedlist>

    </sect1>

    <sect1>

      <title>UnixWare source delivery</title>

      <para>The source code delivery for UnixWare on Intel/386 is organized in

        several subdirectories which reflect the dependency of sources upon

        CPU and hardware architectures.</para>

      <para>When building a new system, one of the first steps is to construct

        a WORK tree. This tree is created from a merge between different parts

        of the original source tree, with symbolic links to the source files,

        depending on CPU/architecture configuration. Along with this WORK

        tree, a TOOLS and a MACH trees are also created, which together

        constitute the private environment for a build. The TOOLS tree is used

        for the tools and libraries used for internal build purpose. The MACH

        tree will contain at the end of a build all the components which

        constitute the actual system we built.</para>

      <para>A set of environment variables defines the current paths for the

        WORK, TOOLS and MACH trees. This has been used to create different

        build environments for each of the NAT-NAT, DRA-NAT and DRA-DRA

        phases.</para>

      <para>The disk space required for building a system is as

        follows:</para>

      <para>Original source tree: 240 MB</para>

      <para>For each of the NAT-NAT, DRA-NAT and DRA-DRA phases:</para>

      <para>WORK tree: 350 MB</para>

      <para>TOOLS tree:	35 MB</para>

      <para>MACH tree: 220 MB</para>

      <para>In order to keep on-line the NAT-NAT and DRA-NAT builds,

        approximately 1.5 GB of disk space has been taken on NFS servers. We

        did not need to perform the last steps at the end of a build, which

        consist in making packages and images of binary deliveries, because we

        were not producing a system for distribution. This would have required

        an additional disk space of about 800 MB per build.</para>

      <para> For the VSX4 validation suite, we needed more than 300 MB of disk

        space to keep two environments for the validation of the native and

        DRA-NAT systems.</para>

    </sect1>

    <sect1>

      <title>TenDRA technology</title>

      <para>Replacement of one compiler by another, and moreover,

        compatibility between object files produced by two different

        compilers, are often difficult issues. But in the case of the TenDRA

        compiler this has been straightforward.</para>

      <para>TenDRA compilers are designed to be compatible with the native

        compiler of the platform they are targeted to, generating the same

        internal format for data, using the same calling conventions, ....

        This allowed us to link together binary files from one or the other

        compiler, without any problems.</para>

      <para>Also, the command line options for the TenDRA compilers reflect as

        much as possible the options for the corresponding native compiler.

        So, we had very few modifications to make to the option line in order

        to replace the native cc compiler by the TenDRA tcc one. These changes

        have been implemented by a front-end shell script which emulates a

        call to the native compiler by a call to tcc. </para>

      <para>It appeared early on that some source files from the UnixWare

        distribution were not strictly ANSI compliant. For example, a function

        declared in a header file using ANSI syntax was actually defined using

        K&amp;R syntax.  Therefore we had to use the relaxing options:</para>

      <para>-not_ansi -nepc</para>

      <para>Using the TenDRA compiler with the native system header files, the

        DRA-NAT mode, required the additional option:</para>

      <para>-Ysystem</para>

      <para>Adaptation to special compilation options local to some makefiles

        are discussed in section 7.</para>

    </sect1>

  </chapter>

  <chapter>

    <title>Results of Project phases</title>

    <para>The NAT-NAT and DRA-NAT project phases, described in

      <link linkend="project-phase">Description of Project phases</link>,

      have been completed with a good level of success. The DRA-DRA phase has

      been limited to cover a hundred or so commands. In the next paragraphs

      we detail the results of each of the phases of the project.</para>

    <sect1>

      <title>Installation phase</title>

      <para>This phase was not concerned with the TenDRA technology, so it did

        not produce any noteworthy results.</para>

    </sect1>

    <sect1>

      <title>NAT-NAT phase</title>

      <para>The objective of this phase was to check that the system can be

        completely built with the native system compiler from the source

        delivery.  Indeed, this phase was very helpful to set-up the build

        environment, and to evaluate the resources we would need for the next

        phases.</para>

    </sect1>

    <sect1>

      <title>DRA-NAT phase</title>

      <para><emphasis>DRA-NAT build</emphasis></para>

      <para>During this phase we compiled the whole system, as for the NAT-NAT

        build, using the TenDRA technology as a standard compiler. Therefore,

        we used the system header files instead of the architecture neutral

        headers.  At completion of this task, we obtained the following

        results:</para>

      <itemizedlist>

        <listitem>

          <para>65% of the library source files could be successfully compiled

            with the TenDRA technology.</para>

        </listitem>

        <listitem>

          <para>About 65% of the kernel C code could be successfully compiled

            with the TenDRA technology.</para>

        </listitem>

        <listitem>

          <para>Nearly all the commands and tools could be compiled with the

            TenDRA technology.</para>

        </listitem>

      </itemizedlist>

      <para>These figures need some explanation to see why the TenDRA

        technology failed to compile some of the files. There were essentially

        three cases where the technology could not compile the files:</para>

      <itemizedlist>

        <listitem>

          <para>support of assembly language instructions inlined in C

            programs.</para>

          <para>Such code is not per-se architecture neutral and it is

            therefore clearly beyond the TenDRA technology goals.</para>

          <para>Assembly code was present in only 2% of the library source

            files, but was used in 22% of the files for the kernel

            components.</para>

        </listitem>

        <listitem>

          <para>support of special alignments for fields in structures

            (#pragma pack(n) directive)</para>

          <para>This feature was only relevant for Intel386 architecture. It

            was present in about 33% of the library C source files (Mostly for

            the Netware related libraries) and in 12% of the kernel C source

            files.</para>

        </listitem>

        <listitem>

          <para>Code written in C++ language.</para>

          <para>There were also some parts of the system which used the C++

            language. Since a C++ producer is not yet available for the TenDRA

            technology, we could not compile these files.</para>

          <para>Files with C++ code represented about 9% of the source code

            for the libraries and about 5% of the source code for the

            commands.</para>

        </listitem>

      </itemizedlist>

      <para>In summary, excluding the C++ files, about 85% of the C source

        files could be successfully compiled with the TenDRA technology.</para>

      <para>Apart from these issues, we found very few problems in the TenDRA

        compiler, 8 problems in fact, which were usually fixed or bypassed

        very quickly. These are included in the Appendix.</para>

      <para><emphasis>DRA-NAT validation</emphasis></para>

      <para>The DRA-NAT built kernel was customized for actual hardware and

        successfully booted.</para>

      <para>The VSX4 tests have been built and exercised first on the native

        UNIX version. On this beta release, approximately 6,000 tests are

        successful while a hundred of them are not. Three libraries are

        involved (libc being the most important one), and best results are got

        using the shared version of them.</para>

      <para>The following system software configuration was then used for

        recompiling and rerunning the VSX4 tests:</para>

      <itemizedlist>

        <listitem>

          <para>TenDRA compiler in DRA-NAT mode</para>

        </listitem>

        <listitem>

          <para>DRA-NAT commands</para>

        </listitem>

        <listitem>

          <para>DRA-NAT libraries for compile-time link-edits</para>

        </listitem>

        <listitem>

          <para>DRA-NAT version of dynamically-linked libc at

            runtime.</para>

        </listitem>

        <listitem>

          <para>DRA-NAT kernel</para>

        </listitem>

      </itemizedlist>

      <para>With this configuration, the VSX4 tests level of success is the

        same as for the native system. Two modules of the dynamically linked

        libc library were temporarily replaced by their native version in

        order to cure problems with the sed command.</para>

      <para>The AIMIII benchmark was used to exercise the native and DRA-NAT

        systems. At a medium user load level (simulated by the benchmark),

        i.e. 30-60 users, the performance of the two systems is similar:

        variations are below 3%.</para>

      <para>During the DRA-NAT validation phase, only 7 problems in TenDRA

        technology were encountered. These are included in the

        Appendix.</para>

    </sect1>

    <sect1>

      <title>DRA-DRA phase</title>

      <para>A base API has been created from a merge between the svid3 and

        xpg4 APIs, which are included in the TenDRA technology delivery. This

        allowed us to compile 57 commands (out of approximately 600). This

        demonstrated the need for a custom extension API for compiling most

        UnixWare commands. With the present extension API, 46 additional

        commands have been compiled.</para>

      <para>During this phase, two minor problems in the TenDRA technology

        were identified (these are included in the Appendix).</para>

    </sect1>

  </chapter>

  <chapter>

    <title>Methods applied</title>

    <sect1>

      <title>Correction of compilation problems</title>

      <para><emphasis>Native C compiler</emphasis></para>

      <para>As stated in <link linkend="project-environment">\xa4  4., page

        8</link>, binaries made by native and TenDRA compilers are

        interoperable. So a straightforward method to bypass a problem with

        TenDRA in DRA-NAT mode is to compile the ``guilty'' source file with

        the native compiler.  This method has been used in cases where tcc

        lacked a feature, e.g.  assembly language inlining, or when a bug in

        the code generated by tcc was identified but not yet fixed.</para>

      <para><emphasis>Source code modifications</emphasis></para>

      <para> In some cases, minor changes were applied to a source file (under

        #ifdef __ANDF__ conditions) when code rewriting was necessary to avoid

        a problem.  Example: function f() is defined with no argument but is

        called sometimes with one argument in original source; the revised

        source will be:</para>

      <programlisting>

      int f() { }

      /* .... */

      #ifndef __ANDF__

      	f(1);

      #else

      	f();

      #endif

      </programlisting>

      <para>In other cases, we had to modify some Makefiles. In the DRA-NAT

        build, this was necessary for sources which contained assembly

        instructions for example (or included a header file which used the

        same feature). When building the libraries and commands, the relevant

        Makefiles were modified. When building the kernel, a more elegant

        method has been used: a ''rulefile'', included by all the Makefiles,

        has been modified to check, prior to compiling a .c file, if a file

        with the same name plus a specific suffix .NATIVE existed. If so, the

        native compiler was called. In addition, shell scripts were written to

        create lists of source files, which were dependent on header files

        known to contain assembly code (or #pragma pack), and to create the

        .NATIVE files according to these lists.</para>

      <para>All these modifications on source files and Makefiles were done

        through a patch procedure:</para>

      <itemizedlist>

        <listitem>

          <para>In order to patch a file, from the WORK tree, two copies of

            the file are made in a patch tree, one for modifications and the

            other for keeping the reference version.</para>

        </listitem>

        <listitem>

          <para>Then the initial file, usually a link into the source tree, is

            replaced by a link to the copy for modifications.</para>

        </listitem>

        <listitem>

          <para>Once this has been done, the initial file can be modified,

            while the initial version of the file is saved.</para>

        </listitem>

      </itemizedlist>

    </sect1>

    <sect1>

      <title>Identification of TenDRA built object files</title>

      <para>In order to control the elements of the systems which were built

        in DRA-NAT mode, it was helpful to insert a special signature in

        object files created by the TenDRA compiler. An ident directive has

        been added to the assembly files generated by the TenDRA compiler,

        with the following pattern:</para>

      <para>/andf/bin/trans386:   (ANDF) 3.0 03/22/95</para>

      <para>Such a pattern can be extracted by mcs or strings commands from

        binary files (executables and libraries) in order to control a

        posteriori the number of modules actually compiled by the TenDRA

        technology.</para>

      <para>The file we modified in the TenDRA source delivery is the

        <filename>src/installer/trans386/trans.c</filename> file, and the

        change we made is located in the main() function:</para>

      <programlisting>

          init_all();

          if (diagnose)

             out_diagnose_prelude();

          TRASH d_capsule();

          /* change start */

          outs(``.ident \''@(#)/andf/bin/trans386:   (ANDF) 3.0 03/22/95\'''');

          outnl();

          /* change end */

          while (weak_list)

              /* ... */

      </programlisting>

    </sect1>

    <sect1>

      <title>Kernel and shared libraries</title>

      <para><emphasis>Configuring the DRA-NAT kernel for actual

        hardware</emphasis></para>

      <para>The normal way to build a kernel is to create, from the set of

        object files built for the kernel part of the system, a kernel which

        is not targeted to any particular hardware. Then, the system must be

        packaged on a tape and floppies, in order to be installed. An

        installation procedure would then be used to load the system and

        interactively configure the generic kernel to the actual hardware.

        However, going all the way through this procedure would have required

        a lot of time and disk space.</para>

      <para>We preferred to use a more simple and incremental way of building

        and configuring a new kernel:</para>

      <itemizedlist>

        <listitem>

          <para>We dedicated a development machine for the kernel

            testing.</para>

        </listitem>

        <listitem>

          <para>We copied the /etc/conf tree into the ``MACH'' tree of DRA-NAT

            build. This tree holds the kernel binary components and the kernel

            configuration files.</para>

        </listitem>

        <listitem>

          <para>We replaced (partially or totally) the original binaries by

            their DRA-NAT version.</para>

        </listitem>

        <listitem>

          <para>We then rebuilt a kernel, with the idbuild command.</para>

          <para>Note that the idbuild command is sensitive to the environment

            variables defining the current build ``MACH'' tree (ROOT and MACH

            variables).</para>

        </listitem>

      </itemizedlist>

      <para><emphasis>Progressive switching from native to DRA-NAT

        kernel</emphasis></para>

      <para>The method described above has proven very useful for easy fault

        isolation in case of a system crash.</para>

      <para>Kernel components are divided into two parts: those parts of the

        base kernel (<filename>/stand/unix</filename>) and those

        dynamically-loaded (from <filename>/etc/conf/mod.d</filename>). We

        started by replacing only one, non-critical, component of the kernel.

        Then, we replaced some dynamically-loaded modules by their DRA-NAT

        version. We continued by replacing other base kernel modules and

        concluded with the remaining dynamically-loaded modules. More than ten

        intermediate kernels were built and exercised during this

        process.</para>

      <para>Prior to exercising these kernels, emergency recovery floppies

        were made.  They could be (and have been) used to repair manually the

        hard disk stand or root file systems when the normal boot process from

        the hard disk failed.</para>

      <para>In order to switch to a new kernel, built in

        <filename>$ROOT/$MACH/etc/conf/cf.d/unix</filename>, it has to be

        copied into <filename>/stand</filename> (for example under the name

        <filename>unix.dranat</filename>).  The new dynamically-loaded kernel

        modules, built in the

        <filename>$ROOT/$MACH/etc/conf/modnew.d</filename> directory, have

        also to be moved to the <filename>/etc/conf.d/mod.d</filename>

        directory in order to be loaded at the next system reboot.  This

        latter operation should be done while the system is quiescent, i.e.

        after bringing it in single user mode and just before rebooting. To

        boot the alternative base kernel, the boot sequence is interrupted and

        an alternate kernel name is supplied by means of the KERNEL=name

        command.</para>

      <para><emphasis>Switching from native to DRA-NAT shared

        libraries</emphasis> </para>

      <para>For exercising a shared library made during the DRA-NAT build,

        e.g. the dynamically loaded libc, relevant files in

        <filename>/usr/lib</filename> and <filename>/usr/ccs/lib</filename>

        were replaced by links to either the reference or the DRA-NAT versions

        of them.  Care must be taken with these links, as shown by an example

        in <link linkend="dra-nat-validation">\xa4  7.4, page 19</link>.</para>

    </sect1>

    <sect1>

      <title>VSX4 validation</title>

      <para><emphasis>Installing VSX4 and building tests for NAT-NAT &amp;

        DRA-NAT systems validation</emphasis></para>

      <para>The VSX4 test suite is a rather large and complex application.

        Before actually running the tests, a number of steps have to be

        performed:</para>

      <itemizedlist>

        <listitem><para>We loaded the VSX4 test suite on a local disk which we

            added on one of the machines. Some hardware and system configurations

            were also needed in order to satisfy the VSX4

            requirements.</para></listitem>