| 2 |
7u83 |
1 |
<!-- Crown Copyright (c) 1998 -->
|
|
|
2 |
<HTML>
|
|
|
3 |
<HEAD>
|
|
|
4 |
<TITLE>Tokens and APIs</TITLE>
|
|
|
5 |
</HEAD>
|
|
|
6 |
<BODY TEXT="#000000" BGCOLOR="#FFFFFF" LINK="#0000FF" VLINK="#400080" ALINK="#FF0000">
|
|
|
7 |
<A NAME=S78>
|
|
|
8 |
<H1>TDF Guide, Issue 4.0 </H1>
|
|
|
9 |
<A HREF="guide13.html">
|
|
|
10 |
<H3>January 1998</H3>
|
|
|
11 |
<IMG SRC="../images/next.gif" ALT="next section"></A>
|
|
|
12 |
<A HREF="guide11.html">
|
|
|
13 |
<IMG SRC="../images/prev.gif" ALT="previous section"></A>
|
|
|
14 |
<A HREF="guide1.html"><IMG SRC="../images/top.gif" ALT="current document"></A>
|
|
|
15 |
<A HREF="../index.html"><IMG SRC="../images/home.gif" ALT="TenDRA home page">
|
|
|
16 |
</A>
|
|
|
17 |
<IMG SRC="../images/no_index.gif" ALT="document index"><P>
|
|
|
18 |
<HR>
|
|
|
19 |
<DL>
|
|
|
20 |
<DT><A HREF="#S79"><B>10.1 </B> - Application programming interfaces</A><DD>
|
|
|
21 |
<DT><A HREF="#S80"><B>10.2 </B> - Linking to APIs</A><DD>
|
|
|
22 |
<DT><A HREF="#S81"><B>10.2.1 </B> - Target independent headers, unique_extern
|
|
|
23 |
</A><DD>
|
|
|
24 |
<DT><A HREF="#S82"><B>10.3 </B> - Language programming interfaces</A><DD>
|
|
|
25 |
</DL>
|
|
|
26 |
<HR>
|
|
|
27 |
<H1>10 <A NAME=0>Tokens and APIs</H1>
|
|
|
28 |
All of the examples of the use of TOKENs so far given have really
|
|
|
29 |
been as abbreviations for commonly used constructs, e.g. the EXP OFFSETS
|
|
|
30 |
for fields of structures. However, the real justification for TOKENs
|
|
|
31 |
are their use as abstractions for things defined in libraries or application
|
|
|
32 |
program interfaces (APIs). <P>
|
|
|
33 |
<A NAME=S79>
|
|
|
34 |
<HR><H2>10.1. Application programming interfaces</H2>
|
|
|
35 |
APIs usually do not give complete language definitions of the operations
|
|
|
36 |
and values that they contain; generally, they are defined informally
|
|
|
37 |
in English giving relationships between the entities within them.
|
|
|
38 |
An API designer should allow implementors the opportunity of choosing
|
|
|
39 |
actual definitions which fit their hardware and the possibility of
|
|
|
40 |
changing them as better algorithms or representations become available.<P>
|
|
|
41 |
The most commonly quoted example is the representation of the type
|
|
|
42 |
FILE and its related operations in C. The ANSI C definition gives
|
|
|
43 |
no common representation for FILE; its implementation is defined to
|
|
|
44 |
be platform-dependent. A TDF producer can assume nothing about FILE;
|
|
|
45 |
not even that it is a structure. The only things that can alter or
|
|
|
46 |
create FILEs are also entities in the Ansi-C API and they will always
|
|
|
47 |
refer to FILEs via a C pointer. Thus TDF abstracts FILE as a SHAPE
|
|
|
48 |
TOKEN with no parameters, make_tok(T_FILE) say. Any program that uses
|
|
|
49 |
FILE would have to include a TOKDEC introducing T_FILE:<P>
|
|
|
50 |
<PRE>
|
|
|
51 |
make_tokdec(T_FILE, empty, shape())
|
|
|
52 |
</PRE>
|
|
|
53 |
and anywhere that it wished to refer to the SHAPE of FILE it would
|
|
|
54 |
do:<P>
|
|
|
55 |
<PRE>
|
|
|
56 |
shape_apply_token(make_tok(T_FILE), ())
|
|
|
57 |
</PRE>
|
|
|
58 |
Before this program is translated on a given platform, the actual
|
|
|
59 |
SHAPE of FILE must be supplied. This would be done by linking a TDF
|
|
|
60 |
CAPSULE which supplies the TOKDEF for the SHAPE of FILE which is particular
|
|
|
61 |
to the target platform.<P>
|
|
|
62 |
Many of the C operations which use FILEs are explicitly allowed to
|
|
|
63 |
be expanded as either procedure calls or as macros. For example, putc(c,f)
|
|
|
64 |
may be implemented either as a procedure call or as the expansion
|
|
|
65 |
of macro which uses the fields of f directly. Thus, it is quite natural
|
|
|
66 |
for putc(c, f) to be represented in TDF as an EXP TOKEN with two EXP
|
|
|
67 |
parameters which allows it to be expanded in either way. Of course,
|
|
|
68 |
this would be quite distinct from the use of putc as a value (as a
|
|
|
69 |
proc parameter of a procedure for example) which would require some
|
|
|
70 |
other representation. One such representation that comes to mind might
|
|
|
71 |
be to simply to make a TAGDEC for the putc value, supplying its TAGDEF
|
|
|
72 |
in the Ansi API CAPSULE for the platform. This might prove to be rather
|
|
|
73 |
short-sighted, since it denies us the possibility that the putc value
|
|
|
74 |
itself might be expanded from other values and hence it would be better
|
|
|
75 |
as another parameterless TOKEN. I have not come across an actual API
|
|
|
76 |
expansion for the putc value as other than a simple TAG; however the
|
|
|
77 |
FILE* value stdin is sometimes expressed as:<P>
|
|
|
78 |
<PRE>
|
|
|
79 |
#define stdin &_iob[0]
|
|
|
80 |
</PRE>
|
|
|
81 |
which illustrates the point. It is better to have all of the interface
|
|
|
82 |
of an API expressed as TOKENs to give both generality and flexibility
|
|
|
83 |
across different platforms.<P>
|
|
|
84 |
<A NAME=S80>
|
|
|
85 |
<HR><H2>10.2. Linking to APIs</H2>
|
|
|
86 |
In general, each API requires platform-dependent definitions to be
|
|
|
87 |
supplied by a combination of TDF linking and system linking for that
|
|
|
88 |
platform. This is illustrated in the following diagram giving the
|
|
|
89 |
various phases involved in producing a runnable program.<P>
|
|
|
90 |
<CENTER><IMG SRC="../images/guide3.gif"></CENTER>
|
|
|
91 |
<P>
|
|
|
92 |
There will be CAPSULEs for each API on each platform giving the expansions
|
|
|
93 |
for the TOKENs involved, usually as uses of identifiers which will
|
|
|
94 |
be supplied by system linking from some libraries. These CAPSULEs
|
|
|
95 |
would be derived from the header files on the platform for the API
|
|
|
96 |
in question, usually using some automatic tools. For example, there
|
|
|
97 |
will be a TDF CAPSULE (derived from <stdio.h>) which defines
|
|
|
98 |
the TOKEN T_FILE as the SHAPE for FILE, together with definitions
|
|
|
99 |
for the TOKENs for putc, stdin, etc., in terms of identifiers which
|
|
|
100 |
will be found in the library libc.a. <P>
|
|
|
101 |
<A NAME=S81>
|
|
|
102 |
<H3>10.2.1. Target independent headers, unique_extern</H3>
|
|
|
103 |
Any producer which uses an API will use system independent information
|
|
|
104 |
to give the common interface TOKENs for this API. In the C producer,
|
|
|
105 |
this is provided by header files using pragmas, which tell the producer
|
|
|
106 |
which TOKENs to use for the particular constructs of the API . In
|
|
|
107 |
any target-independent CAPSULE which uses the API, these TOKENs would
|
|
|
108 |
be introduced as TOKDECs and made globally accessible by using make_linkextern.
|
|
|
109 |
For a world-wide standard API, the EXTERNAL "name" for a
|
|
|
110 |
TOKEN used by make_linkextern should be provided by an application
|
|
|
111 |
of unique_extern on a UNIQUE drawn from a central repository of names
|
|
|
112 |
for entities in standard APIs; this repository would form a kind of
|
|
|
113 |
super-standard for naming conventions in all possible APIs. The mechanism
|
|
|
114 |
for controlling this super-standard has yet to be set up, so at the
|
|
|
115 |
moment all EXTERN names are created by string_extern.<P>
|
|
|
116 |
An interesting example in the use of TOKENs comes in abstracting field
|
|
|
117 |
names. Often, an API will say something like "the type Widget
|
|
|
118 |
is a structure with fields alpha, beta ..." without specifying
|
|
|
119 |
the order of the fields or whether the list of fields is complete.
|
|
|
120 |
The field selection operations for Widget should then be expressed
|
|
|
121 |
using EXP OFFSET TOKENs; each field would have its own TOKEN giving
|
|
|
122 |
its offset which will be filled in when the target is known. This
|
|
|
123 |
gives implementors on a particular platform the opportunity to reorder
|
|
|
124 |
fields or add to them as they like; it also allows for extension of
|
|
|
125 |
the standard in the same way.<P>
|
|
|
126 |
The most common SORTs of TOKENs used for APIs are SHAPEs to represent
|
|
|
127 |
types, and EXPs to represent values, including procedures and constants.
|
|
|
128 |
NATs and VARIETYs are also sometimes used where the API does not specify
|
|
|
129 |
the types of integers involved. The other SORTs are rarely used in
|
|
|
130 |
APIs; indeed it is difficult to imagine <I>any</I> realistic use of
|
|
|
131 |
TOKENs of SORT BOOL. However, the criterion for choosing which SORTs
|
|
|
132 |
are available for TOKENisation is not their immediate utility, but
|
|
|
133 |
that the structural integrity and simplicity of TDF is maintained.
|
|
|
134 |
It is fairly obvious that having BOOL TOKENs will cause no problems,
|
|
|
135 |
so we may as well allow them. <P>
|
|
|
136 |
<A NAME=S82>
|
|
|
137 |
<HR><H2>10.3. Language programming interfaces</H2>
|
|
|
138 |
So far, I have been speaking as though a TOKENised API could only
|
|
|
139 |
be some library interface, built on top of some language, like xpg3,
|
|
|
140 |
posix, X etc. on top of C. However, it is possible to consider the
|
|
|
141 |
constructions of the language itself as ideal candidates for TOKENisation.
|
|
|
142 |
For example, the C for-statement could be expressed as TOKEN with
|
|
|
143 |
four parameters
|
|
|
144 |
<A NAME=footnote77 HREF="footnote.html#77">*</A>. This TOKEN could
|
|
|
145 |
be expanded in TDF in several different ways, all giving the correct
|
|
|
146 |
semantics of a for-statement. A translator (or other tools) could
|
|
|
147 |
choose the expansion it wants depending on context and the properties
|
|
|
148 |
of the parameters. The C producer could give a default expansion which
|
|
|
149 |
a lazy translator writer could use, but others might use expansions
|
|
|
150 |
which might be more advantageous. This idea could be extended to virtually
|
|
|
151 |
all the constructions of the language, giving what is in effect a
|
|
|
152 |
C-language API; perhaps this might be called more properly a language
|
|
|
153 |
programming interface (LPI). Thus, we would have TOKENs for C for-statements,
|
|
|
154 |
C conditionals, C procedure calls, C procedure definitions etc.
|
|
|
155 |
<A NAME=footnote78 HREF="footnote.html#78">*</A>.
|
|
|
156 |
<P>
|
|
|
157 |
The notion of a producer for any language working to an LPI specific
|
|
|
158 |
to the constructs of the language is very attractive. It could use
|
|
|
159 |
different TOKENs to reflect the subtle differences between uses of
|
|
|
160 |
similar constructs in different languages which might be difficult
|
|
|
161 |
or impossible to detect from their expansions, but which could allow
|
|
|
162 |
better optimisations in the object code. For example, Fortran procedures
|
|
|
163 |
are slightly different from C procedures in that they do not allow
|
|
|
164 |
aliasing between parameters and globals. While application of the
|
|
|
165 |
standard TDF procedure calls would be semantically correct, knowledge
|
|
|
166 |
of that the non-aliasing rule applies would allow some procedures
|
|
|
167 |
to be translated to more efficient code. A translator without knowledge
|
|
|
168 |
of the semantics implicit in the TOKENs involved would still produce
|
|
|
169 |
correct code, but one which knew about them could take advantage of
|
|
|
170 |
that knowledge.<P>
|
|
|
171 |
I also think that LPIs would be a very useful tool for crystalising
|
|
|
172 |
ideas on how languages should be translated, allowing one to experiment
|
|
|
173 |
with expansions not thought of by the producer writer. This decoupling
|
|
|
174 |
is also an escape clause allowing the producer writer to defer the
|
|
|
175 |
implementation of a construct completely to translate-time or link-time,
|
|
|
176 |
as is done at the moment in C for off-stack allocation. As such it
|
|
|
177 |
also serves as a useful test-bed for TOKEN constructions which may
|
|
|
178 |
in future become new constructors of core TDF. <P>
|
|
|
179 |
<HR>
|
|
|
180 |
<P><I>Part of the <A HREF="../index.html">TenDRA Web</A>.<BR>Crown
|
|
|
181 |
Copyright © 1998.</I></P>
|
|
|
182 |
</BODY>
|
|
|
183 |
</HTML>
|