Subversion Repositories planix.SVN

.TH HTML 2

.SH NAME

parsehtml,

printitems,

validitems,

freeitems,

freedocinfo,

dimenkind,

dimenspec,

targetid,

targetname,

fromStr,

toStr

\- HTML parser

.SH SYNOPSIS

.nf

.PP

.ft L

#include <u.h>

#include <libc.h>

#include <html.h>

.ft P

.PP

.ta \w'\fLToken* 'u

.B

Item*	parsehtml(uchar* data, int datalen, Rune* src, int mtype,

.B

	int chset, Docinfo** pdi)

.PP

.B

void	printitems(Item* items, char* msg)

.PP

.B

int	validitems(Item* items)

.PP

.B

void	freeitems(Item* items)

.PP

.B

void	freedocinfo(Docinfo* d)

.PP

.B

int	dimenkind(Dimen d)

.PP

.B

int	dimenspec(Dimen d)

.PP

.B

int	targetid(Rune* s)

.PP

.B

Rune*	targetname(int targid)

.PP

.B

uchar*	fromStr(Rune* buf, int n, int chset)

.PP

.B

Rune*	toStr(uchar* buf, int n, int chset)

.SH DESCRIPTION

.PP

This library implements a parser for HTML 4.0 documents.

The parsed HTML is converted into an intermediate representation that

describes how the formatted HTML should be laid out.

.PP

.I Parsehtml

parses an entire HTML document contained in the buffer

.I data

and having length

.IR datalen .

The URL of the document should be passed in as

.IR src .

.I Mtype

is the media type of the document, which should be either

.B TextHtml

or

.BR TextPlain .

The character set of the document is described in

.IR chset ,

which can be one of

.BR US_Ascii ,

.BR ISO_8859_1 ,

.B UTF_8

or

.BR Unicode .

The return value is a linked list of

.B Item

structures, described in detail below.

As a side effect, 

.BI * pdi

is set to point to a newly created

.B Docinfo

structure, containing information pertaining to the entire document.

.PP

The library expects two allocation routines to be provided by the

caller,

.B emalloc

and

.BR erealloc .

These routines are analogous to the standard malloc and realloc routines,

except that they should not return if the memory allocation fails.

In addition,

.B emalloc

is required to zero the memory.

.PP

For debugging purposes,

.I printitems

may be called to display the contents of an item list; individual items may

be printed using the

.B %I

print verb, installed on the first call to

.IR parsehtml .

.I validitems

traverses the item list, checking that all of the pointers are valid.

It returns

.B 1

is everything is ok, and

.B 0

if an error was found.

Normally, one would not call these routines directly.

Instead, one sets the global variable

.I dbgbuild

and the library calls them automatically.

One can also set

.IR warn ,

to cause the library to print a warning whenever it finds a problem with the

input document, and

.IR dbglex ,

to print debugging information in the lexer.

.PP

When an item list is finished with, it should be freed with

.IR freeitems .

Then,

.I freedocinfo

should be called on the pointer returned in

.BI * pdi\f1.

.PP

.I Dimenkind

and

.I dimenspec

are provided to interpret the

.B Dimen

type, as described in the section

.IR "Dimension Specifications" .

.PP

Frame target names are mapped to integer ids via a global, permanent mapping.

To find the value for a given name, call

.IR targetid ,

which allocates a new id if the name hasn't been seen before.

The name of a given, known id may be retrieved using

.IR targetname .

The library predefines

.BR FTtop ,

.BR FTself ,

.B FTparent

and

.BR FTblank .

.PP

The library handles all text as Unicode strings (type

.BR Rune* ).

Character set conversion is provided by

.I fromStr

and

.IR toStr .

.I FromStr

takes

.I n

Unicode characters from

.I buf

and converts them to the character set described by

.IR chset .

.I ToStr

takes

.I n

bytes from

.IR buf ,

interpretted as belonging to character set

.IR chset ,

and converts them to a Unicode string.

Both routines null-terminate the result, and use

.B emalloc

to allocate space for it.

.SS Items

The return value of

.I parsehtml

is a linked list of variant structures,

with the generic portion described by the following definition:

.PP

.EX

.ta 6n +\w'Genattr* 'u

typedef struct Item Item;

struct Item

{

	Item*	next;

	int	width;

	int	height;

	int	ascent;

	int	anchorid;

	int	state;

	Genattr*	genattr;

	int	tag;

};

.EE

.PP

The field

.B next

points to the successor in the linked list of items, while

.BR width ,

.BR height ,

and

.B ascent

are intended for use by the caller as part of the layout process.

.BR Anchorid ,

if non-zero, gives the integer id assigned by the parser to the anchor that

this item is in (see section

.IR Anchors ).

.B State

is a collection of flags and values described as follows:

.PP

.EX

.ta 6n +\w'IFindentshift = 'u

enum

{

	IFbrk =	0x80000000,

	IFbrksp =	0x40000000,

	IFnobrk =	0x20000000,

	IFcleft =	0x10000000,

	IFcright =	0x08000000,

	IFwrap =	0x04000000,

	IFhang =	0x02000000,

	IFrjust =	0x01000000,

	IFcjust =	0x00800000,

	IFsmap =	0x00400000,

	IFindentshift =	8,

	IFindentmask =	(255<<IFindentshift),

	IFhangmask =	255

};

.EE

.PP

.B IFbrk

is set if a break is to be forced before placing this item.

.B IFbrksp

is set if a 1 line space should be added to the break (in which case

.B IFbrk

is also set).

.B IFnobrk

is set if a break is not permitted before the item.

.B IFcleft

is set if left floats should be cleared (that is, if the list of pending left floats should be placed)

before this item is placed, and

.B IFcright

is set for right floats.

In both cases, IFbrk is also set.

.B IFwrap

is set if the line containing this item is allowed to wrap.

.B IFhang

is set if this item hangs into the left indent.

.B IFrjust

is set if the line containing this item should be right justified,

and

.B IFcjust

is set for center justified lines.

.B IFsmap

is used to indicate that an image is a server-side map.

The low 8 bits, represented by

.BR IFhangmask ,

indicate the current hang into left indent, in tenths of a tabstop.

The next 8 bits, represented by

.B IFindentmask

and

.BR IFindentshift ,

indicate the current indent in tab stops.

.PP

The field

.B genattr

is an optional pointer to an auxiliary structure, described in the section

.IR "Generic Attributes" .

.PP

Finally,

.B tag

describes which variant type this item has.

It can have one of the values

.BR Itexttag ,

.BR Iruletag ,

.BR Iimagetag ,

.BR Iformfieldtag ,

.BR Itabletag ,

.B Ifloattag

or

.BR Ispacertag .

For each of these values, there is an additional structure defined, which

includes Item as an unnamed initial substructure, and then defines additional

fields.

.PP

Items of type

.B Itexttag

represent a piece of text, using the following structure:

.PP

.EX

.ta 6n +\w'Rune* 'u

struct Itext

{

	Item;

	Rune*	s;

	int	fnt;

	int	fg;

	uchar	voff;

	uchar	ul;

};

.EE

.PP

Here

.B s

is a null-terminated Unicode string of the actual characters making up this text item,

.B fnt

is the font number (described in the section

.IR "Font Numbers" ),

and

.B fg

is the RGB encoded color for the text.

.B Voff

measures the vertical offset from the baseline; subtract

.B Voffbias

to get the actual value (negative values represent a displacement down the page).

The field

.B ul

is the underline style:

.B ULnone

if no underline,

.B ULunder

for conventional underline, and

.B ULmid

for strike-through.

.PP

Items of type

.B Iruletag

represent a horizontal rule, as follows:

.PP

.EX

.ta 6n +\w'Dimen 'u

struct Irule

{

	Item;

	uchar	align;

	uchar	noshade;

	int	size;

	Dimen	wspec;

};

.EE

.PP

Here

.B align

is the alignment specification (described in the corresponding section),

.B noshade

is set if the rule should not be shaded,

.B size

is the height of the rule (as set by the size attribute),

and

.B wspec

is the desired width (see section

.IR "Dimension Specifications" ).

.PP

Items of type

.B Iimagetag

describe embedded images, for which the following structure is defined:

.PP

.EX

.ta 6n +\w'Iimage* 'u

struct Iimage

{

	Item;

	Rune*	imsrc;

	int	imwidth;

	int	imheight;

	Rune*	altrep;

	Map*	map;

	int	ctlid;

	uchar	align;

	uchar	hspace;

	uchar	vspace;

	uchar	border;

	Iimage*	nextimage;

};

.EE

.PP

Here

.B imsrc

is the URL of the image source,

.B imwidth

and

.BR imheight ,

if non-zero, contain the specified width and height for the image,

and

.B altrep

is the text to use as an alternative to the image, if the image is not displayed.

.BR Map ,

if set, points to a structure describing an associated client-side image map.

.B Ctlid

is reserved for use by the application, for handling animated images.

.B Align

encodes the alignment specification of the image.

.B Hspace

contains the number of pixels to pad the image with on either side, and

.B Vspace

the padding above and below.

.B Border

is the width of the border to draw around the image.

.B Nextimage

points to the next image in the document (the head of this list is

.BR Docinfo.images ).

.PP

For items of type

.BR Iformfieldtag ,

the following structure is defined:

.PP

.EX

.ta 6n +\w'Formfield* 'u

struct Iformfield

{

	Item;

	Formfield*	formfield;

};

.EE

.PP

This adds a single field,

.BR formfield ,

which points to a structure describing a field in a form, described in section

.IR Forms .

.PP

For items of type

.BR Itabletag ,

the following structure is defined:

.PP

.EX

.ta 6n +\w'Table* 'u

struct Itable

{

	Item;

	Table*	table;

};

.EE

.PP

.B Table

points to a structure describing the table, described in the section

.IR Tables .

.PP

For items of type

.BR Ifloattag ,

the following structure is defined:

.PP

.EX

.ta 6n +\w'Ifloat* 'u

struct Ifloat

{

	Item;

	Item*	item;

	int	x;

	int	y;

	uchar	side;

	uchar	infloats;

	Ifloat*	nextfloat;

};

.EE

.PP

The

.B item

points to a single item (either a table or an image) that floats (the text of the

document flows around it), and

.B side

indicates the margin that this float sticks to; it is either

.B ALleft

or

.BR ALright .

.B X

and

.B y

are reserved for use by the caller; these are typically used for the coordinates

of the top of the float.

.B Infloats

is used by the caller to keep track of whether it has placed the float.

.B Nextfloat

is used by the caller to link together all of the floats that it has placed.

.PP

For items of type

.BR Ispacertag ,

the following structure is defined:

.PP

.EX

.ta 6n +\w'Item; 'u

struct Ispacer

{

	Item;

	int	spkind;

};

.EE

.PP

.B Spkind

encodes the kind of spacer, and may be one of

.B ISPnull

(zero height and width),

.B ISPvline

(takes on height and ascent of the current font),

.B ISPhspace

(has the width of a space in the current font) and

.B ISPgeneral

(for all other purposes, such as between markers and lists).

.SS Generic Attributes

.PP

The genattr field of an item, if non-nil, points to a structure that holds

the values of attributes not specific to any particular

item type, as they occur on a wide variety of underlying HTML tags.

The structure is as follows:

.PP

.EX

.ta 6n +\w'SEvent* 'u

typedef struct Genattr Genattr;

struct Genattr

{

	Rune*	id;

	Rune*	class;

	Rune*	style;

	Rune*	title;

	SEvent*	events;

};

.EE

.PP

Fields

.BR id ,

.BR class ,

.B style

and

.BR title ,

when non-nil, contain values of correspondingly named attributes of the HTML tag

associated with this item.

.B Events

is a linked list of events (with corresponding scripted actions) associated with the item:

.PP

.EX

.ta 6n +\w'SEvent* 'u

typedef struct SEvent SEvent;

struct SEvent

{

	SEvent*	next;

	int	type;

	Rune*	script;

};

.EE

.PP

Here,

.B next

points to the next event in the list,

.B type

is one of

.BR SEonblur ,

.BR SEonchange ,

.BR SEonclick ,

.BR SEondblclick ,

.BR SEonfocus ,

.BR SEonkeypress ,

.BR SEonkeyup ,

.BR SEonload ,

.BR SEonmousedown ,

.BR SEonmousemove ,

.BR SEonmouseout ,

.BR SEonmouseover ,

.BR SEonmouseup ,

.BR SEonreset ,

.BR SEonselect ,

.B SEonsubmit

or

.BR SEonunload ,

and

.B script

is the text of the associated script.

.SS Dimension Specifications

.PP

Some structures include a dimension specification, used where

a number can be followed by a

.B %

or a

.B *

to indicate

percentage of total or relative weight.

This is encoded using the following structure:

.PP

.EX

.ta 6n +\w'int 'u

typedef struct Dimen Dimen;

struct Dimen

{

	int	kindspec;

};

.EE

.PP

Separate kind and spec values are extracted using

.I dimenkind

and

.IR dimenspec .

.I Dimenkind

returns one of

.BR Dnone ,

.BR Dpixels ,

.B Dpercent

or

.BR Drelative .

.B Dnone

means that no dimension was specified.

In all other cases,

.I dimenspec

should be called to find the absolute number of pixels, the percentage of total,

or the relative weight.

.SS Background Specifications

.PP

It is possible to set the background of the entire document, and also

for some parts of the document (such as tables).

This is encoded as follows:

.PP

.EX

.ta 6n +\w'Rune* 'u

typedef struct Background Background;

struct Background

{

	Rune*	image;

	int	color;

};

.EE

.PP

.BR Image ,

if non-nil, is the URL of an image to use as the background.

If this is nil,

.B color

is used instead, as the RGB value for a solid fill color.

.SS Alignment Specifications

.PP

Certain items have alignment specifiers taken from the following

enumerated type:

.PP

.EX

.ta 6n

enum

{

	ALnone = 0, ALleft, ALcenter, ALright, ALjustify,

	ALchar, ALtop, ALmiddle, ALbottom, ALbaseline

};

.EE

.PP

These values correspond to the various alignment types named in the HTML 4.0

standard.

If an item has an alignment of

.B ALleft

or

.BR ALright ,

the library automatically encapsulates it inside a float item.

.PP

Tables, and the various rows, columns and cells within them, have a more

complex alignment specification, composed of separate vertical and

horizontal alignments:

.PP

.EX

.ta 6n +\w'uchar 'u

typedef struct Align Align;

struct Align

{

	uchar	halign;

	uchar	valign;

};

.EE

.PP

.B Halign

can be one of

.BR ALnone ,

.BR ALleft ,

.BR ALcenter ,

.BR ALright ,

.B ALjustify

or

.BR ALchar .

.B Valign

can be one of

.BR ALnone ,

.BR ALmiddle ,

.BR ALbottom ,

.BR ALtop

or

.BR ALbaseline .

.SS Font Numbers

.PP

Text items have an associated font number (the

.B fnt

field), which is encoded as

.BR style*NumSize+size .

Here,

.B style

is one of

.BR FntR ,

.BR FntI ,

.B FntB

or

.BR FntT ,

for roman, italic, bold and typewriter font styles, respectively, and size is

.BR Tiny ,

.BR Small ,

.BR Normal ,

.B Large

or

.BR Verylarge .

The total number of possible font numbers is

.BR NumFnt ,

and the default font number is

.B DefFnt

(which is roman style, normal size).

.SS Document Info

.PP

Global information about an HTML page is stored in the following structure:

.PP

.EX

.ta 6n +\w'DestAnchor* 'u

typedef struct Docinfo Docinfo;

struct Docinfo

{

	// stuff from HTTP headers, doc head, and body tag

	Rune*	src;

	Rune*	base;

	Rune*	doctitle;

	Background	background;

	Iimage*	backgrounditem;

	int	text;

	int	link;

	int	vlink;

	int	alink;

	int	target;

	int	chset;

	int	mediatype;

	int	scripttype;

	int	hasscripts;

	Rune*	refresh;

	Kidinfo*	kidinfo;

	int	frameid;

	// info needed to respond to user actions

	Anchor*	anchors;

	DestAnchor*	dests;

	Form*	forms;

	Table*	tables;

	Map*	maps;

	Iimage*	images;

};

.EE

.PP

.B Src

gives the URL of the original source of the document,

and

.B base

is the base URL.

.B Doctitle

is the document's title, as set by a

.B <title>

element.

.B Background

is as described in the section

.IR "Background Specifications" ,

and

.B backgrounditem

is set to be an image item for the document's background image (if given as a URL),

or else nil.

.B Text

gives the default foregound text color of the document,

.B link

the unvisited hyperlink color,

.B vlink

the visited hyperlink color, and

.B alink

the color for highlighting hyperlinks (all in 24-bit RGB format).

.B Target

is the default target frame id.

.B Chset

and

.B mediatype

are as for the

.I chset

and

.I mtype

parameters to

.IR parsehtml .

.B Scripttype

is the type of any scripts contained in the document, and is always

.BR TextJavascript .

.B Hasscripts

is set if the document contains any scripts.

Scripting is currently unsupported.

.B Refresh

is the contents of a

.B "<meta http-equiv=Refresh ...>"

tag, if any.

.B Kidinfo

is set if this document is a frameset (see section

.IR Frames ).

.B Frameid

is this document's frame id.

.PP

.B Anchors

is a list of hyperlinks contained in the document,

and

.B dests

is a list of hyperlink destinations within the page (see the following section for details).

.BR Forms ,

.B tables

and

.B maps

are lists of the various forms, tables and client-side maps contained

in the document, as described in subsequent sections.

.B Images

is a list of all the image items in the document.

.SS Anchors

.PP

The library builds two lists for all of the

.B <a>

elements (anchors) in a document.

Each anchor is assigned a unique anchor id within the document.

For anchors which are hyperlinks (the

.B href

attribute was supplied), the following structure is defined:

.PP

.EX

.ta 6n +\w'Anchor* 'u

typedef struct Anchor Anchor;

struct Anchor

{

	Anchor*	next;

	int	index;

	Rune*	name;

	Rune*	href;

	int	target;

};

.EE

.PP

.B Next

points to the next anchor in the list (the head of this list is

.BR Docinfo.anchors ).

.B Index

is the anchor id; each item within this hyperlink is tagged with this value

in its

.B anchorid

field.

.B Name

and

.B href

are the values of the correspondingly named attributes of the anchor

(in particular, href is the URL to go to).

.B Target

is the value of the target attribute (if provided) converted to a frame id.

.PP

Destinations within the document (anchors with the name attribute set)

are held in the

.B Docinfo.dests

list, using the following structure:

.PP

.EX

.ta 6n +\w'DestAnchor* 'u

typedef struct DestAnchor DestAnchor;

struct DestAnchor

{

	DestAnchor*	next;

	int	index;

	Rune*	name;

	Item*	item;

};

.EE

.PP

.B Next

is the next element of the list,

.B index

is the anchor id,

.B name

is the value of the name attribute, and

.B item

is points to the item within the parsed document that should be considered

to be the destination.

.SS Forms

.PP

Any forms within a document are kept in a list, headed by

.BR Docinfo.forms .

The elements of this list are as follows:

.PP

.EX

.ta 6n +\w'Formfield* 'u

typedef struct Form Form;

struct Form

{

	Form*	next;

	int	formid;

	Rune*	name;

	Rune*	action;

	int	target;

	int	method;

	int	nfields;

	Formfield*	fields;

};

.EE

.PP

.B Next

points to the next form in the list.

.B Formid

is a serial number for the form within the document.

.B Name

is the value of the form's name or id attribute.

.B Action

is the value of any action attribute.

.B Target

is the value of the target attribute (if any) converted to a frame target id.

.B Method

is one of

.B HGet

or

.BR HPost .

.B Nfields

is the number of fields in the form, and

.B fields

is a linked list of the actual fields.

.PP

The individual fields in a form are described by the following structure:

.PP

.EX

.ta 6n +\w'Formfield* 'u

typedef struct Formfield Formfield;

struct Formfield

{

	Formfield*	next;

	int	ftype;

	int	fieldid;

	Form*	form;

	Rune*	name;

	Rune*	value;

	int	size;

	int	maxlength;

	int	rows;

	int	cols;

	uchar	flags;

	Option*	options;

	Item*	image;

	int	ctlid;

	SEvent*	events;

};

.EE

.PP

Here,

.B next

points to the next field in the list.

.B Ftype

is the type of the field, which can be one of

.BR Ftext ,

.BR Fpassword ,

.BR Fcheckbox ,

.BR Fradio ,

.BR Fsubmit ,

.BR Fhidden ,

.BR Fimage ,

.BR Freset ,

.BR Ffile ,

.BR Fbutton ,

.B Fselect

or

.BR Ftextarea .

.B Fieldid

is a serial number for the field within the form.

.B Form

points back to the form containing this field.

.BR Name ,

.BR value ,

.BR size ,

.BR maxlength ,

.B rows

and

.B cols

each contain the values of corresponding attributes of the field, if present.

.B Flags

contains per-field flags, of which

.B FFchecked

and

.B FFmultiple

are defined.

.B Image

is only used for fields of type

.BR Fimage ;

it points to an image item containing the image to be displayed.

.B Ctlid

is reserved for use by the caller, typically to store a unique id

of an associated control used to implement the field.

.B Events