--------------------------------------------------------------------------------

            Dynamically Loadable Module Support library for DJGPP.
            Copyright (C) 2000 by Andrew Zabolotny <bit@eltech.ru>

--------------------------------------------------------------------------------


    Hello there, fellow programmer!

    I believe if you're reading this I don't have to explain why dynamically -
loadable module support is a must for any modern operating system.  Last but
not least, having dynamic libraries support on any operating system is useful
if you want to use LGPL libraries, and don't want to disclose your Ultimate
Know-How contained in other source files linked against those libraries.
Specifically, section 6 of LGPL places several restrictions on `work that uses
the library' which could be worked around IF the LGPLed library is put into a
dynamically-loadable module.

    This library was designed specifically for the Crystal Space project
(see http://crystal.sourceforge.net), but it is very general and useful in
other environments as well.


    License
    -------

    This library is ABSOLUTELY free. That is, no LGPL, no Mozilla, no
credits, no anything (although crediting me is not prohibited :-). Do whatever
you want with it.

    As a consequence, you get ABSOLUTELY no warranties. That is, you get
nothing for nothing.

    (The intention of making this library free is to set a single standard for
dynamically-loadable modules under DJGPP.  I hate all thouse thousands of
non-free similar standards in the software industry, and the only reason why
new and new ones appear is because someone is trying to avoid paying several
megabucks for the joy of using someone else' standard).


    Credits
    -------

    This library is in (a very small) part based on work by Charles Sandmann
and DJ Delorie. DJGPP C library contains a very simple-and-dirty function
called dxeload() and a very simple-and-dirty tool called dxegen which become
the `seed' from which this library has grown.


    How it works
    ------------

    First of all, let's clarify the terms. The original DJGPP dxe package
used the `dxe' extension for dynamic modules (I believe it stands for
`dynamic executable'). I took this extension (why not?) and developed the
version 2 of the DXE format. Thus in the following `DXE' is used as a synonym
for `dynamically loadable executable module'. Also I call them sometimes
`dynamic modules' or simply `modules'.

    There are several ways to use DXE modules. The library allows you either
to load/unload modules at runtime or to link with them (or rather with
so-called `import libraries') at link time (e.g. statically link with dynamic
libraries).

    The library allows to build either dynamic modules that don't have
unresolved symbols as well as dynamic modules that DO have unresolved symbols
(and allows you to resolve them at runtime).

    The first way to use DXE modules is implemented through the dlopen() API.
This API closely mimics the one found on most contemporan Unices, thus you may
even try to compile some Unix apps that uses dynamic libraries without a
single change.

    Statically linking against dynamic libraries allows for a very easy way to
introduce dynamic modules into any program. `Static linking' means that during
linking stage of your program you link against certain library, called a
`import library' in the following, which provides all the functions present in
certain DXE module. The symbols points to some very small wrappers, and when
you call any of those functions, the library is automatically loaded and the
call is redirected further. Also you may load/unload the library manually
by calling some special functions called load_MODNAME and unload_MODNAME
functions (where MODNAME stands for the name of your dynamic module).

    The dxe2gen tool can build these import libraries, so basically you just
take a existing DXE module and generate the corresponding import library for
this module (see the `Import libraries' section). Then you link against it...
and voila! it works.

    Static linkage against dynamic libraries has a drawback: you CANNOT have
exported data inside the library (at least you can't use it). This happens
because the symbols are resolved to the wrappers and not to the actual symbols
inside the module. That is, if you have a variable called `i' in a dynamic
module, and you assign `i = 1' from your program which is statically linked
against that module, you will actually write over the wrapper and not into the
actual location where the contents of `i' are. If you really need to set/get
values of some variables inside a dynamic module, write a couple of set_i and
get_i *functions*.


    Building and installing
    -----------------------

    The package comes in binary-less form :-) thus you will need to build the
library and the dxe2gen tool manually. Its very easy though. I suppose you
already have DJGPP set up correctly (otherwise why you are reading this? :-)
thus you just should type:

	make

    and you will get the libdl.a library as well as the dxe2gen tool (which is
used to build dynamic libraries). Install dxe2gen.exe under X:/djgpp/bin
(where X is the drive where your DJGPP installation is) and libdl.a under
X:/djgpp/lib. Also you should copy the contents of include/ directory into
the X:/djgpp/include directory.

    Also you will need to copy the src/dxe2.ld file into your X:/djgpp/lib
directory. This is a linker script file that tells the GNU linker how a
dynamic module should be built.

    For all you lazy people I have made a special target in makefile, so
basically you don't need to do all the above mentioned operations, but rather
execute a single command:

	make install
	
    This command will build everything and install everything in the respective
places. The above detailed description was put here rather to let you know what
will happen when you type `make install' :-)

    Also the package contains a number of samples/testcases which you may
build to see the dynamic modules in action. To compile them just type:

	make test

    You will end up with a bunch of executables and dxe modules. Find yourself
what they do :-) (the sources are in test/ directory).


    The API
    -------

    The dynamic module loader API is contained within the file dlfcn.h. There
is also a private header file dxe2.h which contains all the non-standard API
functions and structures, but you don't need to include it directly since
dlfcn.h already includes it. Thus, in all programs you just need to
#include <dlfcn.h>. Here are the basic functions of the Unix-like API:

	dxe_h dlopen (const char *filename, int mode);

    This function tries to load a DXE. The `mode' field is a combination of
RTLD_XXX flags, of which just one works (others are ignored but present in
include file for Unix compatibility). The flag is RTLD_GLOBAL which means that
all symbols in this module are made public and subsequently loaded modules
with unresolved symbols will `see' them and will try to find the unresolved
references through them.

    The function returns a handle. The handle is basically an `void *'.
    If the function cannot find a file called `filename' all the components
of a environment variable called `LD_LIBRARY_PATH' is scanned for that file.
For example, you could set it to something like this (I did it at the start of
my djgpp.env file, right after DJDIR variable):

	LD_LIBRARY_PATH=%DJDIR%/dxe;%DJDIR%/lib;c:/usr/dxe;d:/something

    The components of the path are prepended to `filename' and a file with
that name is looked for.

	int dlclose (dxe_h handle);

    Well, guess what... this function CLOSES a dynamic module. Whoa!
The module is unloaded, all pointers that point somewhere inside module are
invalid after that.

	void *dlsym (dxe_h handle, const char *name);

    Return the address of a symbol within a DXE. Both functions and data
symbols are supported. The returned address is just a (void *) thus you
usually will want to typecast it to whatever prototype you have. By specifying
RTLD_DEFAULT in the place of `handle' you have access to the global symbol
scope.

	char *dlerror ();

    This function returns a simple text string (no linefeed at the end) which
describes somehow the reason why last dlopen() has failed.

    --------------

    There are several extensions to the dynamic loader interface, that contain
non-standard functions providing access to some features not found in other
flavours of dynamic module loading facilities.

	int dlregsym (dxe_symbol_table *symtab);

    Register a table of symbols to be exported into the loaded modules.
You can register any number of such tables. When a module with unresolved
external symbols is loaded, all these tables are searched for the respective
symbol. If no one is found, the last-resort handler is called. If even the
last-resort handler cannot return an address, an error condition is raised.

    The effect of dlregsym() is cumulative. That is, you can call it multiple
times to register several export symbol tables, and all of them will be taken
into account when loading a new module.

	int dlunregsym (dxe_symbol_table *symtab);

    This is the inverse of dlregsym(), provided for sake of completeness.
I never had the actual need to unregister a table of symbols that have been
already registered.

	void dlsetres (void *(*errh) (const char *));

    Set the last-resort handler called when a unresolved symbol cannot be
found in all the symbol tables that the dynamic loader have at his disposition.
The handler should return NULL to rise a error condition, otherwise it should
return a valid address. For example, as a last resort, the errh routine could
return the address of a dummy function -- this allows to load modules which
you don't know in advance which unresolved symbols it contains. Of course, the
functions that use this last-resort dummy function will be, most likely,
unuseable but at least you may query the address of some table inside the
module, for example, and process it somehow.

	extern void (*dlerrstatmod) (const char *module);

    This is a *pointer* to a function (e.g. replaceable) containing a pointer
to a function that is being called when static linking fails because of missing
module. Note that due to delayed nature of static linkage, the error can pop up
very late! If you want to check it at startup, call the "load_MODULENAME"
function explicitly. The function should never return.

	extern void (*dlerrstatsym) (const char *module, const char *symbol);

    This is a pointer to a function that is being called when during static
linking the dynamic loader finds that some symbol is missing from dynamic
module. The function should never return.

    --------------

    There are also a number of useful macros, which you may use instead of
explicitely filling structures etc. Here are them:

	DXE_EXPORT_TABLE(name)
	DXE_EXPORT_TABLE_AUTO(name)
	DXE_EXPORT(symbol)
	DXE_EXPORT_END

    These macros allows you to define a table of symbols that are going to be
exported into subsequently loaded modules.  For a example see the `Building a
DXE' section.  Also if you use DXE_EXPORT_TABLE_AUTO instead of
DXE_EXPORT_TABLE the table will be automatically registered with the dynamic
loader during program startup (thus you don't need to call dlregsym(name)
manually).

	DECLARE_STATIC_DXE(name)

    This macro declares the two functions that are present if you are
statically linking against a dynamic library (see the `How it works' section
for details on static linking). Note that `name' should be in capitals!
After declaring the module with the above macro, you can call the load_NAME
and unload_NAME functions to dynamically load and unload the statically linked
dynamic library.


    Building a DXE
    --------------

    To build a DXE module you will need the dxe2gen tool. If you don't have
it, make it by typing "make" in the directory where you see the `Makefile'.
You will end with a dxe2gen.exe tool and with the libdl.a library (which
provides the API described above).

    Now suppose you have some library which you want to turn into a DXE
module. To do this inside the makefile for your library add a rule like this:

mylib.dxe: one.o two.o three.o four.o
	dxe2gen -o $@ $^

    That is, the `dxe2gen -o mylib.dxe one.o two.o three.o four.o' will build
the mylib.dxe dynamically-loadable module. All public (e.g. non-static)
symbols will be exported, and you may query the address for any of them with
the dlsym() API function.

    Now suppose you use some functions from the C library, like strcpy(),
strlen() and so on. In this case dxe2gen will list all those symbols and will
tell you that they are `unresolved', because these functions are not present
in any of the object files you have specified on the command line. From now
on you have several ways to go:

-*- You can link the DXE module with the C library by specifying "-lc" on the
    dxe2gen command line. This effectively will add all the unresolved symbols
    to your DXE module, taking them from the C library. Drawbacks:

    * If your program and/or other modules use same functions, this will
    duplicate the code through all of them.

    * Many functions won't work this way because they refer to other symbols
    in other libraries such as "-lgcc", the later usually refers some symbols
    from crt0.o and you can't link with crt0.o (well, you can but you cannot
    launch crt0.o's initialization routine because it will screw many things
    up). For example, you can't link against -lc to get the printf() routine.
    Most file functions and memory allocation functions are a no-no as well.

-*- Other way is to leave the symbols unresolved and resolve them at runtime.
    To tell dxe2gen to not barf at unresolved symbols, tell him `-U'. This
    will build a DXE module with unresolved symbols.

    Before loading such a module you should provide somehow the missing symbols
    to the DXE loader so that it can fix up all the references to those missing
    functions. If it won't succeed, the loader will fail.

    There are several ways to provide symbol references to loader. The first
    one is to provide a explicit pointer to every function you are going to
    export into dynamically-loadable modules. You can do it with a couple of
    handy macros:

	#include <dlfcn.h>

	DXE_EXPORT_TABLE (exported_symbols)
	  DXE_EXPORT (printf)
	  DXE_EXPORT (strcpy)
	  DXE_EXPORT (strcat)
	  ...
	DXE_EXPORT_END

    Now you should pass this export table to the dynamic linker:

	dlregsym (exported_symbols);

    Allright, now the loader knows these symbols so if any loaded module has
    references to them, it knows how to resolve them. You may call dlregsym
    as much as you want, the symbols are accumulating in a internal table
    of the dynamic loader. Also you may unregister symbols with dlunregsym()
    function (although I don't foresee why someone may want it :-)

    Another way to resolve symbols is to make all exported symbols in one
    module global, thus they get added to the global symbol table and when
    a new module with unresolved symbols is loaded, these shared libraries
    are searched for the corresponding exported symbol. This allows you to
    use symbols from one module as regular `extern's in another module.
    A simple example:

	---------- module A
	void something ()
	{
	  ...
	}
	---------- module B
	extern void something ()
	void something_else ()
	{
	  something ();
	}

    When you link module B, you use the -U switch to suppress the warning
    about symbol `something' being unresolved. Now from your program you
    first load the module A:

	dlopen ("moduleA.dxe", RTLD_GLOBAL);

    then load the module B:

	dlopen ("moduleB.dxe", 0);

    Done, the references are resolved. Note that during first dlopen() call
    we specify the RTLD_GLOBAL flag so that all exported symbols becomes
    visible to other modules.

    ----------------

    Also sometimes you may want to build a DXE module from a ready library.
For example, I have build DXE modules of libjpeg.a, libpng.a and libz.a without
recompiling the libraries. For this you should specify a special option to the
linker: --whole-archive (this is GNU ld option, it is just passed to ld by
dxe2gen):

	dxe2gen -o z.dxe -I libz_i.a --whole-archive -U libz.a \
	  -D "Zlib compression library"

	dxe2gen -o png.dxe -I libpng_i.a --whole-archive -U libpng.a \
	  -D "Portable Network Graphics (PNG) Reference Library"

	dxe2gen -o jpeg.dxe -I libjpeg_i.a --whole-archive -U libjpeg.a \
	  -D "The Independent JPEG Group's JPEG software"

    I recommend calling all import libraries "libsomething_i.a", to avoid
confusion.  Of course, if you don't have long filenames it is not always
possible.  Well, in this case it's up to you how to solve the problem :-)
you may call it for example libsom_i.a.

    Don't forget that all programs linked against libsomething_i.a should be
linked against libdl.a as well.

    Import libraries
    ----------------

    Import libraries are a special kind of libraries (.a) which contains
a number of very small wrappers (just long jumps to the actual functions
inside the dynamic module (before you cry that jumps are ineffective I should
tell you that windoze uses a similar mechanism, and nobody cries windoze is
ineffective, heh-heh)) and two functions: load_MODNAME and unload_MODNAME.

    Initially all jumps are directed to load_MODNAME, thus when you call any
function from a dynamic module it initially arrives to load_MODNAME function.
Then the module is loaded, all jumps are directed to the respective locations
inside the dynamic module, and the last call is restarted.

    As you may understand (and I have mentioned above), this approach does not
work with variables. This is one of the drawbacks of this method.

    To build a import library you should use the -I switch on the dxe2gen
command line. The import library can be built from a ready DXE module as well
as `on-the-fly' during DXE generation process. For example, you can do:

	dxe2gen -I libmy.a -o my.dxe one.o two.o three.o

    as well as

	dxe2gen -o my.dxe one.o two.o three.o
	dxe2gen -I libmy.a my.dxe

    Both above sequences are equivalent.


    Contacts
    --------

    The preffered way to contact me is by e-mail. My address is:

    Andrew Zabolotny, <bit@eltech.ru>

    I also run an ICQ client from time to time, you may find me through
    the search facility of icq (don't remember my icq #).
