                                  HISTORY.TXT
                        (for DosLynx v0.26b, May 2004)
                               by Fred C. Macall
 
Introduction

This is for my fifth release of DosLynx.  In this document, I'll try to take
up where I left off in the last history.txt and not rehash the history of the
previous versions.  However, I will provide a cumulative list of unresolved
bugs, issues, and to do(s) at the end.  If you want to know everything else
there is to know about my previous releases of DosLynx, you can get that
history from:
http://members.nccw.net/fmacall/dlx20/history.txt
http://members.nccw.net/fmacall/dlx22/history.txt
http://members.nccw.net/fmacall/dlx24/history.txt
http://members.nccw.net/fmacall/dlx25/history.txt

Two more machines:  Betty and Charlie, have replaced Booker, since it began
suffering from heat exhaustion, on my home network.  Betty is a '386 based
IBM PS/2 Model L40 SX notebook computer, with 4 MB of ram and a 60 MB disk.
These are the minimums for Windows 3.1, which it can run.  However, I run
only DOS v5.02 on it most of the time.  Betty's addition to my home network
closes a gap, that had existed, in my ability to test DosLynx and other
Internet software on a representative of every significant early generation
of Intel's x86 family of processors.  Betty is connected to Charlie by a
LapLink Parallel Cable and joins my home network via PLIP, like Zeke.

Charlie is an eighty Bogo MIPS Pentium based machine with 64 MB of ram and a
5.5 GB disk.  Naturally, it has an Ethernet connection to my home network.
Presently, I am only running DOS on it.  I run Yet Another PC Bridge (YAPCBR)
on it to give Betty network access.  These things and the rest of my home
network machines are all described in:
http://members.nccw.net/fmacall/PLIP112.TXT .
I debug, test, and use DosLynx on all six of my home network machines.

I've partitioned most of Charlie's disk in a single large Fat32 partition,
which is still largely empty.  This has finally given me an opportunity to
verify how the DosLynx "disk view" value (under) reports such a large
quantity of free disk space.  What I am seeing is as good as I had hoped.
(The disk view value is the second number from the right end of the bottom
line of the DosLynx "desktop".)  The disk view value seems to be staying at
2147418112 as long as the large partition has at least that much space free.
That makes this issue a pretty low priority one.

Old Time Heap Corruption

While updating my Web site in December 2003, I tried using some anchors like:
<A HREF="./#doslynx"> to link to anchors in my Web site's "home page".
I wanted to link back from my site's non-home pages to the home page without
mentioning its actual name:  index.htm.  After all, you don't have to give
that when you access my site.  You specify:  http://members.nccw.net/fmacall/
and the server adds index.htm.  (Or, you might even omit that final / and the
server ends up adding /index.htm.)  Later, when your browser first follows a
link specifying index.htm, it is probably unaware that it has already visited
that page.  That results in a refetch that might be avoided.

Well, when I tested my updated html in a local directory on Sailboard,
following the link given above led DosLynx v0.25b to report Heap Corruption
and shut down!  Tests with earlier versions of DosLynx, going back to
version 0.16a, all demonstrated Heap Corruption and either a shut down or a
hang.  Fixing this was my top priority, when I began work on DosLynx v0.26b,
in January 2004.

I started "walking through" the code involved in the troublesome access.
The first bad spot I came to centered on the cp--; statement in HTParse( ),
in HTPARSE.C.  When a URL's "hostname" was null, as in the present case, that
decrement could leave cp pointing to the octet ahead of the allocated memory
in process.  If that octet contained a period, it would get changed to a
null.  Not too likely.  But, a definite concern.  I put a bandaid on that
booboo and did some more testing.  Of course, the Heap Corruption reappeared.

Some more walking led me to urltodos( ), in URLTODOS.CPP.  It didn't take too
much study to see how much damage this guy could do -- given a cp_url
pointing to a null!  To tame urltodos( ), I've changed the 1 in its
declaration of char * cp_newurl to a 6.  And, prefaced its three cp_url++;
statements with if (cp_url[0]) checks.  None of my tests, since, have been
able to reproduce the earlier Heap Corruption.

FTP Client Optimization

Faithful readers of my history.txt tracts, if any, are sure to recall that my
final answer for FTP directory listing issues, given in DosLynx v0.25b, is
slower than we might like.  Recall that these issues seem to stem from an
ambiguity as to whether a given ftp:// . . . URL refers to a file or to a
directory.  Well, I have realized that ambiguity isn't present in one
frequent enough case.  A reference to an FTP server's root directory.

When an ftp:// . . . URL contains neither a path nor a name, it seems safe to
assume that it must refer to the server's root directory.  I've extended
HTFTPLoad( ), in HTFTP.C, to omit its RETR(ieve) command when it finds that
case.  That eliminates the need for a mini session retry ahead of the LIST or
NLST command used to request a directory report.  And, speeds our access for
the case.  I've also adjusted the code near the beginning of
TURLWindow::URLoader( ), in TURLWIND.CPP, that adds or removes a trailing
slash for some URLs.  The effect of my adjustment is to insure that all
*tp:// . . .  URLs contain at least three slashes.  This may help make
HTFTPLoad( )'s new assumption a little more transparent.

Displaying Local Graphics Files

If there is a common theme to much of the work done for DosLynx v0.26b, it is
something to the effect of "providing optimizations for special cases".
The previous section describes the first of these.  Displaying local graphics
files is a special case of displaying graphics files that has been in need of
optimization, as well.

When a graphics file is received via FTP or HTTP, it often makes sense to
give it a local file name and copy it, on the way to displaying it.
And, that's what DosLynx does.  But, does it make much sense to do that for
graphics files that are already local files?  A slow and unnecessary file
copy process on top of the injury inflicted by inherently slow graphics file
displaying performance has been a real insult with older (read:  slower) PCs!
An insult I became determined to remove.

HTLoadFile( ), in HTFILE.C, is used to access local files and files received
via FTP.  To make much sense of HTFILE.C, you'll probably have to recognize
the none of the symbols DECNET, GOT_READ_DIR, NeXT, NOT_IMPLEMENTED, NO_INIT,
NO_UNIX_IO, USE_DIRENT, nor VMS are defined when making DosLynx.  The symbol
NO_GROUPS does get #define(d) in an #ifdef MSDOS clause.  You may want to
add comments to the #endif(s) in your copy, or delete the unexpanded code,
to be clearer about what is relevant for DosLynx.

My Kludge to avoid copying local graphics files to be displayed begins by
changing HTLoadFile( )'s char * localname to an extern or global declared in
HTFILE.H.  I've also changed it from localname to lclname, in an effort to
get help from the compiler in locating occurrence(s) I might otherwise have
missed.  lclname's actual declaration initializes it to zero.  Alongside of
lclname, I've also added a global:  skpcopy, which is also initialized to
zero.  HTLoadFile( ) may then assign the result of an HTLocalName(addr) call
to lclname.  At label open_file, HTLoadFile( ) then tries to
fopen(lclname, . . . ).

The success of that fopen( ) call, defines the local file case!
When that succeeds, HTLoadFile( ) calls HTParseFile( ), in HTFORMAT.C.
Otherwise, a free(lclname) call gets made.  Afterward, I set lclname to zero
again.  So, lclname's logical value indicates or signals that a local file
is in process.

Early in its processing, HTParseFile( ) makes a call that, for graphics
files, leads to a call to HTHandleImage( ), in HTFWRITE.CPP.
HTHandleImage( ), which may have been called from elsewhere, copies lclname's
logical value into skpcopy.  So, skpcopy's logical value indicates or signals
that a local graphics file is in process.  Fairly early in its processing,
HTHandleImage( )'s gut, contained in HTHnSGut( ) since DosLynx v0.22b, may
perform a new if (skpcopy) clause, which includes an fopen(lclname, "rb")
call.  That replaces the fopen(ca_buffer, "wb") call that otherwise gets
made.  Back in HTParseFile( ), I've added an if (!skpcopy) preface to the
HTFileCopy( ) call being optimized out of the process, in the skpcopy case.

Back in HTLoadFile( ), after HTParseFile( )'s return and a free(lclname)
call, lclname and skpcopy get zeroed again.  The DosLynx user no longer needs
to consider shelling out of DosLynx to view a few local graphics files!

Local Filename Suggestions

Speaking of HTFWRITE.CPP's HTHnSGut( ), I've done some more work on the
local filename suggestion it develops for its call to
TFileDialog::TFileDialog( ), in TFILDLG.CPP in Turbo Vision.  Turbo Vision
quit rejecting HTHnSGut( )'s suggestions after the work done, for
DosLynx v0.25b, "to insure that the filename suggestions given . . . to
'TFileDialog( ) will always include at least a trailing period."
However, the suggestions derived from longer filenames usually ended with a
trailing period, having lost all of the given filename's suffix.

To improve on that result, I've given the filename suggestion development
process an additional stage of truncation.  When the given (long) filename
includes a suffix and a body that is longer than 25 octets, the body is
truncated at 25 octets and combined with the given suffix before being given
a final truncation at 30 octets.  That should insure that up to four of the
leading suffix characters appear in the suggestion given.

Navigate|Reload Current

For some time, I've been acknowledging:  "The Navigate|Reload Current menu
entry could still use some work."  I eased into this work by restoring the
reload_page = 1 ; statement, which had been commented-out of
TURLWindow::handleEvent( )'s case cmReload, in TURLWIN4.CPP, since
DosLynx v0.16a or before.  Then I gave my regular test schedule a few weeks
to turn up any repercussion(s).  None appeared.  However, it slowly became
apparent that something was lacking.  There wasn't any mechanism to insure
that an HText replaced by a Reload wouldn't get used again, in a subsequent
load request.  This issue is a side effect of HTML <BASE HREF= . . . > tag
processing that I've introduced.  It would be easy to resolve if the replaced
HText object could immediately be deleted.  But, it may have to be kept until
its use in other view(s) comes to an end.

To resolve this issue, I've added a variable:  reloaded, to struct _HText,
which is defined in GRIDTEXT.H.  HText_new( ), in GRIDTEXT.CPP, initializes
reloaded to zero.  HText_select( ), also in GRIDTEXT.CPP, sets reloaded when
it finds reload_page set.  HText_select( ) is now perfectly positioned to
determine if reloaded or reload_page prevents an HText's reuse.  Its return
value now reflects that evaluation.  HTLoadDocument( ), in HTACCESS.C, has
been simplified by replacing its test of reload_page with a test of its
HText_select( ) call's return value.

This wasn't quite the end.  After a few more weeks it became apparent that
the test of reload_page, in TURLWindow::URLoader( ) in TURLWIND.CPP, wasn't
quite appropriate.  That test kept the Window's Collection of visitlog items
from being updated in the reload_page case.  That might seem to make sense
because no view is being added to the Window's Collection in the reload_page
case.  However, it may keep the reloaded HText's visitlog item from
accurately indicating where it came from.

I've removed TURLWindow::URLoader( )'s test of reload_page, in order to get
a current visitlog item added to the Window's Collection in that case.
That means the replaced visitlog item needs to be removed from the Window's
Collection.  This is complicated by the possibility that 'URLoader( ) might
not complete a requested Reload.  That may occur when 'URLoader( ) ends up
loading ERROR.HTM.

I've solved this problem by having 'URLoader( ) clear reload_page once it has
reached its download and error cases, which provide no immediate
TNSCp_visited updating.  (The recursive 'URLoader( ) call that loads
ERROR.HTM does update TNSCp_visited.  But, we don't want ERROR.HTM to take
the place of the HText that was to be reloaded.)

This brings us back to TURLWindow::handleEvent( )'s case cmReload.  After its
call to 'URLoader( ), it now checks to see if reload_page is still set.
If so, it gets rid of the replaced HText's TNSCp_visited item.  This takes a
few lines of code.  And, it is being done in over half a dozen places.
So, I've introduced (public) void TURLWindow::discvlen(int n), in new module
TURLWI11.CPP, to discard TNSCp_visited's nth (counted from the last) entry.
A prototype for 'discvlen( ) has been added to the definition of class 
TURLWindow, in TURLWIND.H.  Calls to 'discvlen( ) may now be found in
TURLWIN3.CPP, TURLWIN4.CPP, TURLWIN5.CPP, and TURLWIN8.CPP.

Hotlist Updating

For some time, the processing for the Hotlist|Add Current To Hotlist and
Hotlist|Add Link To Hotlist menu entries or commands has been ending by
emitting some lame messages.  The messages warned that DosLynx might have to
be exited before the Hotlist update could be seen!  The messages were warning
about a real issue that was an undesired effect of caching.  As long as the
Hotlist was on view in some window when an update was performed, it remained
cached and ready to be redisplayed from the cache, in its pre-update form.

I've long wanted to resolve this issue, and the previous section's work led
to that finally being easy to do.

I've added a new const unsigned short int cmRLHotList "command" to
GLOBALS.H.  This command corresponds to a new case in
TDosLynx::handleEvent( ), in TDOSLYN7.CPP, that sets reload_page = 1 and
drops into case cmHotList.  case cmHotList displays the Hotlist in a new
window.  So, case cmRLHotList does that too.  By setting the reload_page
global, case cmRLHotList gets a fresh copy of the Hotlist loaded and insures
that any cached copy of the Hotlist gets marked reloaded.  case cmHotList has
been extended to end by clearing reload_page.

To resolve the issue at hand, it only remained to replace the lame messages
at the end of TURLView::addToHotList( ), in TURLVI28.CPP, with a
TProgram::application->putEvent( ) call signalling the cmRLHotList command.
Instead of the lame messages, the user now sees the updated Hotlist,
presented in a new window.  And, any cached copy of the Hotlist gets marked
reloaded, to keep it from being loaded again.  The user may quickly return to
their previously active window by simply closing the Hotlist window.

Humongous Forms

I had always thought of Forms as relatively small sections of HTML documents.
But, since adding support for the HTML Form <SELECT . . . and <OPTION . . .
tags to DosLynx v0.25b, I've had to elaborate this concept.  Just as some
"favorites list" type HTML documents contain extra long lists of anchors,
some Forms contain extra long lists of Options.  For example, see:
http://www.cmubookstore.com/software.asp .  When I encountered this document
after releasing DosLynx v0.25b, it contained 1995 <OPTION . . . tags
distributed among three Forms!  DosLynx ran out of memory and made a
breakout( ) call while trying to format it.

I've made changes in two areas to make the handling of this kind of document
a little less abrupt.  First, I've increased the allocation parameters
provided for the Select Control's Collection of Options.  That is, in
TCtrl::TCtrl( ) in TCTRL2.CPP, I've increased the parameters in the new
TNSCollection( ) call, made for Select Controls, from 15, 15 to 60, 120.
Second, I've doubled the frequency at which FormatHTML( ), in TURLVIEW.CPP,
checks really safe pool memory, once it has seen an HTML <FORM . . . tag in a
document.  This change halves the memory it might have to allocate to TOption
objects, between checks, and averts the breakout( ) call that was being made.

Zooming the Messages Window

Here is another little special case optimization that I found important.
I have grown very accustomed to having the Messages Window displaying the
last few messages reported, as it usually does.  However, the
Window|Zoom In/Out menu entry or command wasn't maintaining that convention.
A Zoom in and back out (or, was it out and in?) tended to leave earlier
messages on display and proved to be somewhat disconcerting.

Since message posting always restores the Message Window's expected position,
I decided the easiest fix for Zooming would be to recognize its occurrence in
TCapture::handleEvent( ), in TCAPTUR3.CPP, and follow its processing up with
some kind of dummy message.  I added the following, to recognize Zooming,
to the beginning of 'handleEvent( ):
    int zooming = ((TE.what & evCommand) && (TE.message.command == cmZoom)) ;

And I tried the following, to do a dummy post, at the end of 'handleEvent( ):
    if (zooming)
        DumbStream << "" ;
Well, that didn't work.  I verified that DumbStream << "x" ; did produce the
desired effect, however.

After studying the Turbo Vision source and the DosLynx Link Map, I hit on the
following happy ending (i.e.:  one that does work), for 'handleEvent( ):
    if (zooming)
        TT->do_sputn("", 0) ;
So much for the idea of treating Turbo Vision as a "black box"!  Without the
Turbo Vision source, I probably would still be trying to resolve this little
issue.

<BASE HREF= . . . Fixes

When I added HTML <BASE HREF= . . . tag processing to DosLynx, in
version 0.20b, my view of this was that its usual use is to provide the
present document's URL as a base for developing absolute URLs from relative
URLs.  Well, that view is a bit narrow, isn't it?  I was first able to
recognize my mistake while browsing the http://www.sysinternals.com site.
Many of the site's documents contain <BASE HREF=http://www.sysinternals.com/>
tags.  Those don't match most of the site's document URLs, which tend to be
things like:  http://www.sysinternals.com/ntw2k/utilities.shtml .  That is
perfectly valid HTML, of course.

Once DosLynx had loaded such a document, it wasn't able to return to the
site's home page.  Because the cache process, embodied in HTLoadAbsolute( )'s
call to HTLoadDocument( ), both in HTACCESS.C, kept finding other, already
loaded, documents seeming to match the one sought.  The trouble was that my
implementation of the HTML <BASE HREF= . . . tag had given the struct
_HTParentAnchor's char * address variable a dual role.  address always had
served to name or identify a document for purposes such as caching.
It couldn't do that faithfully when my <BASE HREF= . . . processing actually
changed it!  That is, any time a <BASE HREF= . . . tag's URL didn't match the
containing document's URL.

To fix this problem, I've added a char * baseadr variable to the definition
of struct _HTParentAnchor, in HTANCHOR.H.  baseadr takes the role I had tried
to overload onto address.  Along with that addition, I changed the name and
function of HTAnchor_setAddress( ) to HTAnchor_setbaseadr( ), in HTANCHOR.H
and HTANCHOR.C.  And, I added HTAnchor_baseadr( ), for reading baseadr.
It is essentially a copy of HTAnchor_address( ).  Next, I made corresponding
adjustments to HTAnchor_delete( ), HTAnchor_findAddress( ), and
HTAnchor_findChildAndLink( ), all in HTANCHOR.C, HTLoadRelative( ) in
HTACCESS.C, and HTML_start_element( )'s case HTML_BASE, in HTML.C.
These changes fixed the problem with sites like http://www.sysinternals.com .
But, I should have seen another problem coming.

One of my original reasons for adding HTML <BASE HREF= . . . tag processing
was to provide for making local copies of HTML documents work just like the
originals they've been copied from.  To accomplish that, a <BASE HREF= . . .
tag is added to the beginning of each local file copy made.  To work its
magic, this kind of <BASE HREF= . . . tag does have to be able to change a
document's struct _HTParentAnchor address identity!

I've put a horrible Kludge into HTML.C to address (sorry) this issue.
In essence, I'm now allowing one <BASE HREF= . . . tag, preceding an HTML
document, to set both address and baseadr.  All other <BASE HREF= . . . tags
are now limited to setting only baseadr, as suggested above.  To implement
this Kludge, I've added a global variable:  inhtml, to HTML.C.
HTML_new( ) clears inhtml.  HTML_start_element( )'s cases HTML_HTML,
HTML_HEAD, and HTML_BODY set inhtml.  Finally, HTML_start_element( )'s case
HTML_BASE observes inhtml as it calls HTAnchor_setbaseadr( ).  I've given
HTAnchor_setbaseadr( ) an additional parameter to tell it whether or not to
set address, in addition to baseadr.  When case HTML_BASE finds inhtml clear
and does call for address to be set, it also sets inhtml.

Metas, More Or Less

When I put the HTML <META . . . tag content displaying stuff into DosLynx,
in version 0.20b, I briefly wondered if anyone would ever take offence from
that.  Well, Michael Bernardi (http://www.dendraii.co.uk/FAQs/dos-apps.html)
recently wrote to suggest an option to hide the META information.  (Mike says
he didn't take offense, really, but thought it was untidy.)  Mike's comments
came as I had been having some flitting thoughts about presenting an
additional sort of meta content.  I realized my old concern was holding me
up.  And, that Mike's suggestion offered the way out.  I would add a 
configuration option and, thereby, be relatively free to introduce new meta
content, only for those who don't choose to hide it.

I had been thinking about changing the [IMAGE] and [ANCHOR] anchor place 
holders to display filenames (but not paths), from the HTML.  Of course,
these only appear when no ALT= . . . attribute value is provided by the HTML.
So, for some document or other, a presentation like:
. . . [IMAGE] [ANCHOR] . . .
might become:
. . . [spacer.gif] [index.html] . . .

For place holders, the new configuration option will determine which style is
used.  Not whether they appear or not.

While working on DosLynx v0.26b, I discovered a third kind of meta
information I decided to also present.  This discovery came while looking
into a document that seemed to be devoid of any Content.  This document
actually contained plenty of Content.  But, DosLynx was losing it as a result
of the way it was handling the document's following sequence:
    <SCRIPT . . . >
        document.write('<SCRIPT . . . ></SCR' + 'IPT>');
    </SCRIPT>

DosLynx found two <SCRIPT . . . > tags in this.  But, only one </SCRIPT> tag.
This kind of technique has been taught in such places as the book:
JavaScript:  The Definitive Guide, Third Edition, by David Flanagan.
O'Reilly & Associates, Inc., June 1998.  See section 12.2.1.2.  As a result
of missing the obscured </SCRIPT> tag, DosLynx found the remainder of the
troublesome document to be contained within a Script.  Then, the HTML_SCRIPT
cases in HTML_put_character( ) and HTML_put_string( ), in HTML.C, operated to
cause all the Script content not nested in additional HTML to be discarded.

I decided on a two fold approach to resolving this issue.  First, I put a
horrible Kludge into the S_tag case in SGML_character( ), in SGML.C.
The essence of this Kludge is to recognize but ignore <SCRIPT . . . > tag(s)
found nested within <SCRIPT . . . > tag blocks.  This measure alone solved
the problem illustrated above.  By preventing the nested <SCRIPT . . . > tag
from being recognized.  This Kludge seems relatively safe in terms of its
potential for exposing un-paired </SCRIPT> tags, because they'll simply be
ignored.  As you must know by now, I won't mind too much if some extra Script
stuff goes unrecognized and gets presented as Content.

Second, that kind of thinking went even farther.  I decided to designate
Script block content that isn't enclosed in HTML comment delimiters nor other
HTML tags as "meta content".  So, its presentation will depend on the new 
configuration option's setting.  Isn't this a strange kind of meta content,
you might wonder?  Well, I've found that seeing this tells me a lot about a
Web site, in terms of what I can expect from it if my browser won't recognize
Scripts.  Or, how much of their Script(s) may have been blindly copied from
one source or another.  The authors that care about all of their site's users
will tend to be careful to enclose their Script(s) content in HTML comment
delimiters, to keep it from being presented as Content.

Most of the work described in this section would be done in HTML.C.  And, as
reported in the history.txt for DosLynx v0.25b, HTML.C has become one of the
largest modules in DosLynx.  So, I decided to start by seeing if I could
squeeze something out of HTML.C, to make room for the incoming code.  What I
found was the several lines long code sequence beginning with:
                        if((present[HTML_
and ending with something like:
                        else    {
                                HTML_put_string(me, " [IMAGE] ");
                        }

This sequence occurred, with minor variations, in seven places in HTML.C.
I've added the following new function for replacing these sequences, to
HTML.C:
    PRIVATE void HTML_put_value(HTStructured * me, int pad, int ivalnr,
        CONST char * cpstr, CONST BOOL * present, CONST char * * value)

So, for a simpler example, the second replaced sequence in
HTML_start_element( )'s case HTML_IMG was replaced with the call:
            HTML_put_value(me, 1, HTML_IMG_ALT, "[IMAGE]", present, value) ;

Making these seven substitutions did reduce the size of DOSLYNX.EXE by
several hundred octets.  The new code for this section's features then
consumed that savings and only a little more.  One other general improvement
for HTML.C consisted of substituting HTML_put_character( ) calls for all
HTML_put_string( ) calls specifying literal strings containing only a single
character.  There were over a dozen of these.

Implementing this section's features started with adding a global showmetas
variable to GLOBALS.H and GLOBALS.CPP.  Next, commented-out code in 
TDOSLY14.CPP, for parsing the former graphicsmode= configuration option, was
reactivated for the new showmetas= configuration option.  All the rest of the
work for showmetas= configuration was done in HTML.C.

showmetas tests were added to the HTML_SCRIPT cases in HTML_put_character( )
and HTML_put_string( ).  These determine whether the un-commented content of
Script blocks gets presented, or not.

Three showmetas tests were added to HTML_start_element( ), for determining
place holder style.  The first of these is in the HTML_INPUT case, in the
code for the Input TYPE=image case.  The second is in the HTML_A case.
This one determines if a new global char * lastfile variable is given the
address of a [filename] string, or nulled.  lastfile is used in the HTML_IMG
(and HTML_IMAGE) case when it detects nesting within an Anchor block.
The third showmetas test provides the HTML_IMG (and HTML_IMAGE) case with
place holder style when nesting within an Anchor block isn't detected.

Finally, a fourth showmetas test, in HTML_start_element( )'s HTML_META case,
determines whether or not to display HTML <META . . . tag content that
doesn't provide an anchor.

Optimization for Intra-Document Maneuvers

When we follow a link to an anchor within the presently loaded document,
there is no need to reload or reformat that document.  All that is really
necessary is to recognize that the link matches the presently loaded
document's URL, up to but not including the link's #fragment part, if any.
Then, locate the target anchor in the presently loaded document.  Set the
selected anchor pointer (TAp_selected) to match.  Scroll the document's
view to display that anchor.  And, add an appropriate visitlog entry to the
present TURLWindow's Collection of visitlog items (TNSCp_visited).

TURLView::LoadChild( ), in TURLVI17.CPP, was providing much of this
optimization.  But, it missed its chance to stay with the presently loaded
document when the link to be followed didn't end with something like
#fragment.  Or, when an anchor named by the #fragment couldn't be located in
the presently loaded document.  (In both of those cases, 'LoadChild( ) should
simply provide a scroll to the top of the presently loaded document.)
Also, TURLView::LoadChild( )'s code, for locating the target anchor,
duplicated code at the end of TURLView::TURLView( ), in TURLVIE2.CPP.
Finally, there was nothing in DosLynx to avoid reformatting the presently
loaded document upon making a return from such an intra-document link.

As a start toward addressing all of these optimization issues, I decided to
define two more new class TURLView member functions.  These both replaced
sections of code from the end of the TURLView's constructor.  Of course, I've
added their prototypes to the definition of class TURLView, in TURLVIEW.H.

The first of these: void TURLView::setTApsx(const char * fragment), in new
module TURLVI39.CPP, provides the function also needed in 'LoadChild( ).
It locates the first anchor matching the given fragment, in the present
TURLView, and sets TAp_selected accordingly.  It leaves TAp_selected
unchanged if the given fragment can't be located.  A caller may leave, set,
or clear TAp_selected, ahead of its call to 'setTApsx( ), to (pre)determine
the result in that case.

The second new function:
void TURLView::restTApsxndy(ccIndex tapsx, int deltay), in new module
TURLVI41.CPP, provides a function needed to perform an optimized return from
an intra-document link.  It sets TAp_selected to point-out the present
TURLView's tapsxth anchor, if any.  And, it scrolls the present TURLView to
line deltay, if possible.

Next, I updated 'LoadChild( ) to replace its duplicated code with a call to
'setTApsx( ).  And, to improve its logic to recognize the optimization
opportunities it had been missing.  'LoadChild( ) clears TAp_selected ahead
of its new 'setTApsx( ) call.  As it turns out, 'LoadChild( ) is able to make
good use of a 'restTApsxndy(0, 0) call for handling the cases it used to
miss.

Finally, I added the completely missing intra-document return optimization to
case cmLoadParent, in TURLWindow::handleEvent( ) in TURLWIN4.CPP.
Here, recognizing that the "parent" document matches the present document is
accomplished by comparing the URLs, less any #fragment parts, contained in
TNSCp_visited's last two visitlog items.  Given a match, a
TURLV->restTApsxndy( ) call replaces the 'URLoader( ) call that otherwise
gets made.

It is just as well that the Navigate|Reload Current work, described six
sections above, was already done at this point.  Because, as soon as
'LoadChild( ) recognized the opportunities for optimization that it had been
missing, I discovered that I had grown accustomed to one of its former
oversights.  I often use local file copies of favorite URLs as Rolodex cards
for revisiting sites.  I load a local file copy of a site's opening page.
And, would then follow its <BASE HREF= . . . > link, on the top line, to get
to a current copy of the page.  Well, of course, that didn't work anymore.
'LoadChild( ) is now finding the page designated by the <BASE HREF= . . . >
link already loaded.  Now, I use the Navigate|Reload Current menu entry or
command, after loading a local file copy, to accomplish the same kind of
revisit.

Post Download Optimization

I have long been dismayed each time I saw DosLynx reformat the present
window's present view after performing a download.  It seemed as though a
perfectly good copy of that view was sitting on the screen while a lot of
time was being wasted.  TURLWindow::URLoader( )'s post download recursive
'URLoader( ) call, to reload the present view, seemed to be completely
unnecessary.  But, when I tried to remove that call, downloads disabled the
view's scroll bars.  What could the matter be?

Before I reached the full understanding of this problem described below, I
considered trying to make all downloads be performed in new windows.
In essence, this would involve anticipating that a download was about to be
performed and then making a call to TDosLynx::OpenURL( ), which would perform
it.  But, that isn't really practical.  Because, decisions that determine how
a given URL finally gets handled are finely distributed over a number of
routines in the WWW software.

Well, once I saw that the present problem was mostly a difficulty with scroll
bars, I was finally able to recognize a key point that I had been missing.
TURLWindow::URLoader( )'s new TURLView( ) call, for constructing a new view,
contained two calls to the standardScrollBar( ) member function of
Turbo Vision's class TWindow.  These calls were disrupting the present view's
scroll bar set up, even when the new view's construction was abandoned after
performing what turned-out to be a download.

I've resolved this problem by moving those standardScrollBar( ) calls into 
TURLView::TURLView( ), in TURLVIE2.CPP.  Just ahead of its FormatHTML( )
call, which gets made only after it has become clear that a download won't
be performed.  This isn't too elegant because class TURLView doesn't inherit
class TWindow the way class TURLWindow does.  However, for this issue, true
elegance stems from being able to avoid those unnecessary 'URLoader( ) calls!

I've compounded my inelegant approach by using an inelegant implementation.
I didn't want to put anything about class TWindow into TURLVIEW.H because
some of the DosLynx modules that #include it seem close to reaching compiler
capacity limit(s).  To enable TURLView::TURLView( )'s calls to
standardScrollBar( ), I've declared a TURLWindow * TURLWin_cur global
variable, in TURLWIND.CPP.  TURLVIE2.CPP now carries a corresponding
extern TURLWindow * TURLWin_cur declaration.  TURLWindow::URLoader( ) sets
TURLWin_cur = this before making its revised new TURLView(TR, 0, 0, . . . )
call.  The first of TURLView::TURLView( )'s two new standardScrollBar( )
calls actually reads:
  if (!hScrollBar)
      hScrollBar = ((TWindow *)
          TURLWin_cur)->standardScrollBar(sbHorizontal | sbHandleKeyboard) ;
vScrollBar and sbVertical replace hScrollBar and sbHorizontal in the second
new call.

With these changes, I've been able to remove TURLWindow::URLoader( )'s post
download recursive URLoader( ) call, together with a discvlen( ) call.
Downloads are no longer saddled with an unnecessary reformat of the present
view.

Home and End Keys

The 7 and 1 keys (the Numeric Keypad's Home and End keys) weren't working,
were they?  They were being recognized in TURLView::handleEvent( ), in
TURLVIE9.CPP, and changed to kbHome and kbEnd, for handling in Turbo Vision.
The same kind of technique is used for the 9 and 3 keys, which get changed
to kbPgUp and kbPgDn.  Those keys are working, of course.  The difference
may be that the Turbo Vision handling for kbHome and kbEnd tends to treat
those keys as requests to move to the beginning or end of a line.  Not the
beginning or end of a document or list.  Meanwhile, the interpretation of
kbPgUp and kbPgDn is unambiguous.

I've resolved this issue by replacing TURLView::handleEvent( )'s cases '7'
and '1'.  My replacements don't pass a key interpretation buck to
Turbo Vision.  Rather, they use a scrollTo( ) call to explicitly perform the
desired scrolling, to the top or bottom of the document on view.
 
Memory Management Residuals

DosLynx still seems to be just a little less reliable than usual when heap
memory is low, doesn't it?  I think this may be because malloc( ) calls from
within a few of the C/C++ library functions don't get intercepted and handled
by the replacement for the C++ new operator provided in NEW.CPP.
Therefore, those functions are subject to unexpected failure when heap memory
is low.  For more background on these things, see the
Memory Management Background section of the history.txt for DosLynx v0.20b.

After one rare, unexpected, and unrepeatable DosLynx breakout, I audited it
for apparently vulnerable functions.  strdup( ) and tempnam( ) are two
library functions I suspect of being vulnerable to failure when heap memory
is low.  I had inadvertently put about half a dozen strdup( ) calls into
HTML.C over the last few releases of DosLynx.  TTempName::TTempName( ), in
TTEMPNAM.CPP, has always contained a tempnam( ) call.  Its unexpected failure
might have caused the rare breakout I had just observed.

So, I've replaced all calls to strdup( ) with StrAllocCopy( ) calls.  In some
places, initialization had to be added for the target pointer variable
involved.  The DosLynx v0.26b link.map shows that strdup( ) is no longer
being linked into DosLynx.  StrAllocCopy( ) is a macro, #define(d) in 
HTSTRING.H, which expands into a call to HTSACopy( ), in HTSTRING.C.

I've added checking and a Kludge to 'TTempNam( ), for dealing with any 
unexpected failure of tempnam( ).  If the tempnam( ) call should fail, a
new char[640] allocation will be attempted, and the tempnam( ) call retried.
The idea here is to get ReleaseSomeMemory( ), in TDOSLY13.CPP, run. That will
only insure that the new string's allocation succeeds, or a breakout( ) call
gets made from ReleaseSomeMemory( ).  So, after the new string has been 
delete(d), the second tempnam( ) call's return will be checked.  If that
check finds a second failure, a third tempnam( ) call will be made.  At this
point, there should be at least 640 octets of heap memory available, and
tempnam( ) should succeed.  Another check and a new breakout( ) call provide
for a completely unexpected failure of that third tempnam( ) call!
 
At this point, my remaining concerns in this area stem from fputs( ),
fwrite( ), and putc( ) calls in CAPSTDIO.C and HTFWRITE.CPP, an fread( ) call
in HTFORMAT.C, un-captured printf( ) calls in TDOSLYN7.CPP, and vfprintf( )
calls in CAPSTDIO.C. 

Another pending memory management issue has been to "prevent file open
dialogs from being disrupted and restricted by the messages view getting
focus to report a document being freed."  For a long time, my thoughts on
this issue kept leading to ideas that can only be described as overkill.

Finally, I got a more reasonable idea.  Why not use the tnsclhrc mechanism
to protect sequences of calls to the Turbo Vision 'drawView( ) and
'execView( ) functions from the possibility that their memory allocation
requests will lead to document(s) being freed?  As explained in the
history.txt for DosLynx v0.20b, the tnsclhrc mechanism keeps
ReleaseSomeMemory( ) from trying to free any document(s), when doing that
might be dangerous.

Using the tnsclhrc mechanism to protect dialogs from being disrupted by
document freeing isn't perfect.  Because it will occasionally lead to using
really safe pool memory while a cached document might be available for
freeing.  However, this solution does insure that file open dialogs will
never again be disrupted by report(s) of document freeing.  And its cost, in
terms of code to be added, is relatively low.

I've identified all of the 'execView( ) calls in DosLynx.  These are located
in HTALERT.CPP, HTFWRITE.CPP, TDOSLYN3.CPP, TDOSLYN4.CPP, TDOSLY15.CPP,
TURLVIE9.CPP, TURLVI22.CPP, TURLVI23.CPP, TURLVI24.CPP, TURLVI28.CPP,
TURLVI30.CPP, TURLVI40.CPP, and TURLWI10.CPP.  Where these calls are preceded
by a 'drawView( ) call, I've bracketed the couple with a tnsclhrc increment
and decrement.  Elsewhere, I've bracketed the 'execView( ) call, alone.

Updated Score

As explained under the heading:  Keeping Score, in the history.txt for
DosLynx v0.20b, I am maintaining an informal written list of DosLynx issues
and their resolutions.  At the time of this writing, for the release of
DosLynx v0.26b (in May 2004), my list has about 328 issues.  Of these, about
227 (over two thirds) have been resolved or ameliorated.  In the next
section, I'll list a number of the 101 pending issues that haven't been
mentioned above.

To Do

Well, no progress was made "toward providing Forms support in DosLynx", in
the present release.  However, the four "Support the HTML . . ." issues
listed below still represent plenty of work to be done in this area.
Right now, I am planning yet another installment of Forms support and other
improvements and fixes before I tackle https/SSL and cookies support.

Here is a summary of some of the pending issues not discussed anywhere above:

  -  Find and remove memory leaks.  (Memory leaks of even small amounts all
     contribute to memory fragmentation that limits the number of documents
     that can be presented before DosLynx becomes unable to maintain large
     Collection objects.)

  -  Support the HTML <FORM METHOD=post . . . variation.

  -  Support the HTML Form <TEXTAREA . . . tag.

  -  Support the HTML Form <BUTTON . . . tag (like <INPUT TYPE=submit . . .).

  -  Support the HTML Form <INPUT TYPE=file . . . variation.

  -  Provide https/SSL support.  (This is expected by many Web servers that
     support data entry.)

  -  Provide cookies support.  (This, too, may be expected for data entry.)

  -  Find a graceful way for the FTP client to deal with directory reports
     that are too large to be memorized.

  -  Find a way for the File|Save Source and Navigate|Download Selection menu
     entries to handle FTP directory reports that are too large to be
     memorized.

  -  Provide for displaying directory entry details, beyond file name, from
     the WAR-FTP Daemon's DOS style directory reports.

  -  Provide for presenting whatever has been received upon the interruption
     of FTP reception of a presentable file/document.

  -  Provide a configurable "download directory".  (Bypass:  use the command
     line /P option (as demonstrated in DOSLYNX.BA_) and set your DOS default
     drive and path to specify the directory you want to download into.)

  -  Provide for sending e-mail items to multiple recipients.

  -  Provide the e-mail date header's time zone offset field in the standard
     e-mail style, given a TZ=EST5EDT style environment variable setting.

  -  Avoid having the reception of a file at midnight disrupted by an
     unexpected time out.
 
  -  Provide a "disk view" number at the bottom of the screen that isn't
     limited to the 2.1 GB maximum size of a DOS FAT16 partition.  So, the
     free space available for a DosLynx temporary directory in a large DOS
     FAT32 partition won't be under-reported.  (Presently, the disk view
     number stays at 2147418112 as long as the large partition has at least
     that much space free.)

  -  The handling of special characters seems to need work.

  -  Add an HTTP-Referrer line to the HTTP GET command?

Let me know what you think about the way the Forms support is shaping up or
about any other issues you notice with DosLynx v0.26b!

fmacall@nccw.net
