Bug and to-do list for MMiSS Workbench
--------------------------------------

Xml standard == "Extensible Markup Language (XML) 1.0 (Second Edition)"
(available from W3C).

73) 14.05.03
Workbench run remotely comes to a stop inbetween loading HTk images
and asking for user/password.  Is it having problems connecting to the
server?

Unfortunately this has only happened once, and is unreproducible.
Some debug statements in MainMMiSSWorkbench, if it happens often enough
again it should be possible to narrow it down.

72) 04.04.03
Editing objects in a merged view is liable to lead to fetchLink errors.
FIXED

71) 01.04.03
If an object is deleted from a package and we then try to close the package
folder we get a "DeleteAndFindFirst" error.
FIXED

70) 01.04.03
In merging test, new objects are somehow disappearing.
FIXED.  Actually the disappearance was occurring whenever they were checked
in, which isn't exactly good.

69) 01.04.03
Made the XEmacs buffer's modified flag work in a sensible way, also
bind the control-X S to commit and control-X K to quit (like as in
normal XEmacs).
FIXED

68) 01.04.03
Various errors reading back from database.  This turns out to be connected
to the infamous bdb free problem, which I have however now 
FIXED

67) 27.03.03
When an object is inserted manually in XEmacs, it does not get converted into
a magic button when the containing object is saved, meaning that we can't
write to it a second time.
FIXED

66) 27.03.03
Also, EditRef should probably not contain the object link, but instead 
recompute it each time we need to resolve the reference.  
FIXED

65) 27.03.03
Italicisation of objects that indicates editing is lost when the containing
package is closed and opened again.

-- -------------------------------------------------------------------
(Memoirs restarted after a long gap and a number of changes)
-- -------------------------------------------------------------------

64) 4.12.02
I am implementing the folders (after a major change in the object model with
a memory leak which means that elements of the openContents registry in the
type Folders.Folder never get deleted, even when the relevant DisplayedView
is deleted.

63) 8.10.02
There seems to be an erratic bug in daVinci version 2 which causes it not to 
respond to multi(set_context("_C")) commands.  This causes the whole program
to hang in some circumstances but this seems to be not reliably reproducible.

62) 1.10.02
Italicisation added when MMiSS objects are modified.
FIXED.

61) 1.10.02
(ghc5.05 only)
The current version of HaXml has a problem with our DTD in that it expects
a fixed root element, which we obviously can't provide.  I think the solution
may have to be to modify our copy of HaXml to fix this, since Malcolm Wallace
is proving uncooperative.

60) 1.10.02
I have made Emacs keep track of when the contents of an extent are modified.
This means for example that collapsing an unmodified container produces
no warning, and committing only commits the modified containers.
FIXED

59) 27.09.02
See also (5).
We need to improve things when the user does "Close" or "Quit" from the
daVinci menus.

The solution is I think to use the daVinci command 
app_menu(control_file_events) to take control of file menu events from 
daVinci.  This seems to stop the user closing the window manually, or
selecting Close or Quit from the File menu (without Haskell intervening).
Control-X will still bring everything to a halt; I'm not sure if we can or
if it's worth handling this.

----------------------------------------------------------------------
Here follows a list of the outstanding major bug-fixes / improvements which
the existing workbench needs.  This is excluding completely new features
(such as incorporating theorem provers).
----------------------------------------------------------------------

58) 25.09.02
Upgrade in preparation for ghc6.  The major incompatible change is going to
be making all libraries use the new hierarchy.  I am not sure
whether this has been entirely specified yet; for example some of the Posix
stuff we need is not yet in the new hierarchy.

At the same time we should finally fishify everything and switch over to
Haddock as our main documentation tool.

57) 25.09.02
Sort out exactly what is going on with BDB on Linux (see bug 17) and stop it.
The best we can do in the short term (without getting rid of BDB and rolling
our own, or installing some other no doubt equally dodgy system) may just
be to isolate the problem, and post the bug report up the chimney to Santa
Claus, or whoever is responsible for the problem.

FIXED 01.04.03

56) 25.09.02
Write general links manager.  This would allow (a) renaming of objects and
moving them between folders; (b) changing paths for objects; (c) changing
name lists of links from particular objects.  Out of these it would produce
VariableOrderedSet's or something like that which would produce concrete
daVinci changes, so that (for example) these would automatically get updated
whenever (say) a target object moved.

This would be part of uni/types, and manage folder contents as well.

At the same time it would be nice if we could persuade daVinci to generate
edges in the right order.  Unfortunately this seems to be in general not
possible; eg if documents A and B include items 1 and 2 but in different 
orders.

55) 25.09.02
Extend daVinci interface to allow things (like colours and so on) to be
changed more easily.  We need (a) to be able to change them for a particular
node/edge; (b) to be able to change them for all the nodes and edges in a
particular type.  (a) would be useful if we want, say, to change the border
of a node to indicate it is being edited.  (b) if we want to hide all 
folders.

26.09.02
Although we don't (yet) allow colours to be changed, the two changes (a)
and (b) have now been implemented.  There is a minor misfeature in that
hiding edges does not take effect until something is done (for example a 
node being edited) in daVinci2, but this is I think a bug in daVinci2 not
daVinci3.  Therefore I shall for now consider this item
FIXED.

27.09.02
In fact I have discovered (and implemented) a workaround for the daVinci2
bug.

54) 25.09.02
Sort out LaTeX preambles.  At the moment the current hack is to attach
the preamble to every single sub-object.  This is bad because (a) it is
inefficient; (b) it won't work when an object is included by some other 
document with an incompatible preamble (since only the preamble of the
topmost object counts).  In general this is going to be tricky.  Here are
some typical problems:
(a) Documents will typically be composed of atoms with several different
preambles.  We need to merge these preambles somehow.  This requires
possibly name-changing if the name-spaces conflict.  We have to work out
what packages to import; worse we have to work out what order to import them 
in.  (Idea: concatenate all the usepackage lists, then remove all but the
first mention of each package.  If two packages conflict we're sunk, but then
we are anyway.)
(b) If we export to OM-Doc (as the brass hats would like), play around with
the result, and then import back into LaTeX, how on earth are we to preserve
the user's fine-tuned style preferences in the preamble?

At the same time we shouldn't despair here.  I don't think we have to solve the
halting problem if all we have to do is deal with preambles containing
just \usepackage, \(re)newcommand and \def.  And of course
\documentclass[blah blah]{mmiss2}.

----------------------------------------------------------------------
End of list
----------------------------------------------------------------------

53) 25.09.02
It seems to be possible to insert text before a head button in some
circumstances.

This is because we didn't tell Emacs there's anything special about a 
head button.
FIXED

52) 25.09.02
Print from XEmacs now works.
FIXED

51) 24.09.02
The Emacs code used to have a bug in that if the same object was included in
more than one place in the same file (directly or indirectly) it couldn't
distinguish between the corresponding buttons, as the same name was used
for each of them.  

This has been fixed by mangling the names before they get to Emacs.

I have also made a very minor efficiency change, by collapsing EmacsContent
sent to Emacs (so that long sequences of text will be sent all at once.)
FIXED

50) 20.09.02
Improve way buttons are displayed in Emacs.
FIXED

49) 20.09.02
Importing demo.tex caused daVinci to spend forever redrawing each item
one by one.

davinci/DaVinciGraph.hs changed to cluster redrawing operations.
FIXED

48) 20.09.02
daVinci was reporting a Tcl error about extra braces.
It turns out daVinci doesn't like names containing braces.  Changing 
util/UniqueString.hs to avoid braces fixed the problem.
FIXED

47) 19.09.02

Improved tool output display, so the user is only pestered with it if there's
an error.  

Made some very repetitive changes to cope with new finer-grained includes.
FIXED

46) 19.09.02
Fixed bug in Source.hs by which titles of windows (or anything else in
SimpleSource) wasn't getting updated.

45) 19.09.02
Fixed a bug in MMiSSStructureContents which meant some included content
was getting lost.
FIXED

44) 17.09.02
TODO: (1) Export LaTeX and pass it through a filter so it can be displayed on screen.
(2) Add a menu option for the editor which displays exactly what can be seen on the
screen.

18.09.02
(1) FIXED.  There are still some minor problems, like it can't find mmiss2.

43) 17.09.02
Add an option to export LaTeX/XML to a file.
FIXED

42) 17.09.02
Emacs improvements (bug fixes indeed).
(1) Collapsing a container which contains other containers is now banned.
    It used to be allowed, but this would cause the inner container to be
    left locked.
(2) We add Emacs functions for locking/unlocking the buffer making it
    impossible for the user to modify it (at least unless the user hacks
    around the feature) and use them while making modifications.

FIXED

41) 16.09.02
Editing Achim's test file emacs/parser/test_2.tex as XML causes an
Emacs error.

Turns out to be because Emacs Lisp can't handle deep recursion, particularly
in the function uni-scan.  Replacing the neat recursion by a horrible 
iteration seems to fix the problem.
FIXED.

40) 13.09.02
The same-host test accepting calls from Emacs in server/MultiServer.hs
was too strict (because Linux gives "localhost.uni-bremen.de" and not
"localhost").  I have changed it to use the IP address instead.
(There is probably still a lurking security hole if we there's a corrupt
name-server somewhere).
FIXED

39) 13.09.02
Reading in and editing files in LaTeX has now been coded.  The
following errors have been discovered in parser/LaTeXParser.hs
(a) When it finds an error on parsing LaTeX strings it raises
    Fail, rather than returning WithError, crashing the
    repository.
(b) It inserts the preamble even when told not do (given a 
    False argument)
(c) It has a problem parsing \IncludeGroup.
(d) There is a mysterious 
   Fail: No text allowed inside a package!

The good news is that reading a file in LaTeX and inserting it
in Emacs mostly works, even though saving it again doesn't.  So
we have enough for a demonstration. 

38) 12.09.02
The Emacs editing frame is not getting deleted.

13.09.02
Turns out to be simple problem with local variable getting clobbered.
FIXED

37) 12.09.02
Problems with Emacs having difficulty finding the variable uni-allow-changes.

I think this is an Emacs bug.  Stopping uni-allow-changes being a buffer
local variable SEEMS to work around it.

FIXED (hopefully)

36) 12.09.02
Includes in Xml not always converted into magic buttons.

This is because MMiSSEditXml forgot to permit whitespace before "/>".

FIXED

35) 12.09.02
When restoring mmiss-paragraph.1.xml, the link to the included frag2 is not 
shown.

This isn't really a bug.  They can't be displayed because we don't know yet
what variant to show.  Editing and saving the paragraph shows the (current)
links.

FIXED (or rather explained away)

34) 12.09.02
Emacs editing caused problems when control characters were introduced.

This turns out to be the module ParsecToken, which doesn't like control 
characters in its input strings.  Adding EmacsSExp.doControlChars to
filter these seems to have solved the problem.

FIXED

33) 12.09.02
Writing to object a second time complains of variant attributes, even when it
shouldn't.

FIXED.

32) 12.09.02
"Edit Attributes" menu item doesn't seem to work.

Turns out to be typo in changes listed in 26 (accidental circular definition).
FIXED

31) 12.09.02
It's presumably possible to write to an object being edited by importing
another object which specifies it.  Clearly this needs to blocked in
some way.

FIXED

30) 12.09.02
Quitting an MMiSS edit window does not actually tell Emacs to
delete it.
FIXED

29) 12.09.02
The error produced when you attempt to edit a file which is
already being edited wasn't handled very well, as the necessary
Emacs actions (expanding a button or opening an Emacs window) 
were already done.
FIXED

28) 11.09.02
Modifying frag2 inside example mmiss/dtd/mmiss-paragraph.1.xml and then
saving produces a parse error.

12.09.02
FIXED

27) 11.09.02
MMiSSDTDAssumptions needs to be updated to deal with Achim's latest changes to
MMiSS.dtd.
ar
Better wait until MMiSS.dtd has stablised a bit.

Some changes have been made to colours in MMiSS.dtd to make the consistent
with Emacs colours we are about to introduce.

19.09.02.  Probably because of change (47):
FIXED

26) 10.09.02
Rearrange DTD stuff to collect places where we rely on the DTD as far as
possible in MMiSSDTDAssumptions module.

Adapt it to cope with recent changes (several sorts of include and so on).

There is still a bug in this in that Emacs magic buttons don't work.

11.09.02
FIXED

25) 09.09.02
Change object types so that only object types with label are shown in
create-object menu.
FIXED

24) 09.09.02
Links to imported documents aren't shown.
FIXED

23) 06.09.02
Alter mmiss/test/runWorkbench and Makefile.in so that all the users temporary 
files go in the current directory or in /tmp/uni.temp.[USERNAME], thus making
it less likely that different people using the same machine will get into
trouble because their files get in each other way.
FIXED

22) 04.09.02
Incorporate mmiss/parser into the source-file hierarchy, and add an
interface specification for it.
FIXED

21) 04.09.02
On editing file get message
Fail: EmacsBasic: XEmacs error: (error Cannot modify start of a buffer) in command (uni-add-container-buffer "Npara1")

This is the first time this has occurred and it seems to be erratic.
So I'll see if it occurs again (and hope it doesn't).

20) 04.09.02
Collapsing an item in the editor doesn't turn off its read lock.
Solution: put call to finishEdit in.
FIXED

19) 04.09.02
mmiss/test/{Makefile.in,runWorkbench.in} altered so that location of
temporary
 files depends on user.
FIXED

18) 04.09.02
Compiled unoptimised with Linux doesn't start at all but produces
a message saying "Prelude.read: no parse". 

Turns out to be a concurrency bug in the Server code.
FIXED

17) 04.09.02
Compiled optimised with Linux we get a segmentation fault when trying to
display version graph.

Also true when unoptimised, at least after fix for 18 has been applied.
At least that should make it easier to debug.

06.09.02
Investigation reveals that this occurs during bdbclient.c's function
db_retrieve, when it calls the BDB function DB->get.  This then
(perhaps indirectly) calls the BDB function __dbcl_db_get,
in rpc_client/gen_client.c, which in turn calls the XDR function xdr_free
which executes xdr___db_get_reply in rpc_server/db_server_xdr.c.
This calls xdr_bytes on a pointer to the value returned (to free it presumably,
though why this should be necessary I don't know), with the function call
"xdr_bytes(xdrs, (char **)&objp->datadata.datadata_val, (u_int *) &objp->datadata.datadata_len, ~0))".
It is this which falls over; it looks as if it is attempting to free something
that wasn't malloc'd, or has already been free'd.

Temporary workaround: for Linux, set the environment variable MALLOC_CHECK_
when the workbench is running.  In the long term this will have to be reported
to the maintainers of BDB I suppose.

FIXED temporarily.

16) 04.09.02
Totally useless puthash commands in emacs/extents.el corrected (as the
arguments were wrong) and commented out (as puthash should already have
been done).
FIXED

15) 04.09.02
Clicking magic button in dtd/mmiss-paragraph.1.xml (a) fails because it
claims the file is already being edited when it isn't; (b) fails to put button
back when it has failed.
(b) is due to EmacsEdit.openFile not returning a success code and no-one
being prepared to handle its failure; now fixed.

(a) is a silly typo in MMiSSObjects.
FIXED

14) 02.09.02
Intermittently VersionGraph does not start up properly.

03.09.02
After a lengthy investigation it turns out that the reason is that
after ChildProcess.newChildProcess does Posix.forkProcess, we now have
various worker threads doing things like communicating with the server
in both the child and parent process, until the child process manages to
exexve the new process.  This bug has been around ever since Einar's day
at least, but has only emerged now (and partially in item 11) because
I would guess of either the increased complexity (and number of worker threads)
of MMiSSWorkbench, or changes in ghc5.04's scheduler.

This has necessitated a complete rewrite of ChildProcess.newChildProcess's
mechanism for starting a new process, converting it from Haskell into C.
But it's worth it, and the problem seems to be 
FIXED

13) 28.08.02
DTD needs to be modified to allow includes everywhere where atoms and other
insertable items can appear.

Solution agreed 02.09.02.
FIXED

12) 28.08.02
   The button inside dtd/mmiss-paragraph.1.xml is getting inserted into
   the Emacs buffer with the wrong name.  From w.debug:
XEmacs>(MMiSS-init)
XEmacs>(uni-add-container-buffer "Npara1")
XEmacs>(uni-add-button "Npara1" "Hpara1" "para1")
XEmacs>(uni-add-text "Npara1" "<paragraph label=\"para1\"\n  >")
XEmacs>(uni-add-button "Npara1" "Npara1" "para1")
XEmacs>(uni-add-text "Npara1" "</paragraph>")
XEmacs>(uni-bound-container "Npara1")
   so it is wrongly called Npara1, duplicating the name of the container.

04.09.02.
This turns out to be a really simple typo in EmacsEdit.hs.  If only all bugs
were as easy to remove as this.

FIXED

11) 27.08.02
For some reason the workbench has started having trouble
starting daVinci.  If it still happens tomorrow maybe I'll have
to fix it . . 

Output is as follows:

Attempt to start daVinci from path "/home/ger/bin/daVinci" failed
Unexpected response was "Tk: can't re"

28.08.02.  It happened, and is hopefully now fixed.  After extensive
investigation, and help from Mattias, it seems that the Posix.executeFile
function called by ChildProcess and used to start daVinci somehow pangles
unevaluated environment and argument values.  Why this has only been noticed
now I don't know; perhaps something to do with the large size of the 
executable?  
FIXED but see item (14).

10) 27.08.02
Need to alter MMiSSContent/MMiSSObjects to handle both links and references.

9) 27.08.02
Error "Writing textFrag1: Expected an textFragment but found a textFrag1"
Clearly confusing names and xml tags.

Seems to be a bug in MMiSSObjects.simpleWriteToMMiSSObject.
FIXED I hope

8) 27.08.02
Sometimes it seems to hang and nothing is displayed.  w.debug ends
by adding the daVinci rules.  This can usually be "fixed" by trying
again later.  It may only happen when we restart immediately after
quitting.

See item (14).
FIXED

7) 27.08.02
Improved error handling for bad/non-existent. UNIMMISSDTD files.
FIXED

6) 27.08.02
Message "Fail: handleEvents: container NtextFrag1 does not exist"
when quitting after an edit.  Quit does not complete.
FIXED

5) 27.08.02
Choosing "Exit daVinci" from the View menu causes a rather untidy exit.

4) 27.08.02
MMiSSContents.structureContents changed to allow an error to be returned;
MMiSSObjects changed to handle it.
FIXED

3) 26.08.02.  
"Fail: Top-level in Xml does not have a label attribute" when importing
new object.
FIXED see (4).

2)
We assume object names match the NAME production, or that HaXml doesn't notice
if they don't, EG in MMiSSEditXml.hs.  (Compare with sections 3.3.1 and 2.3
of Xml standard).

1)
When editing and we enlarge an object, we always take the variant attributes
of the top object (being edited), not those of any sub-object.  
(7th August 2002)