Current SFI Swarm task-space description

[cgl]

                Swarm-Hive tasks for V1.0 and beyond

  As mentioned in last weeks Swarm Bulletin (The Weekly Pheromone?),
the recent Beta-survey and our distillation of your comments and 
suggestions has helped us to reorganize our Swarm task-space toward 
the goal of completing and making more accessible what's there now 
in Swarm. In this document, I describe what our current high-priority
task set looks like.

We've come up with four general families to which tasks belong:
The four families roughly correspond to hierarchical levels
in the virtual Swarm computation machinery:

1) The "kernel" or "Virtual Machine" (VM)
2) The core supporting libraries and tools that run on the VM (LIBS)
3) The applications that run on the core libs (APPS)
4) The user community that needs to use and learn about 1-3 
        (and even 4) (UC)

------------------------------------------------------------------

Here is a fuller description of each of these areas:

1. The virtual machine or "kernel" (VM)

   The virtual machine provides an "object processor:" a suite of 
libs which manage the creation and concurrent processing of objects.
The specific libs included in VM are defobj, activity, collections, 
and swarmobject.  There is some cross-cutting to other libraries, 
like tkobjc, space, and simtools; but, most of these are not 
fundamental to the VM.

2. libraries, tools, (LIBS)

These are the first layer of software that runs on the
virtual machine and constitute the building blocks for constructing
applications. They rely on the underlying VM, but package up that
low-level functionality in more convenient ways. Many of these entities 
provide generic capability that must be further specialized to be 
useful in a particular application (such as 2D spaces, etc.) 
These provide the "leap-off" points from which people with more 
specialized needs can start.  Examples of this family are things
like the random-number library, the space library, analysis tools,
etc.

3. Applications and domain-specific Libraries (APPS)

  This family includes the "final-products" that are built using
Swarm (both user applications and "example" apps, like heatbugs
or mousetraps.) It also includes libraries of domain-specific 
tools that provide unique functinality, like the GA and Neural-Net 
libs. A Cellular Automata lib, or a proposed set of tools to 
support "game-theory" (e.g., 2x2 payoff matrix) applications. 

   The problem space that Swarm is intended to address should be
illustrated by a well-distributed suite of demos and applications
implemented in Swarm to show how one can harness the features and
flexibility of Swarm in various specific contexts. These are also 
useful in a pedagogical sense for providing examples of usage of 
the various lower-level libs and tools. 

4. the user community (UC)

   This is a bit of a stepchild category.  We've lumped all tasks 
having to do with the user community into this one.  This includes 
releases and their structure, installation mechanics, Web presence, 
fostering user interaction, user-support, and "higher-level"
documentation and instruction (such as a Swarm Manual.)

---------------------------

Of course, the documentation task extends across all of these four
general categories, and includes providing examples of usage of
code and libs at all levels.

--------------------------------------------------------------------

Having laid out the general structure of our task domain, here is
a description of the tasks we have currently set high-priority on
for Swarm Version 1.0, which we *will* release in January '97.

Again, these tasks are motivated primarily by the need to complete
and make more accessible what is there in Swarm already. There are 
many further capabilities we will be adding, but they will have to wait
until we really tighten up and polish the existing capabilities (more
on these extended capabilities later.)

1) The virtual machine (VM)

  Primary tasks for V1.0: 

        I)   Complete Collections Library and documentation
        II)  Complete full support of Zones and documentation
        III) Complete and fully document Activity and Defobj libs.

  Primary responsibility: Roger Burkhart

The Collections library task is straight forward. Currently Zones
are just a pass through to malloc(), but they are intended to 
be much more sophisticated storage managers. In particular, they
need to be smarter about keeping track of what has been allocated
in them so that they can "drop" everything easily and cleanly, and
so that they can serve as the basis for tools like Browsers that
can help users visualize the relationships between the structures
they have built. Zones are also critical for the capability to save
and reload the state of a Swarm application.

The Activity functionality and the Defobj conventions are critical to Swarm,
but are perhaps the least well understood features of Swarm. The Activity
library implements the machinery for the concurrent execution of objects
and full swarms, and its full generality and potential needs to be much better
documented and illustrated. In particular, much of the functionality for
a fully parallel implementation of Swarm is already in place, but is not
being flexed or illustrated by any of the existing apps. A true parallel
version of Swarm will have to wait until after V1.0, but much of the 
machinery for it is already in place, and it will be one of our highest
priorities after V1.0.

The Defobj functionality is also a fundamental feature of Swarm, but needs 
more to be carefully documented and illustrated with examples.

2) Libraries and Tools  (LIBS)

Primary tasks for V1.0:
                
        I)   Upgrade and document the Random Number Library.
        II)  Create higher-level objects for common Activity 
                library functions.
        III) Complete and document a Graph Library.
        IV)  Extend and document the Space Library.
        V)   Extend and document the Probe mechanism.

Primary responsibility: Glen Ropella

  The current Random Number Library is mostly a place-holder for a 
more complete and robust version. The current lib is peculiar in 
some ways and lacks functionality that should be available in a 
generic simulation package.  Sven Thommesen has volunteered to 
provide much of this upgrade in collaboration with us (Thanks Sven!)

  The Activity Library is a very general and complete set of tools
for managing concurrent object execution. However, because of its
generality and power, it is difficult to understand how to use the 
functionality it provides properly. Further, much of the functionality 
is not documented or illustrated in any of our demonstration applications. 
More documentation and usage examples should help enormously.

  A "virtual control panel" is being designed as an object that will 
package the common schedule controller methods and relevant data 
in a common "convenience" object.

   Also, by V1.0, we expect to have a Graph Library that will provide
for the creation and dynamic maintenance of graphs (nodes and links),
and their visualization.

  We will also have a more complete space library, including a discrete 
space supporting real-valued state, support for real-valued coordinates, 
and 3D spaces.

  Probes are very powerful tools for observing and manipulating the
state of objects, but their full functionality and the philosophy 
underlying them is not currently well documented or illustrated.
Again, more documentation and examples will help.


3) Applications and domain-specific tools

Primary tasks for V1.0:

        I)   Extend, upgrade, and more fully document the set of 
                example applications (such as Heatbugs, Mousetrap, etc.)
        II)  Implement and document the support for "Experiment Swarms"
        III) Illustrate and provide more examples of the full
                functionality of Swarm
        IV)  Initiate a Swarm Manual

Primary responsibility: Chris Langton

  Example applications that illustrate the functionality and flexibility
of Swarm in a wide variety of classes of simulation are, and will 
continue to be, important vehicles for helping people to learn about
Swarm and to get their own applications up and running. We have quite
a few applications that have been built to date, either by us or by
users, that illustrate the wide-spectrum of classes of models that
can be constructed using the Swarm machinery. We will be upgrading many 
of the old applications that have fallen by the wayside as Swarm has 
developed over time. We will also be providing new example applications 
to more fully illustrate existing Swarm functionality. Finally, we will 
be providing source-code and documentation for a number of models built 
by Beta-users who have agreed to allow their models to be used in this 
capacity.

  The ability to do production run "experiments", in which a model is 
invoked repeatedly under a variety of initial conditions and parameter
settings is critical to many users (and to us as well!). The 
capability to do this either in interactive graphic mode or in 
batch mode is already there in Swarm, but needs enhanced support
and docuemntation. In particular, this capability depends on
more complete support for Zones in order to drop old invocations 
completely and cleanly. It also depends on more support for
parameter management, and the logging of experimental setup and
results. In the long run, this capability should be extended to
support for the management of multiple Swarm apps running in 
parallel on a network, but the support for this *within* Swarm
will have to wait until after V1.0. (The "Drone" package from
the University of Michaigan Hive supports this capability now
external to Swarm). We will be providing support within the 
serial version of Swarm for such "batch" runs of experiments,
with examples and documentation.

  Also, in this task area, is the problem of motivating and illustrating
the large-scale architecture of Swarm applications. We currently
have been building Apps on the "ObserverSwarm/ModelSwarm" architecture,
but, although this is a convenient way to package up many simulations,
it is certainly not the only useful model, and we will provide
examples of other ways to design the macro-scopic structure
of a simulation. There is nothing "essential" or fundamental
in our "Observer/Model" packaging, it is merely a convention.

Within this family fall a couple of documentation tasks, namely, a problem
space description document and a structure for the Swarm Manual.  The
problem space description document will more fully explain and illustrate
the problem space Swarm was intended to address and possibly provide a 
rudimentary map from a given problem to it's formulation in Swarm.  
We're calling this the "Vision" doc.  The Swarm Manual will be present 
in skeleton form by V1.0 and we feel that many of the existing
docs will simply fall into place given a more coherent overall 
oraganization scheme.  This will provide the structure and "feel" of 
what will finally become a more extensive Swarm Manual.

4) Releases, Installation, and The User Community (UC)

Primary tasks for V1.0:

        I)   Vastly improve the installation process
        II)  Extended support for User Community interaction
        III) Extended "User-area" links and indexing of areas of
                user interest and expertise.
        IV)  SwarmFest: the first Swarm user meeting.

Primary Responsibility: Everyone!

  The installation process was a nightmare for many people. This
was largely due to our dependence on a number of other software
packages, such as gcc, Tk/Tcl, TklObjc, BLT, and so forth. Most of
the problems arose in the installation and/or upgrade of these
required packages. Once they were installed properly, Swarm
itself seemed to install with little trouble for most people. 

  This is a serious problem, which was the primary contributer to
elevated "frustration" on the part of users. Some aspects of this
problem are out of our direct control, as we rely on other software
groups for the installation process of their packages. However, we
should be able to significantly reduce the severity of the "installation 
blues" by providing more intelligent installation and configure scripts.

  For User community support, we will be continuing to upgrade the 
"User" area on the Swarm web-pages, and soliciting more links and
contributions from users. We will also be adding a search engine
over the Swarm documentation pages, and attempting to provide
a better "indexing" scheme for users to find others with specific
interests and/or expertise in using Swarm.
  
  The SwarmFest is currently scheduled for two days in the Feb. 15th-18th
time-frame. We have a number of submitted contributions and suggestions
for presentations, and we will be providing more details in a general
announcement in the next few days. 

-------------------------------------------------------------------------

  The above was a detailed presentation of what we view as our primary
task responsibilities for V1.0 - a summary of our current and most
immediately pressing tasks. 

   After v1.0, many other pressing tasks will be planned for and tackled.

Among these are:

   - Interfaces to other commonly used packages (GIS, Mathematica, ...)
   - Browsers and Editors for schedules, data structures, and classes
   - A graph toolbox with more sophisticated graph operators
   - Revisitation of the review process for Swarm libraries and code
       contributions
   - More thought to a standard experimental style and Swarm's role as
       an intra- and inter-disciplinary communication tool
   - More flesh on the Swarm Manual skeleton
   - Revisitation of the release structure
   - Compiling and using Swarm as dynamically loaded shared libraries
   - Parallel Swarm

Priorities and scheduling of these tasks will have to await completion
of V1.0, (and we expect much discussion of these issues at the
SwarmFest).

-----------------------------------------------------------------------

  Finally, we have to express a huge debt of gratitude to those who have
borne with us so far and helped us enormously, both through the
swarm-support mailing list and the beta-survey. Thanks also for
your understanding of the things we are trying to accomplish and
of the severly limited resources we have available to us for accomplishing
them. We started up the Swarm project realizing that we could only
do so-much, and that we would have to rely heavily on help and participation
from a large user community. We hope that we have provided enough of a 
foundation for users to be able to develop their own multi-agent models 
much more easily and robustly than they could have done on their own from 
scratch.  We are fundamentally committed to the continuing development 
and maintenance of Swarm, but its ultimate success necessarily depends on 
the active participation and contributions of the user-community. 

  We are extremely appreciative of your contributions and help
so far, and hope that we can continue to work together cooperatively
in the evolution of Swarm.

Thanks!

Chris Langton
  for the SFI Swarm Hive....