Documentation for Webvibe Application Version

maintained by Jing Zhang

last updated,11/23/2004


How to compile the server?

How to add collections to Webvibe?

How to do Experiment using current programs?

How to set which part of the xml tag to display in the "Result" panel of the BIBE interface?

How to kill the server on a particular port?

Where are the xml and html files for each eXist collection?

How to modify the configuration of webvibe servlet server?

Other important information?

How to compile the server ?

To check out the new project, you should set the $CVSROOT to "home/webvibe/cvsroot"

$ export CVSROOT=/home/webvibe/cvsroot

Create a directory to store the code

$ mkdir cvs

Then, check out the code with the below command in any of the ISRL machines.

$ cd cvs

$ cvs checkout webvibe_v2

Additional information on running CVS can be found at http://www3.isrl.uiuc.edu/~webvibe/doc/cvsdoc.html

To compile the server,
$ cd webvibe_v2
$ ant

After you compilation, the updated server jar files are under "dist/" directory. To run the server using updated program:
$ cp dist/webvibe-server.jar run/server/
$ cp dist/webvibe-services.jar run/server/webvibe-services.jar

To run the server,
$ cd run
$ cd server
$ ./startserver.sh

How to add collections to Webvibe?

There is a integrated tool to add xml collections to Webvibe, which uses exist as a servlet to index xml files. Actually, when you talk about adding a collection to Webvibe, it means that you need to do the following four tasks:
    1. Index xml files using eXist.
    2. Loading selection list and image selection list to mysql database.
    3. Updating the count information for each selection list and image selection list.
    4. Adding numeric data to mysql database to support numeric search on certain characters.
    5. Add the configuration information in the server configuration file. For this, you could follow the examples in the previous datasets. Here has some more details.

6. Change the structure file for the collection. The lines in this file indicate which elements are lists, imagelist, numberic or free text. These are stored in webvibe/public_html3/structure.

7. Restart the server. This includes the following steps:

(7.1) The server runs under the openkey account so you must be looged in as openkey to control the processes.

$su openkey
(7.2) kill the previous server using command:

The port number that the server runs on is normally 8016. For debugging 8015. To find the process running the port type
    $lsof -i :8016
    the output would be like:

COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
java 11491 openkey 4u IPv4 3799096 TCP *:8016 (LISTEN)


    $kill 11491

The "11491" above is the process id of the previous server. You need to modify this according to the process id in your output.

(7.3) Replace the server configuration file with the one modified in step (5). The configuration file is located at /home/openkey/runserver_v2_www3_eXist1.0/conf/config.xml. If in step (5), you modified this file directly, you can skip this step.

(7.4) Restart the server: there is a cron task which automatically restarts the server every 10 minutes. You can also restart the server with the following command:

   $cd /home/openkey/runserver_v2_www3_eXist1.0/
   $./startServer.sh

 

The webvibe.services.utils.WebvibeManager class does all the things in step 1-4 above in one step. You still need to follow step 5 and step 6 manually.

To add a collection, use:
    $ java -classpath LIBRARY_NEEDED webvibe.services.utils.WebvibeManager -c collectionName xmlFileDirectory dtdFile rootElement characterDir existConfigFilePath  eXistUser eXistPasswd

for example: for the Southeast tree collection the parameters are:
characterDir = /home/openkey/public_html/Plant_Collection/character/xml

To use this tool, the dtdFile and xml files should satisfy some requirements. More specifically, you need to explicitly specify which character is what type. For example, the selection list should be declared with

<!ATTLIST growth_habit
	type (SelectionList) "SelectionList"
>

attribute. The image selection list should be declared with

<!ATTLIST leaf_complexity
	type (ImageSelectionList) "ImageSelectionList"
>

attribute. Note that the spelling for these are case sensetive!! Here is an example dtd file. Also the xml file should be labeled with "type="range"" attribute on numeric characters. Examples include openkey/public_html3/Algae/Algae.dtd and PlantDescription.dtd

To drop a collection from Webvibe, use
$ java -classpath LIBRARY_NEEDED webvibe.services.utils.WebvibeManager -d collectionname existConfigFilePath  eXistUser eXistPasswd

webvibe.services.utils.WebvibeManager also supports properties files to pass in the parameters required.

Sometimes you might want to do the four tasks seperately. Below tells you how.
1. Index xml files using eXist.

webvibe.services.utils.EXISTIndexer can be used to manage eXist collection. You can get the useage information on this by:

$ java -classpath LIBRARY_NEEDED webvibe.services.utils.EXISTIndexer -h

2. Loading selection list and image selection list to mysql database

To load new selection (image selection) list into database, or update the existing ones, use
$ java -classpath LIBRARY_NEEDED webvibe.services.utils.DTD2SelectionList dtdfilepath rootelement characterDir collectionName

when you update the selectin (image selection) list for an existing collection, what this program does is:
    if the previous selection list changes to image selection list, it will delete the coresponding records from the selectionList table and insert new records into selectionImageList table;
    if the previous image selection list changes to selection list, it will delete the coresponding records from the selectionImageList table and insert new records into selectionList table;
    if the URL for an existing image selection list state changes, it will update the URL with the new one. Note, right now one state can only have one URL.

To delete the selection(image selection) list information for a collection, use
$ java -classpath LIBRARY_NEEDED webvibe.services.utils.SelectionListLoader -d collectionName

Note: each time you reload the selection list for a collection, you need to recount the "matches" info by using webvibe.services.utils.EXISTMatchesCounter (see intructions below). Also, if you changed some characters from selection list to image selection list, you will need to re-generate the structure file for the collection so that it will be consistent with the selection list database.

3. To update the number of matches in the eXist collection for selection (image selection) list, use
webvibe.services.utils.EXISTMatchesCounter collectionName existConfigFilePath

4. To add numeric data to mysql database
Follow the instruction on http://www3.isrl.uiuc.edu/~jung/docs/installGuide.html#ADD3.

How to do Experiment using current programs?

(1) Prepare the data:

The experiment will encode the species into some integer to let the subjects identify. Insert this encoding into "experiment" table in mysql database on the server (for example, www3) so that the server can check the subjects' identification is correct.

Field Type Null Note
speciesID int(3) not null The subject will be required to input this when he submit the identification result for server to check.
title varchar(255) not null The file name of the species. For example, "Acer_rubrum.xml"

When a subject ask the server whether his identification is correct, the "specisID" and "title" will be sent to the server and the server will search this table to decide whether it is correct.

(2) Start the client:

Add one parameter "experiment" to the start.sh for linux (or start.bat for windows).

(3) Start the server:

Add two parameters to the startserver.sh, which should be in the flowing form:

$java -CLASSPATH webvibe.server.socket.WebvibeServer PORTNUMBER SERVERCONFIGFILE SERVERCONFIGFILESCHEMAFILE  experiment PATHFOROUTPUTLOGFILE

Below is an example,

$java -CLASSPATH webvibe.server.socket.WebvibeServer 8016 /home/zhang20/webvibe_v2_www3_run/conf/config.xml /home/webvibe/runserver_v2_www3/conf/config.xsd experiment /home/zhang20/webvibe_v2_www3_run/experimentLog.txt

(4) Interpretation of the experiment log file:

All the subjects' behaviors that needs communication with the server will be logged in a single file. Right now there is no logging on the client side. The format of the log is consistent with the client to server protocol at:
http://www3.isrl.uiuc.edu/~jung/docs/developmentGuide.html#PROTOCOL

The only difference is that the new log records the subjectID and speciesID, which is discussed in detail below.
A type: searching
    logged Info:
       A|id|collectionName|queryterms |subjectID= client.subjectID | speciesID=client.speciesID | resultSize | time

B, S and T type:
    logged Info:
       client to server String |"resultSize" | time.

    The "resultSize" is the number of results returned. For "A" type, it is the number of species that matches the query.  For "S" type, it is the number of states that are listed in the selection list. For "T" type, it is the number of thesaurus words that are used to expand the query. For "B" type, it is "null".
    The "time" is the time that the communication between the client and the server occurs.

When a subject checks his/her answer of the identification, is "C" type, which in detail is:
From client to server:
    C| sessionID |client.subjectID |speciesID |  title
From server to client:
    C| sessionID |client.subjectID |speciesID |  title | result| time
       The "result" is one of the following:
          true: the answer is correct.
          Incorrect SpeciesID:  the species ID is not correct.
          Invalid SpeciesID: the speciesID is invalid. In other words, the subject mistyped the speciesID to check the result. In experiment table, there is no such record exists.

How to set which part of the xml tag to display in the "Result" panel of the BIBE interface?

To display the customized tag instead of file name on the document display panel, you need to modify the client side configuration file ("clientconfiguration.xml" file by default) specify the following three paremeters:

   XMLLocation: the location of the original xml files.
   XMLTagToDisplay: the target tag to be displayed.
   BIBEDocService: The BIBE document service URI. It should be "http://www3.isrl.uiuc.edu/telenature/servlet/webvibe.servlet.BIBEDocService" if the server is  not relocated.

The example below display the content of "/Plant_Description/plant_habit_and_life_style/life_span" in the "Title" column.
   <parameter name="database0:XMLLocation" value="http://www.isrl.uiuc.edu/~openkey/Plant_Description/xml/"/>
   <parameter name="database0:XMLTagToDisplay" value="/Plant_Description/plant_habit_and_life_style/life_span"/>
   <parameter name="database0:BIBEDocService" value="http://www3.isrl.uiuc.edu/telenature/servlet/webvibe.servlet.BIBEDocService"/>

If there are no such parameters set for a collection, the display will be the file name, which is the default value.

 

The server for this is a java servlet (installed at /content/web3/users/webvibe/WebvibeServlet/telenature on www3.isrl.uiuc.edu), running under tomcat (installed at /content/web3/users/webvibe/jakarta-tomcat-4.1.29 on www3.isrl.uiuc.edu under "webvibe" account) by default. The class files could be obtained from "dist/webvibe-servlet.jar" in the cvs root and the source files are in "src/webvibe/servlet" directory of the cvs root. You could set up this service as you by copying the jar file to the appropriate place (depending which java servlet server you are using), just as you set up any other servlet. Be sure to change the value of of the "BIBEDocService" parameter in the client configuration file whenever you change the servlet's name or deploying it in a different computer.


How to kill the server on a particular port?

You can use the command lsof or netstat to get the process ID of the server and then kill the server using kill command.

$> lsof -i :port_number

$> kill process-id-of-the-server

or

$> netstat -lpn | grep port_number

$> kill process-id-of-the-server

 Note: The above command can only kill the server when it is running under your account. If you want to kill the public server running under webvibe account, relogin the server as webvibe and then do the above.

To relogin the server:

$> su webvibe

Where are the xml and html files for each eXist collection?

South Eastern Trees (north):

Taxon Files
xml: /content/web3/users/webvibe/public_html/Plant_Description/xml on www3
html: http://www.isrl.uiuc.edu/~openkey/Plant_Description/html/
State Files (for Selection lists)
xml: /home/openkey/public_html/Plant_Collection/state/xml

Flora of North America (fna):
xml: /home/webvibe/FNA_LINKS
html: http://www.isrl.uiuc.edu/~webvibe/FNA_LINKS/

Flora of China (FOC):

xml: /home/webvibe/public_html/FOC/xml/
html: http://flora.huh.harvard.edu:8080/flora/browse.do?flora_id=2&amp;taxon_id=

EcoWatch Butterfly (bfly)

xml: /home/webvibe/bflyxml/
html: http://www3.isrl.uiuc.edu/~jung/bflyxml/display/

Prairie Plant (prairieplant):

xml: /home/openkey/public_html/prairieplant/taxon/xsl/prairieplantxml/
html: http://www.isrl.uiuc.edu/~openkey/prairieplant/taxon/xsl/display_species/html/

Algae Trentepohliales (algaetrentepohliales):

xml: /home/openkey/public_html3/Algae/taxon/xsl/prairieplantxml/
html: http://www.isrl.uiuc.edu/~openkey/prairieplant/taxon/xsl/display_species/html

CIL (CIL):

xml: /home/webvibe/public_html/CILXML/
html: http://www.inquiry.uiuc.edu/bin/unit_update.cgi?command=select&amp;xmlfile=u

How to modify the configuration of webvibe servlet server?

The configuration files for the servlet server are specified in web.xml file. Each time you modify the configuration, such as adding or deleting a collection, you need to restart tomcat server so that it can reload the configurations.
To modify the location of the configuration file, change the value of serverConfigFile variable in web.xml.
To modify the location of the configuration schema file, change the value of serverConfigSchema varible in web.xml.

Other important information?

Here are some misc information about the current webvibe application.
1) Current System status:
The current system is running at port 8016 as socket server. The start shell for it is located at /home/webvibe/runserver_v2_www3_eXist1.0.

2) About eXist:
The current version of eXist used in webvibe is 1.0b. It is installed at "/home/openkey/eXist/eXist-1.0" on www3. The admin password for it is listed in ~home/webvibe/docs. It is used as an embedded model. Refer to http://exist-db.org/deployment.html for more information on how to manage it from exist client command line.
The previous version used (0.9) is backed up at /home/webvibe/eXist/eXist-0.9.1-bak-bak. There is no password for it.
When you want to use a newer version of eXist, remember to check the required library that it will need. This is very important because eXist requires different libraries for different versions, and usually you need to update most of them.

3) eXist LIBRARY_NEEDED
There is a long list of libraries needed to run eXist. Throughout this documentation his list is designated with the variable, LIBRARY_NEEDED. The full definition is
LIBRARY_NEEDED = The value for this variable should contain all the java class (or jar) files that webvibe_v2 will need. More specifically, it should contain all the jar files in the "lib" sub directory in CVS and the jar files for the webvibe server module and service module.  For convenience, you could use the following the value if the content under the directory "/home/webvibe/www3lib_exist1.0" are not changed and webvibe-server.jar and webvibe-services.jar are in current directory:

        webvibe-server.jar:webvibe-services.jar:/home/webvibe/www3lib_exist1.0/ant.jar:/home/webvibe/www3lib_exist1.0/antlr.jar:/home/webvibe/www3lib_exist1.0/commons-logging-1.0.3.jar:/home/webvibe/www3lib_exist1.0/dtdparser121.jar:/home/webvibe/www3lib_exist1.0/excalibur-cli-1.0.jar:/home/webvibe/www3lib_exist1.0/exist.jar:/home/webvibe/www3lib_exist1.0/fastutil-2.52.jar:/home/webvibe/www3lib_exist1.0/axis-1.1.jar:/home/webvibe/www3lib_exist1.0/jaxrpc.jar:/home/webvibe/www3lib_exist1.0/commons-pool.jar:/home/webvibe/www3lib_exist1.0/commons-discovery.jar:/home/webvibe/www3lib_exist1.0/saaj.jar:/home/webvibe/www3lib_exist1.0/jakarta-oro-2.0.6.jar:/home/webvibe/www3lib_exist1.0/jdbc-mysql.jar:/home/webvibe/www3lib_exist1.0/junit.jar:/home/webvibe/www3lib_exist1.0/libreadline-java.jar:/home/webvibe/www3lib_exist1.0/log4j.jar:/home/webvibe/www3lib_exist1.0/resolver-20030708.jar:/home/webvibe/www3lib_exist1.0/rpDBPool_jdk1_4.jar:/home/webvibe/www3lib_exist1.0/xalan-2.5.2.jar:/home/webvibe/www3lib_exist1.0/xerces-2.6.1.jar:/home/webvibe/www3lib_exist1.0/xmldb.jar:/home/webvibe/www3lib_exist1.0/xmlrpc-1.2.jar

4 ) Servlet server and client:
The webvibe servlet server is installed at /content/web3/users/webvibe/WebvibeServlet/telenature. The servlet server is supposed to replace the socket server.
The source code for the  servlet server is  in
$src/webvbie/servlet
The servlet module does not exist before, so all the codes in this package are submitted to the cvs main trunk.

The source code for the servlet client is in "servlet-src" sticky tag. There are two files modified, which are webvibe.application.client.QueryPanel.java and webvibe.application.client.ServerInterface.java. The cvs main trunk are still socket version. To check out the source codes for servlet client, use
$cvs checkout -r servlet-src webvibe_v2/src/webvibe/application/client/QueryPanel.java
$cvs checkout -r servlet-src webvibe_v2/src/webvibe/application/client/ServerInterface.java

To checkout the jar files for servlet client and the servlet server, use
$cvs checkout -r servlet-src webvibe_v2/dist/

5) About the tomcat server on www3.isrl.uiuc.edu:

The tomcat server is installed at /content/web3/users/webvibe/jakarta-tomcat-4.1.29. For documents on tomcat, click here.