There are many sections to this document. Read below to find the relevant part.

You need to make sure CVS and ANT are installed?

Things to copy for a new server

webvibe/public_html/Client/

*.structure

demo.html, demo.php

blank.html

webvibe-client.jar

/corejava

public_html/help

/FamilyDef

/GenusDef

/SpeciesDef

/FNA_XML

/bflyxml

/cgi-bin

/images

put the webvibe server in

webvibe/~bin/webvibe-server.jar

Edit demo.php demo.html to point to correct server, index files.

BIBEClient and BIBEServer

swish-x

mysql -

Thesaurus

Network Services

Apache

Tomcat

Soap

Mozilla (to be used with VNC)

Illinois FNA subset.

Wireless Networking on

iPAQ PocketPC

Handspring PalmOS

Vaio - Windows/me

VNC

Server on TeleBotonistServer

Clients

iPAQ PocketPC

Handspring PalmOS

Vaio - Windows/me

News Client

TeleBotanistServer

iPAQ PocketPC

Handspring PalmOS

Vaio - Windows/me

In order to index files on the web you need to use a web spider.

The swishspider requires a number of Perl libraries. These are listed below:

http://search.cpan.org/search?dist=Compress-Zlib

Compress-Zlib-1.16.tar.gz

http://search.cpan.org/search?dist=libnet

libnet-1.0901.tar.gz

http://search.cpan.org/search?mode=module&query=libnet

Bundle-libnet-1.00.tar.gz

Install LWP From http://search.cpan.org/search?dist=LWP

HTML-Parser-3.25.tar.gz

HTML-Tagset-3.03.tar.gz

http://search.cpan.org/search?mode=module&query=tagset

Differences between swish-e and swish- ex

Functional Differences

Programming Differences

(Insert Hong's documentation here)

The executable is available at /home/webvibe/bin/swishe2.0beta3HCrTxtXml/src/swish-ex

Configuration File

Making an index

Index file = /home/webvibe/index/FNAIndexWDictWithHongSwish

swish-ex -f /home/webvibe/index/butterfly.index -w

'"marginal spots"'

# SWISH format 2.0

# Search words: " marginal spots "

# Number of hits: 4

159 /home/webvibe/public_html/bflyxml/8Phoebis_sennae_m.xml

"8Phoebis_sennae_m.xml" 1941

159 /home/webvibe/public_html/bflyxml/7Phoebis_sennae_f.xml

"7Phoebis_sennae_f.xml" 1942

141 /home/webvibe/public_html/bflyxml/21Speyeria_idalia_f.xml

"21Speyeria_idalia_f.xml" 2015

121 /home/webvibe/public_html/bflyxml/22Speyeria_idalia_m.xml

"22Speyeria_idalia_m.xml" 2116

In xml searches, phrases may be searched as in the following example.

/home/webvibe/users/hong/swish-ex/src/swish-e -T /home/webvibe/public_html/FNA_XML/FNA.dtd -w '"<fna><description>flowers solitary"' -f ~webvibe/index/fnaxml.index

Data structure of swish-ex in indexing and searching xml documents:

Here is an example DTD for article

<!Element Article (Title, Author, Abstract)>

<!Element Title #PCDATA>

<!Element Author (FirstName, LastName)>

<!Element FirstName #PCDATA>

<!Element LastName #PCDATA>

<!Element Abstact #PCDATA>

The data structure used for representing this dtd looks like:

Article Hash

Title

0 Author Hash

Abstract LastName

2 11

Description of the data structure:

Every element that has sub-elements is represented as a hash, for example, element Article is a hash with three key-value pairs; element Author is a hash with two key-value pairs. The author hash is linked to Author cell in Article hash because it is a sub-element of Article element.

The keys are called tagnames. Each tagname has a number associated with it. The numbers are prefixed with their parent elements’ numbers. For example, FirstName has a number 10, where 1 is Author’s number in Article Hash and 0 is the order it gets in Author Hash.

There are several glossaries in this directory including the FNA Glossary

pwd /home/webvibe/FNA/Glossary

mysql -Dvibe -uvibeuser -pthe_password_that_the_staff_know
Thought I would give it to you didn't you.

show tables;

last updated by Karen Medina, 1/3/2003

Processing of the text file.

The file was delivered by Anthony R. Brach in a word document.

This was saved as a text file and copied to the Unix directory ~webvibe/FNA/Glossary under the name SpeciesPlanarumGlossary.txt.

The first ";" was converted to a tab with vi

All references to "'" (single quote) were converted to "\'" (slash quote). This is needed for MySQL.

The program ~webvibe/FNA/Glossary/GlossaryDefinition/InsertSpeciesPlantarum.pl was used to same all of the terms and definitions in a table called species_plantarum_def.

There are 972 definitions in the file.

1) Make a directory called "src" where you wish to keep the development code

#mkdir src

2) Change to that directory

# cd src

3) Copy the latest version of the source out of the cvs source code control system. This will create all of the directories with the source code.

# cvs -d /home/webvibe/cvsroot co webvibe

Your output should look like:

----------------------------

----------------------------

4) Change to the webvibe directory

# cd webvibe

5) Run "ant" to compile the source code and build jar files and store them in webvibe/dist. There will be one for server, client and test. Included jar files are in the webvibe/jar directory.

# ant

Your output should look like:

----------------------------

----------------------------

6) To edit a file, move to the appropriate directory such as server

# cd /home/webvibe/src/webvibe/src/webvibe/server

edit the file

When you are confident of the changes they should be added back into cvs with the

# cvs commit -m "Type a reason for this commit and what you changed"

To recompile go back to the directory containing the build.xml file and run ant again.

# ant

7) To add a new file to the source, create the file, and begin it wilt a line defining its package. For example, a server files should have the line

package webvibe.server;

Follow the example of other files in the directory where you wish to create the new file.

Add the new file to cvs

# cvs add <filename>.

start the test servers with "ant run 8016"

To run on another port, copy ~src/webvibe/dist/webvibe-server.jar to where you want it to reside.

From the command line it can be executed with

java -classpath

/usr/java/lib/mysql_comp.jar:/home/webvibe/bin/server/webvibe-server.jar

webvibe.server.Server1 8016 >> $HOME/logs/cron8016.log%

One question is how the xml files are displayed. One option is to use dynamic XSL to generate javascript or something else to display in the window. The line <XSL processor "http:// www.canis.uiuc.edu/~webvibe/choices/choice.xsl"> which is the incorrect syntax, could make the translation from xml to html or php or whatever it is that needs to be displayed in the choice window. Rather than dynamic translation we could also process this ahead of time do that the files to be displayed would already exist. I think this would be transparent to the java code on the client. It would just have a url that is sticks into a window that it generates. That code would eventually need to return a result string to the java calling function. If the type was "radio" this would only be one item (eg. "blue"). If it was a checkbox there might be several choices made by the user (blue or purple). The java would translate this as a logical OR in the query string.

Non-XML option

1) From Client/swing call soap to with the dtd key as the argument

rpc with a return of an array of strings;

2) Servlet returns string array in the form

dtd;word;url1[|url#]eol

word;url[|url#]eol

...

example:

FNA.dtd;ovate;http://soldev.isrl.uiuc.edu/webvibe/cimages/ovate1.gif| http://soldev.isrl.uiuc.edu/webvibe/cimages/ovate2.gif(eol)

obovate; http://soldev.isrl.uiuc.edu/webvibe/cimages/obovate1.gif| http://imageserver.ncbg..edu/webvibe/cimages/obovate2.gif|(eol)

3) Client needs to restructure the strings into a form/window to display images and get user reply.

Window types: radio button, check box, int,....

Each type needs a different window processor.

There is a security issue for certificate to allow access to multiple image servers.

Server constructs the url from the host+file name. Do we need certificate from the image servers to display the *.gif files?

The keys are of the form

BACKGROUND COLOR|BKGROUNDCOLOR 1 3 2 1 0 0

from bflydtd.structure. The key is

1:3:2:1:0:0

OpenArchive type servers provide character states and values.

Nightly harvester collects values for storage in central database using keys+word to have url for images. Server id used to identify url of images.

Use the Brill tagger in webvibe/etc/tagger

Files may be converted one by one with the following commands run from the directory /home/webvibe/etc/tagger/Bin_and_Data

First tokenizing:

less Species/s_Abies_amabilis.html | tokenize > Species_tokenized/s_Abies_amabilis.tokenized

Then tagging:

tagger LEXICON Species_tokenized/s_Abies_amabilis.tokenized BIGRAMS LEXICALRULEFILE CONTEXTUALRULEFILE -i Species_tagged/s_Abies_amabilis.tagged

There is a script. /home/webvibe/etc/tagger/Bin_and_Data/TokenAndTag.sh that will convert all of the html files in /home/webvibe/etc/tagger/Bin_and_Data/Species into tokenized and tagged files in ~Species_Tokenized and ~/Species_Tagged respectively.

-------- TokenAndTag.sh ----------------

# TokenAndTag.sh

# will convert all of the html files in ~/Species into tokenized and tagged

# files in ~Species_Tokenized and ~/Species_Tagged respectively

#

for name in `ls Species/*.html`

do

basefilename=`basename $name .html`

echo $name

#echo $newname

# newname=`basename $name ".xml"`

cat $name | tokenize > Species_tokenized/$basefilename.tokenized

tagger LEXICON Species_tokenized/$basefilename.tokenized BIGRAMS LEXICALRULEFILE CONTEXTUALRULEFILE -i Species_tagged/$basefilename.tagged

done

-------- TokenAndTag.sh ----------------

grep -i leaf/NN" "blade *

grep -i Leaves\</B\>/NN *

For each file in the directory

# Read though the file

if not end of file templine$ = read a line

read until you find "leaf or leaves"

read up to the word blade,

pick the first adj as a shape word or adv adj., ignore size adj (#-# )

Leaf parts

Blade

Margin

Base

Apex

Examples of forms of Leaf description:

Some start with "Leaves" in bold, some "Leaf Blade" in bold.

Leaves apparently attached to rhizome, 5–25 ´ 1–4 mm; blade linear-lanceolate, base cuneate, margins entire, glabrous, apex round to round-acute.

Leaf blades 1-6 × 1-4 cm, base broadly cuneate, rounded, or cordate, apex acute or short-acuminate to obtuse tip.

Leaves monomorphic or weakly subdimorphic, 10--30 cm. Blade 4-pinnate, 3--12 cm, nearly as wide as long, thin to thick. Ultimate segments narrowly lanceolate to deltate, 2--6 mm; midrib obscure or evident abaxially. Sori of mature blades ± discrete to usually subcontinuous, 3--7(--9) per segment; indusia semicircular to usually elongate and connecting several adjacent sori, margins with 6--10 irregular and prominent teeth and/or lobes.

Leaves monomorphic, erect, 1--1.5 dm; bulblets absent. Petiole 1/6--1/4 length of leaf; scales tan, sparse or falling off early. Blade linear-lanceolate, 1-pinnate, gradually tapered to base. Pinnae ± deltate to ovate, slightly overlapping, in 1 plane, 4--8 mm; base truncate, acroscopic auricle well developed; margins denticulate, not spiny; apex rounded, not dentate; microscales linear, lacking projections, dense on both surfaces. Indusia entire to minutely erose-dentate.

Basal leaves persistent, blades rhombic to deltate or reniform in outline, 3-4×-dissected, 1.1-3.4 × 2-3.1 cm, segments linear, base obtuse, margins crenate, apices of segments narrowly rounded.

--------------------------------------------------------------

Start www.biobrowser.org

Select search databases

Search on a word

3-parted or

-divided

deltate

elliptic

narrowly elliptic

narrowly elongate

broadly elongate

elongate

lance-elliptic

lance-ovate

lanceolate

linear

linear- ?

lobed

oblanceolate

obovate

orbiculate

pinnate

reniform

semicircular

cuneate

Standardize the height to be 150.

If the image is a line drawing save it in gif format. If it is a photo use jpg.

Save the new image to a file named: descriptionword.gif or descriptionword.jpg

If there is more than one example for the same descriptive term use numbers after the descriptive term.. eg. ovate1.gif, ovate2.gif.

If the description is multiword, replace spaces with the underscore character eg. narrowly_elongate.gif

Keep a record of where you got the images. Include the name of the Species and the name of the original file.

Repeat for more images.

You need to copy these files to the proper directory. If you are using unix in the ISRL just save them directly to ~webvibe/FNA/newcharacterimages. this is under your home directory.

If you are on a windows machine in the isrl or another machine anywhere you need to use secure ftp.

open sftp

copy files / drag and drop.

If ssh is not installed on your machine you can get it at the url below for free http://uiarchive.uiuc.edu/content/PC/Communications_and_Networking/SSH/

Move the gif files to the soldev.isrl.uiuc.edu server using sftp (or other secure ftp).

Place in the directory ~webvibe/FNA/newcharacterimages. This directory is also called

~webvibe/FNA/nci. You can place the files in either one.

There is a data entry form at http://soldev.isrl.uiuc.edu:8080/glossary/index.jsp

Images of butterfly characteristics should be stored in files names as above, reflecting the characteristic. For example. there is a <butterfly><taxon><charactertaxon><morphology><wingappendages>no or <butterfly><taxon><charactertaxon><morphology><wingappendages>yes in the Butterfly.DTD. Images might be given names like WingAppendageNo.jpg and WingAppendageYes.jpg. These should be stored on the main Biobrowser server in the directory.

//home/webvibe/public_html/ButterflyCharacters/images

The development directory is usr/local/tomcat/webapps/glossary

The location of the image management stuff is:

http://soldev.isrl.uiuc.edu:8080/glossary/

You need to log in with your username, which I believe is heidorn. In order to use the webpage you have to be in the database as an administrator. There is a section on this page where you can add or remove administrators. The pages are not as automated as I'd like at the moment, but I've spent most of my time this week working on the OAI project. SOAP will definitely work very nicely for it. I'm actually rather surprised that it hasn't been implemented already. It seems like a very elegant way to approach it.

Metadata about the images should be stored in the an xml file fitting the following dtd.

This dtd is based on the NISO: Technical Metadata for Digital Still Imageshttp://www.niso.org/committees/committee_au.html

ButterflyImageImageCharacteristic.dtd

<!-- DTD for Butterfly Image Characteristics-->

<!ELEMENT ImageElements (Description,ImageLocation,CopyrightHolder,Source,DateCr

eated,Species?,DerivedFrom?,Contributor)>

<!ELEMENT Description (#PCDATA)>

<!ELEMENT ImageLocation (URL)>

<!ELEMENT CopyrightHolder (#PCDATA)>

<!ELEMENT Source (#PCDATA)>

<!ELEMENT DateCreated (#PCDATA)>

<!ELEMENT Species (#PCDATA)>

<!ELEMENT DerivedFrom (#PCDATA)>

<!ELEMENT Contributor (ContributorID,ContributorName,ContributorDetails)>

<!ELEMENT ContributorID (#PCDATA)>

<!ELEMENT ContributorName (#PCDATA)>

<!ELEMENT ContributorDetails (#PCDATA)>

Example: WingAppendageNo.xml

<?xml version="1.0"?>

<!DOCTYPE ButterflyCharacteristic SYSTEM "http://soldev.isrl.uiuc.edu/~webvibe/B

utterflyCharacters/ButterflyImageCharacteristic.dtd"

>

<ImageElements>

<Description>wing appendage not present</Description>

<ImageLocation>http://soldev.isrl.uiuc.edu/~webvibe/ButterflyCharacters/images/W

ingAppendageNo.jpg</ImageLocation>

<CopyrightHolder>None</CopyrightHolder>

<Source>Illinois Natural History Museum</Source>

<DateCreated>January 9, 2002</DateCreated>

<Species>Papilio polyxenes</Species>

<DerivedFrom> http://www.canis.uiuc.edu/~webvibe/Butterflies/mSWALTL.jpg</Derive

dFrom>

<Contributor>

<ContributorID>pbheidorn</ContributorID>

<ContributorName>P. Bryan Heidorn</ContributorName>

<ContributorDetails>University of Illinois, GSLIS</ContributorDetails>

</ImageElements>

or

Margins:

crispate

Bei Yu

Nov. 12, 2001

1. Rerun Jun’s code "tparse.pl" on the current data set.
2. Unify the file format as "specification-image-map" order.
3. Add links between family, genus and species.

1. Original data location: /home/webvibe/FNA/
2. Compressed Files List:

• Alismatidae.tar.gz
• Arecidae.tar.gz
• Commelinidae.tar.gz
• Hamamelidae.tar.gz
• Magnoliidae.tar.gz
• Pteridophytes.tar.gz
• Zingiberidae.tar.gz

1. Data:

Working Directory: /home/webvibe/public_html/beiyu/data/

It includes 7 decomressed file directories.

All the big T files are in T_files directory.

All the small t files are in t_files.

Because currently the small t files can’t be included into big T files automatically, I made a program named "Tt_merge.pl" to merge big T files and their corresponded small t files together.

2. Task 1: rerun tparse.pl on the current set.

/home/webvibe/beiyu/code/

1. check which big T files don’t have matched small t files.
2. check which small t files don’t have matched big T files.
3. merge big T files and small t files and store as new T files.

Rerun tparse.pl:

Family	Sub-family	Genus	Sub-genus	Species
76	4	258	4	1331

Family	Sub-family	Genus	Sub-genus	Species
72	0	246	4	1196

Family	Sub-family	Genus	Sub-genus	Species
72	0	252	5	1235

This work is based on the old parsed result. The source data are at:

http://soldev.isrl.uiuc.edu/~webvibe/public_html/Family/

http://soldev.isrl.uiuc.edu/~webvibe/public_html/Genus/

In the course of the experiment, the user needs to press JavaScript buttons to indicate if they believe that the match to the target and what is not a match.

This is accomplished through communication between the main Java-based client application and Javascript. Java opens a new window with JavaScript in one frame and the document being evaluated in the other. Javascript communicates back through a javamessage() ?? call telling Java which button was pushed. The main java application needs to compare the actual target name with the user's selection.

Unfortunately, the user can change the document being displayed by following the hyper links in the document. When the user presses the "This is it" button, they may not be looking at the original document. The currently, being displayed document name needs to be passed back to the main java application. Hong suggested inserting JavaScript buttons into the source documents. This would need to be done with a preprocessor if we can not think of better solution.