Greenstone 2.81 released

David Bainbridge. Thursday, November 13th, 2008

We are pleased to announce that the Windows, GNU/Linux, Mac OS/X and Source distributions of Greenstone v2.81 are now available for download from:

http://www.greenstone.org/download

The main focus has been on multilingual support. Improvements include handling filenames that include non-ASCII characters, accent folding switched on by default for Lucene, and character based segmentation for CJK languages.

This release also features our new installer, which is 100% open source. Previously we had relied on a commercial program for this, which incurred a significant cost in keeping up to date; consequently we decided to develop our own installer, based on the excellent open source installer toolkits already available.

There are many other significant additions in this release, such as the Fedora Librarian Interface (analogous to GLI, but working with a Fedora repository). See the release notes for the complete details. Specific issues fixed in the 2.81 release can be viewed in Greenstone Trac here and here.

This has been a long time coming, thank you for your patience.

As always, please report any problems or bugs to the mailing list.

————–

Thanks to:

John Rose, for help with English GLI help, French translations for GLI and Greenstone.
Maxime Rouast for Greenstone French translations
Celine Guimbertaud for GLI French translations
Yohannes Mulugeta and Abiyot Bayou for Greenstone Amharic translations
Kamal Salih for GLI Arabic translations
Gerhard Riesthuis for Greenstone Dutch translations
Mohan Raj Pradhan for GLI Nepali translations.
Diego Spano for translating the installer’s interface into Spanish.
Xiaofeng Yu for translating the installer’s interface into Mandarin.
Doris Jung for translating the installer’s interface into German.

Greenstone on an iPod wins “best demo” prize

admin. Thursday, July 31st, 2008

A paper by four members of our group entitled Running Greenstone on an iPod won the “best demo” prize at the premier international Digital Libraries conference (JCDL), held in Pittsburgh recently.

We had other successes too. Of ten papers submitted by members of our group, 8 were accepted (well above the odds—the overall success rate was 30%). Here’s a slide the Program Chair presented in the opening ceremony breaking submitted papers down by “continent,” which also included New Zealand, as it has more submissions than Africa, South America, and Australia combined. It was a well-received note of humor, and the audience was quite impressed.

In the DL world, NZ is a continent!

A Fedora Librarian Interface

David Bainbridge. Friday, June 20th, 2008

The ideas encapsulated in the Greenstone Librarian Interface (GLI) are now available for people working with the Fedora Digital Repository system. Currently the software is checked into the SVN repository — we will produce binaries for this as part of the next general Greenstone release. For now, if you’re interested to try it out, follow the instructions for checking out Greenstone (works with either Greenstone 2 or 3). You will also want to install a version of Fedora.

When you check out the Greenstone Librarian Interface part of the SVN install, you also get all the necessary files for the Fedora Librarian Interface . Once you have the code compiled, where you would usually run ‘gli.sh’ (for Mac or Linux) run ‘fli.sh’. For Windows, it is fli.bat

OAI Metadata Analysis Tool updated

Dave Nichols. Thursday, June 5th, 2008

The OAI Visualisation and metadata analysis tool has been upgraded with:

  • lists of potential duplicate values for each element (using approximate string matching, i.e. edit distances)
  • lists of records that are missing particular elements
  • better linking to source item records
  • greatly improved stability

This new alpha 2 version is now running at the same URL:

http://www.nzdl.org/greenstone3/mat

Nightly Snapshot Releases

admin. Wednesday, April 16th, 2008

Starting now, nightly “snapshot” releases of Greenstone3 will be constructed and made available on our snapshots page.

Every night, the most recent revision of the Greenstone3 source code will be automatically checked out, compiled, and packaged up as an easy-to-use installer and put up on our website. These snapshots will be created for Linux and Windows, and soon, Mac. They will be made available in two formats: as an executable Jar, or as a native binary for your operating system. And to make things even easier, there is also a native binary which comes bundled with Java.

As of today, you no longer have to wait for a release to come out before you can take advantage of a new feature of Greenstone3. All you have to do is install the latest snapshot release, and you will have all the features added up to the previous day.

Nightly snapshot releases were made possible by our move to the open source installer Ant Installer. We have been able to construct a release “kit” for each operating system, which automatically creates releases of Greenstone3 using a number of Ant scripts, the Ant Installer software, and a simple executable wrapper program. Since creating a release is now as easy as running a single command, we decided to let the computer create them nightly.

Not surprisingly, the snapshot releases are likely to have a few flaws. These flaws will have two sources: bugs in the release kits, and bugs in the Greenstone code. As far as the release kits are concerned, in time we aim to eliminate all the bugs, so that they produce robust and usable installers. But as for the Greenstone code, we recognise that the most recent revision of the Greenstone code will always have some “bleeding edges”, so we do not expect to ever eliminate all the bugs there. (If you want thoroughly tested releases, go for one of our regular releases, not a snapshot.)

You can help us to perfect the Greenstone3 release kits by downloading and installing a snapshot release, and telling us if you hit any problems. Go on, be brave!

(For those interested, instructions for getting and using the release kits themselves are available at http://wiki.greenstone.org/wiki/index.php/ReleaseKits.)

Also, keep a look out for Greenstone2 snapshot releases and release kits, which are coming soon.

Greenstone Workshops

Dave Nichols. Friday, April 4th, 2008

There will be a 4-day Greenstone Workshop at the University of Malaya (Kuala Lumpur, Malaysia) on 12-15th May 2008. The workshop is hosted by the Faculty of Computer Science and Information Technology and the instructor will be Prof. M.G. Sreekumar – further details are in the workshop brochure (PDF).

Bess Sadler writes that photographs from a Greenstone Workshop in Bulawayo (Zimbabwe) in December 2007 are online.

Waikato visit report from John Rose

admin. Wednesday, March 26th, 2008

I have been a volunteer research associate in the Greenstone team for more than two years, and was very pleased to be able to visit the University of Waikato, at the invitation of Prof. Ian Witten, from 5 to 19 March 2008 (this was also my first visit to New Zealand).

I live in France and have been working, mainly through the internet, to promote the use of Greenstone in developing countries. As a corollary activity, I have also been collaborating with Anna Huang to improve and test the Greenstone language interfaces with emphasis on those needed in developing countries. I had met Ian several times in Paris, and also David Bainbridge, but this visit was my first opportunity to meet the other members of the team.

During my visit I was able to experiment with Greenstone functions which were new to me, discuss problems encountered and future improvements, and consider with the team our strategies for more effectively reaching and involving users in developing countries.

Here are some of the highlights of what was learned and discussed:

Possible problems with Windows XP Home edition

I had followed the instructions for setting up an Apache web server (file library.txt in the Greenstone home directory) under Greenstone 2.80, and found that access to existing collections from the same computer was only possible when the collect sub-directory was shared with all network users (a contradiction since only one user was concerned for client and server).

Similarly, I followed the instructions for installation of the GLI Client and could neither create new collections nor access existing collections.

These two problems were consistent and replicable on my computer for several days, but without explanation they both stopped. I personally feel that there is some interference with the file sharing system under Windows XP Home edition, which mysteriously ended with the many manipulations that were done to understand the problems (there seem to be some internal system user names which may have been involved). Kathy Don is experimenting with Greenstone on this version of Windows. Users who are having similar problems are invited to report them on the Greenstone users list.

The reason for the problem that I was having with the GLI applet was found: the directory where Java SDK was installed was not in the PATH environment variable, which prevented the keytool/jarsigner sequence from functioning. When it was added to PATH, the applet worked fine. I added a warning to this effect in the GLI applet installation instructions.

OAI-PMH

Open Archives Initiative – Protocol for Metadata Handling is a powerful method for open access sharing metadata on the web (see tutorial).

I tested the OAI server under Greenstone 2.80 and it works fine (this is documented only very briefly in the OAI Demo documented example collection, but it’s operation is simple: one needs to have the Web Library – not the Local Library – running and to have previously edited the etc/oai.cfg file according to the instructions found in it.. When this option is active, one or more specified collections serve OAI data to OAI harvesters while the normal web access to these collections continues normally.

I also tested the OAI downloading function as presented in a tutorial on the wiki. This function, potentially very useful for collecting external documents for local Greenstone collections, makes use of the fact that, although OAI-PMH is formally designed only to share metadata, this metadata normally provides information on the location of the original document in the dc.identifier metadata field. But two major constraints were identified:

  • The provision of simple url in this field (as done in the “Rocky” collection at Virginia Tech used in the OAI Demo documented example collection) is not widespread; most OAI repositories provide a handle reference (DSpace) or the url of a webpage containing a link to the original document (EPrints).
  • In the Greenstone version 2.80, the metadata imported under OAI-PMH cannot be edited, justifiable in the sense that they were assigned by the original creator, but inconvenient if documents are to be integrated into a new special collection.

While I was at Waikato, David Brainbridge improved the OAI download facilities to recover the original documents in a all of the above cases, and to convert the metadata to editable form if desired. These improvements will be included in version 2.81 of Greenstone.

Depositor

This undocumented function enables a remote user of a Greenstone web library to submit documents to a collection, and to assign metadata to them, through the web without installing Greenstone or GLI. One need only enable the depositor (by changing “disabled” to “enabled” in the main.cfg file in the etc directory); the Depositor can then be called from a button on the Greenstone home page.

This function should be very useful in creating institutional repositories with Greenstone. It will be documented in version 2.81 (careful: to test it now, you have to assign the user to the “colbuilder” group, even though this has now been replaced by “all-collections-editor” or “personal-collections-editor” for authentication in Greenstone.

Formatting Documents within GLI

If Greenstone users want to manage the formatting of documents in a collection, they are presently obliged to do it outside of GLI (either by reformatting the original document or by creating a formatted html document from the original). Anupama Krishnan has developed a prototype function enabling the user to convert the original document (e.g. in Word or pdf format) to html and subsequently edit it within GLI (for example to define section headings and sub-headings or to improve the style of presentation) before building the collection. This function, to be included in version 2.81, will enable users in many cases to reduce the size of their collections and/or improve the quality of presentation by eliminating the need to present both the original document for display and the html version for searching.

Greenstone3

I was able to install Greenstone3 without any difficulties. It currently performs most of the functions of Greenstone2. The main difference for the basic user is that the formatting language for displaying documents is different, and may appear, at least at first, more complicated than the formatting language of Greenstone2. Dave Nichols is preparing to develop a graphical user interface to facilitate the formatting process, but this will have to await the completion of the basic formatting interface. Given the substantial benefits of Greenstone3 for advanced programmers, and the substantial overhead in maintaining two versions, there is a consensus within the Greenstone team that Greenstone3 should be developed and stabilised as soon as possible to replace Greenstone2.

Updates and documentation

I was able to point out some shortcomings in the latest update (version 2.80):

  • Several of the language interfaces (including Malayalam, Tamil and Telugu) not activated upon installation (the user should add them to the main.cfg file if needed
  • Example collections not updated on Sourceforge (now fixed).

It was agreed that the checklist for issuing new versions should be tightened more closely controlled for future distributions.

In addition we discussed ways to:

Collaboration with users

The Greenstone team is consists overwhelmingly of faculty members who are doing research in the area of digital libraries. Some technical staff (one full time and several part-time, including Ph.D. students) are available to support the research effort, including as appropriate to help incorporate new research results into Greenstone, but resources to ensure support for the international Greenstone community are extremely modest. I participated, in some sense on behalf of the users, in discussions of the Greenstone team on how to improve user support and collaboration within the existing constraints.

The following ideas were expressed:

  • Users as well as developers should be encouraged to use the bug reporting system, which can be used to report interface presentation problems as well as technical problems.
  • The regional and linguistic user communities should be encouraged to participate more actively in helping users in their regions and beyond, while in turn the Greenstone team could work more closely follow and support organised user efforts, especially in the developing countries (already Kathy Don is providing technical support for the southern African network, Anuparma Krishnan for the South Asian network, and Anna Huang for the language interfaces, all with support from myself on the “soft” aspects.
  • The possibility of more closely involving institutions in developing countries in Greenstone research and development activities should be explored. For example, major research thrusts in digitisation of newspapers and in audio-visual collections could perhaps include the development and testing of relevant applications in developing countries.

Workshop Map Started

admin. Thursday, January 24th, 2008

We have just started the Greenstone Workshop Map, which shows the locations of all the Greenstone workshops and tutorials which have been conducted around the world.

Each workshop and tutorial is annotated with the city and country it was held in, the name of the conference it was part of, the date it was held, the university or other establishment which hosted it, and the person or people who conducted it. Just click on a placemark to view its metadata.

The map uses Google Maps, which provides a familiar interface for panning and zooming. We invite you to take a look!

Note: The map is for past events. Future workshops and tutorials appear on the Greenstone calendar, not the map.

OAI Visualisation Tool online

Dave Nichols. Wednesday, January 23rd, 2008

A prototype OAI metadata analysis tool – producing statistics and visualisations of repository metadata – is now online.

Usage notes:

  • when clicking on the links of the left of the visualisation
    you may need to configure your browser to allow popups from nzdl.org
  • producing the analysis report does take some time (e.g. the whole of IDEALS at Illinois takes about 20-25 minutes) – so, initially, we suggest using the max records option to limit the number of items processed

There also is a short feedback survey where you can add comments and suggestions for new features.

Greenstone2.80 Released

admin. Tuesday, December 11th, 2007

The Windows, GNU/Linux, Mac OS/X and Source distributions of Greenstone v2.80
are now available for download from our sourceforge page:
http://sourceforge.net/projects/greenstone or via our download page.

There is a patch to the 2.80 release, which is a newer version of the main.cfg file with additional configuration for several new languages: malayalam, marathi, tamil, telugu, bulgarian, and sinhalese. To install the patch, download the new main.cfg by right clicking this link and select “save link as” (or “save target as”). Then replace the old main.cfg, which is in the ‘etc’ folder of your Greenstone installation.