The GTI (Greenstone Translator Interface) is a Greenstone installation providing a web-based facility to translate the various Greenstone interfaces and websites, including the Greenstone2 and Greenstone3 runtime web interfaces, GLI’s interface, the installer interface, and the greenstone.org website. This facility has been moved to a faster and more reliable server. You can access the new interface at http://gti.greenstone.org. The status page showing the translation status of each module, for each language, is now available at http://gti.greenstone.org/etc/status.html.

The TWSO library is a collection of concert programmes from the Trust Waikato Symphony Orchestra. This has been moved to a new server, so update your bookmarks if you use this collection. It is now available at http://community.nzdl.org/greenstone3/twso/collection/twso/page/about. Note, the http://nzdl.org/twso shortcut now points to the new site. The new server is much faster and more reliable than the previous one.

One of the things we are working on at the moment in Greenstone is 64-bit compatibility (especially on Linux and Mac). Last week while David was away at JCDL I decided to take a deeper look at the issue.

At the most basic level there are two problems. The first is that, in the C and C++ programming languages there exists a data type known as a “long” and it is usually used to store a long integer (a large whole number). On a 32-bit Linux or Mac machine a long is represented using 32 bits (32 ones or zeros), but on a 64-bit Linux or Mac machine a long is represented using 64 bits. This may seem harmless enough, but unfortunately some of our older code has been written assuming that a long is 32 bits and this can stop the code compiling successfully or (in the worst case) cause some fairly hard-to-track-down crashes.

The second problem is fairly similar. Pointers (which are basically big numbers that represent memory addresses) are the same as longs in that they are 32 bits long on a 32-bit machine and 64 bits long on a 64-bit machine. This is only ever an issue when we try and convert a pointer to a number. For example, in some places we try to convert a pointer to an int (32 bit integer on both 32-bit and 64-bit) which doesn’t even compile on 64-bit machines.

There are 3 different ways that we have tried to solve this problem. The first method (the one that we currently use) is to enable an option during compilation that compiles the code as if it were on a 32-bit computer. This method works but has a massive downside that, in order for our code libraries to work together we need to compile most of the rest of the code as if it is 32-bit as well.

The second method is to change all occurrences of long to a data type that is always 32 bits. This effectively makes it exactly the same as the 32-bit version but without the downside of the first method. The downside of making the program run as if it is 32-bit is that you lose the advantages of 64-bit (such as a bigger memory space).

The third method is to only change the areas that cause problems on 64-bit and to leave the rest of code as it is.  This gives us the advantages of 64-bit as well as letting the rest of our code also remain 64-bit. The only downside is the potential to miss areas that are problematic and that might only show up at a later stage.

So last week I spent a lot of time debugging the various errors I encountered trying to implement the third method and for the most part it has been successful.

This week was mostly spent producing test CD versions of 2.84 for Professor Ian Witten. Usually Greenstone CDs are designed so that they can be installed on either Windows, Mac OS, or Linux but due to size constraints (the total size of necessary files was greater than 700MB, which is as much as a standard CD can hold) we were forced to remove that Mac installer as it is unlikely to be necessary in the workshops that Ian will be hosting. Removing the Mac component reduced the size of the ISO to 697MB which was just under the limit. After fixing up a few minor problems with the CD installer it was decided that it would be better if the tutorial sample files were unzipped by default (usually they are compressed in zip format to save space) so that they were more accessible. The problem with this however is that it would have taken us over our 700MB limit.

The documented examples collection (basically a large set of example collections) is usually stored uncompressed on the CD and is copied directly to the Greenstone folder during intallation. It is made up of thousands of small files and we hypothesise that it is one of the biggest contributors to the slow CD installations because of this. By compressing the documented examples into a single file and uncompressing it on installation we can both save space and (potentially) increase the speed of installations. After implementing this (and having the tutorial sample files unzipped) the CD size was reduced to about 650MB which may be enough to reintroduce the Mac installer, we will have to wait till next week to find out.

I have also added authentication pages to the new Greenstone 3 skin (these existed in the old default skin but not in the new skin. Next week I plan to produce a final version of the CD and continue work on Greenstone 3.

This week was mostly spent working on Greenstone 2 for a change.

I upgraded the ExifTool Perl module to the latest version which removed some weirdness we were experiencing with embedded metadata with different character encodings. We use this Perl module to extract embedded metadata in formats such as XMP (Extensible Metadata Platform) and Exif (Exchangeable image file format), as well as metadata from files formats like PDF that have their own embedded metadata formats.

I have also been familiarising myself again with the code we use to generate versions of Greenstone that we can put onto CDs. Professor Ian Witten is heading overseas in the near future and will be running several Greenstone workshops, so he is wanting to a batch of CDs to give out to the participants. Everything appeared to be running smoothly – I was able to install Greenstone off an ISO (basically a file that represents a CD) mounted in a virtual CD drive – but when we then burn that ISO to a real disk the installer does not even start. We get a not-very-helpful error message telling us that there was an error with the error reporting, which certainly doesn’t make for easy debugging.

So next week I will be working on this and probably more Greenstone 3 touch ups.

It was mainly Greenstone 2 bug fixes that week for bugs pointed out by helpful members on the mailing list such as Diego: there was a server.exe crash, a unicode issue with List.pm, and the same plugin also didn’t process the slash that separates metadata fields very well, there was a problem with authentication when using the local library server. All of these are fixed now, and all except for the last require updates of single files, so that they can work fine for Greenstone 2.84 users after that.