greenstone.org greenstone wiki greenstone trac planet greenstone

Archive for the ‘Greenstone2’ Category

xx

Anu’s entry for weeks 11-22 July 2011

ak19. Friday, July 22nd, 2011.
  • Week starting 11 July: Closed ticket 770 to do with multiple pieces of metadata for the same metadata name in GS3. GS3 was previously not consulting the mdoffset field in the index database to work out which of multiple assigned metadata values to display for a particular metadata field. When browsing on that metadata field, it used to display only the first each time, but now displays all values in turn.
  • For the rest of that week and the start of the week thereafter, worked on some items discovered by John Rose and Luigi. They found a bug in the GS2 OAI server that manifested when a GS2 client tried to download docs from it over OAI. The bug had to do with an incorrect URL being generated for the dc.Resource Identifier field. They also requested a minor improvement to the button layout in GLI’s OAI download panel and needed some clarifications on the GS2 OAI server’s behaviour.
  • Continuing on in the week of 18 July: On GLI startup, an information dialog box will show up if the user does not have the PDFBox extension installed (telling them how to get it if they want newer PDF versions processed). A dialog will also appear on startup if the user’s collect home was set to be somewhere outside its default location inside the GS2 installation.
  • In implementing the last, a bug was discovered that had been introduced when implementing the reset-gsdlhome target of the gsicontrol script. The bug interfered with the proper behaviour of setting and loading a custom collecthome when using GLI. It’s now been fixed in such a manner that there’s the added advantage that the intensive operations of the reset-gsdlhome task will not be carried out anymore each time the GS2-server is launched. Instead, the relocation-specific operations are only performed when GSDLHOME has in fact changed since the previous time the GS2-server was launched.
  • The pdfbox-app.jar executable file was changed again: it was returned to being the plain, official 1.5.0 release, without the Greenstone-specific changes regarding the line-separator that had thereafter been committed. Instead, the line.separator is now set as a command-line property when launching the pdfbox-app.jar, as suggested by Dr. Bainbridge, since it was no more than a Java System property that needed to be adjusted for GS’ customisation of PDFBox anyway.
  • Changes have been made to modelcol’s config.cfg (and related changes in runtime-src) to deal with embedded metadata, so that it will now handle the “ex.” prefix of metadata already qualified by a set name, such as ex.dc.something. Further changes were made to runtime-src’s code to not always remove the ex. prefix, since this should be retained for embedded metadata. The handling of embedded metadata by the DSpacePlugin was also slightly modified so that DC metadata in the dublin_core.xml files of DSpace documents get prefixed with “ex.”. This allows these metadata fields to be visible in GLI, while yet being unmodifiable, as they are still extracted (ex) metadata.
  • Tried to reproduce some issues noticed by members of the mailing list.

Blog entry for 19 June - 1st of July

ak19. Tuesday, July 5th, 2011.

Forgot to write entries for the last two weeks

- A lot of time was devoted to ticket 449: after Dr Bainbridge’s initial solution to the problem in javascript, Sam and Veronica spent a lot of their time on it just so that we could get it to do the same in XSLT,  and so at last (yesterday, 4 July) this was finished.

- Sam and Dr Bainbridge noticed the GS2 server’s portnumber would keep incrementing at times if the chosen port was unavailable at that moment. Their ticket specified a way to request preserving the chosen port. So that was implemented some time last week.

- investigated pdf to text on Windows. Ghostscript seems to support ASCII conversion, but Greenstone would need unicode to be preserved. There were Perl solutions as well as open source programs to do this on Windows. For now, PDFBox has been tweaked to use its inbuilt ability to convert PDF to text when this is specified. Also looked into the latest version of AbiWord which Max pointed out as a free and small-sized alternative to MS Office and Open Office for converting docX files.

- the latest updates to acku and areu collections were uploaded

Anu’s entry for the week of Mon 13 Jun 2011

ak19. Friday, June 17th, 2011.

Since Dr Bainbridge was away this week and because I was at an impasse regarding my last ticket for GS3, I had already decided last Friday to consider two requests that seemed feasible and just required a bit of investigation:

1 . Diego noticed that use_sections didn’t work with the PDFBox extensions. So some changes were then made to the PDFBox code to generate the html in a paged fashion and adjust PDFPlugin to handle it. It turned out that changes to the PDFBox code were unnecessary after all, and the latest binary was all that was needed to work with the updated PDFPlugin: the latest PDFBox jar file was inserting page separator elements already and the PDFPlugin merely needed to pick up on that.

2.  Professor Witten had suggested that if someone had Word or Office 2007 installed on their Windows, windows_scripting should be able to convert docx to html for us without requiring Open Office. Last week I had tried out whether this was already possible: but word2html which used native windows scripting cropped the “docx” extension down to “doc” and declared it couldn’t be found. It was not possible to get at the VB source code to modify it, so the next idea was to find some WSH scripts on converting docx (WSH tends to be switched on on Windows by default). There was a WSH script on the web for extracting all the *text* from a docx. It wasn’t quite what was wished for, since formatting would be lost.

Fortunately for us, Veronica recalled the existence of a VBscript for WSH that promised to do just what we needed: docx to html. After she located it for us, all that was required were some modifications to integrate it with gsConvert.pl to get things to work in the default case: where Office/Word 2007 was installed. It worked fine on XP. Then some further changes for error handling needed to be inserted on Word not being installed on a machine. Having got the error output to go to STDERR from the VBscript, it now did pretty well on the Vista where there was no Word either.

It still needs to be tested for how it acts on a Windows which has a version of Word predating the docX format.

3. The idea is to expand the VBS script to have subroutines to handle xlsx and pptx files as well. Some bit of the code for pptx is already working (opening the document), but there may be some differences between opening or saving things in Word and Powerpoint, as the universal Office SaveAs method wasn’t working for me.

4. Temporarily fixed a bug in GS’s classifiers which was noticed on the mailing list and sent the tentative fix to the notifier:

When a user enters non-English characters for a buttonname, perl does not preserve them and so it displays wrong in the browser. The fix required me to assume that the user would have input this in UTF-8, for which I got the perl to work with it now. But need to talk to Dr Bainbridge about whether my assumption was reasonable before commiting the code for all.

There were some questions in the mailing list which I finally got round to answering. There is still Diego’s request for implementing the “allvalues” option in the List classifier to look at, and number 3 above.

Anu’s entry for week of 6 June 2011

ak19. Monday, June 13th, 2011.

Mainly small odds and ends. From making sure that the GS2 OAI server was validating against a new online validator (at which point the resumptionToken functionality was retested), very minor bug fixes such as making sure images in PagedImage collections built with xml item files won’t get reprocessed by ImagePlugin and some other questions on the mailing list. Spent time investigating how to implement use_sections with the PDFBox to PDFPlugin (can try updating the PDFBox code to split on a page at a time) and on Friday was (still unsuccessfully) trying to figure out problems on circumventing hard-coded GS2 {If} format statements in metadata so that things still work with GS3, as in ticket http://trac.greenstone.org/ticket/449.

Anu’s entry for week beginning 30 May 2011

ak19. Friday, June 3rd, 2011.

Fixed a server crashing bug reported on the mailing list (bug was traced to GSDLQueryLex.cpp).Finally worked out a rudimentary way to get an Exact Phrase option in the GS interface for the Web Administrator of our university library, who was faced with this problem. Got GS2 to pass all the remaining OAI validation tests at last. And also got the earliestDatestamp working for GS2’s OAI server as it should (it works out the earliestDatestamp in the manner that GS3 was changed to do it). This means it no longer always returns the unix epoch time of 1970 for all Greenstone OAI repositories, as it used to.

Anu’s entry for week beginning 23 May 2011

ak19. Tuesday, May 31st, 2011.

Fixed a server crashing bug, answered some questions on the mailing list, looked into a GS3 thumbnail issue which ultimately Dr Bainbridge ended up solving, and started on fixing the validation issues with the OAI server for Greenstone 2.

Anu’s entry - week ending Fri 20 May 2011

ak19. Monday, May 23rd, 2011.

Mainly bug fixing.

There were a couple of outstanding unicode bugs that needed fixing (the rest were fixed the week before) such as in MetadataXMLPlugin. There were a few changes that had been made to GS3 which caused it to cease compiling properly which also required fixing. Finally, there was also the ticket where GS3’s List Users didn’t display all the details for a new user. That’s now been fixed too.

25 April - 6 May

ak19. Monday, May 9th, 2011.

Most of the last two weeks were spent on making the final changes to the work Dr Bainbridge had already done to get Greenstone to work again if you have moved the Greenstone installation. It is now the case that if you relocate your GS2 installation that running the Greenstone Server Interface (GSI) will get the server successfully running from its new location, on both Linux and Windows. Underneath, there are differences (because the apache web server for Linux and Mac has its current location fixed into many of its files, which then need to be adjusted upon relocation), but the different operating systems provide the same “reset-gsdlhome” target in gsicontrol.bat and .sh, which is what the GSI calls whenever this application’s launch script is run. The reset-gsdlhome target can also be called from the command-line. The changes that needed to be made for this had the nice side-effect that gsicontrol can be run from any directory.

At the end of last week, we started looking at the GS284 bugs that Diego found.

Share your documents in Facebook or Twitter

Diego Spano. Thursday, April 7th, 2011.

Greenstone has a new macro that lets you share documents in social networks or email systems, using Addtoany tool . The new macro is called _shareme_  that belongs to package Global in document.dm. The macro accepts two parameters: _1_ is the title of the link, and _2_ is the link to share. For _2_, the [srclink] is the default option, but any other metadata can be used too. The only requirement is that the value of that metadata must contain a well-formed URL that begins with “http://”.

If _2_ is left blank, then the link will point to the Greenstone version of the document.

You have to edit your format statement and add something like this:_shareme_([dc.Title],[srclink])

and then you will see “Share+, Facebook, Twitter, Mail , LinkedIn” icons.

_Share_ Icons

There is also a brief version called _sharemesmall_ that requires the same parameters and only shows Share+ icon.

_Sharemesmall_ Icons

The macro code is available with version 2.84. If you are using v.2.83 or earlier you have to edit document.dm file and add the following block:

**** Macro code - Begin ****

package Global

# Social network support
# Defined here in document, as the most likely place this will be used in
# within a document view, however its package is 'Global' because you
# might equally want this in a search or browse list

# _1_ = e.g. title
# _2_ = [srclink] or left empty.  If left empty, then it will share the internal GS document

_sharemescript_ {

<script type="text/javascript">
function fullDomainURL(localURL)
\{
return window.location.protocol+'//'+window.location.host+localURL;
\}
</script>

<script type="text/javascript">
var a2a_config = a2a_config || \{ \};
a2a_config.linkname = "_1_";

_If_(_2_,
var srclink = \'_2_\';

//If metadata value is a valid URL that starts with xxx://
// (e.g. any protocol\, http, https\, ftp ...) then that will be the link to share
if (srclink.match(/^[^:]+:\\\/\\\//i)) \{
a2a_config.linkurl = srclink;
\}
else \{
//if metadata value is [srclink] then we have to cut off the 'href' tag label
var href = srclink.match(/href=\"([^\"]*)\"/);
a2a_config.linkurl = fullDomainURL(href[1]);
\}
,
//if no metadata was passed as link\, then the GS version of the document will be used.
a2a_config.linkurl = fullDomainURL("_gwcgi_")+ "?c=_cgiargc_&a=d&d=_cgiargd_";
)
</script>
}

_shareme_ {

<div style=\'padding-left:50px;\' class=\'a2a_kit a2a_default_style\'>
_sharemescript_(_1_,_2_)
<center>
<a class=\'a2a_dd\' href=\'http://www.addtoany.com/share_save\'>Share</a>
<span class=\"a2a_divider\"></span>
<a class=\'a2a_button_facebook\'></a>
<a class=\'a2a_button_twitter\'></a>
<a class=\'a2a_button_email\'></a>
<a class=\'a2a_button_linkedin\'></a>
</center>
<script type=\"text/javascript\" src=\"http://static.addtoany.com/menu/page.js\"></script>
</div>
}

_sharemesmall_ {

<span style=\'padding-left:8px;\' class=\'a2a_kit a2a_default_style\'>
_sharemescript_(_1_,_2_)
<a class=\'a2a_dd\' href=\'http://www.addtoany.com/share_save\'>Share</a>
<script type=\"text/javascript\" src=\"http://static.addtoany.com/menu/page.js\"></script>
</span>
}

**** Macro code - End ****

Greenstone 2.84 released!

ak19. Friday, April 1st, 2011.

After last week’s bug discovery got fixed at the start of this week (there were issues with HTML files that had non-English filenames interlinking on a Mac OS), we went back to testing the Greenstone binaries on Windows, Linux and Mac. Finally, after uploading all the files onto SourceForge and adjusting the pages there as well as updating Greenstone.org’s own download page, we succeeded in releasing Greenstone 2.84 today!

To grab the Greenstone 2.84 binary for your operating system, visit the download page at Greenstone.org. This page also has the source distributions available in zip and tar.gz formats. Otherwise, you can always expand your binary installation with source code by grabbing the “source-component” archive files from the same download page.

The Greenstone 2.84 Release Notes contain installation instructions as well as details on how to use the latest Greenstone extensions like the PDFBox extension (for later versions of PDF) and OpenOfficeConverter (which can handle the latest Office docx format).

xx