Category Archives: Information management

Feeding Zotero

Zotero is where I keep an eclectic library of background to works in progress, works in incubation, and topics of interest. Zotero 2.0, which is still in beta, allows registered users to synchronize citations between computers and a page on the Zotero server itself. The citations are stored as a library with collections in subfolders, with feeds available at both levels, and can be made public. I liked the idea of pulling feeds from my collections into relevant pages, like my Women scientists in fiction page. It’s a work in progress, but figured I’d make notes before I forgot the convolutions and the useful links found along the way.

Currently there is no option to sync or make public just part of a library. That’s coming, but at the moment it is all or nothing. It is possible, using a Zotero-generated API key, to access a feed from an individual collection (folder) within a private library, but (naturally) click-through is not allowed so the full citation is not accessible. The Lifestream plugin gave a certificate error, when I attempted to add the Zotero feed as a generic blog, but the ZoteroFeedWidgit worked handily out of the box, installing a compact  Zotero feed in the sidebar. It’s not there now; in the sidebar was not where I wanted it, but it is an option for those wishing a sidebar feed. There’s an attached CSS file which grants the user some latitude in styling. The recommended syntax for calling the feed is:

http://feedaddress/top?key=APIkey

To get the feed address, bring up the page of the individual collection (ie, folder or subfolder). At the bottom of the page, there is a standard RSS symbol with a link to subscribe to this feed. Right click and copy the link; it will be different for each collection.

An offered workaround was to create a separate public collection with a separate registration. To enable me to use both my larger private collection and smaller public collection simultaneously, I have the first in Firefox and the second in Flock – otherwise I would have to relaunch Firefox to switch collections. The feed from the public collection (no https) works in Lifestream (in fact, it has taken over my Lifestream for the moment, with a great bolus of migrated articles). Still no in-page feeds, which will require use of a plugin that enables the execution of php in a page. So as I said, a work in progress. I might look at a more generic means of mashing together a Delicious feed and a Zotero feed – the first is very useful for links, the second obligatory for properly formed citations.

For more discussion on Zotero feeds and websites, there’s this thread on the forum.

TiddlyWiki: a demonstration

I expressed my enthusiasm for TiddlyWikis in general terms in [cref favorites-tiddlywiki a previous post], since I find them mindbogglingly useful (to borrow a phrase) for keeping running notes on just about any topic. As I set up a TiddlyWiki to capture notes for a course project, it occurred to me that I might contribute a step by step description of how I go about customizing a new Tiddlywiki.

First step is to download an empty TiddlyWiki from http://www.tiddlywiki.com, courtesy of a large, friendly, blue button. That places a file called empty.html on my Desktop, so the next step is to rename it to something appropriate.

Then I open the page in Firefox, remembering to permit Javascript. The default view shows a GettingStarted tiddler front and centre, which invites the user to set the SiteTitle and SiteSubtitle (which should be self-explanatory), and stipulate the tiddlers to appear on the MainMenu and DefaultTiddlers (which are opened when the site is launched). Click on the images to show at full size.

Before I modify any content, I select options in the sidebar, opening up the options submenu. I enter the name I wish to use to sign my edits, uncheck SaveBackups and check AutoSave. The browser will issue a warning if one attempts to navigate away from unsaved changes, but I prefer to have my changes saved automatically rather than risk a distracted click on the wrong button. SaveBackups tends to generate a confusing bunch of files, although there is the option to put these in a separate subdirectory from the main file, which might be useful, and is accessed through AdvancedOptions. Changes to options are saved in the form of a cookie, so will reset to default when cookies are deleted.

Back under GettingStarted, I click SiteTitle to open that tiddler, click edit (beside the title) and enter “Demo Tiddlywiki”. A similar edit to SiteSubtitle changes that to “making tiddlywiki show its stuff”. The other two I leave, since I have some more customization to do.

Modifying the appearance: adding a template

The style is a default, and I have a couple of styles I prefer, so in a separate tab in Firefox I browse over to to TiddlyThemes and call up the Haemoglobin template. Built in to default TiddlyWiki is the ImportTiddlers plugin, accessed either through the list of Shadowed Tiddlers under the More tab on the right sidebar, or under the Import menu in the backstage toolbar. The backstage toolbar is accessed by clicking on backstage, which either appears in the extreme upper right hand corner of the TiddlyWiki page, or which can be made to appear by moving the mouse over that spot. Enter the URL for the desired template in the box provided, select the desired tiddlers – in this case HaemoSideBarOptions, PageLayout, contentFooter, and StyleSheet.

If ImportTiddlers does not work, I cut and past the contents of these Tiddlers from source into my new TiddlyWiki. It may be important to do so in a certain order: for the Haemoglobin style, the side bar should be added before the page layout, or the link to create new tiddlers disappears. The new page layout places the contents of the MainMenu in the coloured strip across the top.

Adding functionality through plugins: addition of the tagCloud plugin

I find a tag cloud a compact and effective index, and use the tagCloud plugin, available from TiddlyTools, to generate a tag cloud. Using ImportTiddlers, or manually, I load or copy the plugin over to my new tiddlywiki. Manual loading involves copying and pasting the code from TiddlyTools into a new tiddler in my tiddlywiki named (exactly) tagCloud and tagged as (exactly) systemConfig. The tag systemConfig is required for Tiddlywiki to recognize the plugin.

I then create an tiddler for the tag cloud itself called Topics (Tag cloud) (or some variant of) and containing the call to the tagCloud plugin, <<tagCloud systemConfig>>. The error that appears disappears on page reload.

Then I add the tag cloud tiddler to the DefaultTiddlers and MainMenu, using [[Topics (Tag cloud)]], the double square bracket being needed to that the Tiddlywiki code will recognize the two words as a cross reference. Now the tag cloud appears front and centre on loading, and a link to it appears in the main menu, which in the restyled Tiddlywiki runs right to left.

Creating new content

To illustrate the creation of a tiddler, here is the open edit box for one called What’s this?, showing the markup required to create an external link, in this case to Wikipedia. Making a new entry involves clicking new tiddler, on the upper right sidebar and, presented with the editing panel, entering a unique entry title, the text and the tags to go with the text. In the body of the entry, anything that in the form of WikiWords or enclosed in [[double square brackets]] is treated as a link to a current or potential entry (current entries are bold, nonexistant entries are italic). The escape character, to avoid unwanted link creation, is ~, eg, ~TiddlyWiki will not create a link. In the tag box, any single word or phrase in double square brackets, is used as a tag. More information about Tiddlywiki markup, including links and images, is found at TiddlyWiki for the rest of us.

The styled Tiddlywiki, with default tiddlers What’s this? and Topics (Tag cloud), is here:

In recent versions of TiddlyWiki, the styling is controlled entirely from the StyleSheet, which is a great improvement on having it scattered over several tiddlers. The stylesheet itself is CSS, and can be edited for further customization.

Curating

Keeping a tidy TiddlyWiki requires some decisions about preferred terminology, best made prospectively or at least early, and attention to detail. It is case sensitive, so if the title of a tiddler starts with a capital, then all references to that tiddler need to start with a capital. Should nouns in tags be singular, plural, or mixed, eg, article(s)? Tag-variants and non-existant tiddlers tend to proliferate. A list of all tiddlers appears by date of last edit under Timeline on the lower right hand menu and alphabetically under All. Nonexistent tiddlers, ie, those referenced but not defined, appear under More > Missing; this list can be used to hunt down tiddlers spawned by a mis-spelling. Previously used tags appear under the Tags tab, or in the tag cloud. The tag cloud updates when the wiki is reloaded or the tiddler is closed and reopened. Size of the font indicates the frequency of use. Click on a tag, and a drop-down list shows which tiddlers it applies to; this, too, can be used to hunt down tag variants and bring them in line.

I keep a collection of definitions, reviews and useful sites on my Tiddlywiki Backpack page. For some elegant implementations, see: see the Tiddlyspot gallery, and Tiddlywiki in Action.

A few of my favourite tools: Tiddlywiki

This started as part of my earlier post on DokuWiki and MAMP, and then grew to demand a separate entry. Tiddlywiki, a demonstration of über-Javascripting, goes places other wikis cannot. It is a completely self-contained, platform- and server-independent wiki, capable of being run from any rewritable media and on any system that supports a modern browser. Everything, script, entries and plug-ins, is included within a single .html file; installation consists of downloading the empty file, setting a couple of options (name and saving), and starting to edit.

Why? Because I as I said before, I like wikis, I work cross-platform, and I find I work best by separating out my projects. So, one Tiddlywiki per project. To date I have run Tiddlywikis on all three major operating systems, though using Firefox by preference as a browser.

A series of articles by Mark Gibbs at Network World (Jan – Feb 2009) covers the basics. Some of his readers express reservations about the quality of the documentation, but it occurs to me that that in part may be because much of the documentation appears in Tiddlywiki form (see, for example, tiddlywiki.com). A potential user coming to it without previous experience may need a few moments to decipher the conventions of the interface (in a traditional tiddlywiki, look to the right hand menu for actions and lists of previous entries), as well as come to terms with the jargon (although, in 2009, “tiddler” cannot seem any more whimsical than “tweet”).  Tiddlywiki for the rest of us provides a more conventional presentation of the basics, and tiddlywiki.org‘s documentation is built on a MediaWiki engine. Admittedly there are aspects, like the styling, that are intricate beyond simple documentation. I have reached the point that I can tinker with other peoples’ styles, but have not progressed beyond that; fortunately, there are a couple of styles I quite like.

My uses are admittedly rather bland: keeping ongoing research notes on emerging treatments in a disease area, keeping bibliographies, keeping a off-line, but still portable list of resources for my medical/regulatory writing activity, keeping a concordance for a loose series of short stories. Other people are much more adventurous, using tiddlywikis for home, education and commercial websites, presentations, weblogs, course notes, course materials, bibliographies, and developing and presenting scholarly papers – see the Tiddlyspot gallery, and Tiddlywiki in Action. As tiddlywiki is open source, developers have produced a number of variants, including a popular one one that incorporates the tasks-projects-place structure of Dave Allen’s Getting Things Done (GTD) strategy, complete with checkable to-do lists; I’ve flirted with this, but have always returned to Backpack.

My own collection of Tiddlywiki Resources page is here, on Backpack. I started it after I spent far too long hunting around for a site that contained a plug-in I knew I’d seen in passing and wanted to include. My page on Tiddlyspot (a service for hosting Tiddlywikis online, worth an entry in itself), on the German Imperial Navy, shows a number of my favourite modifications, including restyling to the Blueberry template (from Tiddlythemes) and the addition of a tag cloud.

A few of my favourite tools: DokuWiki and MAMP

From the first time I met one, I took to wikis for information gathering like a bluebottle fly to an old fish-head, partly because of their non-linear nature, and partly, since they were web-based, I could style and lay them out to suit myself. With a wiki, I can start collection information at any point and with the most rudimentary framework, grow and restructure at will, design my own menus, and create multiple cross-links between entries. My present wiki engines of choice are DokuWiki and Tiddlywiki. DokuWiki was originally designed for documentation in small companies, which makes it ideal for my present challenge of keeping up with my learning on four different stats packages (R, SAS, STATA and WinBUGS). I keep programming notes, reusable code snippets, a personal Rosetta stone of command-equivalents, and lists of links and references in the form of a DokuWiki run on a MAMP server (see below) on my MacBook.

Familiarity breeds comfort, but DokuWiki’s advantages are that I have found it straightforward to install and configure (particularly now that the user access controls are provided through a web interface), it is FOSS and platform independent, the developer is actively maintaining it, there is an active user community, the mark-up syntax is easy to work with and readable in the raw, and it does not require a database but writes everything to flat-files and folders. The latter, for me, is a key point: at my scale of operations, flat-files are just fine, and they have the major advantage that they are program- and platform-independent. There’s no database to get corrupted (been there, done that). There’s no vendor lock-in. If I ever want to migrate, I probably won’t be alone and someone will have written a tool; even if they haven’t, flat-files, meet Perl; I know you know each other .

For comparisons of wiki software, see the relevant page on wikipedia and WikiMatrix. A sampling of reviews: From Open Source Magazine (Badger, 2007), and Linux.com (Popov, 2008). Implementations I’ve noticed around the web are DokuWiki’s own wiki and the Zotero documentation.

As for the platform, I’ve used the built-in OSX server in the past, but turning on the OSX server involved opening ports that I didn’t necessarily want open, even if I had been confident in my editing of .htaccess. So when I found out about WAMP, I went looking for a Mac version. MAMP, which is intended as a development server with Apache, PHP, MySQL, and Perl. Download and installation of MAMP were straightforward with the help of an on-line tutorial and instructions, and from there it was a matter of setting up and configuring DokuWiki according to the instructions, with a little help from the equivalent instructions for WordPress – blogging software is a possible alternative to a wiki for keeping notes, if they are more linear.

A few of my favourite tools: Zotero

Zotero, developed under the aegis of George Mason University’s Center for History and New Media (CHNM), is a Firefox plug-in for reference management. It offers an emerging cross-platform FOSS alternative to the mature and widely-used proprietary bibliographic software Endnote and Reference Manager. Zotero has been reviewed in the Journal of the Medical Library Association (Thomas E. Vanhecke, 2008), Inside Higher Ed (Scott McLemee, 2007), and the New York Times (Olivia Judson, 2008) – and many others (Google search).

The screenshot on the left shows the layout of the Zotero interface, which can be toggled on and off from a button on the lower right hand corner of the browser window. By default, it expands to fill half the browser screen, although it can be expanded to fill the full screen. The left pane contains the list of collections (folders and subfolders) within the library, and the tag filter search box. The middle pane contains the contents of the selected library, listed by creator, title, and journal abbreviation. The right hand panel contains the bibliographic information and notes for a single, selected entry. Control buttons and search-box are across the top of the Zotero pane. Very important: the icon to add bibliographic entries is in the address bar, at the top of the browser window. From an interface point of view, I’d quite like more user-control over positioning and panel layout. It’s not the ideal setup for a small widescreen computer, unusuable for any length of time on the 7” eeePC (taking things to an admitted extreme), and cramped on a 12-13” laptop – although I admit I probably wouldn’t have noticed before getting spoiled by 19-20” screens. But the ability to reshuffle and close windows would be a luxury.

Zotero maintains a SQLite database of citation information that can be scraped from a large number of database and journal sites.To date, I have mainly used it to collect results from literature and web searches; both single articles and indexed lists (eg, the results of Google Scholar searches) can be added via a single click of a small icon in the address bar. The filters for core sites such as PubMed, Highwire Press, PubMed Central, Google Scholar, WorldCat and JSTOR have all performed well in my hands, as have those for journals such as NEJM, Blood, Lancet and a number of others, with the caveat that in such a complex ecosystem, things will get updated, and do get broken – but tend to be fixed promptly.

Web pages can be collected as links or saved as screenshots, though I’m so familiar with print to PDF on a Mac that if I want to keep an article that only exist as a web-page I tend to print it to a PDF and then bring it into Zotero. PDFs can be imported in a number of ways, including dragging and dropping from the desktop into the window, and automatically, if linked without access restriction from a cited article. A recent update has given Zotero the ability to retrieve PDF metadata from Google Scholar, and complete indexing is possible using free third-party plugins. I tend to import abstracts from PubMed first, to be able to bring in the MeSH headings as tags – saves much tagging and ensures a consistent vocabulary for search. I can add additional tags if need be. Then if there is a free PDF of the article available, I re-import from the publisher’s site, drag the PDF from that entry to the entry from PubMed, and delete the duplicate.

The addition of PDFs carries the caveat that the database can grow very quickly. I’ve not done a thesis or book-sized project in it yet, but I have done article-sized projects, not to mention adding to ever-growing accumulations of materials of interest. Within libraries, materials can be tagged and files in folders and subfolders. It is possible to keep multiple databases, ie, one per project or subject, accessing them through preferences. At present, only one database can be accessed at a time, and Firefox must be relaunched to switch databases. An alternative approach is to keep separate Firefox profiles, and switch between those. I’ve actually done both, but I’m not the only person who thinks easier access to multiple libraries is a good idea.

For the creation of bibliographies, Zotero has plug-ins available to work with (certain versions) of Word and OpenOffice. Bibliographies can also be manually generated by right-clicking on selected entries in the middle pane of the database.

Recently, with Zotero 2.0 beta, the developers added the ability to  synchronize Zotero libraries across multiple computers (via a Zotero.org account), and the ability to share libraries between members of groups, and make them publicly visible. Here is the group for “All Things Zotero“. Having registered at Zotero.org, I effortlessly synced one of my libraries between my Mac running FF 3.5, and my eeePC running FF 3.05 – simply a matter of entering details in the preferences pane in both computers and clicking the green sync icon.

In pushing into new areas, and in taking advantage of developments in Firefox, the developers have sacrificed some backward compatibility: Zotero 2.0 is not compatible with Firefox versions before 3.0, and I notice when I upgraded I was warned that it was, as far as my databases are concerned, one way. I hope that stabilizes, at least to some extent – my older G4 PowerBook (2005), for instance, cannot be upgraded to Firefox 3.