Download/backup BibTeX library from CiteULike, including all the attached PDF

CiteULike is a convenient solution for bibliographic management, but sometimes an off-line replica of the bibliography is also needed. This can be achieved by exporting the CiteULike library in BibTeX format, furthermore BibDesk (and probably other managers too) can be synchronized directly with the CiteULike server.

While it is easy to export a .bib file from CiteULike, downloading all the attached files and having the .bib linking to the downloaded files is a different story.

BibTeX and JSON exports of the CiteULike library can be retrieved from:

  http://www.citeulike.org/json/user/USERNAME

  http://citeulike.org/bibtex/user/USERNAME

The BibTeX export can be read directly into a bibliographic manager, but the JSON export contains more information than the .bib, in particular it contains the location of the attached PDFs. So the idea of the Python script linked below is to do the following: 

1. download the CiteULike library in BibTeX and JSON formats
2. parse the JSON export and download all the attachments
3. modify the .bib file to include links to the downloaded copies of the attachments

the links in the .bib file should work for BibDesk and JabRef.

Before running the script:

• setup CITEULIKE_USERNAME and CITEULIKE_PASSWORD variables in the script
• verify you have wget and pybtex installed

Download the Python script:  citeulike_backup.zip

Resources:

http://www.macresearch.org/how_do_you_organize_digital_article_collections

http://www.andrewberman.org/projects/sync/

http://www.academicproductivity.com/2009/citeulike-bibdesk-sync-your-references-and-live-smarter/ 

http://wiki.citeulike.org/index.php/Importing_and_Exporting#JSON

---

Gory details from http://wiki.citeulike.org/index.php/Importing_and_Exporting#JSON:
# save session cookies

> wget -O /dev/null --keep-session-cookies  --save-cookies cookies.txt --post-data="username=xxxx&password=yyyy&perm=1" http://www.citeulike.org/login.do

# download bibtex with private comments and download an attachment

> wget -O export.bib --load-cookies cookies.txt http://www.citeulike.org/bibtex/user/xxxx
> wget --load-cookies cookies.txt http://www.citeulike.org//pdf/user/xxxx/article/123456/891011/some_99_paper_123456.pdf