Upgrading to Pelican 3.5 and Python 3.

I recently updated my Pelican blog setup, going from Pelican 3.3 to Pelican 3.5. I skipped Pelican 3.4, mostly by circumstance rather than intention — by the time I got around to upgrading, Pelican 3.5 had been released!

Although I wasn’t in any great rush to upgrade anyway, as I was rather diligently procrastinating the task of reinstalling the Linux box I’d been using for my Pelican work, and this is something I wanted to do first. But by the time I had overcome my procrastination (not to mention general reluctance to mess with Things That Are Working), I’d discovered that Pelican 3.5 was out. So perhaps in a way my procrastination resulted in some fortuitous timing.

For the most part things went surprisingly smoothly.

Since I was essentially reinstalling (and reconfiguring) things from scratch on a new Linux installation (Mint 17), I decided to have a go at using Python 3, as up until now I’d only ever used Pelican with Python 2.7. I used the system’s Python 3.4 (and “pip3” via the system’s package manager).

Not being sure if virtualenv/virtualenvwrapper played nice with Python 3, I thought I’d try the native virtual environment support in Python 3: venv.

Here, I ran into a bit of a problem. When trying to create a new virtual environment I’d get an error like this:

Error: Command '['/home/rob/python/newenv/bin/python3.4', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1

This appears to be caused by an issue in Python 3.4 (affecting Debian and derivatives, at least).

The solution here fixed the problem.

After that I was able to create a new virtual environment for Pelican with pyvenv:

mkdir venv
cd venv
pyvenv-3.4 pel
source pel/bin/activate

From this point on things were fairly straightforward. I didn’t encounter any issues with using Python 3 instead of Python 2 to install and configure Pelican:

pip3 install pelican markdown

The above command installed Pelican 3.5. With that done, I did a dummy run of pelican-quickstart, then compared the various generated files to that of version 3.3, to get an idea if any important differences needed to be addressed.

There wasn’t much. Here is a diff of the Makefiles:

1,2c1,2
< PY=python
< PELICAN=pelican
---
> PY?=python3
> PELICAN?=pelican
27a28,29
> GITHUB_PAGES_BRANCH=gh-pages
>
90c92
<     rsync -e "ssh -p $(SSH_PORT)" -P -rvz --delete $(OUTPUTDIR)/ $(SSH_USER)@$(SSH_HOST):$(SSH_TARGET_DIR) --cvs-exclude
---
>     rsync -e "ssh -p $(SSH_PORT)" -P -rvzc --delete $(OUTPUTDIR)/ $(SSH_USER)@$(SSH_HOST):$(SSH_TARGET_DIR) --cvs-exclude
99c101
<     s3cmd sync $(OUTPUTDIR)/ s3://$(S3_BUCKET) --acl-public --delete-removed
---
>     s3cmd sync $(OUTPUTDIR)/ s3://$(S3_BUCKET) --acl-public --delete-removed --guess-mime-type
105,106c107,108
<     ghp-import $(OUTPUTDIR)
<     git push origin gh-pages
---
>     ghp-import -m "Generate Pelican site" -b $(GITHUB_PAGES_BRANCH) $(OUTPUTDIR)
>     git push origin $(GITHUB_PAGES_BRANCH)

(Note that a Python 2 environment was used to generate the Pelican 3.3 files, while Python 3 was used for Pelican 3.5. I’m not sure if that’s significant, I don’t think it was.)

There appear to be only minor changes to the deployment targets. The only one of note being the addition of the -c option to the rsync command.

Here is a diff of pelicanconf.py:

8a9,10
> PATH = 'content'
>
11c13
< DEFAULT_LANG = u'en'
---
> DEFAULT_LANG = 'en'
16a19,20
> AUTHOR_FEED_ATOM = None
> AUTHOR_FEED_RSS = None

So pretty much the only change I had to make was disable author feeds, which I had no use for.

Both publishconf.py were identical.

The develop_server.sh script had many minor changes — almost all of them to the wording of text in echo statements. I just copied over my old version with the latest.

There was only one problem I ran into during the upgrade, and it’s actually a problem I first noticed after I upgraded to Pelican 3.3, which changed the way static files were handled and introduced the EXTRA_PATH_METADATA setting. I had a file with a HTML extension that I simply wanted copied into the root directory of my generated website, without being processed in any way. The file wasn’t actually a valid HTML file though. It was just an arbitrary text file, however it was generated by an external tool, so I couldn’t change the contents of the file, or its file extension. My initial attempt to configure this resulted in something like this in my pelicanconf.py:

STATIC_PATHS = ['images',
                'other/robots.txt',
                'other/favicon.ico',
                'other/arbitrary.html',
                ]

EXTRA_PATH_METADATA = {
    'other/robots.txt': {'path': 'robots.txt'},
    'other/favicon.ico': {'path': 'favicon.ico'},
    'other/arbitrary.html': {'path': 'arbitrary.html'},
    }

That worked for the first two files, which were copied across correctly to the root directory of the generated website. But the file with the HTML extension wasn’t copied, and Pelican output this error message:

ERROR: Skipping other/arbitrary.html: could not find information about 'title'

My initial attempts at finding a solution had been unsuccessful, so I had been ignoring this error for a while. But I eventually noticed this file not being copied to the right place was causing some issues, which motivated me to once again try and find a solution.

I came across some discussions of this problem in these two issues on GitHub.

Turns out that Pelican accepts HTML files as source input, just like Markdown and reStructuredText, which is probably why it was attempting to process my arbitrary (non) HTML input file, and complaining about a missing title. This is all well and good, but I can imagine many valid use cases where it might be desirable to have HTML content just copied over as part of site generation without any processing (like content generated by tools external to Pelican). Fortunately, Pelican provides a way to skip processing of all HTML files via the READERS setting:

READERS = {'html': None}

This solved the problem and got rid of that error message. After adding the above line to my pelicanconf.py all my external, non-generated files ended up where I wanted them in the output directory, and I finally had a “neat” build without any mysterious warning or errors.

Overall, the upgrade went fairly smoothly.

I didn’t encounter any backward compatibility issues with Pelican 3.5. Apart from the minor tweaks above to Makefile and pelicanconf.py, there were no significant changes required to either my configuration or (custom) theme.

I did, however, encounter a minor glitch, which I think had something to do with caching.

Pelican 3.5 features caching to speed up site generation, which is nice and usually works quite well. It doesn’t appear to be perfect, though. At one point, while messing around with theme modifications, I noticed that some parts of my website were not being generated properly. In particular, there were internal links that were pointing to wrong (non-existent) URLs. At first I thought I had introduced some weird theme issue, but that didn’t appear to be the case.

I deleted the cache directory. Then I did a make clean followed by a make html That fixed the problem — the regenerated website was fine. I’m therefore assuming it had something to do with the caching. I wasn’t able to reproduce the problem though, so for now it remains a vague observation that the caching can sometimes cause issues.