Reading With The Lights Out
Taking Substack into the wilderness...and back
I begin this post with a caveat - the end result is not what I hoped for; not because it’s impossible, but because I did not look before I leapt. Read on to experience the highs and lows of my attempt to solve a technical problem, before stumbling head-first into a human problem.
A journey begins
One of the frustrating things about the modern world is that losing internet access for any length of time can quickly expose just how extraordinarily unprepared your brain is for a lack of constant stimulation. I’m sure there are thousands of studies on why actually, some time offline can do you some good, but this is 2023, and I’m finally excited about reading and writing again, so I hoisted my banner high and declared: not today, science!
Instead of allowing my brain to finally get some rest, today I would fix (kind of) a problem that Substack has: no offline support.
What’s the issue?
Substack is fantastic, as I’m sure you’d agree, but it could do with a few improvements. The first among these, and the one which I’d be tackling today, is that Substack is a publishing platform which doesn’t let you actually publish to anywhere, which means you can’t easily access Substack content anywhere but officially supported channels (i.e. the website or reader app). Unless you’re permanently online, or paying for some kind of conversion service like Instapaper or Matter (no disrespect to either of those services, but I’m more of a DIY kind of guy!), then you’re probably out of luck. Articles on Substack remain on Substack, and the next time you find yourself with too much time and too little internet on your hands, you’ll just have to do what people used to do before the internet: sit silently on your own, allowing the crushing horror of the natural world to slowly consume you.
You mean there’s a better way?
E-Readers have been around since the late 1990s, with the first to gain widespread adoption being Amazon’s Kindle - introduced in 2007. This will not be news to most people reading this post, but for those who are unfamiliar - E-Readers offer a variety of features which make them perfect for anybody who reads a lot:
High storage capacity means you can take thousands of books / publications with you on one device, with no need for an internet connection
Extremely long battery life, making them ideal for travelling long distances or extended periods without power
E-Ink displays are much easier on the eyes than your computer, phone or tablet screen.
There are many valid criticisms to be made of E-Readers - in particular their rather closed, proprietary nature - but, for the sake of content, let’s ignore all of those and instead pretend that they’re perfect and good!
Now that you’re a convert, allow me to propose to you the following: Substack publications on an E-Reader would be a cool thing to have. Agreed? Let’s do it!
Getting down to business
Let’s say you have an E-Reader already. It’s probably a Kindle, but it could be one of any number of devices. Since most widely-used readers are essentially walled gardens, most people don’t need to worry about the frustrating and unnecessarily complicated world of e-book file formats - but I’m afraid that I must, dear reader, because I’d set my sights on a problem and there’s no satisfaction to be had in not following through.
I won’t bore you with the detail of the vast array of technical considerations we could account for when deciding on a file format we could use: I decided to use ePub. It’s widely supported and handles everything I needed to worry about for this experiment. It is unfortunately not supported by Kindle devices, but there are many conversion tools available, including one from Amazon. Still, not something I’d need to concern myself with!
For my purposes, my target device was a Supernote A5X - which is first and foremost a note-taking device running on Android, but which also supports e-book formats including ePub. It also has a Kindle app available, which means it’s basically replaced the Kindle for me. Though I’m not a prolific note-taker, I take enough notes in my professional life that managing stacks of paper notebooks is something I no longer care to do.
So now that I knew what I wanted: Substack articles in the ePub format, the next question was: how do I do that?
My first strategy would have been to use the officially supported API, but unfortunately, Substack doesn’t have one, so that was a dead-end. Come on, Substack, help a guy out!
When there’s no nice way to do something like this, what are our options?
I could just manually visit whatever Substack articles I wanted, and save them to PDF via the web browser. PDF is also widely supported on E-Readers, but the format is a mess and this is a decidedly unsatisfying solution to the problem at hand.
The more automation-friendly option is to go fully retro and scrape the website via a script. This is incredibly fragile and almost impossible to support long-term, but getting this to work does make you feel like a hacker, so let’s channel Elliot Alderson and get cracking!
Hacking away
I had an end goal in sight, and visions of myself sitting peacefully, reading Substack articles to my heart’s content anywhere I pleased. I would even be able to make hand-written notes directly on the page! An absolute dream, we can all agree.
I researched what was already available, in the hope of saving myself some work, or preferably all of it (any seasoned programmer will tell you that being lazy is a core personality trait of those who succeed in this game) - and found some code which did exactly what I needed: take a URL and spit out an ePub file. Unfortunately, it appeared to have been abandoned, and no longer worked. This is, paradoxically, sometimes a joy: there’s not a whole lot of fun to be had from just having everything handed to you on a plate. Sometimes fixing something that’s broken, or adding the functionality you want to some existing software, is much more rewarding than actually achieving the end goal itself.
I spent a few hours debugging the code and identifying where things were going wrong. Once I knew where the problems lay, it was fairly straightforward to fix things up once more. I even took the liberty to add a feature - directly embedding images within the ePub file itself. All in all, I probably spent 4 or 5 hours tinkering away until I could take this thing out for a test drive:
It worked! Correct text formatting, embedded images, portable file format, all ready to be taken on my travels. All in all, the perfect Substack companion for anybody who wants to kick back and make some notes on whatever they’re reading. A reader’s tool. A writer’s tool. Perfection.
Not so fast, sunshine
Unfortunately, in my eagerness to make life a little more interesting, a little freer, a little more human - I had neglected to do the one thing you’re supposed to do: look before you leap.
As it turns out, Substack does not want you to do exactly what I had just spent several hours doing. They don’t want you to do it so much that it’s against their Terms of Service - which is why I’m not explaining to you how I did it or encouraging you to do it yourself. Read it again: you should not do anything I’ve written about here.
Foiled once again, by the machinations of our corporate overlords.
These situations are incredibly frustrating for those of us who like to fiddle with the tech around us. It’s of course understandable that a business like Substack wants to keep things under control, and there are obvious copyright and intellectual property rights which need to be respected - but I do feel like there should be explicit personal-use clauses for this kind of thing.
In a world which is absolutely dominated by technology, it often feels like the walls are closing around us, rather than liberating our spirit and creativity. These are not problems of tech - but problems of people. The urge to be free, to be masters of our own domain, to influence the world around us and invent, create, and explore; it feels like the technology at our disposal should be making all of this easier, more accessible and achievable. Instead, we find ourselves constantly penned in. Platforms lock themselves down, new restrictions are invented and added to documents nobody but lawyers ever read, and our world becomes a little smaller, a little more hostile to creative freedom, and a little less human.
This is an insignificant problem to face in a world currently on fire, but as somebody who believes that we have an instinctive urge to be free, I can’t help but feel that the cumulative effect of these minor disappointments, wherever we look, is a death-by-1000-cuts kind of deal.
Ultimately, the exercise wasn’t a total loss. I had fun, and learned a few new things along the way, but next time I’ll remember to RTFM before committing myself to a task!
Turns out I’m a little less Elliot Alderson, and a little more Boris Grishenko: