Not even a year ago, I decided to start using Hugo as the engine for this blog. This has mostly served me well, except for the fact that it relies on me having some kind of access to a console, a text editor, and my Bitbucket account, which made posting stuff while travelling a bit harder, so I opted instead for writing drafts, and then staggering their posts — which is why you now see that for the most part I post something once every three days, except for the free ideas.
Hugo was sold to me as a static generator for blogs, and indeed when I looked into it, that’s what it was clearly aiming on being. Sure the support for arbitrary taxonomies make it possible to use it in slightly different setups for a blog, but it was at that point seriously focusing on blog, and a few other similar site types. The integration with Disqus was pretty good from the start, as much as I’m not really happy about that choice, and the conversion proceeded mostly smoothly, although it took me weeks to make sure the articles were converted correctly, and even months in I dedicated a few nights a month just to go through the posts and make sure their formatting was right, or through the tags to collapse duplicates.
All in all, while imperfect, it was not as horrible as having to maintain my own Typo fork. Until last week.
I finally decided that maintaining a separate website for the projects is a bad idea. Not just for the style being out of sync between the two, but most importantly because I barely ever update that content, as most of my projects are dead or have their own website already (like Autotools Mythbuster) or they effectively are just using their GitHub repository as the main description, even though it pains me. So the best option I found is to just build the pages I care about into Hugo, particularly using a custom taxonomy for the projects, and be done with it. Except.
Except that to be able to do what I had in mind, I needed a feature that was committed after the version of Hugo I froze myself at, so I had to update. Updates with Typo were always extremely painful because of new dependencies, and new features, and changes to the database layout, and all those kind of problems. Certainly Hugo won’t have these problems! Except it decided not to be able to render the theme I was using, as one function got renamed from RSSlink to RSSLink.
That was an easy fix; a bit less easy at first was figuring out that someone decided that RSS feeds should include, unconditionally, the summary of the article, not the full text, because, and I quote: «This is a somewhat breaking change, but is what most people expect from their RSS feeds.»
I’m not sure what these “most people” are. And I’d say that if you want to change such as default, maybe you want it to be an option, but that does not seem to be Hugo’s style, as I’ll show later. But this is not why I’m angry. I’m angry because changing the RSS from full content to summary is a very clear change in impression.
An RSS feed that has full article content, is an RSS feed for a blog (or other site) that wants to be read. You can use this feed to syndicate on Planets (yes they still exist), read it on services like Feedly, or NewsBlur (no they did not all disappear with the death of Google Reader), and have it at hand on offline readers on your mobile devices, too.
RSS feeds that only carry summaries, are there to drive traffic to a site. And this is where the nasty smell around SEOs and similar titles come back in from below the door. I totally understand if one is trying to make a living off their website they want to be able to bring in traffic, which include ads views and the like. I have spoken about ads before, and though I recently removed it from the blog altogether for lack of any useful profit, I totally empathise with those who actually can make a profit and want people to see their ads.
But the fact that the tools decide to switch to this mode make me feel angry and sick, because they are no longer empowering people to make their view visible, they are empowering them to trick users into opening a website, to either get served ads, or (if they are geek enough to use Brave) give bitcoin to the author.
As it turns out, it’s not the only thing that happen to have changed with Hugo, and they all sound like someone decided to follow the path of WordPress, that went from a blogging engine to a total solution for managing websites — which is kind of what Typo did when becoming Publify. Except that instead of going to a general website solution, they decided to one-up all of them. From the same release notes of the version that changed the RSS feed defaults:
Hugo 0.20 introduces the powerful and long sought after feature Custom Output Formats; Hugo isn’t just that “static HTML with an added RSS feed” anymore. Say hello to calendars, e-book formats, Google AMP, and JSON search indexes, to name a few ( #2828 ).
Why would you want to build e-book formats and calendars with the same tool you used to build a blog with? Sure, if it actually was practical I could possibly make Autotools Mythbuster use this, but I somehow doubt it would have enough support for what I want to get out of the output, so I don’t even want to consider that for now. But all in all, it looks like widening a little too much the target field.
Anyway, I went and reverted the changes for my local build of Hugo. I ended up giving up on that by the way, and just applied a local template replacement instead, since that way I could also re-introduce another fix I needed for the RSS that was not merged upstream (the ability to put the taxonomy data into the feed, so you can use NewsBlur’s intelligence trainer to filter out some of my blog’s content). Of course maintaining a forked copy of the builtin template also means that it can break when I update if they decided that it should be FeedLink next time around.
Then I pushed the new version, including the updated donations page – which is not redirected from the old one yet, still working on that gone – and stopped looking too much onto it. I did this (purposefully) in the 3-days break between two posts, so that if something broke I would have time to fix it, but it looked everything was alright.
Until I noticed that I somehow flooded Planet Gentoo with a bunch of posts dating back up to 2006! And someone pinged me on Hangouts for the same reason. So I rolled back to the old version (that did not solve the flooding unfortunately), regenerated, and started digging what happened.
In the version of Hugo I used originally, the RSS feeds were fixed to 15 items. This is a perfectly reasonable debug for a blog, as I didn’t go anywhere near it even at at the time I was spending more time blogging than sleeping. But since Hugo is no longer targeting to be a blog engine, that’s not enough. “News” sites (and I use it in quote, because too many of those are actually either aggregators of other things, or just outright scammers, or fake news sites) would have many more than that per day, so 15 is clearly not a good option for them. So in Hugo 0.19 (the version before the one that changed to use summary), this change can be found:
Make RSS item limit configurable #3035
This is reasonable. The default is kept to 15, but now you can change it in the configuration file to whatever you want it to be, be it 20, 50, or 100.
What I did not notice at that point, was from the following version:
Raise the default rssLimit #3145
That sounds still good, no? It raises the limit. To what?
Of course this is perfectly fine for small websites that have a hundred or two pages. But this blog counts over 2400 articles, written over the span of 12 years (as I have recovered a number of blog posts from previous platforms, and I’m still always looking to see if I can find the backups with the posts of my 15 years old self). It ended up generating a 12MB RSS feed with every single page published up to them.
So what am I doing now? That, unfortunately, I’m not sure. This is the kind of bad upgrade path that frustrated the heck out of me with Typo. Unfortunately the only serious alternative I know to this is WordPress, and that still does not support Markdown unless you use a strange combinations of plugins and I don’t even want to get into that.
I am tempted to see what’s out there for Ruby-based blog engines, although at this point I’m ready to pay for a solution that works native on AWS EC2¹, to avoid having to manage it myself. I would like to be able to edit posts without requiring me a console and git client, and I would like to have an integrated view of the comments, instead of relying on Disqus², which at least a few people hate, and I don’t particularly enjoy.
For now, I guess I’ll have to be extra careful if I want to update Hugo. But at least I should be able to not break this again so easily as I’ll be checking the output before and after the change.
Update (2017-08-14): it looks like this blog post got picked up by Internet’s own peanut gallery that not only don’t seem to understand why I’m complaining here (how many SEOs there?), but also appear to have started suggesting with more or less care a number of other options. I say “more or less” because some are repeats or aimed at solving different problems than mine. There are a few interesting ones I may spend some time looking into, either this week or the next while on the road.
Since this post is over a month old by now, I have not been idle and started instead trying out WordPress, with not particularly stunning results either. I am though still leaning towards that option, because WordPress has the best upgrade guarantees (as long as you’re not using all kind of plugins) and it solves the reliance on Disqus by having proper commenting support.
¹ Update (2017-08-14) I mean running the stack, rather than pushing and storing to S3. If there’s a pre-canned EC2 instance I can install and run, that’ll be about enough for me.
² don’t even try suggesting isso. Maybe my comments are not big data, but at 2400+ blog posts, I don’t really want to have something single-threaded that access a SQLite file!
Thank you for still caring about RSS. It’s how I read your blog; the RSS entries get converted into email messages which I read in Claws. Including full article content saves me from having to hit the actual blog site unless I’m commenting ;-).
I’m curious about what you’ll find. I’ve been relatively happy with Pelican (http://docs.getpelican.com/… but the console/git client workflow is a problem for me too (sometimes I write posts in Google Keep first; sometimes I just don’t write them). Ghost seems to be positioning itself as a WordPress alternative, but I haven’t really used it much, and it’s clearly designed with email in mind instead of RSS.
I actually used your RSS feed earlier today to test the new Gtk+ feed reader that I’m working on! Your feed is in pristine working order and you even have working cache directives. I can see from this post that you actually care about your feed unlike … uhm, the web at large now days.A dynamic site generator (or a feed proxy) would let you use delta feeds to slim down your feed updates even further.I also used to use a static generator. The main advantage of a dynamic site generator like WordPress for me is being able to schedule when posts go live. I usually write like ten articles in the span of two days, and then don’t write anything for ten days.Something that live on the web where I can quickly login and fix typos without requiring a very specific setup with all my data and tools is something that just enables me to write better and more often.Static is great in principle. However, I’ve stumbled upon literally hundreds of blogs in the last year where the owner proclaimed “I’ve switched to X static generator!” and then newer wrote another word again. People clearly don’t really want to deal with all the extra work and inconvenience.
Yeah, mileages always vary.
For me it was a big relief to be able to write blog posts in nano with markdown when I switched to a static blog generator from WordPress with my little blog. Just recently I switched to Hugo and I am quite happy with it. Of course I also changed the standard behavior from excerpts in the feed to full post in the feed as I see it like the blog author about that. And I can understand, that the changing from 15 to unlimited posts is a bit crazy and might cause trouble for some people. But hey, nothing is perfect and at least you can change it. You can also file a bug to raise the developers awareness to be more careful about potential breaking changes in the future.
For now I am quite happy with Hugo as the downside to be not able to post from every computer is not so important to me.
One of the problems I found when I looked at Ghost is that it’s effectively Hugo with more dynamics, but still no comments. And I really want to get rid of Disqus if I can because I like my comments. Indeed one of the problems of my current set up is that I have no way to search through posts and comments, and particularly when comment threads become interesting, I want to be able to find them.
I would like to call myself a forerunner of static sites generation 😉 I have used static sites for my “actual website” since 2005 at least, well before the current trend. But for blogs it becomes… hard.The schedule is indeed a very nice thing, and I used to use it heavily on Typo, until it broke. Right now I use “futures” in Hugo, but it requires me SSHing into the server to re-build it. I originally meant to have this rebuilt once per day, but it turns out it takes over 30 seconds to rebuild my whole site. I guess their benchmarks are aimed to much smaller sites, if not in posts and length of posts, in tags and cross-linking.I see they NewsBlur supports delta feeds, so that may be nice as well. Although I wonder how that works with ModSecurity and similar anomaly detection techs. There is also a json feed format nowadays: http://blog.newsblur.com/po… and Hugo does not seem to support it. WordPress has a plugin, but I don’t know I want to deal with too many WordPress plugins :(Oh yeah I have almost decided that I’ll move to WordPress by now. I’ll play around with it over the weekend.
I used to build locally, push to a staging folder on a remote server, add an entry to atq telling the remote server when to copy the staging folder to the public folder. I then built upon that and started creating a whole set of staging folders and a long at quote. Let me just say that it quickly became quite the nightmare.Delta feeds is mostly an extension of HTTP cache revalidation. I have some ~200 feeds in Newsblur, and they all update right on schedule. (Which is more impressive than it sounds considering how crap the formatting of many feeds are … .) It doesn’t look like requesting delta feed causes problems anywhere. There aren’t all that many websites that support it, however.JSONfeed isn’t anything anyone should worry about. The web don’t need another syndication format and JSONfeed doesn’t bring anything new to the table.My last words on migrating from a static generators to WordPress is a silly quote from a blog post I wrote about the experience last year:• “[Static site generator] makes programmer-perfectionist-Daniel happy.• WordPress makes enjoys-writing-articles-Daniel happy.”
I take it all back. You have a couple of double-escape-encoding issues in your current feed:” and “
Are you sure this is double-escaping? It looks alright to me when parsing the XML. Since needs to be encoded, it makes sense that ” is used.Although I have some concerns about Hugo converting markdown with entities instead of using proper UTF-8 given that it has to use it for my name anyway 😐
I was going to suggest Pelican as well. You could work around the console/git client workflow by doing it all on Github which has a pretty decent editor built in. Then you do need to set up some webhook to deploy it. If you want even less setup then Github Pages with Jekyll might be an option but I have no experience with that.
Yes, I’m sure. The only named entities in XML are quot, amp, apos, lt, and gt. Which is why you end up with the double-escape sequence. So processing a text node with this double-escaped sequence, you’d end up with the literal string ” instead of the character ”. Example.Hugo is trying to use a named entity from HTML in an XML document. However, your processor realizes that won’t work and escapes the ampersand since it’s not used as part of a supported entity. Your processor should ideally have unescaped the HTML entity to the correct Unicode character or alternatively have converted it to a number or hex escaped entity (so ldquo would become #8220 and rdquo as #8221.)Hugo could also have added the missing entity names in XML processing instructions, but by that point you’re starting to get pretty far away from just using Unicode for what it’s meant for.
I don’t plan on paying *even more* to add GitHub to the list of services I need to run a blog. Seriously building those Rube Goldberg machines is not the kind of thing I have interest in, at this point.Also, none of that solve the comments problem. At this point I’m more interested in solving all of it in a go, if I have to change.
For disqus like comments I recently found this https://github.com/adtac/co… , looks nice but I didn’t tested it.
That looked nice (well except for nodejs, but meh), until…It does not have voting, spam-protection, moderationYeah no spam protection and no moderation are deal breaker with the amount of posts I have 😕
I believe pages are always free, but I totally understand the argument of wanting a complete solution just to reduce the maintenance burden. Just because you could, doesn’t mean you should 🙂
Pages are free, but private repositories aren’t.I don’t want to publish the whole blog in repository’s form, because some of the posts are left as drafts for a few days, both because I want to rework them, and because they may be tied to me having the clearance to write about something, checking my sources, etc. Plus all the history if stuff I may have gotten wrong in the drafts, and don’t want to make a fool of myself.
Jekyll is some really good stuff, and importantly, defaults to some really good defaults:https://jekyllrb.com/Really worth a look, though you still need access to a text editor an git. I don’t really think I ever come across scenarios where I don’t, but you could always use GitLab, which has a web-based editor for files (and basically commits straight to your repo). You can also set up CI to either manually or automatically push changes live.
@Flameeyes:disqus My team is building a WordPress-like CMS for Hugo sites. I’m curious if this would help make the Hugo experience better for you (commits to your Bitbucket, supports media uploads, Markdown, etc). https://forestry.io/
Or use the open source GitLab Pages https://about.gitlab.com/fe…
I am using hexo.js and had some problems converting themes and rendering output. There are a lot of static website building solutions out there but is is hard to find a simple one that works and has clear documentation! Thanks for sharing your struggles, could totally relate after spending a couple weeks getting my own site going
Interesting article! I had a look, and noticed no one has recommended Lektor (https://www.getlektor.com/ yet. It’s from the creator of Flask, and is written in Python. It’s a static site generator, but cleverly also has a simple admin interface. Very trivial to set up pushing to AWS, and manage the whole setup. The best part which I haven’t seen in any other generator, is the ability to define models for the pages – it’s truly simple, yet powerful.
Please see the other replies on this thread: I have no intention to use a *different* static site generator. GitLab is not an editor, so that would be the worst-of-both-worlds: terrible editing interface *and* no online editing without anything more than a password. So, no thanks.
I’m not going to push to AWS. Not now, not ever. And this does not seem to solve any of the problems I pointed out at in my post.
GitLab’s builtin-editor would mostly be a safeguard for when you have no editor/git access though.But you, didn’t notice you didn’t want to change generator — your call there. 🙂
Ah, my bad – I misunderstood your “at this point I’m ready to pay for a solution that works native on AWS” statement. Well, hope you manage to find something that works for you in the end!
You’re right that statement is misleading, sorry about it! I’ll see to update it.What I meant is “I’ll pay for something that allows me to just run the whole thing as a stack on AWS (namely EC2).”
Okay that might be the kind of thing I’m interested to look for, I’ll take a look, thanks!
I don’t know how it would work for a big blog like yours, but Discourse can be used as a comment solution, and I like it for my own use. As for your problem with GitHub private repositories being expensive, Gitlab private repositories are free. Moreover, Gitlab pages can use any hooks provided by Docker (that covers all the static generators) so you can use their online editor in markdown or your own, just push and you’re good to go.
In 6 months, did you find any better solution than hugo?
Haven’t been looking actively much, particularly as I have been busy with a move across countries.What I know is that my Hugo pull request got stale twice, so I don’t think I’ll try upgrading any time soon, unless I really need a new feature. And WordPress still has enough security misdesigns that I’m afraid switching to that too :/
I like isso but I’m not even close to 2400+ posts. 🙂
As long as you go with wordpress you can use their comments but if you might go back to some static engine something like this might also be feasable: https://github.com/skx/chronicle2/blob/be57f400302250a1162a69c1f09a5852de964d7a/COMMENTS.md
Sorry if that was mentioned already but I didn’t see it while skimming through the other comments.
I also like Hugo but I can totally understand your complaining. I also miss some stability there and have to change stuff after updates way too often. Just yesterday I realized all posts where added to tag-feeds, even if not having that tag added. I fixed this yesterday and now my main feed is broken…
It might also be that I did something wrong while fixing, but why did I have to fix it in the first place? Before the tag-feeds worked well.