Embracing the IndieWeb

software •  meta •  blog •   •   •   •  ~8 min read
Hugo transition complete, it is time to go Indie

Well, the experiment with Hugo and css flexbox has been (I think) a success. This site is now fully written in the former, and uses the latter. I’m pleased with how this has turned out, despite my (perhaps unfair) ranting at Hugo previously. Site is back up and running, I’ve converted all my posts1, and the only thing I’ve broken is the rss-feeds!

Inflicting all that pain on myself wasn’t without purpose, however, and it also gave me time to think, more particularly about the place technology has in my life and my relationship to it. I’ve kept to my twitter pledge and this site is now the product of all the time I’ve saved in doing so. Going back to Not Being the Product, I’ve tried to move away from captured silos, and towards being ‘self-sufficient’ – not always terribly succesfully, I must admit, but I’ve always had that intention – the silos kept winning.

So, given that I had to do a full site rewrite for Hugo, I thought I’d take another look at the state of the indieweb. I’d done this before, but had given up rather hastily. Now I found myself agreeing more strongly than ever with the IndieWeb philosophy, and I was surprised to see that what had been vague ideas were now concrete proposals backed up by solid implementations. It helps that right around the time I was considering this, webmentions, long an indieweb feature, became a fully fledged W3C Recommendation. If its good enough for them…

At the same time, I read more deeply about the IndieWeb, and came to a greater understanding of the definition of POSSE2. In short: by syndicating without explicitly linking back to the original posts on my own site, I was, to coin a phrase, doing it wrong.

Identity

The first piece of the indieweb puzzle is establishing your own identity. There’s been a lot of movement in this area since the last time I looked at IndieWeb, in particular the microformats2 specification integrates the metadata directly into HTML5 in precisely the manner that the old microdata and RDFa didn’t.

I initially decided to markup my identity credentials using both rel="me" attributes in my social media links in my sidebar, as this was the simplest approach. Later I found out that, while this is necessary3, it is by no means sufficient, enter h-card. This is part of microformats2 and, after learning about the spec, I implemented it in the footer strap of my site. The process (using golang templates) wasn’t too difficult, and the final partial/footer.html is as follows:

    <footer>
        <p class="feet h-card vcard">{{ .Site.Title }} &copy; 
            <a href={{ .Site.BaseURL }} 
                class="p-name u-url url author metatag" rel="me">
                {{ .Site.Author.name }}
            </a>
                <img src="{{ .Site.BaseURL }}{{ .Site.Author.photo }}" 
                class="u-photo" style="display: none"> 
                <a class="p-nickname u-email email metatag" rel="me" 
                href="mailto:{{ .Site.Author.email }}">
                {{ .Site.Author.nick }}
            </a> 
                2011-{{ .Now.Format "2006" }} &bull; Licenced under 
                <a href="{{ .Site.BaseURL }}page/licence" 
                class="metatag">
                {{ .Site.Author.licence }}
            </a> 
        </p>

While I had now taken the first steps to establish my identity on the IndieWeb, I still needed to go further in order to utilise the services IndieWeb members have created - services that are and will be key for this site in the future.

Those services rely on a process called IndieAuth, which takes the data from your site’s h-card and rel="me" links, then leverages authentication protocols of existing services to bootstrap your own identity. As they say on the site:

IndieAuth is part of the IndieWeb movement to take back control of your online identity. Instead of logging in to websites as “you on Twitter” or “you on Facebook”, you should be able to log in as just “you”. We should not be relying on Twitter or Facebook to provide our authenticated identities, we should be able to use our own domain names to log in to sites everywhere.

Getting myself validated through IndieAuth was already taken care of by the rel="me" links, but some services needed me to declare IndieAuth as my authorization endpoint; this was a simple matter of adding one line to my html headers:

    <link rel="authorization_endpoint" 
        href="https://indieauth.com/auth">

The next step was to mark up my blog posts with the metadata used, microformats2 h-entry. This is the entire basis of the indie-web, using classes for information such as titles, authorship and content. The microformats wiki has a detailed article on the use of h-entry; by way of example, the markup for the articles on my sites consists of this golang template:

        <article class="h-entry">
          <h1 class="p-name">
            {{ .Title }}
          </h1>
            {{ partial "metadata.html" . }}
            {{- if .Params.summary }}
          <div class="floatright p-summary">
                {{ .Params.summary }}
          </div>
            {{- end }}
          <div class="e-content">
            {{ .Content }}
          </div>
        </article>

All that taken care of, it was time to move on to actually, you know, using the IndieWeb.

Webmentions

Thanks to Pelle Hesseman I’ve now gotten webmentions running succesfully on this site. Because I use a static site generator, this is a little more involved than it would be on a dynamic site which can use SQL dBases in realtime. The only real solution is to embed a script to the static pages, and poll for webmentions at pageload time. I’d been struggling with getting this process working, but #indieweb pointed me at Pelle’s own webmention endpoint, designed specifically for static sites4 and with a working and mature codebase. Again, I had to add a single line in my headers:

    <link rel="webmention" 
    href="https://webmention.herokuapp.com/api/webmention" />

The script which I need is (thankfully) provided by Pelle on his site, and I was able to sucessfully validate and test it all using webmention.rocks

Escaping the Silos

Once I had webmention support on my site, I needed some way of getting people’s reactions to my posts on social media out of the silos of twitter and google+ and onto my own site. This is a far harder nut to crack, and not one that I felt capable of dealing with on my own. #indieweb to the rescue yet again.

Bridgy is another indieweb service, one which is specifically designed to do what I wanted to do: Create a back-feed so that the reactions of my friends within a silo come back out of the silo and on to my site. As I’d already set myself up for IndieAuth, getting Bridgy active was a trivial matter. I’ve set up the back-feed, and you cane see some of the results in more recent posts here, such as New Years View I or The Cold Pre Dawn.

I haven’t yet tried Bridgy’s Publish Function, this will be the first post I’ll have attempted to syndicate in this manner. If you came here because of a tweet from me: Hey! It worked!

Just an aside to point out that it upsets me that I can syndicate to twitter, or (if I was a member) facebook or medium, but there’s no way to syndicate to google plus, as there’s no gplus write API. Nice way to Do No Evil™, guys!

Future Direction

The next stages of this journey are, of necessity, more complex.

  • I would like to get my own webmention endpoint working. I’ve been looking over the code for Pelle’s endpoint, and I think that’s something I should investigate doing for myself. That would mean I’m not relying on his (fabulous and wonderful) service, real independence of content.

  • I want to be able to use micropub to create posts on my own site. To start with, I’d like to use the protocol to create my microposts and syndicate them automatically, rather than using the POSSE-as-a-service of Bridgy. Again, real independence.

  • I want to become more involved with the IndieWeb community. I was hanging out in their slack channel for a week before I realised it was just a feed of an IRC channel! Time and again, when I’ve had a question or hit a brickwall, I’ve been helped, politely and efficiently, by people like snarfed, aaronpk, kevinmarks and voxpelli. That’s the type of community I enjoy, and one I’m determined to contribute to, in some capacity or another.

Wrap Up

The point of all of this is to escape the silos into which the “User As A Product” vendors want us to stay. To ensure that our own words remain ours, we need to identify ourselves as authors of those words, we need to decentralize control of our identities and words to our own control. It is a process, one which is not simple, and one which, honestly, most people don’t know about and couldn’t care less.

I’ve used the words ‘trivial’, ‘simple’ and ‘just’ in my descriptions above, but that’s how I perceived the difficulty of the task. This isn’t my first time on the merry-go-round; I have greater than baseline knowledge about html and the underlying structures of the web. As it stands, embracing the indieweb is something which demands that greater level of knowledge.

Until such time as the principles that underlie the IndieWeb become part of the core of how people evaluate the entire Web, until people acquire the knowledge to make the process simpler and more immediate, that’s going to remain the case. I hope that by providing details of my own experience, and placing my Hugo theme on github, I can make a contribution in my own small way.

It astonishes me how far the IndieWeb has come since I last looked at it. They’ve accomplished some incredible things, and I’m excited to partake of the journey.


  1. I ended up using some fairly kludgey scripts to convert all my data to the correct YAML format; I’ve posted the most important one as a gist on github. [return]
  2. Publish Own Site, Syndicate Elsewhere [return]
  3. Correctly established rel="me" links are essential for IndieAuth [return]
  4. In his own case, he’s using jekyll [return]
Have you written a response to this? Let me know the URL:

Thank you. So I believe we have to rely on Twitter or similar for "comment system", and feed back the tweet replies/likes as mentions via brid.gy -> webmention.io -> ?? -> my blog. I am trying to figure out the ?? part at the moment.
Website transition to Hugo is complete! Now for the next stage: It is time to go Indie! https://ascraeus.org/embracing-the-indieweb/
Over three years ago I spent a long weekend building my very own, basic but functional webmentions implementation. When I switched the site to Hugo from my homegrown pile of static-blog-building scripts last year, I put rebuilding webmention support way, way on the back burner, and never got back around to it. I really wasn’t fully geared up to make that system work with Hugo, so today with this writeup from Daniel Goldsmith I used the webmention endpoint from Pelle Wessman, to rebuild all of that functionality in a couple of hours!1 👍! It should now be easy to post comments here via any system that will send webmentions, such as one’s own indieblog site, as well as via replies to entries that are syndicated to my micro.blog feed. This is cool! Looks like I have a couple of metadata/microformat adjustments to make so that my mentions appear correctly on the other side. Stand by. [return]
Hi Daniel, i added a function ( not without pain 😆 ) to create thumbnails with php-gd on the media-endpoint scipt that you give me. You could maybe be interested. https://framagit.org/jee/media-endpoint
This post is in response to Diana’s comment on an earlier post, whose excerpts are quoted and answered. Additional notes on the ‘experiment’ are added and will serve as useful reminders about what was done. Started at zero knowledge of implementation : okay, not really. >> Specifically, since you didn’t know what the things was, it follows that you couldn’t have wanted it because of its characteristics so what remains is at best “someone’s recommendation of it and my trust in that someone”. This could have been phrased better to avoid conveying absolute ignorance. I meant I had never done it before, and indeed managing WordPress actively is in itself new, baring general explorations in the past. The goal was to implement interaction in a static website. I read up on the concepts of #indieweb first and have also mentioned Kaushal Modi’s site as the first inspiration. If you see his site – he also has the feature of selecting text and creating URL’s to the selction, on his static blog. I was already using Hugo, but was unable to implement the entire workflow / services as a self-hosted system. My first action was to replicate the workflow on a hugo site as described here, and I was successful. Note that it was relatively easy to switch themes, with content intact and formulate a new config.toml for Hugo. Next, I was able to find and (eventually) correctly use a nodejs program, webmentions.app for sending out webmentions, as well as pingbacks and trackbacks. This had to fired after a post was published, and realisation then dawned that it needed to be hooked to a database, to make sure repeated pings were not sent out. Here wordpress is superior and enables a reponse that the trackback has been registered. The hosted webmentions.io endpoint that is commonly used does not make such a distinction and leaves this responsibility to the machine sending out webmentions. The RSS feeds were modified to include the entire content to facilitate the program to comb these. The issue still remained of receiving webmentions, and then the troublesome notion of displaying these comments on the post, and perhaps maintaining some sort of database of the comments. The latter aspect of setting up a pleasant display of the content is still very much in the development area, and dependent on individual skills. Since atleast ‘predominantly self-hosted’ was a requirement by this point, I took to – cloning webmention.io’s source code. This had a db and could function for both receiving and sending. This is based in Ruby which I have never used. I did almost get it going, but there were too many loose ends still left. At this point, I switched to wordpress, and after some hiccups due to a missing PHP dependency – I was able to get going. Worth noting that I was able to run the Hugo server on the VPS in a terminal on tmux monitoring ~/hugo-sr/ and publishing to ~/hugo-pub, which was in turn sym-linked to the publish_html folder which is served by Apache as the website you see. I could export blog posts in org files from local Emacs directly to a custom location on the VPS, including different document types (post/page) – the whole of which is version controlled (github). Export would automatically happen when the document was saved. This does save a lot of time, and has the least number of dependencies. Evaluation of procedure, viability and trust >> whose recommendation did you follow, why exactly do you trust them on this, how did you evaluate your trust” : Though it was not a recommendation, and was my own decision – I did have a bunch of people to follow. For example, I use Kaushal’s excellent Emacs packages and have interacted with him in the past. I have mentioned that the indieweb wiki is the best place to start. It contains numerous examples and links to working implementations. See this article, whose author has also created nanopub (mentioned in my post), an open source package that can be used to self host a micropub endpoint. I just learned from the post that webmentions is now a full-fledged W3C recommendation, and also that he uses a heroku app for the webmentions endpoint. I use a bunch of plugins that use the dynamic db behind WP to manage the background processes. FWIW: based on recent discussions in #ossasepia, I am sceptical about the value of a recommendation in the software engineering / technology world. I can only continue to explore the concepts to understand them for what they truly are. The people and blogs I’ve mentioned above are examples of indiewebified sites, built with different approaches and toolsets, and each have varying levels of documentation on it, and also share some common threads. Additionally, most people have wiki pages which talk of and upcoming projects. The latter and more is discussed on IRC channels. Refer to this conversation : https://s.ragavan.co/2019/08/551/, and the comments. Side note: so the entire comment is transferred and my site posts show up on Kaushal’s mastodon instance. There are dings in the formatting and threading, and there are several rough edges, but you can see that you control more of the data, while still posting to silos, from your own website. The trust was based on exploring the references mentioned both above and in my earlier article, which showed the methods were feasible. This was further was evaluated further by directly implementing what I understood from their posts / work / interactions. >> “more information without following the link to source” is rather a minus than a plus” Yes I agree. However, typically – (in the networks it is possible to connect with) it does appear the entire content is transferred. The theme + possibly the plugins have to be customised / tweaked to extract and display as desired. I do not possess that skill now. The blogs/sites provided as references are examples of how it can be done rather well, over time and exploration of the code. Though I do not particularly subscribe to ‘micro’ blogging – I have found it is an easy way to maintain a web scrapbook of sorts. I think it ultimately adds value as a representation of my ‘social’ activities, besides the fact that all my responses are stored on my website, and cannot ‘unhappen’ (Refer your post on TheOdinProject), unless I delete my posts. If the external network in quiestion shuts down – context of response will be lost in certain cases, but not entirely, depending on what precautions are taken. Additional notes on the components: Indieweb is about establishing an identity through your website, and enabling 2-way communication between blogs/websites/external networks while enabling ownership / control of data. Enabling this 2-way communication is actually optional. The ‘improved’ communication happens via adding appropriate custom fields and meta data to posts, depending on the ‘base’ categorisations (an extension of taxonomies) called “kinds” (enabled by the Kinds plugin of WP. For example: Note, Article, Bookmark, Like, Reply etc), which can be further customised, (and used to post directly to a particular project with ease). The format containing such relevant metadata is called microformat2, and is used to add additional meta data to posts. The WP Indieweb plugin is like a R metapackage with some settings, and facilitates installation of the recommended / and some mandatory plugins. The ‘indieauth’ authorisation token is generated by the indieauth WP plugin (converting WP to an indieauth server). The identification is on the basis of identifying rel=me links in the website headers and post contents, which can be implemented via relatively easy theme modifications. I started with a modified version of the original WP 2016 theme, called IW16 (functional as you can see here, but under development). All the content is sent from (syndicated) or stored on my website, and WP plugins run additional ‘servers’ and on-the-spot ‘endpoint’ + token generators to enable authentication, send out pingbacks/webmentions and syndicate content. The concept of POSSE is IMO more important than webmentions – i.e posting all content on my website and syndicating, i.e replicating with content on other websites enables me to have my data and my activities on my website. i.e. website and content lives as long I have the server up. Now the Microblogging (sharing a picture/like/bookmark/response to anything – via my site), has to be done by a micropublishing (micropub) client to post directly to the website. Quill is a hosted micropub client, an open source webapp (that I can host on my server for myself if desired). Quill bookmarklets mentioned in my post connect to Aaron’s hosted Quill instance. The authentication for this is via my personal domain name + WP login, and a on-the-spot token generated by the indieauth plugin ( WP becomes auth server), not an external service. This is the case with all the components in terms of authentication. This is called using RelMeAuth, to provide authentication to Oauth processes to external networks (twitter/flickr etc), and is explained in detail in this article. The same is done via the indigenous app on iOS, also mentioned in my earlier article. When I click the above bookmarklets – I can tick options to additionally syndicate the content to the networks I’ve registered with, or not. The entire content goes to my website and these networks. If somebody responds or likes my syndicated post on twitter or wherever, it is fed back to my site. This is enhanced (in appearance and formatting) by the semantic-linkbacks plugin, which leverages the microformat2 specification, which can be built into themes. In addition to syndication of your website post to other networks – backfeed is also required – wherein a service polls your social networking accounts and feeds back likes, comments etc to your website. In this case, both the syndication and the backfeed is being done by somebody’s hosted service called Brid.gy, and their FAQ details their motivation + background, and the github docs talk about ways to add other silo’s like Tinder for example. Note this polling is done where it is possible – and it is not possible with Facebook (who cares) and Linked in (this would have been valuable). Twitter/Mastodon by itself is sufficient for me. In particular, the data science community is quite active on Twitter, not that I am active there by any measure. >> How many accounts/register/store-it-with-X-app instances are in the above? Bridgy and Quill are the only external services being used, in addition to numerous plugins, even beyond indieweb. My website (WP Login) is used as my identity to connect everything. The plugins are potential weakpoints, covered in recourses. Conditions of failure and Recourses >> ..what are the conditions under which you’ll conclude that you misplaced your trust in this instance? In general – multiple options and reasonable documentation exists to build your own version of the components, and even contribute to the development. Therefore, it appears the only way my trust would be completely misplaced, is if the community suddenly vanishes. General conditions of failure and recourses are outlined below: my VPS being down: Well, Linode does restart the webserver if it crashes. As of now, Manual backups are made of WP and etc. More could certainly be done here, and has been initialised. This is slightly easier with Hugo since the entier website and configuration can be version controlled with git repos. It would be interesting to figure out a controlled and automated manner to backup the WP website. The indieauth plugin not functioning to make WP an indieauth server: Alternative implementations are available, and instructions to build my own. The Brid.gy syndication + backfeed polling and publishing service going down: As such it can be self-hosted and the source is availble on github. But there appear to be few other alternatives for this point of failure. My site will continue to function fine. Twitter or some network shut down their API access or even altogether like Orkut – If so – then I atleast have my content on my website, and others comments as well. indieweb movement / all indiepress plugins fail: wordpress will continue to function. Networks have in fact shut down, like tumblr, Orkut etc. The classifications introduced by the Kinds plugin, like Articles, Notes, Bookmarks will vanish. All the content remain as blog posts / pages / comments etc, and they may just not look as well formatted. If wordpress itself fails – all the content is anyway being created and maintained in org documents. I can publish all the documents as a simple html based website, or opt for static site generators like Hugo, as I was using earlier. Unless WP data is accessible – I may lose comment information, and maybe some post categorisation. Shiny new things in the future >> And what happens when in a few more years (months?) there’ll be yet another improvement “recommended” and all that? If it adds value, and conforms to the long term idea of a self sustaining system (that I eventually put together on my own) – I would consider it. Indieweb as a movement is relatively new, but the concepts are still appealing, in terms of being able to stay away from external silos. There is still a lot of ground to cover, and holes in the tools being used, as I have already seen. An ability to design a website that can display webmentions/pinhback and cross site content is required, which I don’t possess, other than what has been done so far.