Ideas of connected ideas

Commonplace book, Memex, Hypertext, Hypercard, Zettelkasten, Wiki, … they all share the same idea: we want to have a central place to store knowledge and ideas (a.k.a. “External brain”), and they should be organically connected to each other, just like they are represented in our mind through mental associations. These connections (associations) make information more discoverable and nudge new thoughts by combining ideas together.

Variations#

The Commonplace book is the oldest form: a personal notebook where you jot down whatever seems worthwhile (quotes, proverbs, ideas) regardless of topic. The practice goes back to ancient culture; People like Newton and Darwin kept one. Just by the nature of putting everything into a single place, you tend to stumble upon connections between disparate topics.

The Zettelkasten (“slip box”) replaces the notebook with index cards, and the connections become explicit. Its most famous user is the sociologist Niklas Luhmann, who accumulated close to 100,000 cards and credited them for his highly productive career, which produced roughly 50 books and 550 articles. Each card gets a fixed ID, that forces each new card to be filed next to a related one rather than under a topic, and cards point to each other by number. In other words, links instead of headings, or kind of a hypertext on paper. How to Take Smart Notes is the popular introduction.

The Memex moved the idea to a machine, at least on paper. Vannevar Bush described it in As We May Think (1945): a desk stuffed with microfilm, where the user could tie any two documents together and build “associative trails” through them, an explicit attempt to mimic associative memory. Nobody really built one, but Douglas Engelbart and Ted Nelson both point to it as their inspiration.

Hypertext can be thought as the memex made with digital computers. Nelson coined the word in 1960s; Engelbart’s NLS demonstrated hypertext editing in the 1968 “Mother of All Demos”, and Tim Berners-Lee’s World Wide Web (proposed in 1989, released in 1991) is the version that overshadowed the previous attempts. The web is the idea of connected ideas at planetary scale, though not a personal one.

Hypercard (Apple, 1987) was an attempt to bring it to the personal computer. Bill Atkinson’s “software erector set” organized information as stacks of cards, like a Rolodex, that anyone could link and script in HyperTalk, a language that reads like casual English. It was discontinued in 2004, but its fingerprints are everywhere: Brendan Eich drew on HyperTalk for Javascript, and Ward Cunningham traces the wiki to a HyperCard stack he built in the late 1980s.

The Wiki put the whole thing on the web and let people edit. Cunningham’s WikiWikiWeb went online at c2.com in 1995 (“wiki” is Hawaiian for “quick”): pages are edited in a plain browser, formatted with lightweight markup, and linked with almost no effort. The huge success of Wikipedia (2001) further popularized the idea, to the point that “wiki” now mostly evokes an encyclopedia rather than a personal notebook.

Each variation answers the same two questions differently: where do the notes live (a notebook, a box of cards, a desk, a screen, a server), and how do they point at each other (headings, numbers, trails, links). The tools below are just the latest answers.

Tools#

What are useful tools for creating such a system?

Physical or digital?#

One can use a physical notebook or a slip box (Zettelkasten), or other analog solutions, all of which have their own charms. However, it is hard to deny that these analog are limited especially in the capacity for creating and updating connections between notes and retrieving them quickly, compared with digital tools.

Markdown#

Probably the simplest digital system is just a folder with a bunch of text files. Markdown is a really good upgrade to plain text files because of its support for wiki links and other useful formatting rules. There have been many formatting systems for personal Wiki (e.g., media wiki, moinmoin, etc.), but it seems like the formatting is converging into Markdown, likely due to the The Rule of Least Power.

Obsidian#

Once you have a bunch of linked text files, they can be managed in many different ways. At the end of the day, they are just text files. However, among those, Obsidian may be the best tool (browser + editor). It assumes Markdown flat files that are connected through Wikilinks. It provides powerful features such as searching, backlink-updating, metadata, and an active plugin ecosystem.

Specialized tools#

There are many powerful commercial tools too. Such tools tend not to use markdown to provide more powerful features, but that power may actually be a drawback (see The Rule of Least Power again).

Roam research, notion, etc. all provide a nice twist to the classic Wiki. Notion is especially popular with its ability to create all kinds of databases and hierarchically organized pages. But, I don’t really use them because I think they are not as good as obsidian in the core functionalities while being bloated & less portable. The most crucial factor for me is that the data (and its structure) is tied to their service, not flat files that I can manage however I see fit.

Syncing, gardening, and publishing#

Any digital personal knowledge system requires syncing across devices as well as constant “tending” or “gardening”. Then you may want to create a publishing system for sharing some of the knowledge that you accumulated.

Syncing can mostly be solved by treating notes like code. A folder of markdown files is a natural fit for Git, so version history, backup, and sync come for free (The Rule of Least Power again). I came across Obsidian while trying to build my own wiki on flat markdown files and Git, which became YYiki. YYiki and Obsidian read from and write to the same repository, and that is the whole point of flat files: any other software can use the same “database”. Obsidian provides syncing service, which is not exactly necessary if you use Git. The biggest value of the syncing feature is for iOS devices.

“Gardening” is simply the process of revisiting pages and updating them to be useful. It can be removing outdated information, adding new links, or connecting to other new pages. Luhmann reportedly spent more time on his cards than on his books; notes that are never revisited become what Ahrens calls a “graveyard for thoughts”. Both Wiki and Zettelkasten communities use the same metaphor, “gardening”: revisit old pages, connect orphans, fix broken links, flesh out stubs.

Publishing is the last step: sharing a subset of the vault without maintaining a second system. Obsidian Publish is tied to Obsidian and costs money; general-purpose static site generators (Jekyll, Hugo, …) assume a blog and make you restructure your notes. So I wrote foliate, a static site generator that points at the vault as it is: a page with public: true gets a URL, published: true additionally lists it in indexes and search, and everything else stays private. The same vault also holds my homepage. YYiki started as a dynamic wiki with online editing, but the complexity and security risk did not justify it; it has since shrunk to a read-only static site generator (foliate), which is how the page you are reading was built.