Looks cool, but as I have been on this knowledge management / productivity journey like everybody. Here are my findings:
If you are reasonably comfortable with computers / Unix.
- You need to first rely on a directory structures, filenames, plaintext, lists and maybe markdown. Stick with a "File over app", Unix approach.
- Try to sort things with universal concepts: locations, things, people, events, metrics, howtos. A bit like the 5Ws approach.
- Leverage good Unix tools: unix commands, make/justfiles, (rip)grep, git, fzf, etc.
- Do not try to solve the problem through the Web. Because you will end up trying to solve web problems instead of basic knowledge management and productivity issues.
- The smartphone/touchscreen is a major problem, but as with the Web do not try to solve it. Use your file manager or even fzf in termux can be adapted to be reasonably usable on a touchscreen.
Something I have been wondering about is the "backlink" feature. It would be cool to link items/notes together through references.
What I would be looking for is a Unix tools that can scan my text files for references to other files in the hierarchy.
I've been experimenting with Flatnotes (https://github.com/Dullage/flatnotes) for a while and really like the design. No notebooks or even folders, just a single directory with markdown files and decent search & tagging. It feels a lot like what happened to email when we gave up all the up-front structuring with deep hierarchies and just said index it and we'll find it when we need to.
The project is just "good enough" for what I need, and aside from tiny bugs whenever I find a gap I can either work around it or live without. Constraints are a powerful motivator for both creativity and getting stuff done.
Tagging is a strictly more powerful tool than hierarchies, at least with same amount tags vs directories/categories, because an item can be tagged using multiple different tags, but can only be in one directory, unless you create duplicates or symlinks or whatever.
I find the opposite. Being forced to consider hierarchical categorisation leads to a more powerful system in practice. It helps me create a mental reference to the item being stored. And it often causes me to see a better way to form an item in the first place — eg maybe this note is really two notes, maybe this idea can actually just be discarded, etc. Or, on rare occasions, a new item doesn’t fit anywhere (but is important) so I need to tweak the hierarchy itself — and that’s a good thing, once in a while, as it can lead to creative insights about the domain as a whole.
I’ve found tagging systems usually become a kind of dark swamp where things go never to be seen again. The lack of structure means I have little memory of what’s gone into the system, so I end up with too much duplication of ideas and inconsistency of style (mess). All this makes it uninviting and difficult to ‘explore’, so I don’t use it much except as a dumping ground, and the swampiness compounds over time.
Hierarchy doesn’t make sense if you care about multiple dimensions, which I often do with notes.
Sometimes I want to look at all notes relating to c++, sometimes everything related to a personal project. Directories don’t support that without symlinking everything that mentions c++ into a folder for that.
Everything is a hierarchy, even the tags in your mind do belong under some tree/node like imagination. Tag is essentially just what you described "duplicates or symlinks or whatever."
> but can only be in one directory, unless you create duplicates or symlinks or whatever.
Non-sym links do exactly this. Starting with your file in foo;
ln foo/file bar/file
will have the file (really a reference-counted pointer) appear in both directories. It's the same location on disk and there's no overhead until you run out of inodes.
MacOS has had first-class support for file tagging for years: You can add arbitrary tags either from “Save as” dialogs or from Finder, and can then browse by tags in Finder or search through your file system filtered by tags.
Does the native file tagging see much adoption? I'm a lifelong Mac user and tried using the tags when they first launched, but eventually it was too much work and I resorted to using search to find what I'm looking for. Not macOS native search but tools like LaunchBar, Alfred, et al or apps like EasyFind.
I have gone through the whole journey similar to you as well. It is so frustrating to keep searching for that perfect app only to find that every other app lacks something or the other. For the last year or so, I've settled on just plain markdown and the directory structure for hierarchy just like you.
I use my preferred editor (Emacs) to modify the files. I get to use all the functionality like key-bindings, search, version control. I push my notes to my self-hosted Forgejo instance (but it can be Github). Forgejo already has a web-interface the notes and links just work there.
On my laptop, I have mdbook configured to watch the directory and build the notes into static website. So if I just want to go through by notes or read something I can use that as well.
I tried finding solutions to take notes on my phone. But I realised that if its longer than a few lines than I prefer using my laptop anyways. I only ever read my notes on the phone. I've setup the same mdbook behind VPN so I can access it on my phone as well. If I ever want to modify notes on-the-go then I can use the Forgejo web interface as well.
You should check out the HyWiki part of the Emacs Hyperbole hypertext package available from melpa or elpa-devel. It utilizes Org mode for notes and WikiWords to automatically interlink them as you type. The WikiWords work in any kind of text buffer as well, even in programming file comments. It even lets you link to many other kinds of Emacs entities like bookmarks, Org IDs and headings, etc. One command to build a web-based wiki. With a little practice, it might amaze you.
This is exactly the conclusion I came to. I still use Obsidian for a lot of things, but I've developed my own task management software that uses plain text files and fzf for everything: https://codeberg.org/ngp/tsk
It has worked great for me! That being said, sync and mobile usage is still a bit of a sore spot with it. It works beautifully with Termux, but I wish there was some way to slap a basic UI on top of a CLI application for mobile. Tk/Tcl is the closest but there's only options on Android (I'm mostly an iOS user) and even then it's not really ideal. For sync, I at least have a reasonable plan: IMAP4 or git-based sync. The roadmap and tasks for the project are tracked in-repo with itself, so it definitely works.
I'll have to look at Termux GUI again. I've tried poking around with Tasker and it's Termux plugin but it seems so confusing and poorly documented that I'd rather just write a native app at that point lol
If you want some windows-forms-esq app development, I highly recommend checking out Flutter Flow. It isn’t exactly cheap, but I’ve used it for some basic personal apps.
As far as I know, it's the opposite (using Skia to render). Flutter for the web is a second-class citizen, whereas for mobile apps, it's quite productive.
Or, you could just fire up Zim Desktop Wiki [1], a local application you can run on any one of the major OS, and be mostly done with it. It gives you bullet 1, 2, 3[2], 4, and backlinks out of the box.
You get 5 if you are willing to compromise. Since it's just all text files in directory trees, you can sync the tree to your mobile phone (e.g with syncthing) and just use any text editor.
I'll leave my imcomplete mind dump below for posterity but I'll make it known that while writing it, I did realise that what I've invented is a text editor.
----------------------------------
I had an idea for a deeply unix-y note taking software a while ago and I've continued to think it would actually work very well.
Notes could either be an sqlite database (nets you compression, performance, easy backups, etc.) or just text files on the filesystem (easy searching, less lock-in, whatever). The "editor" just creates a new temporary file and opens it in `$EDITOR`. You can optionally name it after, otherwise it's named the current date.
Want linking? Use a filesystem-aware editor. Want syntax highlighting? Use an editor with syntax highlighting. Want to sync your notes? Combine it with Syncthing. Want to search your notes? fzf.
After using Bear.app and Things.app for a while, I've moved firmly in the file over app camp. Apps can be enjoyable to use until you don't have access to the app anymore (platform, longevity,...). Now I'm using Emacs with org files, but could just as well use any tools that favors files. Any novel use case I want, I can script it out in a moment, while still enjoying the benefits of viewing my information on most platforms.
> Something I have been wondering about is the "backlink" feature. It would be cool to link items/notes together through references. What I would be looking for is a Unix tools that can scan my text files for references to other files in the hierarchy
The only thing you need is some kind of structure, like the wiki link format or org link format, and you could 'rgrep' the notes directory for a particular link (vim and emacs can put the result in a buffer).
I'm quite happy with Obsidian on Linux, especially due to great plugin ecosystem.
Granted that it would have been great if the clients were open-source; but Obsidian is one of those non-FOSS apps which are so consumer oriented that they get an exception from me.
I sync the files from Obsidian using syncthing to Android & use Markor for editing. But this wouldn't be possible on iOS.
I just use TexStudio. I don't use proper tex files, but it never complains unless I were to try and compile. But their navigation tree and tabbed interface for multiple files makes organizing things really nice.
I can have multiple instances at once, and it's all managed via files and my filesystem.
Eventually I want to build a web frontend to index and reference my data, but for now the above approach is working well.
Unlike Obsidian, TexStudio is open-source, which I consider a plus.
I've been using this for a couple of years on my home-server.
It's an Obsidian knock-off. It's pretty janky and the documentation is lacking. It's open-source which is nice... But the company behind it is ??? I don't know. They are Chinese but I couldn't find much about them.
I use it because I can self-host it, it has most of the features Obsidian has, and I can use it in a web-browser from anywhere - which is the biggest feature for me that Obsidian lacks.
It seems like they borrowed heavily from Notion, Obsidian and RemNote, as far as I can tell (wouldn’t call it a knock-off though, since there are sooo many apps in this space that you don’t really know who came up with what anymore). But the app doesn’t feel janky to me at first glance, it definitely feels more responsive than Notion and less “slippy” than RemNote. Although it is quite noisy with all the tooltips popping up immediately.
My first impression is that they really wanted to include everything (even RemNote-like spaced-repetition flashcards, Notion-like Databases and of course there has to be AI too) and it seems like they did a pretty decent job at that. I also appreciate that there are so many export options, even for Org-Mode (preserving internal links, images, code-blocks, etc.).
I like that it provides a solid, feature-rich alternative to all the cloud-first, closed-source apps in this space. But it may be too distracting/overwhelming for my use-cases with all the advanced layout capabilities and features though. Tana is a similar all-in-one solution that is really well done (and more innovative than this one), but I always seem to gravitate toward more focussed apps.
I assume you want to self host in order to sync locally, right? If that's the case, you can use Obsidian with git plugin and you get sync + versioning (all what git offers).
Logseq is floss(gnu license, closurescript) and almost perfect for me but it's totally mysterious to me what's stopping it from serving a web interface that serves files on the web server. It's an electron app with an HTTP API but for some reason the web demo only opens files clientside with filesystem API
Anyway, one of these days I'll fork it and make it work for me. I also have a perverted desire to change the serialization format from markdown to XML so I can manipulate the graph with other tools that talk xpath, xquery, basex.
They are doing a rewrite of the app to have a SQLite DB storage for notes instead of markdown files. I think once that is released it will be available in a web browser. Currently in alpha testing, I access the application through the browser.
I also installed it on my computer to give it a try, but then as you mentioned I could not find who are behind it other than it's based in China. So decided to just keep with Obsidian
The first profile has a README (zh-CN, easily translated) introducing themselves as a married couple as well as their career trajectory leading to their current company. Googling the company name leads me to https://www.tianyancha.com/company/3153162387 showing the company’s legal structure, the legal names of the couple, their address, etc. (again, zh-CN but easily translated). Looks like I can view their financial reports too if I have a subscription.
The profiles also link to their social media accounts (on their own dev-focused community).
What more is there to know? At least it’s more than the average ShowHN asking for your email and sometimes credit card. I don’t understand these “couldn’t find much about them” claims.
The link you posted gives me "According to relevant legal regulations, access is temporarily not supported in your current location." and "If your device or the Wi-Fi environment you are in is using a VPN service, please disable it and try again."
I'm not using any VPN. Normal internet from Germany
I vaguely recall this being part of a tit-for-tat thing between China and the anti-Chinese. There have been movements to restrict Chinese access to FOSS, because forking FOSS lowers Chinese dependence on the West, along with (ironic) accusations that the "authoritarian" Chinese are limiting access to Western tech products. I thought there was some sort of legislative or judicial outcome that came out of it, but no luck with a quick google.
-----
U.S. restriction on Chinese use of open-source microchip tech would be hard to enforce - October 13, 2023
> U.S. lawmakers are pressuring the administration of President Joseph Biden to place restrictions on RISC-V to prevent China from benefiting from the technology as it attempts to develop its semiconductor industry.
China’s Use of Foreign Open-Source Software, and How to Counter It - April 2, 2024
> Democratic governments also need to reassess which products should not be made open-source because they’re at risk of being weaponized by malign actors.
Whatever the US did, Europe would do. Anybody in the US or Europe working on a FOSS project with Chinese contributors that they're friendly with? Has anything happened recently?
TianYancha is a corporate data aggregation website, it has nothing to do with FOSS. Your post is such a clumsy attempt to steer the conversation into Anti-Americanism/Westernism. Like really blatant lol.
They open sourced it and you can self hosted. I mean, does it even matter where they are from? Why it’s automatically suspicious when you know the authors are Chinese.
True the origin does not matter, but it would be better to have more transparency about the contributors even if it's an open source tool. Because you can still get injected with a malicious code when an update is pushed.
But I agree we should not categorize according to geolocation, but more transparency would be better irrespective of the location in any project.
for web-browser access I host Obsidian with kasmvnc. It is not ideal for use on the phone, but from a tablet it works, and I can host it with Vivaldi in a webpanel for quick edits.
I keep seeing these note-taking tools pop up again and again. I am heavily invested into Logseq already and the new database version is coming out soon. Unless these tools provide import and export utilities to convert your notes between the most popular applications like Logseq and Obsidian, only new users will use them and people with very few notes who can put in the time to move.
After forcing myself to learn and adapt to some note-taking system, I too didn't find it useful for me yet. But i keep pushing myself through because I still believe that there must be some value that I could take from taking notes. Just I didn't find a system that suits me well...? One thing that i find the real benefit/value from all this learn and adapt is "writing as a way to think" (is it by feynmann?). When doing complicated work, writing really help to ease your cognitive load and help you find the gaps in your line of thought. But, I couldn't find a suitable method when dealing with general/every day note-taking. I still have that "graveyard for thought" problem when writing general notes
What is the benefit of the database version? I like that the backend is in text files. I guess there could be some performance/compression benefits to be had, but given the amount of writing a mortal is likely to do over a lifetime, I am not immediately sold on the idea.
The licensing is kind of strange. Self-hosted syncing is supposedly a paid feature, and there are indeed license checks in the code, but unlike your typical open core product with a separately licensed ee/ directory, this one including license checking code is entirely under AGPL, and FAQ specifically stresses “SiYuan is completely open source”. As such one can patch out the license check (literally a one line change if I’m not mistaken, I only scanned the code) and still completely license-compliant, and one can even distribute binaries of the patched out version, though that would be a major jerk move. So, is that intended? Only selling convenience of prebuilt images?
You guessed it right, only CONVENIENCE of prebuilt images are sold, but due to Apple's developer account restrictions, it's actually better to use the official version if you don't want to spend $100 to build the ios version.
In addition, there are indeed two better-known, unofficial patches with paid restrictions removed, one is an experimental version by a third-party developer, which is submitted upstream after testing the functionality, at github.com/siyuan-community/siyuan, and the other is a patch that simply removes the paid functionality. It's at github.com/EightDoor/siyuan.
Also, there was actually a pr from a community developer to add digital certificates to ensure that unofficial packaged binaries wouldn't use the official cloud service, since the plugin bazaar uses the official cdn, which is actually not a small amount of traffic cost, but that pr was rejected.
My company does exactly this intentionally for everything we sell at our extension store. Everything under LGPL, license checks that you could bypass with not much effort.
It allows selling actually free software and can work well if you do this well.
Yeah I know this is a fairly well-trodden model recommended by FSF (okay, maybe I shouldn’t have called it kinda strange). I’m not confident it works out in most cases, though. So I’m wondering if it’s intentional in this case.
Actually, if you have other examples of this happening in the wild, I'd be quite interested.
The other widespread example I have in mind (which seems to work well) is WordPress extensions with premium features, but I've not encountered this too often neither.
I believe making paid open source code to handle interoperability with something inherently closed / paid (if not outright expensive) is a good approach. Would be users are already used to pay for a related provider anyway, and it's not like the features are so useful without the functionalities of the other provider.
A very cool entertainment one (vs other good examples given so far) that got some nice attention on HN earlier this year [0] is the videogame Shattered Pixel Dungeon. Full GPL3, wonderful fun with a lot of great effort put in and very active ongoing development. You can hack away on it and compile it yourself, but then there are also convenience paid versions on the all the major platforms that seem to do reasonably well.
I know there are some open source Mac apps sold for a couple bucks but you can also build for yourself.
That said, I assumed it’s fairly well-trodden mostly because FSF for as long as I can remember approved/recommended something along the lines of selling builds but completely opening the source. I guess RHEL arguably falls into the camp as well, but they sell more than builds.
The Ardour DAW is kinda like this. Downloads of prebuilt images through the website are paid, but you can download and build yourself from source. (Most Linux users probably have free access via distro package manager anyway.)
I can also recommend Trilium Notes [1], which I have been happily using for years. It's currently in "maintenance mode", which I personally see as a feature (no risk of bloatware).
Self-hosted, great webapp, optional native clients and works offline.
My approach to knowledge management has been super simple. Everything is in git, in a big directory structure i've organically grown over time. I have so many notes.. 7 to 8 thousands of them.
Keeping it in git has allowed me flexibility to use it how i see fit. Sometimes i'm in emacs/nevom. Most the time i'm in sublime text because I have workspaces per project/concept.
Finally, for mobile, I push them all to my own gitea instance. I rarely need access to my notes on mobile but it's been something i've been thinking about more.
If anyone has any recommendations on how to read a directory of text files on android, i'm all ears.
When I was on Android, I used Markor for text editing and Termux for command line git.
For iOS, I now use Working Copy for git commit/push/pull, and Obsidian to edit text, and I like Obsidian's interface a lot for making quick Markdown edits.
I believe Obsidian has an Android version as well, but I have not used it. I'm not sure if there is an Android version for Working Copy but there may be similar GUI git apps.
I’ve been self-hosting it for three months now, and I am happy. I came from Joplin; I lost the offline access, but it’s much more “expressive” and nice, so to speak. I don’t miss any other feature, even in the pro version. Doesn’t depend on other docker containers, I just used authelia for auth, and while it uses its own file format I backup the data volume and it’s possible to export manually in simple markdown. The web UI is responsive for mobile use and yes, I am happy. Fits my case better than the others.
Same here. I am self-hosting it on the web on my VPS instance so I can access it from any device and anywhere in the world. And then, I use a self-hosted S3 instance to sync all my installations on devices, including OSX and Android/iPhone.
Sync has worked wonderfully well.
I felt with the number of features it has, this program simply knocks out Obsidian in comparison. I would say it is more of a Notion equivalent.
I am currently working on a small self-hosted companion for Siyuan that lets users share notes publicly.
What I miss from this type of apps, including notion, is the ability to "inherit" properties from databases.
I would like, for example, to create a superdb with basic task properties (title, deadline) and then create subdatabases that add more project-specific properties.
This, I'll be able to create a single view with all my tasks for the day in a single place, but also add all the info I need for individual tasks.
Unfortunately, for some reason, nobody (that I know of) built something like this into their apps.
Tana has had this for a long time with super tags, and Logseq has it (at least in the test builds of their DB version; I can’t speak to the traditional MD-based version) in their sub-tagging functionality.
I agree that it would be great across the board, though!
Trilium has attribute inheritance and is scriptable. The default task implementation uses templates though, not inheritance. So you would need to script it.
What is the good solution to sync text files? I haven't found a good solution that is open source and self-hosted. Ideally, eventually, my setup should give me real time syncing the same level as multiple people editing the same file. But that is a stretch goal.
I am currently using Syncthing but unfortunately it's client app has been removed from Google Play [1]. I am managing with Syncthing-Fork.
Open source / Free software comes with all the usual benefits:
- you can fork and adapt to your needs
- should the original authors stop developing it or take a direction you don't like but your started depending on it, someone can take over the development of a fork
- you can study how it works
- you can reuse some of the code to build an alternative product
- you can contribute patches if the project accepts them
- if you have to migrate, even if the format is specific, you can at least check how it works
That thing being open source is a big plus, and Obsidian using a standard format is a big plus.
That's like saying "you do not need oracle to use your bytes on disk". I may be mistaken but doesn't obsidian provide a ton of functionality on top of markdown?
No, just a little bit. Your docs remain 98% compatible with other Markdown editors. The functionality is mainly around a.) better UI for editing Markdown, b.) plugin ecosystem (optional), c.) paid sync (optional, and achievable otherwise e.g. SyncThing, your own rsync script, etc), and d.) optional (and very bad) "publish-as-a-website" feature.
The plugins are no small thing though. I replaced Anki with a spaced repetition plugin, now my notes can be flashcards and I don't need to maintain two separate apps.
Also the UI for search and navigation is much better than just a collection of Markdown files. I rolled my own note system using Markdown before this. Obsidian is way better.
If you only use standard Markdown. But the advocacy often focuses on DataView and a thousand other features that are simply not available without Obsidian.
True, but it seems very obvious that if you add a bunch of plugin dependencies (or even one), you are deliberately choosing to forego "standard" Markdown (there's actually no such thing, but roughly speaking).
That's why they're plugins.
I do use the Excalidraw plugin, but nothing much else, and that is why I have an easy time making my Obsidian notes web-accessible (currently, via SilverBullet, but any tool that makes markdown web-editable will work — as long as you don't go nuts with plugins, that is).
Having said that, I think the reason Obsidian "failed" — to the extent that Notion and some others have massively more adoption amongst organizations larger than me and my gray beard – is that they failed to combine their (super awesome) files-and-folders approach with a web editable solution.
They thought - obviously wrongly, in hindsight — that web accessible would be enough.
It's not. It's the 10%, Notion etc cover the 90% (but with fairly bad tradeoffs, they have export and it works, but you can't easily interop with your data where it lives).
But I've had such an easy time making my Obsidian web-editable that I suspect in a few minutes (or months) Obsidian will be like heyyyyy... edit yo vault via web yo — and for free! and then we will all be like woo Obsidian boo Notion!!
The code is indeed entirely AGPL, I looked at license headers in the specific files implementing paid features. I wonder if it’s an oversight. See my other comment https://news.ycombinator.com/item?id=42514304 for details.
It doesn't have to be free everything to qualify for open source. The source and licence are available. You can actually bypass payment if you want to do that as the source ia there.
Yes, if SiYuan stops being developed, you can still get your notes exported as markdown files. Since SiYuan is open-source, you can also use the internal code to parse the JSON format notes.
Yes, your existing installations on your devices, whatever they are, can read your data as long as you use the correct Repo keys that are used to encrypt stuff.
In the worst case, you can deploy the docker instance with older images to keep going on.
Also you need a license for anything work related. Since I code in my work time and my free time I can't separate this clearly. To be compliant I would need a license
Obsidian has built-in support for markdown, images, PDFs, canvas (via JSON Canvas which they developed and open sourced https://obsidian.md/canvas), and others.
For databases, you can add fields/properties both in the markdown frontmatter or in the text and query it via very popular plugins:
Do not rely on the tooling; forced structure might work for some, but it never worked for me. I eventually settled on markdown and structure this myself. As sunshine-o also said, tooling around this can help to retrieve information, but it is not essential if you organize. but this is different for everybody.
I eventually started to use obsidian, but also nextcloud notes, quillpad and webdav to get access to my notes. I am happy, but it took a long time to get to something that worked.
this tool looks 'complicated'... as it needs an infrastructure to operate.
I use folders for my data. a-z. This is the basic directory structure. b->books->management for example.
Does not solve the fundamental problem of a file system that you also could file this under m->management->books
I use recoll to index and find things. https://www.recoll.org/ And yes, I have a 20 TB HDD. Quite a bit of data and media. If someone knows how to make an encrypte mirror of my drive with https://www.opendrive.com/ I would be interested. No incremental backups, if possible I would like to access it like a drive.
For my notes I just use light speed fast nvpy with simplenote.com to synchronized with several computes. https://github.com/cpbotha/nvpy
My notes don't follow the a-z approach but have regular headlines. I use nvpy as a mixture between a to do list and a knowledge database. For to do I have recently thought of using paper cards for kanban, but I am not sure. Suggestions welcome.
I'm a fan of tables with feature parity with Notion! As much as everyone says "we should have one step above Excel somehow", there aren't too many apps that have put a lot of thought into that.
I am using markdown files in a private GitHub repo for personal knowledge management. For me this works just fine... Is there any feature you think I might miss from tools like these?
This is what I do with my Obsidian repo. Feels like best of both worlds to me. I have my files in open format in my repo. And I use Obsidian primarily for the editing experience. If I want to quickly capture a thought on my phone, I use the GitHub app to create an issue in that repo and process it later.
I see the value in backlinks (I assume you mean backlinks between said markdown files, right?), and graphview might come in handy once in a while...but am not seeing the value brought by databases here? Unless the db becomes part of a top/management layer for said files, which adds other things like maybe possibly faster search or easier query of content (that is, easier than find/grep, etc.)? Not knocking if DBs can be used, with all apologies, am genuinely curious how DBs could help here? (Context is that if this person is using simple markdown files in folders/repos, then they probably don't need/want more than that and typical file management tools)
When people mean DBs in these note taking tools, they are usually referring to the Notion like databases: a lightweight Airtable-like table allowing a few data manipulations and analysis. Basically a CSV file with a nice UI on top if you will
The number of apps that offer note-taking functionality is exploding. It solidifies Emacs' community's point of view that all these custom tailored solutions become a limiting factor for many. But to learn Emacs and Elisp is not trivial.
* Any particular resources you followed to learn Elisp.
* What sort of time investment did it require for you to build your own system based out of Emacs?
Have been hosting this with Docker for ~2years. It's quite stable now except from some expected minor bugs on firefox (because they "don't have time to maintain").
I believe convenience and actually wanting to give something back to the creators would have many people pay for pro features, since it’s actually “Pay once, use for life”, which is a rare and welcome sight in this subscription-flooded world. And it still leaves people with low income the option to (legally?) circumvent payment. Not sure how sustainable this is as a business model, but I think it’s pretty nice compared to being forced into continuous payment.
I tried to give it a shot a few months back, but the lack of vim bindings were a blocker for me, and seems like it isn't planned yet. Hope to try it sometime in the future.
Looks cool, but as I have been on this knowledge management / productivity journey like everybody. Here are my findings:
If you are reasonably comfortable with computers / Unix.
- You need to first rely on a directory structures, filenames, plaintext, lists and maybe markdown. Stick with a "File over app", Unix approach.
- Try to sort things with universal concepts: locations, things, people, events, metrics, howtos. A bit like the 5Ws approach.
- Leverage good Unix tools: unix commands, make/justfiles, (rip)grep, git, fzf, etc.
- Do not try to solve the problem through the Web. Because you will end up trying to solve web problems instead of basic knowledge management and productivity issues.
- The smartphone/touchscreen is a major problem, but as with the Web do not try to solve it. Use your file manager or even fzf in termux can be adapted to be reasonably usable on a touchscreen.
Something I have been wondering about is the "backlink" feature. It would be cool to link items/notes together through references. What I would be looking for is a Unix tools that can scan my text files for references to other files in the hierarchy.
I've been experimenting with Flatnotes (https://github.com/Dullage/flatnotes) for a while and really like the design. No notebooks or even folders, just a single directory with markdown files and decent search & tagging. It feels a lot like what happened to email when we gave up all the up-front structuring with deep hierarchies and just said index it and we'll find it when we need to.
The project is just "good enough" for what I need, and aside from tiny bugs whenever I find a gap I can either work around it or live without. Constraints are a powerful motivator for both creativity and getting stuff done.
Tagging is a strictly more powerful tool than hierarchies, at least with same amount tags vs directories/categories, because an item can be tagged using multiple different tags, but can only be in one directory, unless you create duplicates or symlinks or whatever.
I find the opposite. Being forced to consider hierarchical categorisation leads to a more powerful system in practice. It helps me create a mental reference to the item being stored. And it often causes me to see a better way to form an item in the first place — eg maybe this note is really two notes, maybe this idea can actually just be discarded, etc. Or, on rare occasions, a new item doesn’t fit anywhere (but is important) so I need to tweak the hierarchy itself — and that’s a good thing, once in a while, as it can lead to creative insights about the domain as a whole.
I’ve found tagging systems usually become a kind of dark swamp where things go never to be seen again. The lack of structure means I have little memory of what’s gone into the system, so I end up with too much duplication of ideas and inconsistency of style (mess). All this makes it uninviting and difficult to ‘explore’, so I don’t use it much except as a dumping ground, and the swampiness compounds over time.
Hierarchy doesn’t make sense if you care about multiple dimensions, which I often do with notes.
Sometimes I want to look at all notes relating to c++, sometimes everything related to a personal project. Directories don’t support that without symlinking everything that mentions c++ into a folder for that.
Everything is a hierarchy, even the tags in your mind do belong under some tree/node like imagination. Tag is essentially just what you described "duplicates or symlinks or whatever."
> but can only be in one directory, unless you create duplicates or symlinks or whatever.
Non-sym links do exactly this. Starting with your file in foo;
will have the file (really a reference-counted pointer) appear in both directories. It's the same location on disk and there's no overhead until you run out of inodes.I really like tagging but somehow the concept could never become central to any filesystem:
- I believe BeOS tried,
- MS tried with WinFS but cancelled it in Windows Vista,
- I am sure some cloud storage service bet on it but can't cite any.
Actually tagging is probably mainly successful in cases where we can reliably automate the tagging such as email or photos.
MacOS has had first-class support for file tagging for years: You can add arbitrary tags either from “Save as” dialogs or from Finder, and can then browse by tags in Finder or search through your file system filtered by tags.
Does the native file tagging see much adoption? I'm a lifelong Mac user and tried using the tags when they first launched, but eventually it was too much work and I resorted to using search to find what I'm looking for. Not macOS native search but tools like LaunchBar, Alfred, et al or apps like EasyFind.
Colours, yes – but then again I worked in a print shop that'd been using Apple since MacOS 8, so they might be used to those quirks.
I like tagging for bookmarks in Firefox and derivatives of Firefox.
I have gone through the whole journey similar to you as well. It is so frustrating to keep searching for that perfect app only to find that every other app lacks something or the other. For the last year or so, I've settled on just plain markdown and the directory structure for hierarchy just like you.
I use my preferred editor (Emacs) to modify the files. I get to use all the functionality like key-bindings, search, version control. I push my notes to my self-hosted Forgejo instance (but it can be Github). Forgejo already has a web-interface the notes and links just work there.
On my laptop, I have mdbook configured to watch the directory and build the notes into static website. So if I just want to go through by notes or read something I can use that as well.
I tried finding solutions to take notes on my phone. But I realised that if its longer than a few lines than I prefer using my laptop anyways. I only ever read my notes on the phone. I've setup the same mdbook behind VPN so I can access it on my phone as well. If I ever want to modify notes on-the-go then I can use the Forgejo web interface as well.
You should check out the HyWiki part of the Emacs Hyperbole hypertext package available from melpa or elpa-devel. It utilizes Org mode for notes and WikiWords to automatically interlink them as you type. The WikiWords work in any kind of text buffer as well, even in programming file comments. It even lets you link to many other kinds of Emacs entities like bookmarks, Org IDs and headings, etc. One command to build a web-based wiki. With a little practice, it might amaze you.
This is exactly the conclusion I came to. I still use Obsidian for a lot of things, but I've developed my own task management software that uses plain text files and fzf for everything: https://codeberg.org/ngp/tsk
It has worked great for me! That being said, sync and mobile usage is still a bit of a sore spot with it. It works beautifully with Termux, but I wish there was some way to slap a basic UI on top of a CLI application for mobile. Tk/Tcl is the closest but there's only options on Android (I'm mostly an iOS user) and even then it's not really ideal. For sync, I at least have a reasonable plan: IMAP4 or git-based sync. The roadmap and tasks for the project are tracked in-repo with itself, so it definitely works.
> but I wish there was some way to slap a basic UI on top of a CLI application for mobile
I have only relied on fzf so far but have played with the various components of the Termux:API [0] for basic UIs, and it works well.
There is also Termux:GUI [1] with Bash bindings which allows to build more advanced UIs.
- [0] https://wiki.termux.com/wiki/Termux:API
- [1] https://github.com/termux/termux-gui
I'll have to look at Termux GUI again. I've tried poking around with Tasker and it's Termux plugin but it seems so confusing and poorly documented that I'd rather just write a native app at that point lol
If you want some windows-forms-esq app development, I highly recommend checking out Flutter Flow. It isn’t exactly cheap, but I’ve used it for some basic personal apps.
I kinda doubt FlutterFlow would meet my needs, it looks like it's primarily build on web technologies which is a nonstarter for this.
As far as I know, it's the opposite (using Skia to render). Flutter for the web is a second-class citizen, whereas for mobile apps, it's quite productive.
FlutterFlow is just the UI to build Flutter apps, which yes, is built on web-tech. However, Flutter runs natively on mobile devices.
Or, you could just fire up Zim Desktop Wiki [1], a local application you can run on any one of the major OS, and be mostly done with it. It gives you bullet 1, 2, 3[2], 4, and backlinks out of the box.
You get 5 if you are willing to compromise. Since it's just all text files in directory trees, you can sync the tree to your mobile phone (e.g with syncthing) and just use any text editor.
[1] https://zim-wiki.org/index.html [2] git with an add-on.
I'll leave my imcomplete mind dump below for posterity but I'll make it known that while writing it, I did realise that what I've invented is a text editor.
----------------------------------
I had an idea for a deeply unix-y note taking software a while ago and I've continued to think it would actually work very well.
Notes could either be an sqlite database (nets you compression, performance, easy backups, etc.) or just text files on the filesystem (easy searching, less lock-in, whatever). The "editor" just creates a new temporary file and opens it in `$EDITOR`. You can optionally name it after, otherwise it's named the current date.
Want linking? Use a filesystem-aware editor. Want syntax highlighting? Use an editor with syntax highlighting. Want to sync your notes? Combine it with Syncthing. Want to search your notes? fzf.
I do this except with git, fzf, and a simple CLI tool:
https://codeberg.org/ngp/tsk
Contributions welcome :)
After using Bear.app and Things.app for a while, I've moved firmly in the file over app camp. Apps can be enjoyable to use until you don't have access to the app anymore (platform, longevity,...). Now I'm using Emacs with org files, but could just as well use any tools that favors files. Any novel use case I want, I can script it out in a moment, while still enjoying the benefits of viewing my information on most platforms.
> Something I have been wondering about is the "backlink" feature. It would be cool to link items/notes together through references. What I would be looking for is a Unix tools that can scan my text files for references to other files in the hierarchy
The only thing you need is some kind of structure, like the wiki link format or org link format, and you could 'rgrep' the notes directory for a particular link (vim and emacs can put the result in a buffer).
For backlinks, have a look at Andy Matuschak's note-link-janitor: https://github.com/andymatuschak/note-link-janitor/
I wrote my own version of it a few years ago, but have moved on, and currently use LogSeq instead: https://github.com/cdaven/noteexplorer
Great suggestions.
I'm quite happy with Obsidian on Linux, especially due to great plugin ecosystem.
Granted that it would have been great if the clients were open-source; but Obsidian is one of those non-FOSS apps which are so consumer oriented that they get an exception from me.
I sync the files from Obsidian using syncthing to Android & use Markor for editing. But this wouldn't be possible on iOS.
Obsidian lets you live on top of the filesystem structure using markdown files, and it gives you links, tags, topic clouds, and more.
If you use it with reasonable conventions, you can compile and deploy a web version with ghost or zola or something automatically run in CI.
It's backed up, synced, and comes with desktop and mobile apps. It has vim key bindings and git plugins.
It's a pretty sweet system that works nicely with Unix, and it adapts to multiple navigational and organizational paradigms.
I've tried a lot of things, and nothing has ever come close to Obsidian.
All falls Short when it comes to cross referencing, sub notes , rich capabilities.
For a long while I was trying to keep text notes in a structured set of directories, Tombo.exe made that a lot easier/nicer to use:
https://openhub.net/p/p_7697
sadly, it looks to have dropped off the 'net.
>Something I have been wondering about is the "backlink" feature.
I trust this is why everyone tries to rely on network-like structure.
org-roam gives you backlinks. directory of text files.
I just use TexStudio. I don't use proper tex files, but it never complains unless I were to try and compile. But their navigation tree and tabbed interface for multiple files makes organizing things really nice.
I can have multiple instances at once, and it's all managed via files and my filesystem.
Eventually I want to build a web frontend to index and reference my data, but for now the above approach is working well.
Unlike Obsidian, TexStudio is open-source, which I consider a plus.
I've been using this for a couple of years on my home-server.
It's an Obsidian knock-off. It's pretty janky and the documentation is lacking. It's open-source which is nice... But the company behind it is ??? I don't know. They are Chinese but I couldn't find much about them.
I use it because I can self-host it, it has most of the features Obsidian has, and I can use it in a web-browser from anywhere - which is the biggest feature for me that Obsidian lacks.
It seems like they borrowed heavily from Notion, Obsidian and RemNote, as far as I can tell (wouldn’t call it a knock-off though, since there are sooo many apps in this space that you don’t really know who came up with what anymore). But the app doesn’t feel janky to me at first glance, it definitely feels more responsive than Notion and less “slippy” than RemNote. Although it is quite noisy with all the tooltips popping up immediately.
My first impression is that they really wanted to include everything (even RemNote-like spaced-repetition flashcards, Notion-like Databases and of course there has to be AI too) and it seems like they did a pretty decent job at that. I also appreciate that there are so many export options, even for Org-Mode (preserving internal links, images, code-blocks, etc.).
I like that it provides a solid, feature-rich alternative to all the cloud-first, closed-source apps in this space. But it may be too distracting/overwhelming for my use-cases with all the advanced layout capabilities and features though. Tana is a similar all-in-one solution that is really well done (and more innovative than this one), but I always seem to gravitate toward more focussed apps.
I assume you want to self host in order to sync locally, right? If that's the case, you can use Obsidian with git plugin and you get sync + versioning (all what git offers).
I use syncthing to sync my obsidian vault which is also a git repository between linux, mac and windows.
Works great!
Which git plugin do you like? And have you used the plugin from more than one machine to a repo more than one machine also syncs with?
Sync is premium since some versions. Before it was free, you had only to provide a S3 bucket or a webdav location
Logseq is floss(gnu license, closurescript) and almost perfect for me but it's totally mysterious to me what's stopping it from serving a web interface that serves files on the web server. It's an electron app with an HTTP API but for some reason the web demo only opens files clientside with filesystem API
Anyway, one of these days I'll fork it and make it work for me. I also have a perverted desire to change the serialization format from markdown to XML so I can manipulate the graph with other tools that talk xpath, xquery, basex.
They are doing a rewrite of the app to have a SQLite DB storage for notes instead of markdown files. I think once that is released it will be available in a web browser. Currently in alpha testing, I access the application through the browser.
Why are you not using Obsidian itself? Anything wrong with it?
I also installed it on my computer to give it a try, but then as you mentioned I could not find who are behind it other than it's based in China. So decided to just keep with Obsidian
Took me like a minute to find out who are behind it. Committers on GitHub project point to two profiles:
- https://github.com/88250
- https://github.com/Vanessa219
The first profile has a README (zh-CN, easily translated) introducing themselves as a married couple as well as their career trajectory leading to their current company. Googling the company name leads me to https://www.tianyancha.com/company/3153162387 showing the company’s legal structure, the legal names of the couple, their address, etc. (again, zh-CN but easily translated). Looks like I can view their financial reports too if I have a subscription.
The profiles also link to their social media accounts (on their own dev-focused community).
What more is there to know? At least it’s more than the average ShowHN asking for your email and sometimes credit card. I don’t understand these “couldn’t find much about them” claims.
The link you posted gives me "According to relevant legal regulations, access is temporarily not supported in your current location." and "If your device or the Wi-Fi environment you are in is using a VPN service, please disable it and try again."
I'm not using any VPN. Normal internet from Germany
It is also unavailable from Canada.
Same in the US.
And Australia.
I vaguely recall this being part of a tit-for-tat thing between China and the anti-Chinese. There have been movements to restrict Chinese access to FOSS, because forking FOSS lowers Chinese dependence on the West, along with (ironic) accusations that the "authoritarian" Chinese are limiting access to Western tech products. I thought there was some sort of legislative or judicial outcome that came out of it, but no luck with a quick google.
-----
U.S. restriction on Chinese use of open-source microchip tech would be hard to enforce - October 13, 2023
> U.S. lawmakers are pressuring the administration of President Joseph Biden to place restrictions on RISC-V to prevent China from benefiting from the technology as it attempts to develop its semiconductor industry.
https://thechinaproject.com/2023/10/13/u-s-bar-on-chinese-us...
-----
China’s Use of Foreign Open-Source Software, and How to Counter It - April 2, 2024
> Democratic governments also need to reassess which products should not be made open-source because they’re at risk of being weaponized by malign actors.
https://chinaobservers.eu/chinas-use-of-foreign-open-source-...
-----
Whatever the US did, Europe would do. Anybody in the US or Europe working on a FOSS project with Chinese contributors that they're friendly with? Has anything happened recently?
TianYancha is a corporate data aggregation website, it has nothing to do with FOSS. Your post is such a clumsy attempt to steer the conversation into Anti-Americanism/Westernism. Like really blatant lol.
Okay yeah appears to be heavily geolocked. Still a lot of information otherwise.
They open sourced it and you can self hosted. I mean, does it even matter where they are from? Why it’s automatically suspicious when you know the authors are Chinese.
True the origin does not matter, but it would be better to have more transparency about the contributors even if it's an open source tool. Because you can still get injected with a malicious code when an update is pushed.
But I agree we should not categorize according to geolocation, but more transparency would be better irrespective of the location in any project.
It's this D and Vanessa - I think this might be Nicknames. But I don't see how they should be more transparent.
They also have a forum and they answer quite quickly
There is source code. Investigate it, don't be a lazy bum.
for web-browser access I host Obsidian with kasmvnc. It is not ideal for use on the phone, but from a tablet it works, and I can host it with Vivaldi in a webpanel for quick edits.
I keep seeing these note-taking tools pop up again and again. I am heavily invested into Logseq already and the new database version is coming out soon. Unless these tools provide import and export utilities to convert your notes between the most popular applications like Logseq and Obsidian, only new users will use them and people with very few notes who can put in the time to move.
Creating a note taking, personal productivity app is an elaborate but common procrastination technique for developers.
After forcing myself to learn and adapt to some note-taking system, I too didn't find it useful for me yet. But i keep pushing myself through because I still believe that there must be some value that I could take from taking notes. Just I didn't find a system that suits me well...? One thing that i find the real benefit/value from all this learn and adapt is "writing as a way to think" (is it by feynmann?). When doing complicated work, writing really help to ease your cognitive load and help you find the gaps in your line of thought. But, I couldn't find a suitable method when dealing with general/every day note-taking. I still have that "graveyard for thought" problem when writing general notes
right up there with creating your own personal website
At least you didn't create a new language for your Personal Home Page and even later rewrote it to be some sort of general hypertext processor.
I feel exposed lol
What is the benefit of the database version? I like that the backend is in text files. I guess there could be some performance/compression benefits to be had, but given the amount of writing a mortal is likely to do over a lifetime, I am not immediately sold on the idea.
The licensing is kind of strange. Self-hosted syncing is supposedly a paid feature, and there are indeed license checks in the code, but unlike your typical open core product with a separately licensed ee/ directory, this one including license checking code is entirely under AGPL, and FAQ specifically stresses “SiYuan is completely open source”. As such one can patch out the license check (literally a one line change if I’m not mistaken, I only scanned the code) and still completely license-compliant, and one can even distribute binaries of the patched out version, though that would be a major jerk move. So, is that intended? Only selling convenience of prebuilt images?
You guessed it right, only CONVENIENCE of prebuilt images are sold, but due to Apple's developer account restrictions, it's actually better to use the official version if you don't want to spend $100 to build the ios version.
In addition, there are indeed two better-known, unofficial patches with paid restrictions removed, one is an experimental version by a third-party developer, which is submitted upstream after testing the functionality, at github.com/siyuan-community/siyuan, and the other is a patch that simply removes the paid functionality. It's at github.com/EightDoor/siyuan.
Also, there was actually a pr from a community developer to add digital certificates to ensure that unofficial packaged binaries wouldn't use the official cloud service, since the plugin bazaar uses the official cdn, which is actually not a small amount of traffic cost, but that pr was rejected.
My company does exactly this intentionally for everything we sell at our extension store. Everything under LGPL, license checks that you could bypass with not much effort.
It allows selling actually free software and can work well if you do this well.
Yeah I know this is a fairly well-trodden model recommended by FSF (okay, maybe I shouldn’t have called it kinda strange). I’m not confident it works out in most cases, though. So I’m wondering if it’s intentional in this case.
> I know this is a fairly well-trodden model
Actually, if you have other examples of this happening in the wild, I'd be quite interested.
The other widespread example I have in mind (which seems to work well) is WordPress extensions with premium features, but I've not encountered this too often neither.
Edit: thanks all for your examples :-)
I want to add a niche example. The famous Thunderbird addon owl for exchange (https://addons.thunderbird.net/en-US/thunderbird/addon/owl-f...) where you can open the source of the addon and change the licence checks to always return true.
I believe making paid open source code to handle interoperability with something inherently closed / paid (if not outright expensive) is a good approach. Would be users are already used to pay for a related provider anyway, and it's not like the features are so useful without the functionalities of the other provider.
A very cool entertainment one (vs other good examples given so far) that got some nice attention on HN earlier this year [0] is the videogame Shattered Pixel Dungeon. Full GPL3, wonderful fun with a lot of great effort put in and very active ongoing development. You can hack away on it and compile it yourself, but then there are also convenience paid versions on the all the major platforms that seem to do reasonably well.
----
0: https://news.ycombinator.com/item?id=39773641
I know there are some open source Mac apps sold for a couple bucks but you can also build for yourself.
That said, I assumed it’s fairly well-trodden mostly because FSF for as long as I can remember approved/recommended something along the lines of selling builds but completely opening the source. I guess RHEL arguably falls into the camp as well, but they sell more than builds.
The Ardour DAW is kinda like this. Downloads of prebuilt images through the website are paid, but you can download and build yourself from source. (Most Linux users probably have free access via distro package manager anyway.)
If I remember correctly, other examples of OSS code with premium addons and license check, on the top of my head: Bitwarden, Metabase, EmailEngine
> though that would be a major jerk move
Would it? If they don't want people to do it they could just use a proprietary license.
I can also recommend Trilium Notes [1], which I have been happily using for years. It's currently in "maintenance mode", which I personally see as a feature (no risk of bloatware).
Self-hosted, great webapp, optional native clients and works offline.
https://github.com/zadam/trilium
As another fan of Trilium Notes for years, I just want to add that there is now also an active community fork continuing the work on the tool.
https://github.com/TriliumNext/Notes
I can second the recommendations i've read here.
My approach to knowledge management has been super simple. Everything is in git, in a big directory structure i've organically grown over time. I have so many notes.. 7 to 8 thousands of them.
Keeping it in git has allowed me flexibility to use it how i see fit. Sometimes i'm in emacs/nevom. Most the time i'm in sublime text because I have workspaces per project/concept.
Finally, for mobile, I push them all to my own gitea instance. I rarely need access to my notes on mobile but it's been something i've been thinking about more.
If anyone has any recommendations on how to read a directory of text files on android, i'm all ears.
When I was on Android, I used Markor for text editing and Termux for command line git.
For iOS, I now use Working Copy for git commit/push/pull, and Obsidian to edit text, and I like Obsidian's interface a lot for making quick Markdown edits.
I believe Obsidian has an Android version as well, but I have not used it. I'm not sure if there is an Android version for Working Copy but there may be similar GUI git apps.
I’ve been self-hosting it for three months now, and I am happy. I came from Joplin; I lost the offline access, but it’s much more “expressive” and nice, so to speak. I don’t miss any other feature, even in the pro version. Doesn’t depend on other docker containers, I just used authelia for auth, and while it uses its own file format I backup the data volume and it’s possible to export manually in simple markdown. The web UI is responsive for mobile use and yes, I am happy. Fits my case better than the others.
Same here. I am self-hosting it on the web on my VPS instance so I can access it from any device and anywhere in the world. And then, I use a self-hosted S3 instance to sync all my installations on devices, including OSX and Android/iPhone. Sync has worked wonderfully well.
I felt with the number of features it has, this program simply knocks out Obsidian in comparison. I would say it is more of a Notion equivalent. I am currently working on a small self-hosted companion for Siyuan that lets users share notes publicly.
What I miss from this type of apps, including notion, is the ability to "inherit" properties from databases.
I would like, for example, to create a superdb with basic task properties (title, deadline) and then create subdatabases that add more project-specific properties.
This, I'll be able to create a single view with all my tasks for the day in a single place, but also add all the info I need for individual tasks.
Unfortunately, for some reason, nobody (that I know of) built something like this into their apps.
It sounds like you're looking for a project/task management tool rather than a knowledge base?
Most complex tools have task types, some are customizable, some not - e.g. Jira would do what you describe?
I think Tana can do this with the "supertags" feature.
https://tana.inc/docs/supertags
Tana has had this for a long time with super tags, and Logseq has it (at least in the test builds of their DB version; I can’t speak to the traditional MD-based version) in their sub-tagging functionality.
I agree that it would be great across the board, though!
Trilium has attribute inheritance and is scriptable. The default task implementation uses templates though, not inheritance. So you would need to script it.
What is the good solution to sync text files? I haven't found a good solution that is open source and self-hosted. Ideally, eventually, my setup should give me real time syncing the same level as multiple people editing the same file. But that is a stretch goal.
I am currently using Syncthing but unfortunately it's client app has been removed from Google Play [1]. I am managing with Syncthing-Fork.
[1] https://github.com/syncthing/syncthing-android/issues/2064
I use org-mode for note taking and personal knowledge base.
In my notes.org file, I collect all kind of snippets, knowledge, ideas, how-tos, and such stuff.
Outlines make it possible to hide parts of the text in the buffer.
A headline starts with one or more stars. A heading has one star, a sub-heading two, etc.
Use the linking feature in ordmode to link items/notes together through references.
Stick with a "File over app", Unix approach. You need to first rely on a directory structures, filenames, plaintext, lists and orgmode.
Leverage Unix tools: unix commands, (rip)grep, git, fzf, etc.
This looks nice but nearly identical to Obsidian. How does it differ from Obsidian's features?
It seems to be open source, which is a big plus compared to obsidian
Why is it a big plus, genuine question? You do not need Obsidian to use your Markdown written content and are not vendor locked in as such.
Open source / Free software comes with all the usual benefits:
- you can fork and adapt to your needs
- should the original authors stop developing it or take a direction you don't like but your started depending on it, someone can take over the development of a fork
- you can study how it works
- you can reuse some of the code to build an alternative product
- you can contribute patches if the project accepts them
- if you have to migrate, even if the format is specific, you can at least check how it works
That thing being open source is a big plus, and Obsidian using a standard format is a big plus.
The same reason it is a plus for any other open source software. You can modify it and fork it if it's discontinued.
That's like saying "you do not need oracle to use your bytes on disk". I may be mistaken but doesn't obsidian provide a ton of functionality on top of markdown?
No, just a little bit. Your docs remain 98% compatible with other Markdown editors. The functionality is mainly around a.) better UI for editing Markdown, b.) plugin ecosystem (optional), c.) paid sync (optional, and achievable otherwise e.g. SyncThing, your own rsync script, etc), and d.) optional (and very bad) "publish-as-a-website" feature.
The plugins are no small thing though. I replaced Anki with a spaced repetition plugin, now my notes can be flashcards and I don't need to maintain two separate apps.
Also the UI for search and navigation is much better than just a collection of Markdown files. I rolled my own note system using Markdown before this. Obsidian is way better.
If you only use standard Markdown. But the advocacy often focuses on DataView and a thousand other features that are simply not available without Obsidian.
True, but it seems very obvious that if you add a bunch of plugin dependencies (or even one), you are deliberately choosing to forego "standard" Markdown (there's actually no such thing, but roughly speaking).
That's why they're plugins.
I do use the Excalidraw plugin, but nothing much else, and that is why I have an easy time making my Obsidian notes web-accessible (currently, via SilverBullet, but any tool that makes markdown web-editable will work — as long as you don't go nuts with plugins, that is).
Having said that, I think the reason Obsidian "failed" — to the extent that Notion and some others have massively more adoption amongst organizations larger than me and my gray beard – is that they failed to combine their (super awesome) files-and-folders approach with a web editable solution.
They thought - obviously wrongly, in hindsight — that web accessible would be enough.
It's not. It's the 10%, Notion etc cover the 90% (but with fairly bad tradeoffs, they have export and it works, but you can't easily interop with your data where it lives).
But I've had such an easy time making my Obsidian web-editable that I suspect in a few minutes (or months) Obsidian will be like heyyyyy... edit yo vault via web yo — and for free! and then we will all be like woo Obsidian boo Notion!!
But we'll see
So not much of a value proposition then is what you are saying? Why use it at all?
Well I'm not an evangelist of the app or anything, but I use it because it has the best UI I've found so far for editing markdown collections.
Not fully open source. It seems you need to pay to get the privilege to "Export PDF/Image with watermark".
https://b3log.org/siyuan/en/pricing.html
The code is indeed entirely AGPL, I looked at license headers in the specific files implementing paid features. I wonder if it’s an oversight. See my other comment https://news.ycombinator.com/item?id=42514304 for details.
It doesn't have to be free everything to qualify for open source. The source and licence are available. You can actually bypass payment if you want to do that as the source ia there.
Right but it's a bit unusual. I was expecting the "pro" features to be in a separate repo or something.
I don't think I would even bother to update a local patch on every releases. I'll just use another foss app without silly "pro" features.
The file format isn't markdown, instead a proprietary format in JSON
I'm not an Obsidian user, but I agree with their "file over app" philosophy.[1]
If SiYuan stops being developed, are the files still readable/parse-able?
[1]: https://stephango.com/file-over-app
Yes, if SiYuan stops being developed, you can still get your notes exported as markdown files. Since SiYuan is open-source, you can also use the internal code to parse the JSON format notes.
Yes, your existing installations on your devices, whatever they are, can read your data as long as you use the correct Repo keys that are used to encrypt stuff. In the worst case, you can deploy the docker instance with older images to keep going on.
It's open source, so yes, and you can export files to Markdown anyway.
Is it not open source?
It locks your efforts to the vendor/project which is different to Obsidian
It is still open source, not proprietary, and you can export files to Markdown anyway.
Obsidian sync feature is paid service, this project you can setup your own service
There are free alternatives, e.g https://github.com/vrtmrz/obsidian-livesync
Also you need a license for anything work related. Since I code in my work time and my free time I can't separate this clearly. To be compliant I would need a license
I use Syncthing with Obsidian. Free and open source.
You can just sync Obsidian with nextcloud, Dropbox or whatever
Obsidian has a free git plugin.
Its editor is fully WYSIWYG, so it does not switch to raw Markdown when you want to modify something.
Isn't Obsidian limited to markdown files? This uses a different format so it can add more features like databases.
No, Obsidian is quite more powerful.
Obsidian has built-in support for markdown, images, PDFs, canvas (via JSON Canvas which they developed and open sourced https://obsidian.md/canvas), and others.
For databases, you can add fields/properties both in the markdown frontmatter or in the text and query it via very popular plugins:
https://github.com/blacksmithgu/obsidian-dataview
There are tons of community plugins that support all kind of stuff: tasks, kanban, LLM/Copilot, graph analysis of links, charts.
It can also be extended in JS, both writing your own plugins or via a few plugins that allow limited JS support.
---
Obsidian is actually quite good as a NoCode prototyping platform for personal apps :-)
E.g. CRUD:
- Use templates, via Templater: to define the content of your data
- Use links and tags to define relations and connections
- Use dataview or graphs for views
- There are even plugins to define buttons and the actions they perform, if you need commands
Do not rely on the tooling; forced structure might work for some, but it never worked for me. I eventually settled on markdown and structure this myself. As sunshine-o also said, tooling around this can help to retrieve information, but it is not essential if you organize. but this is different for everybody.
I eventually started to use obsidian, but also nextcloud notes, quillpad and webdav to get access to my notes. I am happy, but it took a long time to get to something that worked.
this tool looks 'complicated'... as it needs an infrastructure to operate.
I use folders for my data. a-z. This is the basic directory structure. b->books->management for example.
Does not solve the fundamental problem of a file system that you also could file this under m->management->books
I use recoll to index and find things. https://www.recoll.org/ And yes, I have a 20 TB HDD. Quite a bit of data and media. If someone knows how to make an encrypte mirror of my drive with https://www.opendrive.com/ I would be interested. No incremental backups, if possible I would like to access it like a drive.
For my notes I just use light speed fast nvpy with simplenote.com to synchronized with several computes. https://github.com/cpbotha/nvpy
My notes don't follow the a-z approach but have regular headlines. I use nvpy as a mixture between a to do list and a knowledge database. For to do I have recently thought of using paper cards for kanban, but I am not sure. Suggestions welcome.
> Does not solve the fundamental problem of a file system that you also could file this under m->management->books
You can use symbolic links to connect directories and file to multiple parents.
I know. But this is asking and would get messy very fast. And what if an item has 10 attributes?
John Doe has Birthday on December 12th would have at least 5 already.
I'm a fan of tables with feature parity with Notion! As much as everyone says "we should have one step above Excel somehow", there aren't too many apps that have put a lot of thought into that.
I am using markdown files in a private GitHub repo for personal knowledge management. For me this works just fine... Is there any feature you think I might miss from tools like these?
This is what I do with my Obsidian repo. Feels like best of both worlds to me. I have my files in open format in my repo. And I use Obsidian primarily for the editing experience. If I want to quickly capture a thought on my phone, I use the GitHub app to create an issue in that repo and process it later.
Graph view, backlinks and databases.
I see the value in backlinks (I assume you mean backlinks between said markdown files, right?), and graphview might come in handy once in a while...but am not seeing the value brought by databases here? Unless the db becomes part of a top/management layer for said files, which adds other things like maybe possibly faster search or easier query of content (that is, easier than find/grep, etc.)? Not knocking if DBs can be used, with all apologies, am genuinely curious how DBs could help here? (Context is that if this person is using simple markdown files in folders/repos, then they probably don't need/want more than that and typical file management tools)
When people mean DBs in these note taking tools, they are usually referring to the Notion like databases: a lightweight Airtable-like table allowing a few data manipulations and analysis. Basically a CSV file with a nice UI on top if you will
Ah-ah, Ok, i can see that something like that could help. Thanks for the clarification!
org-roam is pretty good. I came from Obsidian and wanted to build out some extra personal features for Bible notes.
I made the switch to org-roam because i wanted to use elisp to develop instead of JS obsidian plugins.
It took me a while to get over the learning curve but I’m very happy with it
The number of apps that offer note-taking functionality is exploding. It solidifies Emacs' community's point of view that all these custom tailored solutions become a limiting factor for many. But to learn Emacs and Elisp is not trivial.
* Any particular resources you followed to learn Elisp.
* What sort of time investment did it require for you to build your own system based out of Emacs?
* How are you syncing your files?
Have been hosting this with Docker for ~2years. It's quite stable now except from some expected minor bugs on firefox (because they "don't have time to maintain").
A wiki. Am I mistaken or is the term just not in trend any more?
Not bad ! Is there way to migrate existing evernote notebooks/notes to this ?
It seems AGPL but also has a paid tier which makes me wonder what their business model is.
Is it only the hassle of self hosting that stops potential customers from having the paid features for free?
I believe convenience and actually wanting to give something back to the creators would have many people pay for pro features, since it’s actually “Pay once, use for life”, which is a rare and welcome sight in this subscription-flooded world. And it still leaves people with low income the option to (legally?) circumvent payment. Not sure how sustainable this is as a business model, but I think it’s pretty nice compared to being forced into continuous payment.
Welp it appears from what I could find on their website that the Pro tier is needed to use all features even for selfhosted instances.
Am I missing something here or couldn’t people just fork this and pull off a vaultwarden?
Obsidian. That is all.
can't bring myself to trust anything about privacy from a software with a weird-ass Chinese name.
Notion.so, but free. Nice!
I can't find how their multi-user collaboration looks like. Do they even have one?
Apparently it's on their roadmap: https://github.com/orgs/siyuan-note/projects/1/views/1
I absolutely love this program and recommended it multiple times on HN.
notes.txt
add choice of encryption, vcs, backup. solved for many decades
[dead]
[dead]
I tried to give it a shot a few months back, but the lack of vim bindings were a blocker for me, and seems like it isn't planned yet. Hope to try it sometime in the future.