PhoenixKitKnowledgeBase.Storage (PhoenixKitKnowledgeBase v0.1.0)

Copy Markdown View Source

The filesystem layer. Files on disk are the source of truth; Postgres is an index of them.

Layout

<root>/system/<vault-slug>/…                    organisation vaults
<root>/users/<email-slug>/<vault-slug>/…        personal vaults

Inside a vault, directories are folders and <Title>.md is a note — the same shape Obsidian uses, so mv, rm, rg, git and Obsidian itself all work on the tree directly.

Configure the root (defaults to priv/knowledge_base under the running app's working directory, which is the host Phoenix project in dev):

config :phoenix_kit_knowledge_base, root: "/www/app/priv/knowledge_base"

Path safety

Every path that reaches the filesystem is built from user-supplied text — note titles, folder names, vault slugs — so containment is enforced twice, in two different ways, because this is the one place in the module where a bug is a server compromise rather than a bad page:

  1. safe_relative/1 rejects the input outright — absolute paths, any . or .. segment, and NUL bytes never become a path at all.
  2. resolve/2 expands the joined result and re-checks that it is still under the base. That catches anything the first pass didn't anticipate. It is a check on the string: Path.expand/1 is lexical and follows no symlinks, which is why the walk refuses to follow them instead.

Symlinks inside a vault are skipped by the walk rather than followed, so a planted link can't pull /etc/passwd into someone's search index — and since only walked paths are ever indexed, nothing in the app can name a path through one. The vault directory itself may be a symlink — that is how you point a vault at an Obsidian vault that already exists somewhere else.

Atomic writes

write_note/3 writes to a temporary file in the same directory and renames it into place. rename/2 within a filesystem is atomic, so a reader — rg, a rescan, Obsidian — never observes a half-written note, and a crash mid-write leaves the previous version intact rather than a truncated file.

Summary

Types

A file found by list_notes/1: path relative to the vault directory, plus cheap change-detection metadata.

Functions

The directory a vault's files live in, relative to root/0.

Create a folder inside a vault.

Delete a folder and everything under it. Notes inside go with it — the caller warns first.

A path without its markdown extension.

Turns an email address into a readable, safe directory segment.

Empty the vault's trash.

Create a vault's directory (and any missing parents).

The markdown file extension this module reads and writes.

The folder a note at this path sits in: a relative path, or "" for the vault root.

The characters a note or folder name may not contain, for showing in an error.

SHA-256 of a note's bytes, hex encoded. Used for change and rename detection.

Every directory in a vault, as paths relative to it.

Every markdown file in a vault, with the metadata rescan compares.

Everything in the vault's trash, newest first.

Move a folder, which is how both renaming and reparenting are done.

Move or rename a note.

True when a note's file already exists.

The relative path a note with this title, in this folder, should live at.

Delete one trashed item for good.

Read a note's raw markdown.

Joins a validated relative path onto a base and confirms the result is still inside it.

Move a trashed note back to where it came from.

The configured root directory for all vaults.

Validates a caller-supplied relative path.

A note's current size, mtime and content hash — everything rescan needs to decide whether the index is stale, and what the stale-write guard compares against.

Whether bytes can be indexed as a note's text: valid UTF-8 with no NUL.

The title a note at this path has — its filename without the extension.

The trash directory name, relative to a vault.

Move a whole folder into the trash, subtree intact.

Move a note into the vault's trash instead of unlinking it.

Validates one path segment as a filename or folder name.

Resolve a path inside a vault, applying both containment checks.

Absolute path to a vault's directory.

Write a note atomically and return its fresh stat.

Types

entry()

@type entry() :: %{
  relative_path: String.t(),
  mtime: DateTime.t(),
  size: non_neg_integer()
}

A file found by list_notes/1: path relative to the vault directory, plus cheap change-detection metadata.

Functions

build_directory(atom, vault_slug, owner_email)

@spec build_directory(:system | :user, String.t(), String.t() | nil) :: String.t()

The directory a vault's files live in, relative to root/0.

Computed once when the vault is created and then stored on the row, never recomputed: a personal vault's path contains a slug of its owner's email, and silently relocating someone's notes because they changed their email address would be a spectacular way to lose a vault.

create_folder(directory, folder_path)

@spec create_folder(String.t(), String.t()) :: :ok | {:error, term()}

Create a folder inside a vault.

delete_folder(directory, folder_path)

@spec delete_folder(String.t(), String.t()) :: :ok | {:error, term()}

Delete a folder and everything under it. Notes inside go with it — the caller warns first.

drop_extension(path)

@spec drop_extension(String.t()) :: String.t()

A path without its markdown extension.

Case-insensitive on purpose: the walk accepts Note.MD as readily as Note.md (markdown?/1 downcases before comparing), so a title derived with a case-sensitive strip would come out as "Note.MD" — and a bare [[Note]] would never resolve to it. Strip the same way the walk matches.

email_slug(email)

@spec email_slug(String.t() | nil) :: String.t()

Turns an email address into a readable, safe directory segment.

fotkin@gmail.com becomes fotkin-gmail-com. Falls back to user for input with nothing usable in it, so the path is always well-formed — the vault slug underneath it is what actually keeps the directory unique.

empty_trash(directory)

@spec empty_trash(String.t()) :: :ok | {:error, term()}

Empty the vault's trash.

ensure_vault(directory)

@spec ensure_vault(String.t()) :: :ok | {:error, term()}

Create a vault's directory (and any missing parents).

extension()

@spec extension() :: String.t()

The markdown file extension this module reads and writes.

folder_from_path(relative_path)

@spec folder_from_path(String.t()) :: String.t()

The folder a note at this path sits in: a relative path, or "" for the vault root.

forbidden_characters()

@spec forbidden_characters() :: String.t()

The characters a note or folder name may not contain, for showing in an error.

hash(content)

@spec hash(binary()) :: String.t()

SHA-256 of a note's bytes, hex encoded. Used for change and rename detection.

list_folders(directory)

@spec list_folders(String.t()) :: {:ok, [String.t()]} | {:error, term()}

Every directory in a vault, as paths relative to it.

list_notes(directory)

@spec list_notes(String.t()) :: {:ok, [entry()]} | {:error, term()}

Every markdown file in a vault, with the metadata rescan compares.

Skips dotfiles and dot-directories (.obsidian, .git, .trash) and skips symlinks rather than following them.

list_trash(directory)

@spec list_trash(String.t()) :: {:ok, [map()]} | {:error, term()}

Everything in the vault's trash, newest first.

Each entry reports where it came from and when it was deleted, both read straight out of the path.

move_folder(directory, from, to)

@spec move_folder(String.t(), String.t(), String.t()) :: :ok | {:error, term()}

Move a folder, which is how both renaming and reparenting are done.

There is one operation here because on a filesystem there is one operation: renaming research to archive and moving it into 2026/ are both a directory rename, and pretending otherwise would mean two code paths that can disagree.

Four things are refused rather than attempted:

  • the vault root, in either position — moving it means moving the vault
  • a target that already exists, so nothing is merged or clobbered
  • moving a folder into its own subtree, which orphans the directory being moved along with everything under it
  • anything outside the vault, by the same containment check as every other path here

The caller reindexes afterwards. This only moves bytes.

move_note(directory, from, to)

@spec move_note(String.t(), String.t(), String.t()) :: :ok | {:error, term()}

Move or rename a note.

Refuses to overwrite an existing file: a rename that silently destroys another note is worse than a rename that fails.

note_exists?(directory, relative_path)

@spec note_exists?(String.t(), String.t()) :: boolean()

True when a note's file already exists.

note_relative_path(title, folder_path \\ nil)

@spec note_relative_path(String.t(), String.t() | nil) ::
  {:ok, String.t()} | {:error, term()}

The relative path a note with this title, in this folder, should live at.

The filename is the title, plus .md — Obsidian's scheme, so a note created here is indistinguishable from one created there.

purge(directory, trash_path)

@spec purge(String.t(), String.t()) :: :ok | {:error, term()}

Delete one trashed item for good.

read_note(directory, relative_path)

@spec read_note(String.t(), String.t()) :: {:ok, String.t()} | {:error, term()}

Read a note's raw markdown.

resolve(base, relative)

@spec resolve(String.t(), String.t() | nil) ::
  {:ok, String.t()} | {:error, :unsafe_path}

Joins a validated relative path onto a base and confirms the result is still inside it.

The second, redundant-looking check is the point: safe_relative/1 reasons about the string, this reasons about the resolved path. Either alone has been enough to produce a traversal bug in somebody's codebase.

restore(directory, trash_path, original_path)

@spec restore(String.t(), String.t(), String.t()) :: :ok | {:error, term()}

Move a trashed note back to where it came from.

Refuses when something already occupies the original path: restoring must never overwrite a note that was written since.

root()

@spec root() :: String.t()

The configured root directory for all vaults.

Relative configuration is resolved against the current working directory, which in a Phoenix app is the project root — so the default lands on priv/knowledge_base inside the host app.

safe_relative(path)

@spec safe_relative(String.t() | nil) :: {:ok, String.t()} | :error

Validates a caller-supplied relative path.

Returns {:ok, normalised} or :error. The empty string is valid and means "the vault root", which is how a note at the top level is addressed.

stat_note(directory, relative_path)

@spec stat_note(String.t(), String.t()) :: {:ok, map()} | {:error, term()}

A note's current size, mtime and content hash — everything rescan needs to decide whether the index is stale, and what the stale-write guard compares against.

text?(content)

@spec text?(binary()) :: boolean()

Whether bytes can be indexed as a note's text: valid UTF-8 with no NUL.

Postgres stores neither invalid UTF-8 nor NUL in a text column, and refuses the whole row rather than the offending byte. A file is free to hold either — a stray binary renamed .md, a Latin-1 export — so the indexer asks first and reports the file, where an insert would raise.

title_from_path(relative_path)

@spec title_from_path(String.t()) :: String.t()

The title a note at this path has — its filename without the extension.

trash_dir()

@spec trash_dir() :: String.t()

The trash directory name, relative to a vault.

trash_folder(directory, folder_path)

@spec trash_folder(String.t(), String.t()) :: {:ok, String.t()} | {:error, term()}

Move a whole folder into the trash, subtree intact.

trash_note(directory, relative_path)

@spec trash_note(String.t(), String.t()) :: {:ok, String.t() | nil} | {:error, term()}

Move a note into the vault's trash instead of unlinking it.

The trashed copy keeps its original path under a per-deletion timestamp directory — .trash/1756150000/research/Ideas.md. That shape is doing real work: the original path is recoverable verbatim (restore is a move, with no name to un-mangle), and deleting, recreating and deleting the same note again produces two independent copies rather than one overwriting the other.

A missing file yields {:ok, nil}: the desired state is "not in the vault", and there is nothing to recover.

validate_filename(name)

@spec validate_filename(String.t() | nil) ::
  {:ok, String.t()}
  | {:error,
     :blank
     | :leading_dot
     | :trailing_dot
     | :too_long
     | :reserved
     | {:forbidden_characters, [String.t()]}}

Validates one path segment as a filename or folder name.

Returns {:ok, trimmed} or {:error, reason}. Rejects rather than repairs — an earlier version silently stripped offending characters, which meant typing Ideas: Part 2 produced a note called Ideas Part 2 with no explanation and a title that no longer matched what was typed. Obsidian reports the problem and lets you fix it; so does this.

The rules are Obsidian's: no * " \ / < > : | ? (path-hostile), no [ ] # ^ (link-syntax breakers), nothing that would make the file hidden or ambiguous.

Note the asymmetry with indexing. This governs names this app creates. A file that already exists on disk is indexed whatever it is called — the filesystem allowed it, Obsidian may have made it, and refusing to show someone's note because of its name would be worse than the name.

vault_child(directory, relative_path)

@spec vault_child(String.t(), String.t()) ::
  {:ok, String.t()} | {:error, :unsafe_path}

Resolve a path inside a vault, applying both containment checks.

Public because the contexts need it, but note the shape: it takes the vault's stored directory rather than a caller-chosen base, so there is no way to ask it about a path outside the root.

vault_path(directory)

@spec vault_path(String.t()) :: String.t()

Absolute path to a vault's directory.

write_note(directory, relative_path, content)

@spec write_note(String.t(), String.t(), String.t()) ::
  {:ok, map()} | {:error, term()}

Write a note atomically and return its fresh stat.

Temp file in the same directory, then rename — a rename within one filesystem is atomic, so nothing ever reads a partial note and a crash mid-write leaves the old content in place.