100 Commits
Author SHA1 Message Date
Antonio SJ Musumeci 8e87bcbff2 docs: Add Unraid migration guidance for mergerfs
- Add a top-level Migration page covering both migration directions.
- Explain filesystem reuse, parity constraints, partition requirements,
  and unlocking encrypted Unraid disks.
- Describe intact ZFS pool imports and subdirectory branch reuse,
  distinguishing them from cases requiring a data copy.
- Add navigation and link the guide from the Unraid comparison.
2026-10-03 16:52:25 -05:00
Antonio SJ Musumeci fcd7527c9d docs: document mergerfs tradeoffs and web UI management
* Add: a comprehensive downsides FAQ with mitigations and practical
  priorities.
* Document: mergerfs-webui and link it from setup and configuration
  guides.
* Replace: obsolete mergerfs.fsck references with fsck.mergerfs
  guidance.
2026-10-01 20:14:52 -04:00
Antonio SJ Musumeci 83d1630f55 docs: rework and expand project comparisons 2026-09-22 14:50:47 -05:00
Antonio SJ Musumeci cfb925e5ae docs: clarify branch resolution and mount ordering 2026-09-21 13:46:38 -05:00
Antonio SJ Musumeci 515b0549ef docs/performance: explain mergerfs IO performance vs RAID
Describe why mergerfs does not aggregate branch performance like
RAID/ZFS (whose aggregate throughput is also capped by shared
interconnects) and instead provides independent per-branch
performance: best case A+B+C across concurrent streams, worst case
single-stream limited to one branch's speed.
2026-09-05 16:00:43 -04:00
Antonio SJ Musumeci 7ec6f8e2f2 Manage cloned /dev/fuse fds in sessions
- Store cloned read-worker fds on fuse_session and close them there.
- Set up clone fallback through session helpers used by the loop.
- Carry per-request fds through receive, process, and reply paths.

Tests show significant performance increases in highly concurrent
workloads on a high core count system. Upto 15% better on reads and
27% on writes.
2026-05-14 20:13:07 -05:00
Antonio SJ Musumeci 5f7fa17ad8 Inline ioprio disabled fast path
- Expose the ioprio enabled flag as an atomic for relaxed inline checks
- Inline SetFrom construction so disabled read/write paths avoid a function call
- Move priority application into a slow helper used only when ioprio is enabled
2026-05-14 00:22:06 -04:00
Antonio SJ Musumeci 30d59d4458 Use shared_mutex for fuse log config
- Guard fuse_cfg_t log file and filepath shared pointers with a shared mutex
- Replace atomic shared pointer loads and stores with shared and unique locks
2026-05-14 00:22:06 -04:00
Antonio SJ Musumeci 02dc36080c Use shared_mutex for ReadDir state
- Replace atomic shared pointer loads/stores with shared/unique locking.
- Store the configured readdir mode directly as std::string.
- Copy the active ReadDir implementation under lock before invoking it.
2026-05-14 00:22:06 -04:00
Antonio SJ Musumeci 8734847968 Guard branch snapshots with shared_mutex
Important changes:
- Replace atomic shared_ptr operations with shared_mutex-protected snapshots.
- Retry copy-on-write branch updates when another writer publishes first.
- Keep long-running parsing and read-only filesystem checks outside branch locks.
2026-05-14 00:22:06 -04:00
Antonio SJ Musumeci 608cd00764 Use atomic open file reference counts
- Use relaxed increment under the map bucket lock when duplicating file handles.
- Use acq_rel decrement when releasing and checking for final references.
- Load moved ref_count with relaxed ordering during map insertion.
2026-05-14 00:22:06 -04:00
Antonio SJ Musumeci b13727abde Document readdir merge mutex scope
- Explain that the mutex is per readdir call.
- Document that only dirent deduplication and list append are serialized.
- Clarify that per-branch opendir and getdents64 work runs outside the critical section.
2026-05-14 00:22:06 -04:00
Antonio SJ Musumeci 231aac15cd Use shared write locks until ENOSPC relocation
- Replace the per-file Mutex with std::shared_mutex for shared write access.
- Run direct and cached writes under a shared lock while the fd is stable.
- Retry failed writes under a unique lock before move-on-ENOSPC relocation.
2026-05-14 00:22:06 -04:00
Antonio SJ Musumeci 8b17d5f35e Synchronize node updates with path release
- Add helpers that release path locks and update node state under f.lock.
- Update getattr and setattr stat-cache invalidation while preserving ESTALE handling.
- Rework create, open, tmpfile, and set_path_info to mutate lookup/open state under the shared lock.
2026-05-14 00:22:06 -04:00
Antonio SJ Musumeci 8608b2be46 Clarify releasedir readdir synchronization barrier
- Document the lock/unlock pair as an acquire/release barrier.
- Explain that releasedir waits for in-flight readdir calls before destroy/free.
- Warn against removing the synchronization despite it not being a critical section.
2026-05-14 00:22:06 -04:00
Antonio SJ Musumeci 1e0bd54989 Move statvfs cache to use shared mutex
Its read heavy and micro benchmarks show no regression on the low end
and 30%+ throughput improvements on the highend.
2026-05-14 00:22:06 -04:00
Antonio SJ Musumeci 56285fc6e5 Pass annobin override to RPM toolset builds
Important changes:
- Add Makefile RPMBUILD and RPMBUILD_FLAGS overrides for the rpm target.
- Detect /opt/rh GCC toolsets before RPM packaging.
- Pass --undefine _annotated_build to avoid Rocky 8 annobin plugin failures.
2026-05-13 17:01:41 -04:00
Antonio SJ Musumeci dfe053a0ae Require C++20-capable build toolchains
Important changes:
- Switch mergerfs and vendored libfuse Makefiles to -std=c++20.
- Select the latest available GCC/G++ or toolset in container builds.
- Verify compiler C++20 support during package installation.
- Fix dynamic SysLog formatting for fmt C++20 checks.
2026-05-13 13:55:43 -05:00
Antonio SJ Musumeci 12ac368e8b Preserve CPPFLAGS in release builds
- Append -DNDEBUG to existing CPPFLAGS in the release target.
- Keep caller-provided build flags for Debian packaging.
2026-05-09 19:28:56 -05:00
Antonio SJ Musumeci c8f32f9f25 update build system and dependency configuration 2026-05-09 18:55:23 -05:00
Antonio SJ Musumeci ce3737ff33 Add new OS build files 2026-05-07 23:35:14 -04:00
Antonio SJ Musumeci 84e4cdeb7e make all integration tests self-contained with ephemeral mounts
- Add mergerfs_mount() context manager in posix_parity.py that creates
  temporary branch dirs in tests/.test_tmp/, mounts an ephemeral mergerfs
  instance, and cleans up after use.
- Add find_mergerfs(), find_fusermount(), mount_mergerfs(), unmount_mergerfs()
  helpers.
- Update run-tests to invoke each TEST_* script without a mountpoint arg.
- Update README.md to document the new `python3 tests/run-tests` invocation.
- Fix 44 tests to use mergerfs_mount() instead of expecting a pre-mounted path.
- Add tempfile import to 10 tests that were missing it.
- Fix TEST_mount_lifecycle to use mergerfs_mount() context manager.
- Fix TEST_posix_tmpfile missing join import.
- Fix TEST_cfg_xattr_modes error message framing (no "(known bug)").
- Fix TEST_posix_copy_file_range: loop-copy full size, compare st_size and
  SHA256 hash; add NativeTmp for space management; route native temp to
  tests/.test_tmp/ to avoid ENOSPC on /tmp.
- Fix TEST_posix_getattr_fgetattr: st_cmp skips st_size for dirs; add
  compare_ino param for safe inode comparison when inodecalc=passthrough.
- Fix TEST_unlink_rename, TEST_use_* tests: namespace cleanup, consistent
  variable naming, None check for cmpfunc.
- Fix TEST_readlink_semantics: remove type annotations to support older
  Python; wrap in mergerfs_mount() try/except.
- Add flush=True to run-tests print so test names show immediately.
2026-05-07 21:47:40 -04:00
Antonio SJ Musumeci 6d6d73c7b3 Use unmount() on FreeBSD for ENOTCONN umount 2026-05-07 17:49:24 -04:00
Antonio SJ Musumeci bea31543a0 update fuse_lib_tmpfile() node management to match fuse_lib_create() 2026-05-06 20:49:46 -04:00
Antonio SJ Musumeci 99835a0178 Document advisory locking limitation and update deprecated options
- Add documentation explaining that advisory file locking is not supported

- Add remember-nodes to deprecated options mapping

- Add comment explaining stat_crc32b fingerprint invalidation behavior

- Fix heading level for Host vs Container identity section
2026-05-06 20:49:46 -04:00
Antonio SJ Musumeci 342a1efcfb Add QEMU binfmt setup and fix static installer
- Fix typo in static.installer: /pkg/ -> /pkgs/

- Add missing mkdir -p /build before makeself

- Mount pkgs directory via podman build -v so static.installer can access

  previously built static tarballs without relying on bind mounts relative

  to the build context

- Register qemu-user-static binfmt handlers for cross-compilation

- Make --git-repo optional when using --setup
2026-05-06 20:49:46 -04:00
Antonio SJ Musumeci c587a127fb add missing <new> include for std::nothrow in fuse_session.cpp 2026-05-06 20:49:46 -04:00
Antonio SJ Musumeci f1034a3ad6 Remove timed node removal 2026-05-06 20:49:46 -04:00
Antonio SJ Musumeci 8117998205 Remove unused lock operation plumbing 2026-05-06 20:49:46 -04:00
Antonio SJ Musumeci cc780bc521 libfuse: remove per-node lock bookkeeping 2026-05-06 20:49:46 -04:00
Antonio SJ Musumeci 14b7c8e607 Use stat fingerprint as cache validity sentinel
- Reserve a zero stat CRC to represent an invalid cache fingerprint.

- Include device, size, inode, mtime, and ctime data in the fingerprint.

- Recompute the fingerprint during auto-cache open before keeping cache.
2026-05-06 20:49:46 -04:00
Antonio SJ Musumeci 3ff9794fff libfuse: fix use-after-free in fuse_lib_ioctl 2026-04-30 12:51:12 -05:00
Antonio SJ Musumeci f71e27644e vendor moodycamel queue and upgrade to 1.0.5 2026-04-30 08:43:42 -04:00
Antonio SJ Musumeci 943f91ff98 Strengthen mergerfs integration tests
- Make the test runner fail on failed tests and report explicit skips
- Strengthen assertions for O_DIRECT, LUP policy, EXDEV fallback, and xattrs
- Fix cleanup and file descriptor handling across test scripts
2026-04-29 20:25:20 -04:00
Antonio SJ Musumeci ef1e01b5a5 Add unit and integration tests for recently changed code
Unit tests (29 new in tests/tests.cpp):
- str::eq() and str::startswith() char* nullptr safety
- str::from() u64/s64/int suffix parsing (K/M/G/T, overflow, invalid)
- num::humanize() byte formatting (0, 512, 1K, 1M, 1G, 1T, non-exact)
- RmdirErr priority error accumulation (ENOTEMPTY/EEXIST override rules)
- HashSet inline-to-heap growth, empty string, and large insertion
- fs::inode algorithm switching (valid/invalid names, passthrough raw ino,
  distinct inodes across algorithms)
- RND::rand64() bounds (range, max, shrink_to_rand_elem)
- Branches::Impl copy and move assignment re-links default_minfreespace

Integration tests (5 new in tests/):
- TEST_policy_lup: LUP policy create and search
- TEST_rmdir_enotempty_priority: rmdir returns ENOTEMPTY for non-empty dirs
- TEST_open_after_unlink: read/write via fd after unlink
- TEST_api_xattr: mergerfs xattr API (fullpath, relpath, basepath, allpaths)
- TEST_movefile_enospc: moveonenospc configuration round-trip

Also: remove xattr nosys checks from TEST_cfg_xattr_modes, reformat
integration test scripts to use temp_dir/cleanup_dir helpers, and
minor parameter alignment changes in fuse_getxattr, fuse_init,
fuse_rename.
2026-04-29 17:41:00 -04:00
Antonio SJ Musumeci 50c0627b2b Better handle make sanitize option 2026-04-29 12:24:10 -04:00
Antonio SJ Musumeci 660f556ac6 Don't return success on ENOTEMPTY rmdir errors 2026-04-29 12:23:49 -04:00
Antonio SJ Musumeci 997b036064 Reject invalid policies 2026-04-29 00:34:00 -04:00
Antonio SJ Musumeci 9308a3729c When file unlinked check open_files map
Inadvertantly removed. Didn't realize that the kernel would not always
send file handle when unlinked.
2026-04-28 16:04:26 -04:00
Antonio SJ Musumeci 7616d8b1ac Rework readlink to not depend on cstr nul termination 2026-04-26 01:37:08 -04:00
Antonio SJ Musumeci a86092b3c2 Don't push docs for RC tags 2026-04-23 20:46:55 -05:00
Antonio SJ Musumeci e613d9415f Better same file check for fsck 2026-04-22 22:28:56 -05:00
Antonio SJ Musumeci 1a8eb326af Tweak docs on NFS and identity management 2026-04-21 13:20:07 -05:00
Antonio SJ Musumeci 9fb1642e93 Fix docs regarding "noforget" 2026-04-20 23:25:47 -05:00
Antonio SJ Musumeci 8be86ee611 Remove mkdocs build github workflow 2026-04-19 23:48:13 -05:00
Antonio SJ Musumeci ff48ff6047 Handle/ignore SIGHUP 2026-04-17 11:14:05 -05:00
Antonio SJ Musumeci d5a2187713 Fix make race condition 2026-04-16 18:20:00 -05:00
Antonio SJ Musumeci 4c9a9d7e15 Upgrade rapidhash 2026-04-12 23:44:23 -05:00
Antonio SJ Musumeci 425e30b50f Use nanosleep rather than yield and return EINTR 2026-04-11 11:30:57 -05:00
Antonio SJ Musumeci 3905355522 Detect OS for release-sample make target 2026-04-08 20:48:02 -05:00
Antonio SJ Musumeci 9e2e1afafa Update docs on limiting drive spinup 2026-04-06 14:01:20 -05:00
Antonio SJ Musumeci 6d218fa87f Misc updates and fixes 2026-04-04 02:46:24 -05:00
Antonio SJ Musumeci 3b10406f95 Add more details to project comparisons 2026-04-04 02:17:06 -05:00
Antonio SJ Musumeci f2c8e5d9eb Add FUSE request_timeout
Not yet hooked up to config parser to set as need to evaluate usage.
2026-04-01 23:20:05 -05:00
Antonio SJ Musumeci b581c5d5e7 Add support for FUSE_COPY_FILE_RANGE_64 2026-04-01 23:20:05 -05:00
Antonio SJ Musumeci 1ebc8cfa5f Enable statx with file handle 2026-03-31 19:53:17 -04:00
Antonio SJ Musumeci acfa797503 Misc formatting changes 2026-03-31 19:53:17 -04:00
Antonio SJ Musumeci 7fc20457ca Rework open file resource management 2026-03-31 19:53:17 -04:00
Antonio SJ Musumeci ecee84b323 Improve proj comparisons + new SnapRAID details 2026-03-31 19:51:28 -04:00
Antonio SJ Musumeci b8776d205d Update docs; add nofs, policyfs and greyhole 2026-03-27 22:54:39 -05:00
Antonio SJ Musumeci 06433a0548 Fallback to fusermount3 where available
Also general cleanup of mount_generic.
2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci f4b602340b Add more integration tests 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci a78aef36e0 Additional check for ENOTCONN on mount 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 0d05523dfd Move maintainence thread func take u64 rather than int
signed rollover is UB
2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 0cfab60f37 replace raw C-strings with std::string in converted libfuse files
util/mount.mergerfs.cpp:
- Remove xstrdup/xrealloc wrappers entirely
- Convert type, source, options, command, setuid to std::string
- Replace add_arg() pointer-rewriting with shell_quote() returning std::string
- Replace add_option() with simple std::string += concatenation
- Fix const_cast UB on string literal assignment

lib/helper.cpp:
- Convert add_default_subtype local subtype_opt: malloc+snprintf+free -> std::string construction

lib/mount_generic.h:
- Convert fuse_mount_sys locals source/type: malloc+strcpy+strcat -> std::string
- Remove 4 unnecessary free() calls (RAII cleanup)
- .c_str() at mount() and fuse_mnt_add_mount() boundaries

lib/mount_util.h:
- Rewrite fuse_mnt_resolve_path internals using std::string: rfind/substr
  instead of manual pointer arithmetic, strrchr, strlen chains
2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 21966ade97 Convert vendored/libfuse C sources to C++17
Convert all C source files in vendored/libfuse/ to C++17:
- lib/crc32b.c    -> crc32b.cpp  (const void* cast)
- lib/fuse_opt.c -> fuse_opt.cpp (designated init, void* arithmetic)
- lib/fuse_signals.c -> fuse_signals.cpp
- lib/helper.c   -> helper.cpp  (void* data cast)
- lib/mount.c    -> mount.cpp  (thin wrapper)
- lib/mount_generic.c -> mount_generic.h (static inline, #pragma once)
- lib/mount_bsd.c -> mount_bsd.h    (static inline, #pragma once)
- lib/mount_util.c -> mount_util.h   (static inline, #pragma once)
- util/fusermount.c -> fusermount.cpp
- util/mount.mergerfs.c -> mount.mergerfs.cpp

Key changes:
- Replace C99 designated initializer in fuse_opt_parse with field-by-field init
- Fix void* pointer arithmetic via char* base + offset pattern
- All C-style (T*)voidptr casts replaced with static_cast/reinterpret_cast
- Remove redundant #define _GNU_SOURCE (already in FUSE_FLAGS)
- Wrap __clone2 declaration in extern "C" for ia64 path
- Update Makefile: empty SRC_C, build utils with $(CXX) instead of $(CC)
- Remove stale .c files now replaced by .cpp/.h counterparts

All binaries build cleanly: mergerfs, mergerfs-fusermount, mount.mergerfs
2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 96eb51eecb Move fuse_session to C++ and make exit flag atomic 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 881147daa2 Fix null ptr handling issue in ioctl processing 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 0f1a8e9b69 Fix error condition in fs_copyfile 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 08f9cc6932 Fix readlink implementation, add more tests 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 3099b15db1 More aggressive error handing with pagesize queries 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 07b1e6bdba Auto-commit fuse_msgbuf.cpp 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 7763c7384a Use fatal::abort() rather than assert in places 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 61365d226d Move scope_guard to vendored/ 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci f33e78bc24 Cleanup procfs and abort on critical issues 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 7f52623536 Cleanup branches and add more unit tests 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci a30a4df9c0 Cleanup config and add tests 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 1e0d5aa91d fs_wait_for_mount cleanup 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci d09fadbc28 fs_inode cleanup 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 21ce889449 Fixup listxattr error handling 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 9d1a4696e9 Small improvements to fs::statvfs_cache 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci ed0cbcf064 Cleanup fuse_readlink 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 076e359891 Improve thread pool 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci bf98d3f3a6 Fix possible error handling issue with errno 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci c45c5ce920 Make cfgfile depth calculation thread friendly 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci e0a892f0d0 Improve error handling when querying pagesize 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 164b7f73c9 Misc fixes and improvements to thread pool 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 3a45be4cd8 Rework string utils, more test cases 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 107a3acf95 Better error handling in pagesize config type 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci a5eafb8972 Check string is not empty before resize in Config::keys 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 8199cc8a8b Fix error handling in fuse chmod, chown, setxattr, removexattr 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci e5d897f3dc Ensure zero initialized timeouts structs 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci c0af09f7d8 Fix incorrect type in fs_read and write 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 1b1e14ca29 Remove dead branch in fuse_reply_data 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 82b89aa099 Change session->exited to sig_atomic_t 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci aa91e1b2ba Initialize stat buffer and add stat call in fuse_mount_sys 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 82e81afcdc Add NULL check for cmsg in receive_fd to prevent segfault 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 1a5c39f813 Fix buffer overflow in readlink by reserving space for null terminator 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci fce1803d42 Use RND class for shrink_to_rand_elem and add assertions 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci 5a9a17afcb fix objpool: add size tracking, improve allocation matching; fix msgbuf: clear pool on size change 2026-03-24 21:13:34 -04:00