darlinghq/darling-copyfile
1
0
Fork 0
mirror of https://github.com/darlinghq/darling-copyfile.git synced 2024-11-23 04:39:58 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Update Source To copyfile-173.40.2

Browse Source
This commit is contained in:
Thomas A 2023-01-29 21:59:54 -08:00
parent eca07904e0
commit e2b53baa1c
15 changed files with 847 additions and 116 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

126
copyfile.3
View File

@ -125,13 +125,15 @@ file had extended attributes but no ACLs, the return value would be
.It Dv COPYFILE_PACK
Serialize the
.Va from
file. The
file.
The
.Va to
file is an AppleDouble-format file.
.It Dv COPYFILE_UNPACK
Unserialize the
.Va from
file. The
file.
The
.Va from
file is an AppleDouble-format file; the
.Va to
@ -168,7 +170,8 @@ file. (This is only applicable for the
.Fn copyfile
function.) No error is returned if
.Xr remove 3
fails. Note that
fails.
Note that
.Xr remove 3
removes a symbolic link itself, not the
target of the link.
@ -185,8 +188,8 @@ This flag is equivalent to (COPYFILE_EXCL | COPYFILE_ACL | COPYFILE_STAT | COPYF
| COPYFILE_NOFOLLOW_SRC).
Note that if cloning is successful, progress callbacks will not be invoked.
Note also that there is no support for cloning directories: if a directory is provided as the source,
an error will be returned. Since this flag implies COPYFILE_NOFOLLOW_SRC, symbolic links themselves will
be cloned instead of their targets.
an error will be returned.
Since this flag implies COPYFILE_NOFOLLOW_SRC, symbolic links themselves will be cloned instead of their targets.
(This is only applicable for the
.Fn copyfile
function.)
@ -197,9 +200,9 @@ This flag is equivalent to (COPYFILE_EXCL | COPYFILE_ACL | COPYFILE_STAT | COPYF
| COPYFILE_NOFOLLOW_SRC).
Note that if cloning is successful, progress callbacks will not be invoked.
Note also that there is no support for cloning directories: if a directory is provided as the source and
COPYFILE_CLONE_FORCE is not passed, this will instead copy the directory. Since this flag implies COPYFILE_NOFOLLOW_SRC,
symbolic links themselves will be cloned instead of their targets. Recursive copying however is
supported, see below for more information.
COPYFILE_CLONE_FORCE is not passed, this will instead copy the directory.
Since this flag implies COPYFILE_NOFOLLOW_SRC, symbolic links themselves will be cloned instead of their targets.
Recursive copying however is supported, see below for more information.
(This is only applicable for the
.Fn copyfile
function.)
@ -216,16 +219,19 @@ if sparse copying cannot be performed for any reason; otherwise, an error is ret
This is a convenience macro, equivalent to
.Dv (COPYFILE_NOFOLLOW_DST | COPYFILE_NOFOLLOW_SRC) .
.It Dv COPYFILE_RUN_IN_PLACE
If the src file has quarantine information, add the QTN_FLAG_DO_NOT_TRANSLOCATE flag to the quarantine information of the dst file. This allows a bundle to run in place instead of being translocated.
If the src file has quarantine information, add the QTN_FLAG_DO_NOT_TRANSLOCATE flag to the quarantine information of the dst file.
This allows a bundle to run in place instead of being translocated.
.It Dv COPYFILE_PRESERVE_DST_TRACKED
Preserve the UF_TRACKED flag at
.Va to
when copying metadata, regardless of whether
.Va from
has it set. This flag is used in conjunction with COPYFILE_STAT, or COPYFILE_CLONE (for its fallback case).
has it set.
This flag is used in conjunction with COPYFILE_STAT, or COPYFILE_CLONE (for its fallback case).
.El
.Pp
Copying files into a directory is supported. If
Copying files into a directory is supported.
If
.Va to
is a directory,
.Va from
@ -269,7 +275,8 @@ parameters are pointers to
.It Dv COPYFILE_STATE_SRC_FILENAME
.It Dv COPYFILE_STATE_DST_FILENAME
Get or set the filename associated with the source (or destination)
file. If it has not been initialized yet, the value will be
file.
If it has not been initialized yet, the value will be
.Dv NULL .
For
.Fn copyfile_state_set ,
@ -385,7 +392,8 @@ entered. (That is, none of the filesystem objects contained
within the directory have been copied yet.)
.It Dv COPYFILE_RECURSE_DIR_CLEANUP
The object being copied is a directory, and all of the
objects contained have been copied. At this stage, the destination directory
objects contained have been copied.
At this stage, the destination directory
being copied will have any extra permissions that were added to
allow the copying will be removed.
.It Dv COPYFILE_RECURSE_ERROR
@ -403,15 +411,15 @@ The second argument to the call-back function will indicate
the stage of the copy, and will be one of the following values:
.Bl -tag -width COPYFILE_FINISH
.It Dv COPYFILE_START
Before copying has begun. The third
parameter will be a newly-created
Before copying has begun.
The third parameter will be a newly-created
.Vt copyfile_state_t
object with the call-back function and context pre-loaded.
.It Dv COPYFILE_FINISH
After copying has successfully finished.
.It Dv COPYFILE_ERR
Indicates an error has happened at some stage. If the
first argument to the call-back function is
Indicates an error has happened at some stage.
If the first argument to the call-back function is
.Dv COPYFILE_RECURSE_ERROR ,
then an error occurred while processing the source hierarchy;
otherwise, it will indicate what type of object was being copied,
@ -437,13 +445,14 @@ values:
The copy will continue as expected.
.It Dv COPYFILE_SKIP
This object will be skipped, and the next object will
be processed. (Note that, when entering a directory.
returning
be processed.
(Note that, when entering a directory, returning
.Dv COPYFILE_SKIP
from the call-back function will prevent the contents
of the directory from being copied.)
.It Dv COPYFILE_QUIT
The entire copy is aborted at this stage. Any filesystem
The entire copy is aborted at this stage.
Any filesystem
objects created up to this point will remain.
.Fn copyfile
will return -1, but
@ -474,11 +483,13 @@ Note that recursive cloning is also supported with the
.Dv COPYFILE_CLONE
flag (but not the
.Dv COPYFILE_CLONE_FORCE
flag). A recursive clone operation invokes
flag).
A recursive clone operation invokes
.Fn copyfile
with
.Dv COPYFILE_CLONE
on every entry found in the source file-system object. Because
on every entry found in the source file-system object.
Because
.Fn copyfile
does not allow the cloning of directories, a recursive clone will
instead copy any directory it finds (while cloning its contents).
@ -503,10 +514,12 @@ Note that if the source path ends in a
.Va /
its contents are copied rather than the directory itself (like cp(1)).
The behavior of a recursive copy on a directory hierarchy also depends
on the contents of the destination. If the destination is a directory,
the source directory (or its contents, if the source path ends in a
on the contents of the destination.
If the destination is a directory, the source directory (or its contents,
if the source path ends in a
.Va /
) will be copied into it. If the destination exists but is not a
) will be copied into it.
If the destination exists but is not a
directory, and the source is a non-empty directory, the copy will fail;
the exact error set depends on the flags provided to
.Fn copyfile
@ -518,9 +531,11 @@ and
.Fn fcopyfile
will also use a callback to report data (e.g.,
.Dv COPYFILE_DATA )
progress. If given, the callback will be invoked on each
progress.
If given, the callback will be invoked on each
.Xr write 2
call. The first argument to the callback function will be
call.
The first argument to the callback function will be
.Dv COPYFILE_COPY_DATA .
The second argument will either be
.Dv COPYFILE_PROGRESS
@ -560,7 +575,8 @@ when finished with each individual attribute.
may be called for all of the extended attributes, before
the first callback with
.Dv COPYFILE_PROGRESS
is invoked.) Any attribute skipped by returning
is invoked.)
Any attribute skipped by returning
.Dv COPYFILE_SKIP
from the
.Dv COPYFILE_START
@ -621,6 +637,28 @@ changes while the copy is occurring, the results are undefined.
does not reset the seek position for either source or destination.
This can result in the destination file being a different size
than the source file.
.Sh EXAMPLES
.Bd -literal -offset indent
/* Initialize a state variable */
copyfile_state_t s;
s = copyfile_state_alloc();
/* Copy the data and extended attributes of one file to another */
copyfile("/tmp/f1", "/tmp/f2", s, COPYFILE_DATA | COPYFILE_XATTR);
/* Convert a file to an AppleDouble file for serialization */
copyfile("/tmp/f2", "/tmp/tmpfile", NULL, COPYFILE_ALL | COPYFILE_PACK);
/* Release the state variable */
copyfile_state_free(s);
/* A more complex way to call copyfile() */
s = copyfile_state_alloc();
copyfile_state_set(s, COPYFILE_STATE_SRC_FILENAME, "/tmp/foo");
/* One of src or dst must be set... rest can come from the state */
copyfile(NULL, "/tmp/bar", s, COPYFILE_ALL);
/* Now copy the same source file to another destination file */
copyfile(NULL, "/tmp/car", s, COPYFILE_ALL);
copyfile_state_free(s);
/* Remove extended attributes from a file */
copyfile("/dev/null", "/tmp/bar", NULL, COPYFILE_XATTR);
.Ed
.Sh ERRORS
.Fn copyfile
and
@ -688,40 +726,18 @@ parameter.
In addition, both functions may set
.Dv errno
via an underlying library or system call.
.Sh EXAMPLES
.Bd -literal -offset indent
/* Initialize a state variable */
copyfile_state_t s;
s = copyfile_state_alloc();
/* Copy the data and extended attributes of one file to another */
copyfile("/tmp/f1", "/tmp/f2", s, COPYFILE_DATA | COPYFILE_XATTR);
/* Convert a file to an AppleDouble file for serialization */
copyfile("/tmp/f2", "/tmp/tmpfile", NULL, COPYFILE_ALL | COPYFILE_PACK);
/* Release the state variable */
copyfile_state_free(s);
/* A more complex way to call copyfile() */
s = copyfile_state_alloc();
copyfile_state_set(s, COPYFILE_STATE_SRC_FILENAME, "/tmp/foo");
/* One of src or dst must be set... rest can come from the state */
copyfile(NULL, "/tmp/bar", s, COPYFILE_ALL);
/* Now copy the same source file to another destination file */
copyfile(NULL, "/tmp/car", s, COPYFILE_ALL);
copyfile_state_free(s);
/* Remove extended attributes from a file */
copyfile("/dev/null", "/tmp/bar", NULL, COPYFILE_XATTR);
.Ed
.Sh SEE ALSO
.Xr listxattr 2 ,
.Xr getxattr 2 ,
.Xr listxattr 2 ,
.Xr setxattr 2 ,
.Xr acl 3
.Sh HISTORY
The
.Fn copyfile
API was introduced in Mac OS X 10.5.
.Sh BUGS
Both
.Fn copyfile
functions lack a way to set the input or output block size.
.Pp
Recursive copies do not honor hard links.
.Sh HISTORY
The
.Fn copyfile
API was introduced in Mac OS X 10.5.

292
copyfile.c
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2004-2019 Apple, Inc. All rights reserved.
* Copyright (c) 2004-2020 Apple, Inc. All rights reserved.
*
* @APPLE_LICENSE_HEADER_START@
*
@ -202,6 +202,41 @@ does_copy_protection(int fd)
return ((sfs.f_flags & MNT_CPROTECT) == MNT_CPROTECT);
}
static bool
path_does_copy_protection(const char *path)
{
struct statfs sfs;
if (statfs(path, &sfs) == -1) {
char parent_path[MAXPATHLEN];
if (errno != ENOENT)
return false;
// If the path doesn't exist,
// try to get its parent path and re-attempt the statfs().
if (dirname_r(path, parent_path) == NULL)
return false;
if (statfs(parent_path, &sfs) == -1)
return false;
}
return ((sfs.f_flags & MNT_CPROTECT) == MNT_CPROTECT);
}
static int
do_copy_protected_open(const char *path, int flags, int class, int dpflags, int mode)
{
// The passed-in protection class is meaningful, so use open_dprotected_np().
if (path_does_copy_protection(path)) {
return open_dprotected_np(path, flags, class, dpflags, mode);
}
// Fall-back to regular open().
return open(path, flags, mode);
}
static void
sort_xattrname_list(void *start, size_t length)
{
@ -1077,6 +1112,74 @@ static int copyfile_clone(copyfile_state_t state)
return ret;
}
/*
* Check if two provided paths are identical,
* and if we're able to determine that, return true.
*/
static bool copyfile_paths_identical(const char *src, const char *dst)
{
struct attrlist attrs;
struct statfs sfs;
struct stat src_sb, dst_sb;
char volroot[MAXPATHLEN + 1];
struct {
uint32_t length;
vol_capabilities_attr_t volAttrs;
} volattrs;
char *real_src_path = NULL, *real_dst_path = NULL;
// Common case: the destination does not exist.
if ((stat(dst, &dst_sb) == -1) || (stat(src, &src_sb) == -1))
return false;
// If both files exist, then we next try to check file IDs.
// This requires that the underlying filesystem support persistent file IDs.
if (statfs(src, &sfs) == -1)
return false;
strlcpy(volroot, sfs.f_mntonname, sizeof(volroot));
memset(&attrs, 0, sizeof(attrs));
attrs.bitmapcount = ATTR_BIT_MAP_COUNT;
attrs.volattr = ATTR_VOL_CAPABILITIES;
if (getattrlist(volroot, &attrs, &volattrs, sizeof(volattrs), 0) == -1)
return false;
// If the underlying devices are not the same, then the files are not the same.
if (src_sb.st_dev != dst_sb.st_dev)
return false;
if ((volattrs.volAttrs.capabilities[VOL_CAPABILITIES_FORMAT] & VOL_CAP_FMT_PERSISTENTOBJECTIDS) &&
(volattrs.volAttrs.valid[VOL_CAPABILITIES_FORMAT] & VOL_CAP_FMT_PERSISTENTOBJECTIDS)) {
// The underlying source filesystem supports persistent file IDs,
// so if our two files have the same file ID on the same device,
// they are identical.
return (src_sb.st_ino == dst_sb.st_ino);
}
// Finally, if we don't support persistent file ID's,
// we fall back to path comparisons.
real_src_path = realpath(src, NULL);
if (real_src_path == NULL)
goto exit;
real_dst_path = realpath(dst, NULL);
if (real_dst_path == NULL)
goto exit;
if (strncasecmp(src, dst, MAXPATHLEN) == 0)
return true;
exit:
if (real_src_path)
free(real_src_path);
if (real_dst_path)
free(real_dst_path);
return false;
}
/*
* the original copyfile() routine; this copies a source file to a destination
* file. Note that because we need to set the names in the state variable, this
@ -1134,6 +1237,20 @@ int copyfile(const char *src, const char *dst, copyfile_state_t state, copyfile_
COPYFILE_SET_FNAME(src, s);
COPYFILE_SET_FNAME(dst, s);
if (!(s->flags & COPYFILE_CHECK)) {
// We have no work to do if `src` and `dst` point to the same place.
if (copyfile_paths_identical(src, dst)) {
// ...but return an error if requested to do so.
if (s->flags & COPYFILE_EXCL) {
s->err = EEXIST;
goto error_exit;
}
ret = 0;
goto exit;
}
}
if (s->flags & COPYFILE_RECURSIVE) {
ret = copytree(s);
goto exit;
@ -1678,7 +1795,7 @@ static int copyfile_unset_acl(copyfile_state_t s)
*/
static int copyfile_open(copyfile_state_t s)
{
int oflags = O_EXCL | O_CREAT | O_WRONLY;
int oflags = O_EXCL | O_CREAT;
int islnk = 0, isdir = 0, isreg = 0;
int osrc = 0, dsrc = 0;
int prot_class = PROTECTION_CLASS_DEFAULT;
@ -1749,6 +1866,26 @@ static int copyfile_open(copyfile_state_t s)
if (s->dst && s->dst_fd == -2)
{
/*
* Per <rdar://60074298>, only open files for writing if we expect
* to modify the file's content. This avoids undesirable side effects
* of O_WRONLY when we're only modifying metadata/attributes.
*
* The calls needed by COPYFILE_METADATA (e.g. fchown(), fchmod(),
* fchflags(), futimes(), fsetattrlist(), f{set,remove}xattr(), and
* acl_set_fd()) are safe to use with O_RDONLY descriptors. They
* operate on the underlying vnode_t, as path-based variants do.
* These usually only need write permissions in st_mode or ACLs.
*/
const copyfile_flags_t writable_flags = (COPYFILE_DATA | COPYFILE_DATA_SPARSE);
if (COPYFILE_PACK & s->flags) {
oflags |= O_WRONLY; // always writes file content
} else if (COPYFILE_UNPACK & s->flags) {
oflags |= O_RDONLY; // only updates metadata
} else {
oflags |= (writable_flags & s->flags) ? O_WRONLY : O_RDONLY;
}
/*
* COPYFILE_UNLINK tells us to try removing the destination
* before we create it. We don't care if the file doesn't
@ -1841,7 +1978,8 @@ static int copyfile_open(copyfile_state_t s)
return -1;
}
set_cprot_explicit = 1;
} else while((s->dst_fd = open_dprotected_np(s->dst, oflags | dsrc, prot_class, 0, s->sb.st_mode | S_IWUSR)) < 0)
} else while((s->dst_fd = do_copy_protected_open(s->dst, oflags | dsrc, prot_class,
0, s->sb.st_mode | S_IWUSR)) < 0)
{
/*
* We set S_IWUSR because fsetxattr does not -- at the time this comment
@ -2053,10 +2191,13 @@ static int copyfile_data_sparse(copyfile_state_t s, size_t input_blk_size, size_
if (!(s->flags & COPYFILE_DATA_SPARSE)) {
// Don't attempt this unless the right flags are passed.
return ENOTSUP;
} else if (src_size <= 0) {
} else if (src_size < 0) {
// The file size of our source is invalid; there's nothing to copy.
errno = EINVAL;
goto error_exit;
} else if (src_size == 0) {
// This is a zero-length file: no work to do.
goto exit;
}
// Since a major underlying filesystem requires that holes are block-aligned,
@ -2761,6 +2902,8 @@ static int copyfile_stat(copyfile_state_t s)
return 0;
}
#define MAX_GETXATTR_RETRIES 3
#define MAX_XATTR_BUFFER_SIZE (32 * 1024 * 1024) // 32 MiB
/*
* Similar to copyfile_security() in some ways; this
* routine copies the extended attributes from the source,
@ -2774,31 +2917,53 @@ static int copyfile_stat(copyfile_state_t s)
static int copyfile_xattr(copyfile_state_t s)
{
char *name;
char *namebuf, *end;
char *namebuf = NULL;
char *end;
ssize_t xa_size;
void *xa_dataptr;
ssize_t bufsize = 4096;
ssize_t xa_bufsize = 4096;
ssize_t namebuf_size = 0;
ssize_t asize;
ssize_t nsize;
ssize_t list_size;
int ret = 0;
int look_for_decmpea = 0;
int tries_left = MAX_GETXATTR_RETRIES;
/* delete EAs on destination */
if ((nsize = flistxattr(s->dst_fd, 0, 0, 0)) > 0)
dst_restart:
if ((list_size = flistxattr(s->dst_fd, 0, 0, 0)) > 0)
{
if ((namebuf = (char *) malloc(nsize)) == NULL)
return -1;
else
nsize = flistxattr(s->dst_fd, namebuf, nsize, 0);
/* this is always true on the first call: no buffer yet (namebuf_size == 0) */
if (list_size > namebuf_size) {
if (list_size > MAX_XATTR_BUFFER_SIZE) {
copyfile_warn("destination's xattr list size (%zu) exceeds the threshold (%d); trying to allocate", list_size, MAX_XATTR_BUFFER_SIZE);
}
namebuf_size = list_size;
void *tdptr = namebuf;
if (nsize > 0) {
if ((namebuf =
(void *) realloc((void *) namebuf, namebuf_size)) == NULL)
{
if (tdptr) {
free(tdptr);
}
return -1;
}
}
list_size = flistxattr(s->dst_fd, namebuf, namebuf_size, 0);
if ((list_size < 0) && (errno == ERANGE) && (tries_left > 0)) {
/* `namebuf` is too small - try again */
tries_left--;
goto dst_restart;
} else if (list_size > 0) {
/*
* With this, end points to the last byte of the allocated buffer
* This *should* be NUL, from flistxattr, but if it's not, we can
* set it anyway -- it'll result in a truncated name, which then
* shouldn't match when we get them later.
*/
end = namebuf + nsize - 1;
end = namebuf + list_size - 1;
if (*end != 0)
*end = 0;
for (name = namebuf; name <= end; name += strlen(name) + 1) {
@ -2809,15 +2974,24 @@ static int copyfile_xattr(copyfile_state_t s)
fremovexattr(s->dst_fd, name,0);
}
}
free(namebuf);
} else
if (nsize < 0)
{
if (errno == ENOTSUP || errno == EPERM)
return 0;
else
return -1;
}
else if (list_size < 0)
{
if (namebuf) {
free(namebuf);
}
if (errno == ENOTSUP || errno == EPERM) {
return 0;
} else {
return -1;
}
}
if (namebuf) {
free(namebuf);
}
namebuf = NULL;
namebuf_size = 0;
tries_left = MAX_GETXATTR_RETRIES;
#ifdef DECMPFS_XATTR_NAME
if ((s->flags & COPYFILE_DATA) &&
@ -2829,24 +3003,50 @@ static int copyfile_xattr(copyfile_state_t s)
#endif
/* get name list of EAs on source */
if ((nsize = flistxattr(s->src_fd, 0, 0, look_for_decmpea)) < 0)
src_restart:
if ((list_size = flistxattr(s->src_fd, 0, 0, look_for_decmpea)) <= 0)
{
if (errno == ENOTSUP || errno == EPERM)
if (namebuf) {
free(namebuf);
}
if (list_size == 0) {
return 0;
else
} else if (errno == ENOTSUP || errno == EPERM) {
return 0;
} else {
return -1;
} else
if (nsize == 0)
return 0;
}
}
if ((namebuf = (char *) malloc(nsize)) == NULL)
return -1;
else
nsize = flistxattr(s->src_fd, namebuf, nsize, look_for_decmpea);
/* this is always true on the first call: no buffer yet (namebuf_size == 0) */
if (list_size > namebuf_size) {
if (list_size > MAX_XATTR_BUFFER_SIZE) {
copyfile_warn("source's xattr list size (%zu) exceeds the threshold (%d); trying to allocate", list_size, MAX_XATTR_BUFFER_SIZE);
}
namebuf_size = list_size;
void *tdptr = namebuf;
if ((namebuf =
(void *) realloc((void *) namebuf, namebuf_size)) == NULL)
{
if (tdptr) {
free(tdptr);
}
return -1;
}
}
if (nsize <= 0) {
free(namebuf);
return (int)nsize;
list_size = flistxattr(s->src_fd, namebuf, namebuf_size, look_for_decmpea);
if ((list_size < 0) && (errno == ERANGE) && (tries_left > 0)) {
/* `namebuf` is too small - try again */
tries_left--;
goto src_restart;
} else if (list_size <= 0) {
if (namebuf) {
free(namebuf);
}
return (int)list_size;
}
/*
@ -2855,11 +3055,11 @@ static int copyfile_xattr(copyfile_state_t s)
* set it anyway -- it'll result in a truncated name, which then
* shouldn't match when we get them later.
*/
end = namebuf + nsize - 1;
end = namebuf + list_size - 1;
if (*end != 0)
*end = 0;
if ((xa_dataptr = (void *) malloc(bufsize)) == NULL) {
if ((xa_dataptr = (void *) malloc(xa_bufsize)) == NULL) {
free(namebuf);
return -1;
}
@ -2875,17 +3075,22 @@ static int copyfile_xattr(copyfile_state_t s)
if (strncmp(name, XATTR_QUARANTINE_NAME, end - name) == 0)
continue;
tries_left = MAX_GETXATTR_RETRIES;
get_restart:
if ((xa_size = fgetxattr(s->src_fd, name, 0, 0, 0, look_for_decmpea)) < 0)
{
continue;
}
if (xa_size > bufsize)
if (xa_size > xa_bufsize)
{
if (xa_size > MAX_XATTR_BUFFER_SIZE) {
copyfile_warn("xattr named %s has size (%zu), which exceeds the threshold (%d); trying to allocate", name, list_size, MAX_XATTR_BUFFER_SIZE);
}
void *tdptr = xa_dataptr;
bufsize = xa_size;
xa_bufsize = xa_size;
if ((xa_dataptr =
(void *) realloc((void *) xa_dataptr, bufsize)) == NULL)
(void *) realloc((void *) xa_dataptr, xa_bufsize)) == NULL)
{
free(tdptr);
ret = -1;
@ -2893,8 +3098,13 @@ static int copyfile_xattr(copyfile_state_t s)
}
}
if ((asize = fgetxattr(s->src_fd, name, xa_dataptr, xa_size, 0, look_for_decmpea)) < 0)
if ((asize = fgetxattr(s->src_fd, name, xa_dataptr, xa_bufsize, 0, look_for_decmpea)) < 0)
{
if ((errno == ERANGE) && (tries_left > 0)) {
/* `xa_dataptr` is too small - try again */
tries_left--;
goto get_restart;
}
continue;
}

20
copyfile.xcodeproj/project.pbxproj
View File

@ -7,7 +7,9 @@
objects = {
/* Begin PBXBuildFile section */
096213F7239827D0005847FC /* identical_test.c in Sources */ = {isa = PBXBuildFile; fileRef = 096213F6239827D0005847FC /* identical_test.c */; };
098AF3B622692BF300F9BA42 /* stat_test.c in Sources */ = {isa = PBXBuildFile; fileRef = 098AF3B522692BF300F9BA42 /* stat_test.c */; };
3EF9FA5F2418B6BA003B43E8 /* readonly_fd_test.c in Sources */ = {isa = PBXBuildFile; fileRef = 3EF9FA5E2418B6BA003B43E8 /* readonly_fd_test.c */; };
721D4F071EA95283000F0555 /* copyfile.c in Sources */ = {isa = PBXBuildFile; fileRef = FCCE17C1135A658F002CEE6D /* copyfile.c */; };
721D4F081EA95290000F0555 /* xattr_flags.c in Sources */ = {isa = PBXBuildFile; fileRef = 72406E621676C3C80099568B /* xattr_flags.c */; };
72406E631676C3C80099568B /* xattr_flags.c in Sources */ = {isa = PBXBuildFile; fileRef = 72406E621676C3C80099568B /* xattr_flags.c */; };
@ -18,6 +20,7 @@
72B4C0F41676C47D00C13E05 /* copyfile_private.h in Headers */ = {isa = PBXBuildFile; fileRef = 72B4C0F31676C47D00C13E05 /* copyfile_private.h */; settings = {ATTRIBUTES = (Private, ); }; };
72EAA3B016A72F4500833E98 /* xattr_flags.h in Headers */ = {isa = PBXBuildFile; fileRef = 72EAA3AF16A72F4500833E98 /* xattr_flags.h */; settings = {ATTRIBUTES = (Public, ); }; };
86EF9F0A1834018C00AAB3F3 /* xattr_properties.h in Headers */ = {isa = PBXBuildFile; fileRef = 86EF9F091834018C00AAB3F3 /* xattr_properties.h */; settings = {ATTRIBUTES = (Private, ); }; };
D11048A22455AE7900E8F465 /* xattr_test.c in Sources */ = {isa = PBXBuildFile; fileRef = D11048A12455AE7900E8F465 /* xattr_test.c */; };
FCCE17C3135A658F002CEE6D /* copyfile.c in Sources */ = {isa = PBXBuildFile; fileRef = FCCE17C1135A658F002CEE6D /* copyfile.c */; };
FCCE17C4135A658F002CEE6D /* copyfile.h in Headers */ = {isa = PBXBuildFile; fileRef = FCCE17C2135A658F002CEE6D /* copyfile.h */; settings = {ATTRIBUTES = (Public, ); }; };
/* End PBXBuildFile section */
@ -35,9 +38,13 @@
/* End PBXCopyFilesBuildPhase section */
/* Begin PBXFileReference section */
096213F5239827D0005847FC /* identical_test.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = identical_test.h; sourceTree = "<group>"; };
096213F6239827D0005847FC /* identical_test.c */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.c; path = identical_test.c; sourceTree = "<group>"; };
098AF3B422692BF300F9BA42 /* stat_test.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = stat_test.h; sourceTree = "<group>"; };
098AF3B522692BF300F9BA42 /* stat_test.c */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.c; path = stat_test.c; sourceTree = "<group>"; };
098AF3B7226A510E00F9BA42 /* copyfile_test.entitlements */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.plist.entitlements; path = copyfile_test.entitlements; sourceTree = "<group>"; };
3EF9FA5D2418B6BA003B43E8 /* readonly_fd_test.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = readonly_fd_test.h; sourceTree = "<group>"; };
3EF9FA5E2418B6BA003B43E8 /* readonly_fd_test.c */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.c; path = readonly_fd_test.c; sourceTree = "<group>"; };
3F1EFD4C185C4EB400D1C970 /* copyfile.xcconfig */ = {isa = PBXFileReference; lastKnownFileType = text.xcconfig; name = copyfile.xcconfig; path = xcodescripts/copyfile.xcconfig; sourceTree = "<group>"; };
721D4F051EA95008000F0555 /* libcopyfile.tbd */ = {isa = PBXFileReference; lastKnownFileType = "sourcecode.text-based-dylib-definition"; name = libcopyfile.tbd; path = Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/usr/lib/system/libcopyfile.tbd; sourceTree = DEVELOPER_DIR; };
72406E621676C3C80099568B /* xattr_flags.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = xattr_flags.c; sourceTree = "<group>"; };
@ -53,6 +60,8 @@
72EAA3AF16A72F4500833E98 /* xattr_flags.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = xattr_flags.h; sourceTree = "<group>"; };
861E1C14180F0AF900E65B9A /* xattr_name_with_flags.3 */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = xattr_name_with_flags.3; sourceTree = "<group>"; };
86EF9F091834018C00AAB3F3 /* xattr_properties.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = xattr_properties.h; sourceTree = "<group>"; };
D11048A02455AE7900E8F465 /* xattr_test.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = xattr_test.h; sourceTree = "<group>"; };
D11048A12455AE7900E8F465 /* xattr_test.c */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.c; path = xattr_test.c; sourceTree = "<group>"; };
FCCE17BB135A6444002CEE6D /* libcopyfile.dylib */ = {isa = PBXFileReference; explicitFileType = "compiled.mach-o.dylib"; includeInIndex = 0; path = libcopyfile.dylib; sourceTree = BUILT_PRODUCTS_DIR; };
FCCE17C0135A658F002CEE6D /* copyfile.3 */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = copyfile.3; sourceTree = "<group>"; };
FCCE17C1135A658F002CEE6D /* copyfile.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = copyfile.c; sourceTree = "<group>"; };
@ -89,6 +98,10 @@
isa = PBXGroup;
children = (
726EE9DA1E9423E50017A5B9 /* main.c */,
096213F6239827D0005847FC /* identical_test.c */,
096213F5239827D0005847FC /* identical_test.h */,
3EF9FA5E2418B6BA003B43E8 /* readonly_fd_test.c */,
3EF9FA5D2418B6BA003B43E8 /* readonly_fd_test.h */,
726EE9DE1E9425160017A5B9 /* sparse_test.c */,
726EE9DF1E9425160017A5B9 /* sparse_test.h */,
098AF3B522692BF300F9BA42 /* stat_test.c */,
@ -98,6 +111,8 @@
726EE9E21E946B320017A5B9 /* systemx.c */,
726EE9E31E946B320017A5B9 /* systemx.h */,
098AF3B7226A510E00F9BA42 /* copyfile_test.entitlements */,
D11048A02455AE7900E8F465 /* xattr_test.h */,
D11048A12455AE7900E8F465 /* xattr_test.c */,
);
path = copyfile_test;
sourceTree = "<group>";
@ -201,6 +216,7 @@
developmentRegion = English;
hasScannedForEncodings = 0;
knownRegions = (
English,
en,
);
mainGroup = FCCE17AB135A5FFB002CEE6D;
@ -238,9 +254,12 @@
files = (
098AF3B622692BF300F9BA42 /* stat_test.c in Sources */,
721D4F081EA95290000F0555 /* xattr_flags.c in Sources */,
3EF9FA5F2418B6BA003B43E8 /* readonly_fd_test.c in Sources */,
D11048A22455AE7900E8F465 /* xattr_test.c in Sources */,
721D4F071EA95283000F0555 /* copyfile.c in Sources */,
726EE9DB1E9423E50017A5B9 /* main.c in Sources */,
726EE9E61E946D590017A5B9 /* test_utils.c in Sources */,
096213F7239827D0005847FC /* identical_test.c in Sources */,
726EE9E41E946B320017A5B9 /* systemx.c in Sources */,
726EE9E01E9425160017A5B9 /* sparse_test.c in Sources */,
);
@ -366,6 +385,7 @@
"-ldispatch",
"-lxpc",
);
OTHER_TAPI_FLAGS = "-umbrella System";
SDKROOT = macosx.internal;
"SIM_SUFFIX[sdk=iphonesimulator*]" = _sim;
SUPPORTS_TEXT_BASED_API = YES;

119
copyfile_test/identical_test.c Normal file
View File

@ -0,0 +1,119 @@
//
// identical_test.c
// copyfile_test
//
#include <errno.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <removefile.h>
#include <sys/fcntl.h>
#include <sys/stat.h>
#include <sys/xattr.h>
#include "../copyfile.h"
#include "identical_test.h"
#include "test_utils.h"
#define REGULAR_FILE_NAME "regular_file"
#define REGULAR_DIR_NAME "regular_dir"
#define DUMMY_XATTR_NAME "dummy_xattr"
#define DUMMY_XATTR_DATA "drell"
#define TEST_FILE_DATA "krogan"
static bool verify_src_dst_identical(const char *apfs_test_directory, __unused size_t block_size) {
char regular_file[BSIZE_B] = {0}, folder[BSIZE_B] = {0}, file_inside_folder[BSIZE_B] = {0};
int regular_file_fd, file_inside_folder_fd;
bool success = true;
// The idea here is to verify that copyfile(file1, file1) returns success
// without doing anything.
// There are a few wrinkles - COPYFILE_CHECK still needs to work on these files,
// and we need to make sure that our identity check works on filesystems without
// persistent object identifiers. The first we can easily verify but the second
// is not tested here. Nor are negative tests included (there are an infinite
// number of those, so we rely on the other tests to verify that behavior).
// Create path names.
assert_with_errno(snprintf(regular_file, BSIZE_B, "%s/" REGULAR_FILE_NAME, apfs_test_directory) > 0);
assert_with_errno(snprintf(folder, BSIZE_B, "%s/" REGULAR_DIR_NAME, apfs_test_directory) > 0);
assert_with_errno(snprintf(file_inside_folder, BSIZE_B, "%s/" REGULAR_FILE_NAME, folder) > 0);
// First, verify copyfile(file1, file1),
// where file1 is a regular file.
// Create our regular file.
regular_file_fd = open(regular_file, DEFAULT_OPEN_FLAGS, DEFAULT_OPEN_PERM);
assert_with_errno(regular_file_fd >= 0);
// Write some data to the test file so that we can verify it is not empty.
assert(write(regular_file_fd, TEST_FILE_DATA, sizeof(TEST_FILE_DATA)) > 0);
// Verify copyfile(file1, file1) does nothing.
assert_no_err(copyfile(regular_file, regular_file, NULL, COPYFILE_ALL));
success = success && verify_contents_with_buf(regular_file_fd, 0, (const char *)TEST_FILE_DATA, sizeof(TEST_FILE_DATA));
// Verify copyfile(file1, file1, COPYFILE_EXCL) returns an error.
assert(copyfile(regular_file, regular_file, NULL, COPYFILE_ALL|COPYFILE_EXCL) == -1);
assert(errno == EEXIST);
// Write an dummy xattr to the file to verify COPYFILE_CHECK.
assert_no_err(fsetxattr(regular_file_fd, DUMMY_XATTR_NAME, DUMMY_XATTR_DATA, sizeof(DUMMY_XATTR_DATA), 0, XATTR_CREATE));
// Verify copyfile(file1, file1, ..., COPYFILE_CHECK) works.
assert_no_err(copyfile(regular_file, regular_file, NULL, COPYFILE_CHECK) == COPYFILE_XATTR);
// Now, verify that copyfile(dir1, dir1, COPYFILE_RECURSIVE)
// also returns early. Do this by making sure the contents of a file inside the directory
// do not change after copyfile(COPYFILE_RECURSIVE).
// Create our directory.
assert_no_err(mkdir(folder, DEFAULT_MKDIR_PERM));
// Create a regular file inside that directory.
file_inside_folder_fd = open(file_inside_folder, DEFAULT_OPEN_FLAGS, DEFAULT_OPEN_PERM);
assert_with_errno(file_inside_folder_fd >= 0);
// Write some data to the interior file so that we can verify it is not empty.
assert(write(file_inside_folder_fd, (const char *)TEST_FILE_DATA, sizeof(TEST_FILE_DATA)) > 0);
// Verify copyfile(dir1, dir1, ... COPYFILE_RECURSIVE).
assert_no_err(copyfile(folder, folder, NULL, COPYFILE_RECURSIVE));
success = success && verify_contents_with_buf(file_inside_folder_fd, 0, TEST_FILE_DATA, sizeof(TEST_FILE_DATA));
// Post-test cleanup.
assert_no_err(close(file_inside_folder_fd));
assert_no_err(close(regular_file_fd));
(void)removefile(folder, NULL, REMOVEFILE_RECURSIVE);
(void)removefile(regular_file, NULL, 0);
return success;
}
bool do_src_dst_identical_test(const char *apfs_test_directory, __unused size_t block_size) {
char test_dir[BSIZE_B] = {0};
int test_folder_id;
bool success = true;
printf("START [identical]\n");
// Get ready for the test.
test_folder_id = rand() % DEFAULT_NAME_MOD;
create_test_file_name(apfs_test_directory, "identical", test_folder_id, test_dir);
assert_no_err(mkdir(test_dir, DEFAULT_MKDIR_PERM));
success = verify_src_dst_identical(test_dir, block_size);
if (success) {
printf("PASS [identical]\n");
} else {
printf("FAIL [identical]\n");
}
(void)removefile(test_dir, NULL, REMOVEFILE_RECURSIVE);
return success ? EXIT_SUCCESS : EXIT_FAILURE;
}

14
copyfile_test/identical_test.h Normal file
View File

@ -0,0 +1,14 @@
//
// identical_test.h
// copyfile_test
//
#ifndef identical_test_h
#define identical_test_h
#include <stdbool.h>
#include <stdlib.h>
bool do_src_dst_identical_test(const char *apfs_test_directory, size_t block_size);
#endif /* identical_test_h */

6
copyfile_test/main.c
View File

@ -11,8 +11,11 @@
#include <sys/stat.h>
#include <removefile.h>
#include "identical_test.h"
#include "readonly_fd_test.h"
#include "sparse_test.h"
#include "stat_test.h"
#include "xattr_test.h"
#include "test_utils.h"
#define DISK_IMAGE_SIZE_MB 512
@ -51,11 +54,14 @@ int main(__unused int argc, __unused const char * argv[]) {
// Run our tests.
sranddev();
failed |= do_readonly_fd_test(TEST_DIR, stb.f_bsize);
failed |= do_sparse_test(TEST_DIR, stb.f_bsize);
failed |= do_sparse_recursive_test(TEST_DIR, stb.f_bsize);
failed |= do_fcopyfile_offset_test(TEST_DIR, stb.f_bsize);
failed |= do_preserve_dst_flags_test(TEST_DIR, stb.f_bsize);
failed |= do_preserve_dst_tracked_test(TEST_DIR, stb.f_bsize);
failed |= do_src_dst_identical_test(TEST_DIR, stb.f_bsize);
failed |= do_xattr_test(TEST_DIR, stb.f_bsize);
// Cleanup the disk image we ran our tests on.
if (USING_DISK_IMAGE) {

124
copyfile_test/readonly_fd_test.c Normal file
View File

@ -0,0 +1,124 @@
//
// Copyright (c) 2020 Apple Inc. All rights reserved.
//
#include <err.h>
#include <errno.h>
#include <fcntl.h>
#include <paths.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/acl.h>
#include <sys/attr.h>
#include <sys/stat.h>
#include <sys/time.h>
#include <sys/xattr.h>
#include <unistd.h>
#include "readonly_fd_test.h"
#include "test_utils.h"
static
bool test_readonly_fd_metadata(const char *basedir)
{
char filename[] = ".readonly-ops-XXXXXX";
bool created = false;
bool success = true;
int dirfd = -1;
int tmpfd = -1;
int fd = -1;
acl_t acl = NULL;
static const char test_name[] = "readonly_fd_metadata";
printf("START [%s]\n", test_name);
assert_with_errno((dirfd = open(basedir, O_RDONLY | O_DIRECTORY)) != -1);
assert_with_errno((tmpfd = mkstempsat_np(dirfd, filename, 0)) != -1);
created = true;
assert_with_errno((fd = openat(dirfd, filename, O_RDONLY)) != -1);
close(tmpfd);
tmpfd = -1;
// confirm that writes are disallowed
const char data[] = "failure";
assert(write(fd, data, sizeof(data) - 1) == -1);
// check fchown()
const uid_t uid = geteuid();
const gid_t gid = getegid();
assert_no_err(fchown(fd, uid, gid));
// check fchmod()
assert_no_err(fchmod(fd, 0644));
assert_no_err(fchmod(fd, 0600));
// check fchflags()
assert_no_err(fchflags(fd, UF_HIDDEN));
// check setting timestamps with fsetattrlist
const time_t mtime = 978307200;
const time_t atime = mtime + 1;
struct timeval matimes_usec[] = {{mtime, 0}, {atime, 0}};
assert_no_err(futimes(fd, matimes_usec));
struct attrlist attrlist = {
.bitmapcount = ATTR_BIT_MAP_COUNT,
.commonattr = ATTR_CMN_MODTIME | ATTR_CMN_ACCTIME,
};
struct {
struct timespec mtime;
struct timespec atime;
} matimes_nsec = {{mtime, 0}, {atime, 0}};
assert_no_err(fsetattrlist(fd, &attrlist, &matimes_nsec, sizeof(matimes_nsec), 0));
// check adding and removing xattrs
static const char key[] = "local.test-xattr";
static const char value[] = "local.test-xattr.value";
assert_no_err(fsetxattr(fd, key, value, sizeof(value)-1, 0, 0));
assert_no_err(fremovexattr(fd, key, 0));
// check setting ACLs
assert_with_errno((acl = acl_init(1)) != NULL);
assert_no_err(acl_set_fd(fd, acl));
// log pass/fail before cleanup
if (success) {
printf("PASS [%s]\n", test_name);
} else {
printf("FAIL [%s]\n", test_name);
}
// clean up resources
if (acl) {
acl_free(acl);
}
if (fd != -1) {
close(fd);
}
if (tmpfd != -1) {
close(tmpfd);
}
if (created) {
unlinkat(dirfd, filename, 0);
}
if (dirfd != -1) {
close(dirfd);
}
return success;
}
bool do_readonly_fd_test(const char *apfs_test_directory, size_t block_size __unused)
{
// These tests verify the underlying calls needed for COPYFILE_METADATA
// operations are safe with O_RDONLY file descriptors. If this fails,
// expect <rdar://60074298> to cause many other copyfile() failures.
bool success = true;
success = success && test_readonly_fd_metadata(apfs_test_directory);
return !success; // caller expects nonzero to mean failure
}

13
copyfile_test/readonly_fd_test.h Normal file
View File

@ -0,0 +1,13 @@
//
// Copyright (c) 2020 Apple Inc. All rights reserved.
//
#ifndef readonly_fd_test_h
#define readonly_fd_test_h
#include <stdbool.h>
#include <stddef.h>
bool do_readonly_fd_test(const char *apfs_test_directory, size_t block_size);
#endif /* readonly_fd_test_h */

9
copyfile_test/sparse_test.c
View File

@ -182,12 +182,16 @@ static off_t write_diff_adj_holes(int fd, off_t block_size) {
return 0;
}
static off_t write_nothing(__unused int fd, __unused off_t block_size) {
return 0;
}
typedef struct {
creator_func func; // pointer to function to create a sparse file
const char * name; // null terminated string
} sparse_test_func;
#define NUM_TEST_FUNCTIONS 10
#define NUM_TEST_FUNCTIONS 11
sparse_test_func test_functions[NUM_TEST_FUNCTIONS] = {
{write_start_and_end_holes, "start_and_end_holes"},
{write_middle_hole, "middle_hole"},
@ -198,7 +202,8 @@ sparse_test_func test_functions[NUM_TEST_FUNCTIONS] = {
{write_no_sparse, "no_sparse"},
{write_sparse_odd_offset, "write_sparse_odd_offset"},
{write_sparse_bs_offset, "write_sparse_bs_offset"},
{write_diff_adj_holes, "write_diff_adj_holes"}
{write_diff_adj_holes, "write_diff_adj_holes"},
{write_nothing, "write_nothing"},
};
bool do_sparse_test(const char* apfs_test_directory, size_t block_size) {

4
copyfile_test/stat_test.c
View File

@ -174,8 +174,8 @@ bool do_preserve_dst_tracked_test(const char *test_directory, __unused size_t bl
assert_no_err(copyfile(file_src, file_dst, NULL, COPYFILE_DATA|COPYFILE_STAT|COPYFILE_PRESERVE_DST_TRACKED));
assert_no_err(stat(file_dst, &dst_stb));
success &= (dst_stb.st_size == src_fsize);
success &= (dst_stb.st_flags & UF_TRACKED);
success = success && (dst_stb.st_size == src_fsize);
success = success && (dst_stb.st_flags & UF_TRACKED);
if (success) {
printf("PASS [preserve_dst_tracked]\n");
} else {

102
copyfile_test/test_utils.c
View File

@ -11,10 +11,78 @@
#include <unistd.h>
#include <sys/fcntl.h>
#include <sys/stat.h>
#include <sys/xattr.h>
#include "test_utils.h"
#include "systemx.h"
static bool verify_xattr_content(int fd, const char *xattr_name, const char *expected, ssize_t size) {
// Verify that the file referenced by `fd` has an xattr named `xattr_name`
// of size `size` and with contents equal to `expected`.
char *actual = NULL;
bool equal;
assert(fd > 0 && xattr_name && expected);
assert_with_errno(actual = malloc(size));
assert_with_errno(fgetxattr(fd, xattr_name, actual, size, 0, 0) == size);
equal = (memcmp(actual, expected, size) == 0);
if (!equal) {
printf("xattr %s: content does not match expected\n", xattr_name);
}
free(actual);
return equal;
}
bool verify_fd_xattr_contents(int orig_fd, int copy_fd) {
// Verify that both fd's have the same xattrs.
// We do so by first verifying that `flistxattr()` returns the same size
// for both, and then validating that `copy_fd` has each of the xattrs
// that `orig_fd` has.
char *namebuf = NULL, *xa_buf = NULL, *name, *end;
ssize_t orig_size, copy_size, xa_size;
bool equal = true;
assert((orig_fd > 0) && (copy_fd > 0));
orig_size = flistxattr(orig_fd, 0, 0, XATTR_SHOWCOMPRESSION);
copy_size = flistxattr(copy_fd, 0, 0, XATTR_SHOWCOMPRESSION);
if (orig_size != copy_size) {
printf("xattrlist size: orig_size(%zu) != (%zu)copy_size\n", orig_size, copy_size);
return false;
}
if (orig_size == 0) {
return true;
}
assert_with_errno(namebuf = malloc(orig_size));
assert_with_errno(flistxattr(orig_fd, namebuf, orig_size, 0) == orig_size);
end = namebuf + orig_size - 1;
if (*end != 0) {
*end = 0;
}
for (name = namebuf; name <= end; name += strlen(name) + 1) {
xa_size = fgetxattr(orig_fd, name, 0, 0, 0, 0);
assert(xa_size >= 0);
assert_with_errno(xa_buf = malloc(xa_size));
assert_with_errno(fgetxattr(orig_fd, name, xa_buf, xa_size, 0, 0) == xa_size);
equal = equal && verify_xattr_content(copy_fd, name, xa_buf, xa_size);
free(xa_buf);
if (!equal) {
break;
}
}
free(namebuf);
return equal;
}
bool verify_st_flags(struct stat *sb, uint32_t flags_to_expect) {
// Verify that sb's flags include flags_to_expect.
if (((sb->st_flags & flags_to_expect)) != flags_to_expect) {
@ -26,6 +94,37 @@ bool verify_st_flags(struct stat *sb, uint32_t flags_to_expect) {
return true;
}
bool verify_contents_with_buf(int orig_fd, off_t orig_pos, const char *expected, size_t length)
{
// Read *length* bytes from a file descriptor at a specified position
// and assert that they match *length* bytes from expected.
char orig_contents[length];
bool equal;
assert(orig_fd > 0 && orig_pos >= 0);
memset(orig_contents, 0, length);
errno = 0;
ssize_t pread_res = pread(orig_fd, orig_contents, length, orig_pos);
assert_with_errno(pread_res == (off_t) length);
equal = (memcmp(orig_contents, expected, length) == 0);
if (!equal) {
printf("fd (%lld - %lld) did not match expected contents\n", orig_pos, orig_pos + length);
// Find the first non-matching byte and print it out.
for (size_t bad_off = 0; bad_off < length; bad_off++) {
if (orig_contents[bad_off] != expected[bad_off]) {
printf("first mismatch is at offset %zu, original 0x%llx expected 0x%llx\n",
bad_off, (unsigned long long)orig_contents[bad_off],
(unsigned long long)expected[bad_off]);
break;
}
}
}
return equal;
}
bool verify_fd_contents(int orig_fd, off_t orig_pos, int copy_fd, off_t copy_pos, size_t length) {
// Read *length* contents of the two fds and make sure they compare as equal.
// Don't alter the position of either fd.
@ -33,6 +132,7 @@ bool verify_fd_contents(int orig_fd, off_t orig_pos, int copy_fd, off_t copy_pos
bool equal;
assert(orig_fd > 0 && copy_fd > 0);
assert(orig_pos >= 0);
memset(orig_contents, 0, length);
memset(copy_contents, 0, length);
@ -119,7 +219,7 @@ int create_hole_in_fd(int fd, off_t offset, off_t length) {
void create_test_file_name(const char *dir, const char *postfix, int id, char *string_out) {
// Make a name for this new file and put it in out_name, which should be BSIZE_B bytes.
// Make a name for this new file and put it in string_out, which should be BSIZE_B bytes.
assert_with_errno(snprintf(string_out, BSIZE_B, "%s/testfile-%d.%s", dir, id, postfix) > 0);
}

2
copyfile_test/test_utils.h
View File

@ -35,7 +35,9 @@
#define DIFF_PATH "/usr/bin/diff"
// Test routine helpers.
bool verify_fd_xattr_contents(int orig_fd, int copy_fd);
bool verify_st_flags(struct stat *sb, uint32_t flags_to_expect);
bool verify_contents_with_buf(int orig_fd, off_t orig_pos, const char *expected, size_t length);
bool verify_fd_contents(int orig_fd, off_t orig_pos, int copy_fd, off_t copy_pos, size_t length);
bool verify_copy_contents(const char *orig_name, const char *copy_name);
bool verify_copy_sizes(struct stat *orig_sb, struct stat *copy_sb, copyfile_state_t cpf_state,

84
copyfile_test/xattr_test.c Normal file
View File

@ -0,0 +1,84 @@
//
// xattr_test.c
// copyfile_test
//
#include <unistd.h>
#include <removefile.h>
#include <sys/fcntl.h>
#include <sys/stat.h>
#include <sys/xattr.h>
#include "xattr_test.h"
#include "test_utils.h"
#define SRC_FILE_NAME "src_file"
#define DST_FILE_NAME "dst_file"
#define SMALL_XATTR_NAME "small_xattr"
#define SMALL_XATTR_DATA "drell"
#define BIG_XATTR_NAME "big_xattr"
#define BIG_XATTR_SIZE (20 * 1024 * 1024) // 20MiB
#define DEFAULT_CHAR_MOD 256
static bool copy_and_verify_xattr_contents(const char *src_file, const char *dst_file, int src_file_fd, int dst_file_fd) {
assert_no_err(copyfile(src_file, dst_file, NULL, COPYFILE_XATTR));
return verify_fd_xattr_contents(src_file_fd, dst_file_fd);
}
bool do_xattr_test(const char *apfs_test_directory, __unused size_t block_size) {
char test_dir[BSIZE_B] = {0};
char src_file[BSIZE_B] = {0}, dst_file[BSIZE_B] = {0};
char *big_xattr_data = NULL, buf[4096] = {0};
int test_folder_id;
int src_file_fd, dst_file_fd;
bool success = true;
printf("START [xattr]\n");
// Get ready for the test.
test_folder_id = rand() % DEFAULT_NAME_MOD;
create_test_file_name(apfs_test_directory, "xattr", test_folder_id, test_dir);
assert_no_err(mkdir(test_dir, DEFAULT_MKDIR_PERM));
// Create path names.
assert_with_errno(snprintf(src_file, BSIZE_B, "%s/" SRC_FILE_NAME, test_dir) > 0);
assert_with_errno(snprintf(dst_file, BSIZE_B, "%s/" DST_FILE_NAME, test_dir) > 0);
// Create our files.
src_file_fd = open(src_file, DEFAULT_OPEN_FLAGS, DEFAULT_OPEN_PERM);
assert_with_errno(src_file_fd >= 0);
dst_file_fd = open(dst_file, DEFAULT_OPEN_FLAGS, DEFAULT_OPEN_PERM);
assert_with_errno(dst_file_fd >= 0);
// Sanity check - empty copy
success = success && copy_and_verify_xattr_contents(src_file, dst_file, src_file_fd, dst_file_fd);
// Write a small xattr to the source file.
assert_no_err(fsetxattr(src_file_fd, SMALL_XATTR_NAME, SMALL_XATTR_DATA, sizeof(SMALL_XATTR_DATA), 0, XATTR_CREATE));
success = success && copy_and_verify_xattr_contents(src_file, dst_file, src_file_fd, dst_file_fd);
// Create big xattr data
assert_with_errno(big_xattr_data = malloc(BIG_XATTR_SIZE));
for (int i = 0; i * sizeof(buf) < BIG_XATTR_SIZE; i++) {
memset(buf, rand() % DEFAULT_CHAR_MOD, sizeof(buf));
memcpy(big_xattr_data + (i * sizeof(buf)), buf, sizeof(buf));
}
// Write a big xattr to the source file.
assert_no_err(fsetxattr(src_file_fd, BIG_XATTR_NAME, big_xattr_data, BIG_XATTR_SIZE, 0, XATTR_CREATE));
success = success && copy_and_verify_xattr_contents(src_file, dst_file, src_file_fd, dst_file_fd);
if (success) {
printf("PASS [xattr]\n");
} else {
printf("FAIL [xattr]\n");
}
free(big_xattr_data);
(void)removefile(test_dir, NULL, REMOVEFILE_RECURSIVE);
return success ? EXIT_SUCCESS : EXIT_FAILURE;
}

14
copyfile_test/xattr_test.h Normal file
View File

@ -0,0 +1,14 @@
//
// xattr_test.h
// copyfile_test
//
#ifndef xattr_test_h
#define xattr_test_h
#include <stdbool.h>
bool do_xattr_test(const char *apfs_test_directory, size_t block_size);
#endif /* xattr_test_h */

34
xattr_name_with_flags.3
View File

@ -7,6 +7,7 @@
.Sh NAME
.Nm xattr_preserve_for_intent , xattr_name_with_flags , xattr_name_without_flags ,
.Nm xattr_flags_from_name , xattr_intent_with_flags
.Nd obtain properties related to extended attributes, for use in copying
.Sh LIBRARY
.Lb libc
.Sh SYNOPSIS
@ -23,7 +24,8 @@
.Fn xattr_intent_with_flags "xattr_operation_intent_t" "xattr_flags_t"
.Sh DESCRIPTION
These functions are used in conjunction with copying extended attributes from
one file to another. Various types of copying (an "intent") check flags to
one file to another.
Various types of copying (an "intent") check flags to
determine which is allowed or not.
.Pp
The
@ -32,7 +34,8 @@ function returns an extended attribute name with the appropriate flags encoded
as a string; the
.Fn xattr_name_without_flags
undoes this, giving the name of the extended attribute without the flags
encoding. The slight inverse of that is
encoding.
The slight inverse of that is
.Fn xattr_flags_from_name ,
which will return the flags encoded in a name.
.Pp
@ -62,30 +65,30 @@ named extended attribute should be preserved during a copy for
the given intent.
.Sh INTENT
The type
.Dt xattr_operation_intent_t
is an integral type, which is used to indicate what the intent for the operation
is. The following intent values are defined:
.Vt xattr_operation_intent_t
is an integral type, which is used to indicate what the intent for the operation is.
The following intent values are defined:
.Bl -tag -width XATTR_OPERATION_INTENT_SHARE
.It Dv XATTR_OPERATION_INTENT_COPY
Indicates that the intent is to simply copy from the source to the destination.
E.g., with cp. Most extended attributes should generally be preserved in this
case.
E.g., with cp.
Most extended attributes should generally be preserved in this case.
.It Dv XATTR_OPERATION_INTENT_SAVE
Indicates that intent is to perform a save (perhaps as in a "safe save").
This differs from a copy in that the content may be changing; the destination
may be over-writing or replacing the source, and some extended attributes should
not be preserved during this process.
.It Dv XATTR_OPERATION_INTENT_SHARE
Indicates that the intent is to share, or export, the object. For example,
saving as an attachment in an email message, or placing in a public folder.
Indicates that the intent is to share, or export, the object.
For example, saving as an attachment in an email message, or placing in a public folder.
Sensitive information should probably not be preserved in this case.
.It Dv XATTR_OPERATION_INTENT_SYNC
Indicates that the intent is to sync the object to a service like iCloud Drive.
.El
.Sh FLAGS
Various flags are defined by the type
.Dt xattr_flags_t ;
the currently-defined values for this are
.Vt xattr_flags_t ;
the currently-defined values for this are:
.Bl -tag -width XATTR_FLAG_CONTENT_DEPENDENT
.It Dv XATTR_FLAG_NO_EXPORT
This indicates that the extended attribute should not be exported, or shared.
@ -94,15 +97,16 @@ This is used with
.It Dv XATTR_FLAG_CONTENT_DEPENDENT
This indicates that the extended attribute is tied to the contents of the
file (or vice versa), such that it should be re-created when the contents
are changed. A checksum, for example, should not be copied, and would thus
be marked with this flag.
are changed.
A checksum, for example, should not be copied, and would thus be marked with this flag.
.It Dv XATTR_FLAG_NEVER_PRESERVE
This indicates that the extended attribute should never be copied from a
source object to a destination, no matter what the given intent is.
.It Dv XATTR_FLAG_SYNCABLE
This indicates that the extended attribute should be copied when the file
is synced on services like iCloud Drive. Sync services may enforce additional
restrictions on the acceptable size and number of extended attributes.
is synced on services like iCloud Drive.
Sync services may enforce additional restrictions on the acceptable size and number
of extended attributes.
.El
.Sh EXAMPLE
The following example is a simple function that, given an extended attribute