Magic-Mirror/gecko-dev
1
0
Fork 0
mirror of https://github.com/mozilla/gecko-dev.git synced 2024-11-24 13:21:05 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Bug 1736855 - Add IOUtils Migration Guide to source docs r=Gijs DONTBUILD

Browse Source 
A public gdoc's not bad, but source docs is better.
This place was chosen because the implementation is nearby,
even though it's really support for toolkit and browser stuff.

Also update the docs a tad. `exists` and `setPermissions` are now available!
And we have JSON read/write, too!

Differential Revision: https://phabricator.services.mozilla.com/D129069
This commit is contained in:
Chris H-C 2021-10-21 15:44:59 +00:00
parent bcf621c781
commit 6b06ddd335
2 changed files with 562 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
dom/docs/index.rst
View File

@ -12,3 +12,4 @@ These linked pages contain design documents for the DOM implementation in Gecko.
scriptSecurity/xray_vision
workersAndStorage/index
webIdlBindings/index
ioutils_migration

561
dom/docs/ioutils_migration.md Normal file
View File

@ -0,0 +1,561 @@
# IOUtils Migration Guide
**Improving performance through a new file API**
---
## What is IOUtils?
`IOUtils` is a privileged JavaScript API for performing file I/O in the Firefox frontend.
It was developed as a replacement for `OS.File`, addressing
[bug 1231711](https://bugzilla.mozilla.org/show_bug.cgi?id=1231711).
It is *not to be confused* with the unprivileged
[DOM File API](https://developer.mozilla.org/en-US/docs/Web/API/File).
`IOUtils` provides a minimal API surface to perform common
I/O tasks via a collection of static methods inspired from `OS.File`.
It is implemented in C++, and exposed to JavaScript via WebIDL bindings.
The most up-to-date API can always be found in
[IOUtils.webidl](https://searchfox.org/mozilla-central/source/dom/chrome-webidl/IOUtils.webidl).
## Differences from `OS.File`
`IOUtils` has a similar API to `OS.File`, but one should keep in mind some key differences.
### No `File` instances
At no point does `IOUtils` expose a file descriptor/handle to the caller.
This API operates only on absolute path strings.
Platform-specific file descriptors may have been exposed by instances of `OS.File` through the
[`fd`](https://developer.mozilla.org/en-US/docs/Mozilla/JavaScript_code_modules/OSFile.jsm/OS.File_for_workers#Attributes)
attribute.
This will no longer be possible.
### WebIDL has no `Date` type
`IOUtils` is written in C++ and exposed to JavaScript through WebIDL.
Many uses of `OS.File` concern themselves with obtaining or manipulating file metadata,
like the last modified time, however the `Date` type does not exist in WebIDL.
Using `IOUtils`,
these values are returned to the caller as the number of milliseconds since
`1970-01-01T00:00:00Z`.
`Date`s can be safely constructed from these values if needed.
For example, to obtain the last modification time of a file and update it to the current time:
```js
let { lastModified } = await IOUtils.stat(path);
let lastModifiedDate = new Date(lastModified);
let now = new Date();
await IOUtils.touch(path, now.valueOf());
```
### Some methods are not implemented
For various reasons
(complexity, safety, availability of underlying system calls, usefulness, etc.)
the following `OS.File` methods have no analogue in IOUtils.
They also will **not** be implemented.
- void unixSymlink(in string targetPath, in string createPath)
- string getCurrentDirectory(void)
- void setCurrentDirectory(in string path)
- object open(in string path)
- object openUnique(in string path)
### Errors are reported as `DOMException`s
When an `OS.File` method runs into an error,
it will throw/reject with a custom
[`OS.File.Error`](https://developer.mozilla.org/en-US/docs/Mozilla/JavaScript_code_modules/OSFile.jsm/OS.File.Error).
These objects have custom attributes that can be checked for common error cases.
`IOUtils` has similar behaviour, however its methods consistently reject with a
[`DOMException`](https://developer.mozilla.org/en-US/docs/Web/API/DOMException)
whose name depends on the failure:
| Exception Name | Reason for exception |
| -------------- | -------------------- |
| `NotFoundError` | A file at the specified path could not be found on disk. |
| `NotAllowedError` | Access to a file at the specified path was denied by the operating system. |
| `NotReadableError` | A file at the specified path could not be read for some reason. It may have been too big to read, or it was corrupt, or some other reason. The exception message should have more details. |
| `ReadOnlyError` | A file at the specified path is read only and could not be modified. |
| `NoModificationAllowedError` | A file already exists at the specified path and could not be overwritten according to the specified options. The exception message should have more details. |
| `OperationError` | Something went wrong during the I/O operation. E.g. failed to allocate a buffer. The exception message should have more details. |
| `UnknownError` | An unknown error occurred in the implementation. An nsresult error code should be included in the exception message to assist with debugging and improving `IOUtils` internal error handling. |
### `IOUtils` is async only
`OS.File` provided an asynchronous front-end for main-thread consumers,
and a synchronous front-end for servicer workers.
`IOUtils` only provides an asynchronous API,
which can still be called from both the main thread and service workers.
## `OS.File` vs `IOUtils`
Some methods and options of `OS.File` keep the same name and underlying behaviour in `IOUtils`,
but others have been renamed.
The following is a detailed comparison with examples of the methods and options in each API.
### Reading a file
`IOUtils` provides the following methods to read data from a file. Like
`OS.File`, they accept an `options` dictionary.
```idl
Promise<Uint8Array> read(DOMString path, ...);
Promise<DOMString> readUTF8(DOMString path, ...);
Promise<any> readJSON(DOMString path, ...);
```
#### Options
| `OS.File` option | `IOUtils` option | Description |
| ------------------ | ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| bytes: number? | maxBytes: number? | If specified, read only up to this number of bytes. Otherwise, read the entire file. Default is null. |
| compression: 'lz4' | decompress: boolean | If true, read the file and return the decompressed LZ4 stream. Otherwise, just read the file byte-for-byte. Default is false. |
| encoding: 'utf-8' | N/A; use `readUTF8` instead. | Interprets the file as UTF-8 encoded text, and returns a string to the caller. |
#### Examples
##### Read raw (unsigned) byte values
**`OS.File`**
```js
let bytes = await OS.File.read(path); // Uint8Array
```
**`IOUtils`**
```js
let bytes = await IOUtils.read(path); // Uint8Array
```
##### Read UTF-8 encoded text
**`OS.File`**
```js
let utf8 = await OS.File.read(path, { encoding: 'utf-8' }); // string
```
**`IOUtils`**
```js
let utf8 = await IOUtils.readUTF8(path); // string
```
##### Read JSON file
**`IOUtils`**
```js
let obj = await IOUtils.readJSON(path); // object
```
##### Read LZ4 compressed file contents
**`OS.File`**
```js
// Uint8Array
let bytes = await OS.File.read(path, { compression: 'lz4' });
// string
let utf8 = await OS.File.read(path, {
encoding: 'utf-8',
compression: 'lz4',
});
```
**`IOUtils`**
```js
let bytes = await IOUtils.read(path, { decompress: true }); // Uint8Array
let utf8 = await IOUtils.readUTF8(path, { decompress: true }); // string
```
### Writing to a file
IOUtils provides the following methods to write data to a file. Like
OS.File, they accept an options dictionary.
```idl
Promise<unsigned long long> write(DOMString path, Uint8Array data, ...);
Promise<unsigned long long> writeUTF8(DOMString path, DOMString string, ...);
Promise<unsigned long long> writeJSON(DOMString path, any value, ...);
```
#### Options
| `OS.File` option | `IOUtils` option | Description |
| -------------------- | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| backupTo: string? | backupFile: string? | Identifies the path to backup the target file to before performing the write operation. If unspecified, no backup will be performed. Default is null. |
| tmpPath: string? | tmpPath: string? | Identifies a path to write to first, before performing a move to overwrite the target file. If unspecified, the target file will be written to directly. Default is null. |
| noOverwrite: boolean | noOverwrite: boolean | If true, fail if the destination already exists. Default is false. |
| flush: boolean | flush: boolean | If true, force the OS to flush its internal buffers to disk. Default is false. |
| encoding: 'utf-8' | N/A; use `writeUTF8` instead. | Allows the caller to supply a string to be encoded as utf-8 text on disk. |
#### Examples
##### Write raw (unsigned) byte values
**`OS.File`**
```js
let bytes = new Uint8Array();
await OS.File.writeAtomic(path, bytes);
```
**`IOUtils`**
```js
let bytes = new Uint8Array();
await IOUtils.write(path, bytes);
```
##### Write UTF-8 encoded text
**`OS.File`**
```js
let str = "";
await OS.File.writeAtomic(path, str, { encoding: 'utf-8' });
```
**`IOUtils`**
```js
let str = "";
await IOUtils.writeUTF8(path, str);
```
##### Write A JSON object
**`IOUtils`**
```js
let obj = {};
await IOUtils.writeJSON(path, obj);
```
##### Write with LZ4 compression
**`OS.File`**
```js
let bytes = new Uint8Array();
await OS.File.writeAtomic(path, bytes, { compression: 'lz4' });
let str = "";
await OS.File.writeAtomic(path, str, {
compression: 'lz4',
});
```
**`IOUtils`**
```js
let bytes = new Uint8Array();
await IOUtils.write(path, bytes, { compress: true });
let str = "";
await IOUtils.writeUTF8(path, str, { compress: true });
```
### Move a file
`IOUtils` provides the following method to move files on disk.
Like `OS.File`, it accepts an options dictionary.
```idl
Promise<void> move(DOMString sourcePath, DOMString destPath, ...);
```
#### Options
| `OS.File` option | `IOUtils` option | Description |
| -------------------- | ----------------------------------- | ---------------------------------------------------------------------------- |
| noOverwrite: boolean | noOverwrite: boolean | If true, fail if the destination already exists. Default is false. |
| noCopy: boolean | N/A; will not be implemented | This option is not implemented in `IOUtils`, and will be ignored if provided |
#### Example
**`OS.File`**
```js
await OS.File.move(srcPath, destPath);
```
**`IOUtils`**
```js
await IOUtils.move(srcPath, destPath);
```
### Remove a file
`IOUtils` provides *one* method to remove files from disk.
`OS.File` provides several methods.
```idl
Promise<void> remove(DOMString path, ...);
```
#### Options
| `OS.File` option | `IOUtils` option | Description |
| ---------------------------------------------------------- | --------------------- | ---------------------------------------------------------------------------------------------------------------- |
| ignoreAbsent: boolean | ignoreAbsent: boolean | If true, and the destination does not exist, then do not raise an error. Default is true. |
| N/A; `OS.File` has dedicated methods for directory removal | recursive: boolean | If true, and the target is a directory, recursively remove the directory and all its children. Default is false. |
#### Examples
##### Remove a file
**`OS.File`**
```js
await OS.File.remove(path, { ignoreAbsent: true });
```
**`IOUtils`**
```js
await IOUtils.remove(path);
```
##### Remove a directory
**`OS.File`**
```js
await OS.File.removeDir(path, { ignoreAbsent: true });
```
**`IOUtils`**
```js
await IOUtils.remove(path, { recursive: true });
```
##### Remove an empty directory
**`OS.File`**
```js
await OS.File.removeEmptyDir(path, { ignoreAbsent: true });
```
**`IOUtils`**
```js
await IOUtils.remove(path, { recursive: true });
```
### Make a directory
`IOUtils` provides the following method to create directories on disk.
Like `OS.File`, it accepts an options dictionary.
```idl
Promise<void> makeDirectory(DOMString path, ...);
```
#### Options
| `OS.File` option | `IOUtils` option | Description |
| ----------------------- | ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| ignoreExisting: boolean | ignoreExisting: boolean | If true, succeed even if the target directory already exists. Default is true. |
| from: string | createAncestors: boolean | If true, `IOUtils` will create all missing ancestors in a path. Default is true. This option differs from `OS.File`, which requires the caller to specify a root path from which to create missing directories. |
| unixMode | N/A | `IOUtils` does not support setting a custom directory mode on unix. |
| winSecurity | N/A | `IOUtils` does not support setting custom directory security settings on Windows. |
#### Example
**`OS.File`**
```js
await OS.File.makeDir(srcPath, destPath);
```
**`IOUtils`**
```js
await IOUtils.makeDirectory(srcPath, destPath);
```
### Update a file's modification time
`IOUtils` provides the following method to update a file's modification time.
```idl
Promise<void> touch(DOMString path, optional long long modification);
```
#### Example
**`OS.File`**
```js
await OS.File.setDates(path, new Date(), new Date());
```
**`IOUtils`**
```js
await IOUtils.touch(path, new Date().valueOf());
```
### Get file metadata
`IOUtils` provides the following method to query file metadata.
```idl
Promise<void> stat(DOMString path);
```
#### Example
**`OS.File`**
```js
let fileInfo = await OS.File.stat(path);
```
**`IOUtils`**
```js
let fileInfo = await IOUtils.stat(path);
```
### Copy a file
`IOUtils` provides the following method to copy a file on disk.
Like `OS.File`, it accepts an options dictionary.
```idl
Promise<void> copy(DOMString path, ...);
```
#### Options
| `OS.File` option | `IOUtils` option | Description |
| ------------------------------------------------------------------- | -------------------- | -------------------------------------------------------------------|
| noOverwrite: boolean | noOverwrite: boolean | If true, fail if the destination already exists. Default is false. |
| N/A; `OS.File` does not appear to support recursively copying files | recursive: boolean | If true, copy the source recursively. |
#### Examples
##### Copy a file
**`OS.File`**
```js
await OS.File.copy(srcPath, destPath);
```
**`IOUtils`**
```js
await IOUtils.copy(srcPath, destPath);
```
##### Copy a directory recursively
**`OS.File`**
```js
// Not easy to do.
```
**`IOUtils`**
```js
await IOUtils.copy(srcPath, destPath, { recursive: true });
```
### Iterate a directory
At the moment, `IOUtils` does not have a way to expose an iterator for directories.
This is blocked by
[bug 1577383](https://bugzilla.mozilla.org/show_bug.cgi?id=1577383).
As a stop-gap for this functionality,
one can get all the children of a directory and iterate through the returned path array using the following method.
```idl
Promise<sequence<DOMString>> getChildren(DOMString path);
```
#### Example
**`OS.File`**
```js
for await (const { path } of new OS.FileDirectoryIterator(dirName)) {</p>
...
}
```
**`IOUtils`**
```js
for (const path of await IOUtils.getChildren(dirName)) {
...
}
```
### Check if a file exists
`IOUtils` provides the following method analogous to the `OS.File` method of the same name.
```idl
Promise<boolean> exists(DOMString path);
```
#### Example
**`OS.File`**
```js
if (await OS.File.exists(path)) {
...
}
```
**`IOUtils`**
```js
if (await IOUtils.exists(path)) {
...
}
```
### Set the permissions of a file
`IOUtils` provides the following method analogous to the `OS.File` method of the same name.
```idl
Promise<void> setPermissions(DOMString path, unsigned long permissions, optional boolean honorUmask = true);
```
#### Options
| `OS.File` option | `IOUtils` option | Description |
| ----------------------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| unixMode: number | permissions: usigned long | The UNIX file mode representing the permissions. Required in IOUtils. |
| unixHonorUmask: boolean | honorUmask: boolean | If omitted or true, any UNIX file mode is modified by the permissions. Otherwise the exact value of the permissions will be applied. |
#### Example
**`OS.File`**
```js
await OS.File.setPermissions(path, { unixMode: 0o600 });
```
**`IOUtils`**
```js
await IOUtils.setPermissions(path, 0o600);
```
## FAQs
**Why should I use `IOUtils` instead of `OS.File`?**
[Bug 1231711](https://bugzilla.mozilla.org/show_bug.cgi?id=1231711)
provides some good context, but some reasons include:
* reduced cache-contention,
* faster startup, and
* less memory usage.
Additionally, `IOUtils` benefits from a native implementation,
which assists in performance-related work for
[Project Fission](https://hacks.mozilla.org/2021/05/introducing-firefox-new-site-isolation-security-architecture/).
We are actively working to migrate old code usages of `OS.File`
to analogous `IOUtils` calls, so new usages of `OS.File`
should not be introduced at this time.
**Do I need to import anything to use this API?**
Nope! It's available via the `IOUtils` global in JavaScript (`ChromeOnly` context).
**Can I use this API from C++ or Rust?**
Currently usage is geared exclusively towards JavaScript callers,
and all C++ methods are private except for the Web IDL bindings.
However given sufficient interest,
it should be easy to expose ergonomic public methods for C++ and/or Rust.
**Why isn't `IOUtils` written in Rust?**
At the time of writing,
support for Web IDL bindings was more mature for C++ oriented tooling than it was for Rust.
**Is `IOUtils` feature complete? When will it be available?**
`IOUtils` is considered feature complete as of Firefox 83.