Update documentation

docs/embeddings/configuration/cloud.md

@@ -1,43 +1,55 @@
 # Cloud
 
-This section describes parameters used to sync compressed indexes with cloud storage. These parameters are only enabled if an embeddings index is stored as compressed. They are set via the [embeddings.load](../../methods/#txtai.embeddings.base.Embeddings.load) and [embeddings.save](../../methods/#txtai.embeddings.base.Embeddings.save) methods.
+The following describes parameters used to sync indexes with cloud storage. Cloud object storage, the [Hugging Face Hub](https://huggingface.co/models) and custom providers are all supported.
+
+Parameters are set via the [embeddings.load](../../methods/#txtai.embeddings.base.Embeddings.load) and [embeddings.save](../../methods/#txtai.embeddings.base.Embeddings.save) methods.
 
 ## provider
 ```yaml
 provider: string
 ```
 
-The cloud storage provider, see [full list of providers here](https://libcloud.readthedocs.io/en/stable/storage/supported_providers.html).
+Cloud provider. Can be one of the following:
+
+- Cloud object storage. Set to one of these [providers](https://libcloud.readthedocs.io/en/stable/storage/supported_providers.html).
+
+- Hugging Face Hub. Set to `huggingface-hub`.
+
+- Custom providers. Set to the full class path of the custom provider.
 
 ## container
 ```yaml
 container: string
 ```
 
-Container/bucket/directory name.
+Container/bucket/directory/repository name.
 
-## key
+## Cloud object storage configuration
+
+In addition to the above common configuration, the cloud object storage provider has the following additional configuration parameters.
+
+### key
 ```yaml
 key: string
 ```
 
 Provider-specific access key. Can also be set via ACCESS_KEY environment variable. Ensure the configuration file is secured if added to the file.
 
-## secret
+### secret
 ```yaml
 secret: string
 ```
 
 Provider-specific access secret. Can also be set via ACCESS_SECRET environment variable. Ensure the configuration file is secured if added to the file.
 
-## host
+### host
 ```yaml
 host: string
 ```
 
 Optional server host name. Set when using a local cloud storage server.
 
-## port
+### port
 ```yaml
 port: int
 ```
@@ -57,3 +69,28 @@ region: string
 ```
 
 Optional parameter to specify the storage region, provider-specific.
+
+## Hugging Face Hub configuration
+
+The huggingface-hub provider supports the following additional configuration parameters. More on these parameters can be found in the [Hugging Face Hub's documentation](https://huggingface.co/docs/huggingface_hub/main/en/package_reference/overview).
+
+### revision
+```yaml
+revision: string
+```
+
+Optional Git revision id which can be a branch name, a tag, or a commit hash
+
+### cache
+```yaml
+cache: string
+```
+
+Path to the folder where cached files are stored
+
+### token
+```yaml
+token: string|boolean
+```
+
+Token to be used for the download. If set to True, the token will be read from the Hugging Face config folder.

docs/embeddings/configuration/index.md

@@ -2,13 +2,20 @@
 
 This following describes available embeddings configuration. These parameters are set via the [Embeddings constructor](../methods#txtai.embeddings.base.Embeddings.__init__).
 
+## format
+```yaml
+format: pickle|json
+```
+
+Sets the configuration storage format. Defaults to pickle.
+
 ## path
 ```yaml
 path: string
 ```
 
 Sets the path for a vectors model. When using a transformers/sentence-transformers model, this can be any model on the
-[Hugging Face Model Hub](https://huggingface.co/models) or a local file path. Otherwise, it must be a local file path to a word embeddings model.
+[Hugging Face Hub](https://huggingface.co/models) or a local file path. Otherwise, it must be a local file path to a word embeddings model.
 
 ## method
 ```yaml
@@ -86,6 +93,17 @@ tokenize: boolean
 Enables string tokenization (defaults to false). This method applies tokenization rules that only work with English language text and may increase the quality of
 English language sentence embeddings in some situations.
 
+## instructions
+```yaml
+instructions:
+    query: prefix for queries
+    data: prefix for indexing
+```
+
+Instruction-based models use prefixes to modify how embeddings are computed. This is especially useful with asymmetric search, which is when the query and indexed data are of vastly different lengths. In other words, short queries with long documents.
+
+[E5-base](https://huggingface.co/intfloat/e5-base) is an example of a model that accepts instructions. It takes `query: ` and `passage: ` prefixes and uses those to generate embeddings that work well for asymmetric search.
+
 ## backend
 ```yaml
 backend: faiss|hnsw|annoy