Compare commits

...
Author SHA1 Message Date
Will Schurman 7f102ef29d feat: add EntityField transformation 2022-07-01 13:00:43 -04:00
James Ide 0dbf5e98bb [tests] Run tests on Node 18 instead of Node 12 (#174)
Node 16 is LTS now and 12 is unsupported. 18 is also now released and we will one day run on it so this makes sure one of our major dependencies works on Node 18.

All tests pass, no changes needed.
2022-06-28 17:24:42 -07:00
James Ide 4347d9d577 [npm][ts] Upgrade all minor npm deps incl. TS 4.7 (#176)
Why
---
There were some automated security notices about outdated npm package versions, many of which often can be addressed with minor dependency upgrades.

How
---
Deleted and regenerated yarn.lock, which affects this repo but not consumers of these packages. However, TypeScript 4.7 has a change where it considers stringifying a Symbol to be a potential error. This commit explicitly stringifies keys (entity field names).

I opted for `String(fieldName)` over constraining the field name with `N extends string & keyof TFields` because adding `string &` is "viral" in the sense that lots of files including users of Entity would need to add `string &`. Also, using Symbol field names probably would work fine, they just need to be stringified before printing them.

There were also some Koa router type changes that treat params as `string | undefined` so we need to use `!` in the Entity example router.

Test Plan
---
tsc + unit tests
2022-06-21 00:18:15 -07:00
Will Schurman 9a91b19466 chore: clean up metrics API a bit and add docblocks (#175) 2022-06-08 17:32:07 -07:00
Will Schurman cb7aaad979 docs: update docblock for EntityQueryContext classes (#173) 2022-05-11 11:22:49 -07:00
James Ide 9fcfbf746a feat: update ioredis to v5, make it a devDependency of the cache packages (#172)
Why: ioredis v5 ships with its own TypeScript declarations. The breaking changes from v4 to v5 affect the code that instantiates ioredis, not the code that uses it. As such, I believe that entity is still technically compatible with ioredis v4, but for the sake of keeping our test matrix smaller, this commit uses just ioredis v5 in tests and makes v5 a peer dependency of `entity-cache-adapter-redis`.

Additionally ioredis was a dependency of `entity-cache-adapter-redis`. I changed it to a devDependency and changed the import to `import type` since the package is used only for type checking.

ioredis v5.0.0 changelog: https://github.com/luin/ioredis/releases/tag/v5.0.0. Notable it drops support for Node 10 and for custom Promise libraries, neither of which we use.

Test Plan: building all packages and testing them with Lerna succeeds
2022-04-11 09:44:57 -07:00
James Ide 9c44e8e8d4 Upgrade TS to 4.6.x, use knex 1.0.5 during testing (#171)
Why: Expo's API server will use TS 4.6 and knex 1.0.5. This change makes it so that we run entity's CI tests against these new dependencies.

How: Upgraded the version of typescript in the root package.json. Modified yarn.lock to use knex 1.0.5 instead of updating the db adapter's package.json since knex 1.0.2 still works fine.

Test Plan: Lerna runs tsc in every package (no errors). Unit tests in CI.
2022-04-11 09:16:19 -07:00
Will Schurman d731b0ddb7 v0.25.3 2022-03-12 14:32:00 -07:00
Will Schurman 293868b1a7 fix: lock lru-cache to v6.0.0 (#170) 2022-03-12 13:29:16 -08:00
56 changed files with 3139 additions and 3391 deletions

No files matched your search

+1 -1
View File
@@ -14,7 +14,7 @@ jobs:
ref: ${{ github.event.pull_request.head.sha }}
- uses: actions/setup-node@v1
with:
node-version: 12.x
node-version: 18.x
- name: Cache Node.js modules
uses: actions/cache@v1
with:
+11
View File
@@ -3,6 +3,17 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [0.25.3](https://github.com/expo/entity/compare/v0.25.2...v0.25.3) (2022-03-12)
### Bug Fixes
* lock lru-cache to v6.0.0 ([#170](https://github.com/expo/entity/issues/170)) ([293868b](https://github.com/expo/entity/commit/293868b1a73df9f749216af68b1ae94ac22ac2b9))
## [0.25.2](https://github.com/expo/entity/compare/v0.25.1...v0.25.2) (2022-03-12)
+1 -1
View File
@@ -4,5 +4,5 @@
],
"npmClient": "yarn",
"useWorkspaces": true,
"version": "0.25.2"
"version": "0.25.3"
}
+4 -4
View File
@@ -19,10 +19,9 @@
"@expo/entity": "file:packages/entity",
"@expo/results": "^1.0.0",
"@types/invariant": "^2.2.33",
"@types/ioredis": "^4.26.4",
"@types/jest": "^26.0.10",
"@types/jsbn": "^1.2.29",
"@types/lru-cache": "^7.4.0",
"@types/lru-cache": "^5.1.1",
"@types/node": "^14.6.0",
"@types/uuid": "^8.3.0",
"@typescript-eslint/eslint-plugin": "^4.14.0",
@@ -31,16 +30,17 @@
"eslint": "^7.7.0",
"eslint-config-universe": "^7.0.0",
"eslint-plugin-tsdoc": "^0.2.11",
"ioredis": "^5.0.4",
"jest": "^26.6.3",
"lerna": "^4.0.0",
"lru-cache": "^7.3.1",
"lru-cache": "^6.0.0",
"nullthrows": "^1.1.1",
"pg": "^8.6.0",
"prettier": "^2.3.2",
"ts-jest": "~26.4.4",
"ts-mockito": "^2.6.1",
"typedoc": "^0.21.9",
"typescript": "^4.4.2",
"typescript": "^4.6.3",
"uuid": "^8.3.0"
},
"dependencies": {
@@ -3,6 +3,17 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [0.25.3](https://github.com/expo/entity/compare/v0.25.2...v0.25.3) (2022-03-12)
### Bug Fixes
* lock lru-cache to v6.0.0 ([#170](https://github.com/expo/entity/issues/170)) ([293868b](https://github.com/expo/entity/commit/293868b1a73df9f749216af68b1ae94ac22ac2b9))
## [0.25.2](https://github.com/expo/entity/compare/v0.25.1...v0.25.2) (2022-03-12)
@@ -3,23 +3,26 @@
Cross-request [LRU](https://github.com/isaacs/node-lru-cache) cache adapter for `@expo/entity`. Use
this cache with caution - it is nonstandard. The cache is shared between requests in the node process.
Note: This uses version 6.0.0 of `node-lru-cache` since it the most tuned version for our use case (low TTL + LRU). Upgrading
to 7.x will cause high memory usage for the entity cache adapter use case since it allocates fixed-size data structures up front to tune for the non-TTL use case: https://github.com/isaacs/node-lru-cache/issues/208.
[Documentation](https://expo.github.io/entity/modules/_expo_cache_adapter_local_memory.html)
## Why NOT use this cache
Because this is an in-memory cache, cross-box invalidation is not possible. Do not use this cache
Because this is an in-memory cache, cross-machine invalidation is not possible. Do not use this cache
if you have the following use cases:
- The objects stored have high mutability
- The objects stored are mutable
- Reading a stale object from the cache is not acceptable in your application
- Cross-box invalidation is not possible
- Cross-machine invalidation is not possible
## Typical use cases
If your application sees many requests fetching the same objects, you can save a trip to your cache
cluster and backing datastore by using this in-memory cache. Here are some good use cases:
- The objects stored are mostly immutable
- The objects stored are mostly immutable, and reading a stale object for a short TTL is acceptable
- You have a low TTL setting in your cache
## Usage
@@ -1,6 +1,6 @@
{
"name": "@expo/entity-cache-adapter-local-memory",
"version": "0.25.2",
"version": "0.25.3",
"description": "Cross-request local memory cache adapter for @expo/entity",
"files": [
"build",
@@ -30,9 +30,9 @@
"@expo/entity": "*"
},
"dependencies": {
"lru-cache": "^7.3.1"
"lru-cache": "^6.0.0"
},
"devDependencies": {
"@expo/entity": "^0.25.2"
"@expo/entity": "^0.25.3"
}
}
@@ -7,76 +7,6 @@ export const DOES_NOT_EXIST_LOCAL_MEMORY_CACHE = Symbol('doesNotExist');
type LocalMemoryCacheValue<TFields> = Readonly<TFields> | typeof DOES_NOT_EXIST_LOCAL_MEMORY_CACHE;
export type LocalMemoryCache<TFields> = LRUCache<string, LocalMemoryCacheValue<TFields>>;
type LRUCacheOptionsV7<K, V> = {
/**
* the number of most recently used items to keep.
* note that we may store fewer items than this if maxSize is hit.
*/
max: number;
/**
* if you wish to track item size, you must provide a maxSize
* note that we still will only keep up to max *actual items*,
* so size tracking may cause fewer than max items to be stored.
* At the extreme, a single item of maxSize size will cause everything
* else in the cache to be dropped when it is added. Use with caution!
* Note also that size tracking can negatively impact performance,
* though for most cases, only minimally.
*/
maxSize?: number;
/**
* function to calculate size of items. useful if storing strings or
* buffers or other items where memory size depends on the object itself.
* also note that oversized items do NOT immediately get dropped from
* the cache, though they will cause faster turnover in the storage.
*/
sizeCalculation?: (value: V, key: K) => number;
/**
* function to call when the item is removed from the cache
* Note that using this can negatively impact performance.
*/
dispose?: (value: V, key: K) => void;
/**
* max time to live for items before they are considered stale
* note that stale items are NOT preemptively removed by default,
* and MAY live in the cache, contributing to its LRU max, long after
* they have expired.
* Also, as this cache is optimized for LRU/MRU operations, some of
* the staleness/TTL checks will reduce performance, as they will incur
* overhead by deleting items.
* Must be a positive integer in ms, defaults to 0, which means "no TTL"
*/
ttl?: number;
/**
* return stale items from cache.get() before disposing of them
* boolean, default false
*/
allowStale?: boolean;
/**
* update the age of items on cache.get(), renewing their TTL
* boolean, default false
*/
updateAgeOnGet?: boolean;
/**
* update the age of items on cache.has(), renewing their TTL
* boolean, default false
*/
updateAgeOnHas?: boolean;
/**
* update the "recently-used"-ness of items on cache.has()
* boolean, default false
*/
updateRecencyOnHas?: boolean;
};
export default class GenericLocalMemoryCacher<TFields> implements IEntityGenericCacher<TFields> {
constructor(private readonly localMemoryCache: LocalMemoryCache<TFields>) {}
@@ -86,21 +16,17 @@ export default class GenericLocalMemoryCacher<TFields> implements IEntityGeneric
const DEFAULT_LRU_CACHE_MAX_AGE_SECONDS = 10;
const DEFAULT_LRU_CACHE_SIZE = 10000;
const maxAgeSeconds = options.ttlSeconds ?? DEFAULT_LRU_CACHE_MAX_AGE_SECONDS;
const lruCacheOptions: LRUCacheOptionsV7<string, TFields> = {
return new LRUCache<string, LocalMemoryCacheValue<TFields>>({
max: options.maxSize ?? DEFAULT_LRU_CACHE_SIZE,
maxSize: options.maxSize ?? DEFAULT_LRU_CACHE_SIZE,
sizeCalculation: (value: LocalMemoryCacheValue<TFields>) =>
value === DOES_NOT_EXIST_LOCAL_MEMORY_CACHE ? 0 : 1,
ttl: maxAgeSeconds * 1000, // convert to ms
};
return new LRUCache<string, LocalMemoryCacheValue<TFields>>(lruCacheOptions as any);
length: (value) => (value === DOES_NOT_EXIST_LOCAL_MEMORY_CACHE ? 0 : 1),
maxAge: maxAgeSeconds * 1000, // convert to ms
});
}
static createNoOpCache<TFields>(): LocalMemoryCache<TFields> {
return new LRUCache<string, LocalMemoryCacheValue<TFields>>({
max: 1,
maxSize: 1,
sizeCalculation: () => 10, // make all things larger than max size
max: 0,
maxAge: -1,
});
}
@@ -142,7 +68,7 @@ export default class GenericLocalMemoryCacher<TFields> implements IEntityGeneric
public async invalidateManyAsync(keys: readonly string[]): Promise<void> {
for (const key of keys) {
this.localMemoryCache.delete(key);
this.localMemoryCache.del(key);
}
}
@@ -68,7 +68,7 @@ export default class LocalMemoryCacheAdapter<TFields> extends EntityCacheAdapter
fieldValue: NonNullable<TFields[N]>
): string {
const columnName = this.entityConfiguration.entityToDBFieldsKeyMapping.get(fieldName);
invariant(columnName, `database field mapping missing for ${fieldName}`);
invariant(columnName, `database field mapping missing for ${String(fieldName)}`);
const parts = [
this.entityConfiguration.tableName,
`${this.entityConfiguration.cacheKeyVersion}`,
@@ -5,8 +5,7 @@ describe(GenericLocalMemoryCacher, () => {
it('creates a cache with default options', () => {
const cache = GenericLocalMemoryCacher.createLRUCache();
expect(cache.max).toBe(10000);
expect(cache.maxSize).toBe(10000);
expect(cache.ttl).toBe(10000);
expect(cache.maxAge).toBe(10000);
});
it('respects specified options', () => {
@@ -15,8 +14,7 @@ describe(GenericLocalMemoryCacher, () => {
maxSize: 10,
});
expect(cache.max).toBe(10);
expect(cache.maxSize).toBe(10);
expect(cache.ttl).toBe(3000);
expect(cache.maxAge).toBe(3000);
});
});
@@ -3,6 +3,14 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [0.25.3](https://github.com/expo/entity/compare/v0.25.2...v0.25.3) (2022-03-12)
**Note:** Version bump only for package @expo/entity-cache-adapter-redis
## [0.25.2](https://github.com/expo/entity/compare/v0.25.1...v0.25.2) (2022-03-12)
**Note:** Version bump only for package @expo/entity-cache-adapter-redis
@@ -1,6 +1,6 @@
{
"name": "@expo/entity-cache-adapter-redis",
"version": "0.25.2",
"version": "0.25.3",
"description": "Redis cache adapter for @expo/entity",
"files": [
"build",
@@ -27,12 +27,11 @@
"author": "Expo",
"license": "MIT",
"peerDependencies": {
"@expo/entity": "*"
},
"dependencies": {
"ioredis": "^4.27.3"
"@expo/entity": "*",
"ioredis": ">=5"
},
"devDependencies": {
"@expo/entity": "^0.25.2"
"@expo/entity": "^0.25.3",
"ioredis": "^5.0.4"
}
}
@@ -1,6 +1,6 @@
import { EntityCacheAdapter, EntityConfiguration, CacheLoadResult, mapKeys } from '@expo/entity';
import invariant from 'invariant';
import { Redis } from 'ioredis';
import type { Redis } from 'ioredis';
import GenericRedisCacher from './GenericRedisCacher';
@@ -113,7 +113,7 @@ export default class RedisCacheAdapter<TFields> extends EntityCacheAdapter<TFiel
fieldValue: NonNullable<TFields[N]>
): string {
const columnName = this.entityConfiguration.entityToDBFieldsKeyMapping.get(fieldName);
invariant(columnName, `database field mapping missing for ${fieldName}`);
invariant(columnName, `database field mapping missing for ${String(fieldName)}`);
return this.context.makeKeyFn(
this.context.cacheKeyPrefix,
this.entityConfiguration.tableName,
@@ -1,4 +1,4 @@
import { DateField } from '@expo/entity';
import { DateField, LowercaseStringField } from '@expo/entity';
export const redisTransformerMap = new Map([
[
@@ -22,4 +22,10 @@ export const redisTransformerMap = new Map([
read: (val: any) => (val ? new Date(val) : val),
},
],
[
LowercaseStringField.name,
{
write: (val: any) => (typeof val === 'string' ? val.toLowerCase() : val),
},
],
]);
@@ -3,6 +3,14 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [0.25.3](https://github.com/expo/entity/compare/v0.25.2...v0.25.3) (2022-03-12)
**Note:** Version bump only for package @expo/entity-database-adapter-knex
## [0.25.2](https://github.com/expo/entity/compare/v0.25.1...v0.25.2) (2022-03-12)
**Note:** Version bump only for package @expo/entity-database-adapter-knex
@@ -1,6 +1,6 @@
{
"name": "@expo/entity-database-adapter-knex",
"version": "0.25.2",
"version": "0.25.3",
"description": "Knex database adapter for @expo/entity",
"files": [
"build",
@@ -33,7 +33,7 @@
"knex": "^1.0.2"
},
"devDependencies": {
"@expo/entity": "^0.25.2",
"@expo/entity": "^0.25.3",
"pg": "^8.7.1"
}
}
@@ -7,6 +7,7 @@ import {
TableQuerySelectionModifiers,
TableFieldSingleValueEqualityCondition,
TableFieldMultiValueEqualityCondition,
LowercaseStringField,
} from '@expo/entity';
import { Knex } from 'knex';
@@ -36,6 +37,12 @@ export default class PostgresEntityDatabaseAdapter<TFields> extends EntityDataba
write: (val: any[] | any) => (Array.isArray(val) ? JSON.stringify(val) : val),
},
],
[
LowercaseStringField.name,
{
write: (val: any) => (typeof val === 'string' ? val.toLowerCase() : val),
},
],
]);
}
+8
View File
@@ -3,6 +3,14 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [0.25.3](https://github.com/expo/entity/compare/v0.25.2...v0.25.3) (2022-03-12)
**Note:** Version bump only for package @expo/entity-example
## [0.25.2](https://github.com/expo/entity/compare/v0.25.1...v0.25.2) (2022-03-12)
**Note:** Version bump only for package @expo/entity-example
+2 -2
View File
@@ -1,7 +1,7 @@
{
"name": "@expo/entity-example",
"private": true,
"version": "0.25.2",
"version": "0.25.3",
"description": "An example integration of the @expo/entity framework",
"scripts": {
"tsc": "tsc",
@@ -23,7 +23,7 @@
"author": "Expo",
"license": "MIT",
"dependencies": {
"@expo/entity": "^0.25.2",
"@expo/entity": "^0.25.3",
"apollo-server-koa": "^2.16.1",
"graphql": "^15.3.0",
"koa": "^2.13.0",
@@ -30,7 +30,7 @@ router.get('/', async (ctx: ExampleContext) => {
router.get('/:id', async (ctx: ExampleContext) => {
const viewerContext = ctx.state.viewerContext;
const noteResult = await NoteEntity.loader(viewerContext).loadByIDAsync(ctx.params.id);
const noteResult = await NoteEntity.loader(viewerContext).loadByIDAsync(ctx.params['id']!);
if (!noteResult.ok) {
ctx.throw(403, noteResult.reason);
}
@@ -69,7 +69,7 @@ router.put('/:id', async (ctx: ExampleContext) => {
const viewerContext = ctx.state.viewerContext;
const { title, body } = ctx.request.body;
const noteLoadResult = await NoteEntity.loader(viewerContext).loadByIDAsync(ctx.params.id);
const noteLoadResult = await NoteEntity.loader(viewerContext).loadByIDAsync(ctx.params['id']!);
if (!noteLoadResult.ok) {
ctx.throw(403, noteLoadResult.reason);
}
@@ -90,7 +90,7 @@ router.put('/:id', async (ctx: ExampleContext) => {
router.delete('/:id', async (ctx: ExampleContext) => {
const viewerContext = ctx.state.viewerContext;
const noteLoadResult = await NoteEntity.loader(viewerContext).loadByIDAsync(ctx.params.id);
const noteLoadResult = await NoteEntity.loader(viewerContext).loadByIDAsync(ctx.params['id']!);
if (!noteLoadResult.ok) {
ctx.throw(403, noteLoadResult.reason);
}
@@ -3,6 +3,14 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [0.25.3](https://github.com/expo/entity/compare/v0.25.2...v0.25.3) (2022-03-12)
**Note:** Version bump only for package @expo/entity-full-integration-tests
## [0.25.2](https://github.com/expo/entity/compare/v0.25.1...v0.25.2) (2022-03-12)
**Note:** Version bump only for package @expo/entity-full-integration-tests
@@ -1,7 +1,7 @@
{
"name": "@expo/entity-full-integration-tests",
"private": true,
"version": "0.25.2",
"version": "0.25.3",
"description": "Full redis and knex integration tests for the entity framework",
"scripts": {
"tsc": "tsc",
@@ -27,8 +27,8 @@
"@expo/entity-database-adapter-knex": "*"
},
"devDependencies": {
"@expo/entity": "^0.25.2",
"@expo/entity-cache-adapter-redis": "^0.25.2",
"@expo/entity-database-adapter-knex": "^0.25.2"
"@expo/entity": "^0.25.3",
"@expo/entity-cache-adapter-redis": "^0.25.3",
"@expo/entity-database-adapter-knex": "^0.25.3"
}
}
@@ -3,6 +3,14 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [0.25.3](https://github.com/expo/entity/compare/v0.25.2...v0.25.3) (2022-03-12)
**Note:** Version bump only for package @expo/entity-ip-address-field
## [0.25.2](https://github.com/expo/entity/compare/v0.25.1...v0.25.2) (2022-03-12)
**Note:** Version bump only for package @expo/entity-ip-address-field
@@ -1,6 +1,6 @@
{
"name": "@expo/entity-ip-address-field",
"version": "0.25.2",
"version": "0.25.3",
"description": "IP address EntityField definitions for @expo/entity",
"files": [
"build",
@@ -28,7 +28,7 @@
"ip-address": "^8.1.0"
},
"devDependencies": {
"@expo/entity": "^0.25.2",
"@expo/entity": "^0.25.3",
"@types/jsbn": "^1.2.29"
},
"peerDependencies": {
@@ -1,8 +0,0 @@
import { EntityFieldDefinition } from '@expo/entity';
import { Address4, Address6 } from 'ip-address';
export class IPAddressField extends EntityFieldDefinition<string> {
protected validateInputValueInternal(value: string): boolean {
return Address4.isValid(value) || Address6.isValid(value);
}
}
@@ -0,0 +1,12 @@
import { EntityFieldDefinition } from '@expo/entity';
import { Address4, Address6 } from 'ip-address';
export class IPAddressField extends EntityFieldDefinition<string> {
protected validateAndTransformInputValueInternal(
value: string
): { isValid: false } | { isValid: true; transformedValue: string } {
return Address4.isValid(value) || Address6.isValid(value)
? { isValid: true, transformedValue: value }
: { isValid: false };
}
}
@@ -1,6 +1,6 @@
import { describeFieldTestCase } from '@expo/entity';
import { IPAddressField } from '../EntityFields';
import { IPAddressField } from '../IPAddressField';
describeFieldTestCase(
new IPAddressField({ columnName: 'wat' }),
@@ -12,5 +12,6 @@ describeFieldTestCase(
'1:2:3:4:5:6:7:8/64',
'fedc:ba98:7654:3210:fedc:ba98:7654:3210',
],
['', 'abs', '0.0.0.x/0', '123.21.23', '1:2:3:4:5:6', '198.10/8']
['', 'abs', '0.0.0.x/0', '123.21.23', '1:2:3:4:5:6', '198.10/8'],
[{ in: '192.168.1.1', out: '192.168.1.1' }]
);
@@ -4,4 +4,4 @@
* @module @expo/entity-ip-address-field
*/
export * from './EntityFields';
export * from './IPAddressField';
@@ -3,6 +3,14 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [0.25.3](https://github.com/expo/entity/compare/v0.25.2...v0.25.3) (2022-03-12)
**Note:** Version bump only for package @expo/entity-secondary-cache-local-memory
## [0.25.2](https://github.com/expo/entity/compare/v0.25.1...v0.25.2) (2022-03-12)
**Note:** Version bump only for package @expo/entity-secondary-cache-local-memory
@@ -1,6 +1,6 @@
{
"name": "@expo/entity-secondary-cache-local-memory",
"version": "0.25.2",
"version": "0.25.3",
"description": "Local memory secondary cache for @expo/entity",
"files": [
"build",
@@ -31,8 +31,8 @@
"@expo/entity-cache-adapter-local-memory": "*"
},
"devDependencies": {
"@expo/entity": "^0.25.2",
"@expo/entity-cache-adapter-local-memory": "^0.25.2",
"@expo/entity": "^0.25.3",
"@expo/entity-cache-adapter-local-memory": "^0.25.3",
"nullthrows": "^1.1.1"
}
}
@@ -3,6 +3,14 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [0.25.3](https://github.com/expo/entity/compare/v0.25.2...v0.25.3) (2022-03-12)
**Note:** Version bump only for package @expo/entity-secondary-cache-redis
## [0.25.2](https://github.com/expo/entity/compare/v0.25.1...v0.25.2) (2022-03-12)
**Note:** Version bump only for package @expo/entity-secondary-cache-redis
@@ -1,6 +1,6 @@
{
"name": "@expo/entity-secondary-cache-redis",
"version": "0.25.2",
"version": "0.25.3",
"description": "Redis secondary cache for @expo/entity",
"files": [
"build",
@@ -30,12 +30,10 @@
"@expo/entity": "*",
"@expo/entity-cache-adapter-redis": "*"
},
"dependencies": {
"ioredis": "^4.17.3"
},
"devDependencies": {
"@expo/entity": "^0.25.2",
"@expo/entity-cache-adapter-redis": "^0.25.2",
"@expo/entity": "^0.25.3",
"@expo/entity-cache-adapter-redis": "^0.25.3",
"ioredis": "^5.0.4",
"nullthrows": "^1.1.1"
}
}
+8
View File
@@ -3,6 +3,14 @@
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
## [0.25.3](https://github.com/expo/entity/compare/v0.25.2...v0.25.3) (2022-03-12)
**Note:** Version bump only for package @expo/entity
## [0.25.2](https://github.com/expo/entity/compare/v0.25.1...v0.25.2) (2022-03-12)
**Note:** Version bump only for package @expo/entity
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@expo/entity",
"version": "0.25.2",
"version": "0.25.3",
"description": "A privacy-first data model",
"files": [
"build",
+12 -4
View File
@@ -121,12 +121,20 @@ export abstract class EntityFieldDefinition<T> {
* - EntityLoader.loadByFieldValue - to ensure the value being loaded by is a valid value
* - EntityMutator.setField - to ensure the value being set is a valid value
*/
public validateInputValue(value: T | null | undefined): boolean {
public validateAndTransformInputValue(
value: T | null | undefined
): { isValid: false } | { isValid: true; transformedValue: T | null | undefined } {
if (value === null || value === undefined) {
return true;
return { isValid: true, transformedValue: value };
}
return this.validateInputValueInternal(value);
return this.validateAndTransformInputValueInternal(value);
}
protected abstract validateAndTransformInputValueInternal(
value: T
): { isValid: false } | { isValid: true; transformedValue: T };
public transformInputValue(value: T): T {
return value;
}
protected abstract validateInputValueInternal(value: T): boolean;
}
+78 -22
View File
@@ -6,8 +6,28 @@ import { EntityFieldDefinition } from './EntityFieldDefinition';
* {@link EntityFieldDefinition} for a column with a JS string type.
*/
export class StringField extends EntityFieldDefinition<string> {
protected validateInputValueInternal(value: string): boolean {
return typeof value === 'string';
protected validateAndTransformInputValueInternal(
value: string
): { isValid: false } | { isValid: true; transformedValue: string } {
return typeof value === 'string'
? { isValid: true, transformedValue: value }
: { isValid: false };
}
}
/**
* {@link EntityFieldDefinition} for a column with a JS string type in which
* stored values are lower-cased.
*/
export class LowercaseStringField extends StringField {
protected override validateAndTransformInputValueInternal(
value: string
): { isValid: false } | { isValid: true; transformedValue: string } {
const superValue = super.validateAndTransformInputValueInternal(value);
if (superValue.isValid) {
return { isValid: true, transformedValue: superValue.transformedValue.toLowerCase() };
}
return { isValid: false };
}
}
@@ -16,8 +36,14 @@ export class StringField extends EntityFieldDefinition<string> {
* Enforces that the string is a valid UUID.
*/
export class UUIDField extends StringField {
protected override validateInputValueInternal(value: string): boolean {
return validateUUID(value);
public override validateAndTransformInputValueInternal(
value: string
): { isValid: false } | { isValid: true; transformedValue: string } {
const superValue = super.validateAndTransformInputValueInternal(value);
if (superValue.isValid && validateUUID(superValue.transformedValue)) {
return { isValid: true, transformedValue: superValue.transformedValue };
}
return { isValid: false };
}
}
@@ -25,8 +51,10 @@ export class UUIDField extends StringField {
* {@link EntityFieldDefinition} for a column with a JS Date type.
*/
export class DateField extends EntityFieldDefinition<Date> {
protected validateInputValueInternal(value: Date): boolean {
return value instanceof Date;
protected validateAndTransformInputValueInternal(
value: Date
): { isValid: false } | { isValid: true; transformedValue: Date } {
return value instanceof Date ? { isValid: true, transformedValue: value } : { isValid: false };
}
}
@@ -34,8 +62,12 @@ export class DateField extends EntityFieldDefinition<Date> {
* {@link EntityFieldDefinition} for a column with a JS boolean type.
*/
export class BooleanField extends EntityFieldDefinition<boolean> {
protected validateInputValueInternal(value: boolean): boolean {
return typeof value === 'boolean';
protected validateAndTransformInputValueInternal(
value: boolean
): { isValid: false } | { isValid: true; transformedValue: boolean } {
return typeof value === 'boolean'
? { isValid: true, transformedValue: value }
: { isValid: false };
}
}
@@ -44,8 +76,12 @@ export class BooleanField extends EntityFieldDefinition<boolean> {
* Enforces that the number is an integer.
*/
export class IntField extends EntityFieldDefinition<number> {
protected validateInputValueInternal(value: number): boolean {
return typeof value === 'number' && Number.isInteger(value);
protected validateAndTransformInputValueInternal(
value: number
): { isValid: false } | { isValid: true; transformedValue: number } {
return typeof value === 'number' && Number.isInteger(value)
? { isValid: true, transformedValue: value }
: { isValid: false };
}
}
@@ -54,8 +90,12 @@ export class IntField extends EntityFieldDefinition<number> {
* Enforces that the number is a float (which includes integers in JS).
*/
export class FloatField extends EntityFieldDefinition<number> {
protected validateInputValueInternal(value: number): boolean {
return typeof value === 'number';
protected validateAndTransformInputValueInternal(
value: number
): { isValid: false } | { isValid: true; transformedValue: number } {
return typeof value === 'number'
? { isValid: true, transformedValue: value }
: { isValid: false };
}
}
@@ -64,8 +104,12 @@ export class FloatField extends EntityFieldDefinition<number> {
* Enforces that every member of the string array is a string.
*/
export class StringArrayField extends EntityFieldDefinition<string[]> {
protected validateInputValueInternal(value: string[]): boolean {
return Array.isArray(value) && value.every((subValue) => typeof subValue === 'string');
protected validateAndTransformInputValueInternal(
value: string[]
): { isValid: false } | { isValid: true; transformedValue: string[] } {
return Array.isArray(value) && value.every((subValue) => typeof subValue === 'string')
? { isValid: true, transformedValue: value }
: { isValid: false };
}
}
@@ -73,8 +117,12 @@ export class StringArrayField extends EntityFieldDefinition<string[]> {
* {@link EntityFieldDefinition} for a column with a JS JSON object type.
*/
export class JSONObjectField extends EntityFieldDefinition<object> {
protected validateInputValueInternal(value: object): boolean {
return typeof value === 'object' && !Array.isArray(value);
protected validateAndTransformInputValueInternal(
value: object
): { isValid: false } | { isValid: true; transformedValue: object } {
return typeof value === 'object' && !Array.isArray(value)
? { isValid: true, transformedValue: value }
: { isValid: false };
}
}
@@ -82,8 +130,12 @@ export class JSONObjectField extends EntityFieldDefinition<object> {
* {@link EntityFieldDefinition} for a enum column with a JS string or number type.
*/
export class EnumField extends EntityFieldDefinition<string | number> {
protected validateInputValueInternal(value: string | number): boolean {
return typeof value === 'number' || typeof value === 'string';
protected validateAndTransformInputValueInternal(
value: string | number
): { isValid: false } | { isValid: true; transformedValue: string | number } {
return typeof value === 'number' || typeof value === 'string'
? { isValid: true, transformedValue: value }
: { isValid: false };
}
}
@@ -91,8 +143,10 @@ export class EnumField extends EntityFieldDefinition<string | number> {
* {@link EntityFieldDefinition} for a column with a JS JSON array type.
*/
export class JSONArrayField extends EntityFieldDefinition<any[]> {
protected validateInputValueInternal(value: any[]): boolean {
return Array.isArray(value);
protected validateAndTransformInputValueInternal(
value: any[]
): { isValid: false } | { isValid: true; transformedValue: any[] } {
return Array.isArray(value) ? { isValid: true, transformedValue: value } : { isValid: false };
}
}
@@ -101,7 +155,9 @@ export class JSONArrayField extends EntityFieldDefinition<any[]> {
* Does not do any validation.
*/
export class MaybeJSONArrayField extends EntityFieldDefinition<any | any[]> {
protected validateInputValueInternal(_value: any): boolean {
return true;
protected validateAndTransformInputValueInternal(
value: any
): { isValid: false } | { isValid: true; transformedValue: any } {
return { isValid: true, transformedValue: value };
}
}
+39 -19
View File
@@ -82,12 +82,15 @@ export default class EntityLoader<
fieldName: N,
fieldValues: readonly NonNullable<TFields[N]>[]
): Promise<ReadonlyMap<NonNullable<TFields[N]>, readonly Result<TEntity>[]>> {
this.validateFieldValues(fieldName, fieldValues);
const validatedAndTransformedFieldValues = this.validateAndTransformFieldValues(
fieldName,
fieldValues
);
const fieldValuesToFieldObjects = await this.dataManager.loadManyByFieldEqualingAsync(
this.queryContext,
fieldName,
fieldValues
validatedAndTransformedFieldValues
);
return await this.constructAndAuthorizeEntitiesAsync(fieldValuesToFieldObjects);
@@ -126,7 +129,9 @@ export default class EntityLoader<
const entityResults = await this.loadManyByFieldEqualingAsync(uniqueFieldName, fieldValue);
invariant(
entityResults.length <= 1,
`loadByFieldEqualing: Multiple entities of type ${this.entityClass.name} found for ${uniqueFieldName}=${fieldValue}`
`loadByFieldEqualing: Multiple entities of type ${this.entityClass.name} found for ${String(
uniqueFieldName
)}=${fieldValue}`
);
return entityResults[0] ?? null;
}
@@ -197,16 +202,29 @@ export default class EntityLoader<
fieldEqualityOperands: FieldEqualityCondition<TFields, N>[],
querySelectionModifiers: QuerySelectionModifiers<TFields> = {}
): Promise<readonly Result<TEntity>[]> {
for (const fieldEqualityOperand of fieldEqualityOperands) {
const fieldValues = isSingleValueFieldEqualityCondition(fieldEqualityOperand)
? [fieldEqualityOperand.fieldValue]
: fieldEqualityOperand.fieldValues;
this.validateFieldValues(fieldEqualityOperand.fieldName, fieldValues);
}
const validatedAndTransformedFieldEqualityOperands: FieldEqualityCondition<TFields, N>[] =
fieldEqualityOperands.map((fieldEqualityOperand) => {
if (isSingleValueFieldEqualityCondition(fieldEqualityOperand)) {
return {
...fieldEqualityOperand,
fieldValue: this.validateAndTransformFieldValues(fieldEqualityOperand.fieldName, [
fieldEqualityOperand.fieldValue,
])[0]!,
};
} else {
return {
...fieldEqualityOperand,
fieldValues: this.validateAndTransformFieldValues(
fieldEqualityOperand.fieldName,
fieldEqualityOperand.fieldValues
),
};
}
});
const fieldObjects = await this.dataManager.loadManyByFieldEqualityConjunctionAsync(
this.queryContext,
fieldEqualityOperands,
validatedAndTransformedFieldEqualityOperands,
querySelectionModifiers
);
const uncheckedEntityResults = this.tryConstructEntities(fieldObjects);
@@ -345,17 +363,19 @@ export default class EntityLoader<
});
}
private validateFieldValues<N extends keyof Pick<TFields, TSelectedFields>>(
fieldName: N,
fieldValues: readonly TFields[N][]
): void {
private validateAndTransformFieldValues<
N extends keyof Pick<TFields, TSelectedFields>,
T extends TFields[N]
>(fieldName: N, fieldValues: readonly T[]): readonly T[] {
const fieldDefinition = this.entityConfiguration.schema.get(fieldName);
invariant(fieldDefinition, `must have field definition for field = ${fieldName}`);
for (const fieldValue of fieldValues) {
const isInputValid = fieldDefinition.validateInputValue(fieldValue);
if (!isInputValid) {
invariant(fieldDefinition, `must have field definition for field = ${String(fieldName)}`);
return fieldValues.map((fieldValue) => {
const result = fieldDefinition.validateAndTransformInputValue(fieldValue);
if (result.isValid) {
return result.transformedValue;
} else {
throw new EntityInvalidFieldValueError(this.entityClass, fieldName, fieldValue);
}
}
});
}
}
+27 -12
View File
@@ -81,16 +81,22 @@ abstract class BaseMutator<
protected readonly metricsAdapter: IEntityMetricsAdapter
) {}
protected validateFields(fields: Partial<TFields>): void {
protected validateAndTransformFields(fields: Partial<TFields>): Partial<TFields> {
const result: Partial<TFields> = {};
for (const fieldName in fields) {
const fieldValue = fields[fieldName];
const fieldDefinition = this.entityConfiguration.schema.get(fieldName);
invariant(fieldDefinition, `must have field definition for field = ${fieldName}`);
const isInputValid = fieldDefinition.validateInputValue(fieldValue);
if (!isInputValid) {
const validationResult = fieldDefinition.validateAndTransformInputValue(fieldValue);
if (validationResult.isValid) {
result[fieldName] = validationResult.transformedValue;
} else {
throw new EntityInvalidFieldValueError(this.entityClass, fieldName, fieldValue);
}
}
return result;
}
protected async executeMutationValidatorsAsync(
@@ -183,7 +189,7 @@ export class CreateMutator<
/**
* Commit the new entity after authorizing against creation privacy rules. Invalidates all caches for
* queries that would return new entity and caches the new entity if not in a transactional query context.
* queries that would return new entity.
* @returns authorized, cached, newly-created entity result, where result error can be UnauthorizedError
*/
async createAsync(): Promise<Result<TEntity>> {
@@ -210,11 +216,13 @@ export class CreateMutator<
private async createInternalAsync(
queryContext: EntityTransactionalQueryContext
): Promise<Result<TEntity>> {
this.validateFields(this.fieldsForEntity);
const transformedAndValidatedFieldsForEntity = this.validateAndTransformFields(
this.fieldsForEntity
);
const temporaryEntityForPrivacyCheck = new this.entityClass(this.viewerContext, {
[this.entityConfiguration.idField]: '00000000-0000-0000-0000-000000000000', // zero UUID
...this.fieldsForEntity,
...transformedAndValidatedFieldsForEntity,
} as unknown as TFields);
const authorizeCreateResult = await asyncResult(
@@ -249,7 +257,10 @@ export class CreateMutator<
{ type: EntityMutationType.CREATE }
);
const insertResult = await this.databaseAdapter.insertAsync(queryContext, this.fieldsForEntity);
const insertResult = await this.databaseAdapter.insertAsync(
queryContext,
transformedAndValidatedFieldsForEntity
);
const entityLoader = this.entityLoaderFactory.forLoad(
this.viewerContext,
@@ -382,8 +393,7 @@ export class UpdateMutator<
/**
* Commit the changes to the entity after authorizing against update privacy rules.
* Invalidates all caches for pre-update entity and caches the updated entity if not in a
* transactional query context.
* Invalidates all caches for pre-update entity.
* @returns authorized updated entity result, where result error can be UnauthorizedError
*/
async updateAsync(): Promise<Result<TEntity>> {
@@ -410,9 +420,14 @@ export class UpdateMutator<
private async updateInternalAsync(
queryContext: EntityTransactionalQueryContext
): Promise<Result<TEntity>> {
this.validateFields(this.updatedFields);
const transformedAndValidatedFieldsForEntity = this.validateAndTransformFields(
this.fieldsForEntity
) as TFields;
const entityAboutToBeUpdated = new this.entityClass(this.viewerContext, this.fieldsForEntity);
const entityAboutToBeUpdated = new this.entityClass(
this.viewerContext,
transformedAndValidatedFieldsForEntity
);
const authorizeUpdateResult = await asyncResult(
this.privacyPolicy.authorizeUpdateAsync(
this.viewerContext,
@@ -465,7 +480,7 @@ export class UpdateMutator<
)
);
queryContext.appendPostCommitInvalidationCallback(
entityLoader.invalidateFieldsAsync.bind(entityLoader, this.fieldsForEntity)
entityLoader.invalidateFieldsAsync.bind(entityLoader, transformedAndValidatedFieldsForEntity)
);
const unauthorizedEntityAfterUpdate = new this.entityClass(this.viewerContext, updateResult);
@@ -19,9 +19,24 @@ export type EntityPrivacyPolicyEvaluationContext = {
cascadingDeleteCause: EntityCascadingDeletionInfo | null;
};
/**
* Evaluation mode for a {@link EntityPrivacyPolicy}. Useful when transitioning to
* using Entity for privacy.
*/
export enum EntityPrivacyPolicyEvaluationMode {
/**
* Enforce this privacy policy. Throw upon denial.
*/
ENFORCE,
/**
* Do not enforce this privacy policy. Always allow but log when it would have denied.
*/
DRY_RUN,
/**
* Enforce this privacy policy. Throw and log upon denial.
*/
ENFORCE_AND_LOG,
}
+11
View File
@@ -29,6 +29,12 @@ export abstract class EntityQueryContext {
): Promise<T>;
}
/**
* Entity framework representation of a non-transactional query execution unit.
* When supplied to {@link EntityMutator} and {@link EntityLoader} methods, they will be
* run independently of any running transaction (though mutations start their own
* independent transactions internally when not being run in a transaction).
*/
export class EntityNonTransactionalQueryContext extends EntityQueryContext {
constructor(
queryInterface: any,
@@ -48,6 +54,11 @@ export class EntityNonTransactionalQueryContext extends EntityQueryContext {
}
}
/**
* Entity framework representation of a transactional query execution unit. When supplied
* to {@link EntityMutator} and {@link EntityLoader} methods, those methods and their
* dependent triggers and validators will run within the transaction.
*/
export class EntityTransactionalQueryContext extends EntityQueryContext {
private readonly postCommitInvalidationCallbacks: PostCommitCallback[] = [];
private readonly postCommitCallbacks: PostCommitCallback[] = [];
@@ -95,7 +95,7 @@ class TestLocalCacheAdapter<TFields> extends EntityCacheAdapter<TFields> {
fieldValue: NonNullable<TFields[N]>
): string {
const columnName = this.entityConfiguration.entityToDBFieldsKeyMapping.get(fieldName);
invariant(columnName, `database field mapping missing for ${fieldName}`);
invariant(columnName, `database field mapping missing for ${String(fieldName)}`);
const parts = [
this.entityConfiguration.tableName,
`${this.entityConfiguration.cacheKeyVersion}`,
@@ -288,7 +288,7 @@ describe(EnforcingEntityLoader, () => {
'invalidateFieldsAsync',
'invalidateEntityAsync',
'tryConstructEntities',
'validateFieldValues',
'validateAndTransformFieldValues',
'constructAndAuthorizeEntitiesAsync',
];
expect(loaderProperties).toEqual(expect.arrayContaining(knownLoaderOnlyDifferences));
@@ -13,12 +13,15 @@ import {
EnumField,
JSONArrayField,
MaybeJSONArrayField,
LowercaseStringField,
} from '../EntityFields';
import describeFieldTestCase from '../utils/testing/describeFieldTestCase';
class TestFieldDefinition extends EntityFieldDefinition<string> {
protected validateInputValueInternal(value: string): boolean {
return value === 'helloworld';
protected validateAndTransformInputValueInternal(
value: string
): { isValid: false } | { isValid: true; transformedValue: string } {
return value === 'helloworld' ? { isValid: true, transformedValue: value } : { isValid: false };
}
}
@@ -35,49 +38,97 @@ describe(EntityFieldDefinition, () => {
test('validator returns true when value is null', () => {
const fieldDefinition = new TestFieldDefinition({ columnName: 'wat', cache: true });
expect(fieldDefinition.validateInputValue(null)).toBe(true);
expect(fieldDefinition.validateAndTransformInputValue(null).isValid).toBe(true);
});
test('validator returns true when value is undefined', () => {
const fieldDefinition = new TestFieldDefinition({ columnName: 'wat', cache: true });
expect(fieldDefinition.validateInputValue(undefined)).toBe(true);
expect(fieldDefinition.validateAndTransformInputValue(undefined).isValid).toBe(true);
});
test('validator returns false when value is invalid', () => {
const fieldDefinition = new TestFieldDefinition({ columnName: 'wat', cache: true });
expect(fieldDefinition.validateInputValue('nothelloworld')).toBe(false);
expect(fieldDefinition.validateAndTransformInputValue('nothelloworld').isValid).toBe(false);
});
test('validator returns true when value is valid', () => {
const fieldDefinition = new TestFieldDefinition({ columnName: 'wat', cache: true });
expect(fieldDefinition.validateInputValue('helloworld')).toBe(true);
expect(fieldDefinition.validateAndTransformInputValue('helloworld').isValid).toBe(true);
});
});
describeFieldTestCase(new StringField({ columnName: 'wat' }), ['hello', ''], [1, true, {}, [[]]]);
describeFieldTestCase(
new StringField({ columnName: 'wat' }),
['hello', ''],
[1, true, {}, [[]]],
[{ in: 'hello', out: 'hello' }]
);
describeFieldTestCase(
new LowercaseStringField({ columnName: 'wat' }),
['HELLO', 'hello'],
[1, true, {}, [[]]],
[
{ in: 'low', out: 'low' },
{ in: 'UP', out: 'up' },
]
);
describeFieldTestCase(
new UUIDField({ columnName: 'wat' }),
[uuidv1(), uuidv3('wat', uuidv3.DNS), uuidv4(), uuidv5('wat', uuidv5.DNS)],
[uuidv4().replace('-', ''), '', 'hello']
[uuidv4().replace('-', ''), '', 'hello'],
[uuidv4()].map((u) => ({ in: u, out: u }))
);
describeFieldTestCase(
new DateField({ columnName: 'wat' }),
[new Date()],
[Date.now()],
[new Date()].map((u) => ({ in: u, out: u }))
);
describeFieldTestCase(
new BooleanField({ columnName: 'wat' }),
[true, false],
[0, 1, ''],
[{ in: true, out: true }]
);
describeFieldTestCase(
new IntField({ columnName: 'wat' }),
[1],
['1', 0.5, true],
[{ in: 1, out: 1 }]
);
describeFieldTestCase(
new FloatField({ columnName: 'wat' }),
[1, 0.5, -0.5],
['1'],
[{ in: 3.5, out: 3.5 }]
);
describeFieldTestCase(new DateField({ columnName: 'wat' }), [new Date()], [Date.now()]);
describeFieldTestCase(new BooleanField({ columnName: 'wat' }), [true, false], [0, 1, '']);
describeFieldTestCase(new IntField({ columnName: 'wat' }), [1], ['1', 0.5, true]);
describeFieldTestCase(new FloatField({ columnName: 'wat' }), [1, 0.5, -0.5], ['1']);
describeFieldTestCase(
new StringArrayField({ columnName: 'wat' }),
[[['what']] as any, [[]] as any], // jest test cases need extra wrapping array
['hello']
['hello'],
[{ in: ['hello'], out: ['hello'] }]
);
describeFieldTestCase(
new JSONObjectField({ columnName: 'wat' }),
[{}],
[true, 'hello'],
[{ in: { hello: 1 }, out: { hello: 1 } }]
);
describeFieldTestCase(
new EnumField({ columnName: 'wat' }),
['hello', 1],
[true],
[{ in: 1, out: 1 }]
);
describeFieldTestCase(new JSONObjectField({ columnName: 'wat' }), [{}], [true, 'hello']);
describeFieldTestCase(new EnumField({ columnName: 'wat' }), ['hello', 1], [true]);
describeFieldTestCase(
new JSONArrayField({ columnName: 'wat' }),
[[[1, 2]] as any, [['hello']] as any], // jest test cases need extra wrapping array
[1, 'hello']
[1, 'hello'],
[{ in: [1, 2], out: [1, 2] }]
);
describeFieldTestCase(
new MaybeJSONArrayField({ columnName: 'wat' }),
[1, 'hello', [['hello']]], // jest test cases need extra wrapping array
[]
[],
[{ in: 1, out: 1 }]
);
@@ -41,6 +41,7 @@ describe(EntityLoader, () => {
testIndexedField: 'h1',
intField: 5,
stringField: 'huh',
lowercasedField: 'lw1',
dateField: dateToInsert,
nullableField: null,
},
@@ -49,6 +50,7 @@ describe(EntityLoader, () => {
testIndexedField: 'h2',
intField: 3,
stringField: 'huh',
lowercasedField: 'lw2',
dateField: dateToInsert,
nullableField: null,
},
@@ -87,6 +89,11 @@ describe(EntityLoader, () => {
);
expect(entities.map((m) => m.getID())).toEqual([id1, id2]);
const entitiesLoadedTransformedField = await enforceResultsAsync(
entityLoader.loadManyByFieldEqualingAsync('lowercasedField', 'LW1')
);
expect(entitiesLoadedTransformedField.map((m) => m.getID())).toEqual([id1]);
const entityResultNumber3 = await entityLoader.loadByFieldEqualingAsync('intField', 3);
expect(entityResultNumber3).not.toBeNull();
expect(entityResultNumber3!.enforceValue().getID()).toEqual(id2);
@@ -134,6 +141,7 @@ describe(EntityLoader, () => {
{
customIdField: id1,
stringField: 'huh',
lowercasedField: 'lw1',
intField: 4,
testIndexedField: '4',
dateField: new Date(),
@@ -142,6 +150,7 @@ describe(EntityLoader, () => {
{
customIdField: id2,
stringField: 'huh',
lowercasedField: 'lw2',
intField: 4,
testIndexedField: '5',
dateField: new Date(),
@@ -150,6 +159,7 @@ describe(EntityLoader, () => {
{
customIdField: id3,
stringField: 'huh2',
lowercasedField: 'lw3',
intField: 4,
testIndexedField: '6',
dateField: new Date(),
@@ -231,6 +241,7 @@ describe(EntityLoader, () => {
{
customIdField: id1,
stringField: 'huh',
lowercasedField: 'lw1',
testIndexedField: '1',
intField: 3,
dateField: new Date(),
@@ -34,6 +34,6 @@ export default class EntityInvalidFieldValueError<
fieldName: N,
fieldValue?: TFields[N]
) {
super(`Entity field not valid: ${entityClass.name} (${fieldName} = ${fieldValue})`);
super(`Entity field not valid: ${entityClass.name} (${String(fieldName)} = ${fieldValue})`);
}
}
@@ -34,6 +34,6 @@ export default class EntityNotFoundError<
fieldName: N,
fieldValue: TFields[N]
) {
super(`Entity not found: ${entityClass.name} (${fieldName} = ${fieldValue})`);
super(`Entity not found: ${entityClass.name} (${String(fieldName)} = ${fieldValue})`);
}
}
@@ -11,7 +11,10 @@ import {
timeAndLogLoadEventAsync,
timeAndLogLoadMapEventAsync,
} from '../metrics/EntityMetricsUtils';
import IEntityMetricsAdapter, { EntityMetricsLoadType } from '../metrics/IEntityMetricsAdapter';
import IEntityMetricsAdapter, {
EntityMetricsLoadType,
IncrementLoadCountEventType,
} from '../metrics/IEntityMetricsAdapter';
import { computeIfAbsent, zipToMap } from '../utils/collections/maps';
import ReadThroughEntityCache from './ReadThroughEntityCache';
@@ -57,7 +60,8 @@ export default class EntityDataManager<TFields> {
fieldName: N,
fieldValues: readonly NonNullable<TFields[N]>[]
): Promise<ReadonlyMap<NonNullable<TFields[N]>, readonly Readonly<TFields>[]>> {
this.metricsAdapter.incrementDataManagerCacheLoadCount({
this.metricsAdapter.incrementDataManagerLoadCount({
type: IncrementLoadCountEventType.CACHE,
fieldValueCount: fieldValues.length,
entityClassName: this.entityClassName,
});
@@ -65,7 +69,8 @@ export default class EntityDataManager<TFields> {
fieldName,
fieldValues,
async (fetcherValues) => {
this.metricsAdapter.incrementDataManagerDatabaseLoadCount({
this.metricsAdapter.incrementDataManagerLoadCount({
type: IncrementLoadCountEventType.DATABASE,
fieldValueCount: fieldValues.length,
entityClassName: this.entityClassName,
});
@@ -108,7 +113,9 @@ export default class EntityDataManager<TFields> {
);
if (nullOrUndefinedValueIndex >= 0) {
throw new Error(
`Invalid load: ${this.entityClassName} (${fieldName} = ${fieldValues[nullOrUndefinedValueIndex]})`
`Invalid load: ${this.entityClassName} (${String(fieldName)} = ${
fieldValues[nullOrUndefinedValueIndex]
})`
);
}
@@ -117,7 +124,8 @@ export default class EntityDataManager<TFields> {
return await this.databaseAdapter.fetchManyWhereAsync(queryContext, fieldName, fieldValues);
}
this.metricsAdapter.incrementDataManagerDataloaderLoadCount({
this.metricsAdapter.incrementDataManagerLoadCount({
type: IncrementLoadCountEventType.DATALOADER,
fieldValueCount: fieldValues.length,
entityClassName: this.entityClassName,
});
@@ -24,7 +24,7 @@ export const getDatabaseFieldForEntityField = <TFields>(
entityField: keyof TFields
): string => {
const databaseField = entityConfiguration.entityToDBFieldsKeyMapping.get(entityField);
invariant(databaseField, `database field mapping missing for ${entityField}`);
invariant(databaseField, `database field mapping missing for ${String(entityField)}`);
return databaseField!;
};
@@ -102,7 +102,9 @@ export default class ReadThroughEntityCache<TFields> {
// TODO(wschurman): emit or throw here since console may not be available
// eslint-disable-next-line no-console
console.warn(
`unique key ${fieldName} in ${this.entityConfiguration.tableName} returned multiple rows for ${fieldValue}`
`unique key ${String(fieldName)} in ${
this.entityConfiguration.tableName
} returned multiple rows for ${fieldValue}`
);
continue;
}
@@ -4,7 +4,6 @@ import {
when,
anything,
verify,
anyNumber,
objectContaining,
spy,
anyString,
@@ -13,7 +12,10 @@ import {
} from 'ts-mockito';
import EntityDatabaseAdapter from '../../EntityDatabaseAdapter';
import IEntityMetricsAdapter, { EntityMetricsLoadType } from '../../metrics/IEntityMetricsAdapter';
import IEntityMetricsAdapter, {
EntityMetricsLoadType,
IncrementLoadCountEventType,
} from '../../metrics/IEntityMetricsAdapter';
import NoOpEntityMetricsAdapter from '../../metrics/NoOpEntityMetricsAdapter';
import TestEntity, { testEntityConfiguration, TestFields } from '../../testfixtures/TestEntity';
import {
@@ -519,24 +521,27 @@ describe(EntityDataManager, () => {
).once();
verify(
metricsAdapterMock.incrementDataManagerDataloaderLoadCount(
metricsAdapterMock.incrementDataManagerLoadCount(
deepEqual({
type: IncrementLoadCountEventType.DATALOADER,
fieldValueCount: 1,
entityClassName: TestEntity.name,
})
)
).once();
verify(
metricsAdapterMock.incrementDataManagerCacheLoadCount(
metricsAdapterMock.incrementDataManagerLoadCount(
deepEqual({
type: IncrementLoadCountEventType.CACHE,
fieldValueCount: 1,
entityClassName: TestEntity.name,
})
)
).once();
verify(
metricsAdapterMock.incrementDataManagerDatabaseLoadCount(
metricsAdapterMock.incrementDataManagerLoadCount(
deepEqual({
type: IncrementLoadCountEventType.DATABASE,
fieldValueCount: 1,
entityClassName: TestEntity.name,
})
@@ -565,9 +570,7 @@ describe(EntityDataManager, () => {
)
).once();
verify(metricsAdapterMock.incrementDataManagerDataloaderLoadCount(anyNumber())).never();
verify(metricsAdapterMock.incrementDataManagerCacheLoadCount(anyNumber())).never();
verify(metricsAdapterMock.incrementDataManagerDatabaseLoadCount(anyNumber())).never();
verify(metricsAdapterMock.incrementDataManagerLoadCount(anything())).never();
});
it('throws when a load-by value is null or undefined', async () => {
@@ -9,10 +9,28 @@ export enum EntityMetricsLoadType {
LOAD_MANY_RAW,
}
/**
* Event about a single call to an {@link EntityLoader} method.
*/
export interface EntityMetricsLoadEvent {
/**
* {@link EntityMetricsLoadType} for this load.
*/
type: EntityMetricsLoadType;
/**
* Class name of the {@link Entity} being loaded.
*/
entityClassName: string;
/**
* Total duration of this load, including fetch and construction of entities.
*/
duration: number;
/**
* Number of entities returned for this load.
*/
count: number;
}
@@ -23,13 +41,57 @@ export enum EntityMetricsMutationType {
}
export interface EntityMetricsMutationEvent {
/**
* {@link EntityMetricsMutationType} for this mutation.
*/
type: EntityMetricsMutationType;
/**
* Class name of the {@link Entity} being mutated.
*/
entityClassName: string;
/**
* Total duration of this mutation.
*/
duration: number;
}
export enum IncrementLoadCountEventType {
/**
* Type for when a dataloader load is initiated via the standard load methods
* since all loads go through a dataloader.
*/
DATALOADER,
/**
* Type for when a cache load is initiated due to a dataloader miss.
*/
CACHE,
/**
* Type for when a database load is initiated due to a dataloader and cache miss, when an entity query doesn't support caching, or during a transaction.
*/
DATABASE,
}
/**
* Event used to record dataloader, cache, and database load counts in {@link EntityDataManager}.
*/
export interface IncrementLoadCountEvent {
/**
* Type of this event.
*/
type: IncrementLoadCountEventType;
/**
* Number of field values being loaded for this call.
*/
fieldValueCount: number;
/**
* Class name of the {@link Entity} being loaded.
*/
entityClassName: string;
}
@@ -38,7 +100,13 @@ export enum EntityMetricsAuthorizationResult {
ALLOW,
}
/**
* Event used to record a singe {@link EntityPrivacyPolicy} authorization.
*/
export interface EntityMetricsAuthorizationEvent {
/**
* Class name of the {@link Entity} being authorized.
*/
entityClassName: string;
action: EntityAuthorizationAction;
evaluationResult: EntityMetricsAuthorizationResult;
@@ -69,25 +137,10 @@ export default interface IEntityMetricsAdapter {
logMutatorMutationEvent(mutationEvent: EntityMetricsMutationEvent): void;
/**
* Called when a dataloader load is initiated via the standard
* load methods (not equality conjunction or raw).
* Called when a dataloader, cache, or database load is initiated via the standard
* load methods (not equality conjunction or raw). Most commonly used for logging
* a waterfall to determine dataloader and cache hit rates and ratios.
* @param fieldValueCount - count of field values being loaded for a field
*/
incrementDataManagerDataloaderLoadCount(incrementLoadCountEvent: IncrementLoadCountEvent): void;
/**
* Called when a cache load is initiated via the standard
* load methods (not equality conjunction or raw). Occurs upon a dataloader
* miss.
* @param fieldValueCount - count of field values being loaded for a field
*/
incrementDataManagerCacheLoadCount(incrementLoadCountEvent: IncrementLoadCountEvent): void;
/**
* Called when a database load is initiated via the standard
* load methods (not equality conjunction or raw). Occurs upon a cache
* miss or when fetching an uncacheable field.
* @param fieldValueCount - count of field values being loaded for a field
*/
incrementDataManagerDatabaseLoadCount(incrementLoadCountEvent: IncrementLoadCountEvent): void;
incrementDataManagerLoadCount(incrementLoadCountEvent: IncrementLoadCountEvent): void;
}
@@ -9,9 +9,5 @@ export default class NoOpEntityMetricsAdapter implements IEntityMetricsAdapter {
logAuthorizationEvent(_authorizationEvent: EntityMetricsAuthorizationEvent): void {}
logDataManagerLoadEvent(_loadEvent: EntityMetricsLoadEvent): void {}
logMutatorMutationEvent(_mutationEvent: EntityMetricsMutationEvent): void {}
incrementDataManagerDataloaderLoadCount(
_incrementLoadCountEvent: IncrementLoadCountEvent
): void {}
incrementDataManagerCacheLoadCount(_incrementLoadCountEvent: IncrementLoadCountEvent): void {}
incrementDataManagerDatabaseLoadCount(_incrementLoadCountEvent: IncrementLoadCountEvent): void {}
incrementDataManagerLoadCount(_incrementLoadCountEvent: IncrementLoadCountEvent): void {}
}
@@ -3,7 +3,7 @@ import { result, Result } from '@expo/results';
import Entity from '../Entity';
import { EntityCompanionDefinition } from '../EntityCompanionProvider';
import EntityConfiguration from '../EntityConfiguration';
import { UUIDField, StringField, DateField, IntField } from '../EntityFields';
import { UUIDField, StringField, DateField, IntField, LowercaseStringField } from '../EntityFields';
import EntityPrivacyPolicy from '../EntityPrivacyPolicy';
import ViewerContext from '../ViewerContext';
import AlwaysAllowPrivacyPolicyRule from '../rules/AlwaysAllowPrivacyPolicyRule';
@@ -12,6 +12,7 @@ export type TestFields = {
customIdField: string;
testIndexedField: string;
stringField: string;
lowercasedField: string;
intField: number;
dateField: Date;
nullableField: string | null;
@@ -31,6 +32,9 @@ export const testEntityConfiguration = new EntityConfiguration<TestFields>({
stringField: new StringField({
columnName: 'string_field',
}),
lowercasedField: new LowercaseStringField({
columnName: 'lowercase_field',
}),
intField: new IntField({
columnName: 'number_field',
}),
@@ -86,6 +90,7 @@ export default class TestEntity extends Entity<TestFields, string, ViewerContext
customIdField: testValue,
testIndexedField: 'hello',
stringField: 'hello',
lowercasedField: 'hello',
intField: 1,
dateField: new Date(),
nullableField: null,
@@ -158,7 +158,10 @@ export default class StubDatabaseAdapter<T> extends EntityDatabaseAdapter<T> {
private generateRandomID(): any {
const idSchemaField = this.entityConfiguration2.schema.get(this.entityConfiguration2.idField);
invariant(idSchemaField, `No schema field found for ${this.entityConfiguration2.idField}`);
invariant(
idSchemaField,
`No schema field found for ${String(this.entityConfiguration2.idField)}`
);
if (idSchemaField instanceof StringField) {
return uuidv4();
} else if (idSchemaField instanceof IntField) {
@@ -3,18 +3,32 @@ import { EntityFieldDefinition } from '../../EntityFieldDefinition';
export default function describeFieldTestCase<T>(
fieldDefinition: EntityFieldDefinition<T>,
validValues: T[],
invalidValues: any[]
invalidValues: any[],
transformValues: {
in: T;
out: T;
}[]
): void {
describe(fieldDefinition.constructor.name, () => {
if (validValues.length > 0) {
test.each(validValues)(`${fieldDefinition.constructor.name}.valid %p`, (value) => {
expect(fieldDefinition.validateInputValue(value)).toBe(true);
expect(fieldDefinition.validateAndTransformInputValue(value).isValid).toBe(true);
});
}
if (invalidValues.length > 0) {
test.each(invalidValues)(`${fieldDefinition.constructor.name}.invalid %p`, (value) => {
expect(fieldDefinition.validateInputValue(value)).toBe(false);
expect(fieldDefinition.validateAndTransformInputValue(value).isValid).toBe(false);
});
}
if (transformValues.length > 0) {
test.each(transformValues)(`${fieldDefinition.constructor.name}.transformed %p`, (value) => {
const result = fieldDefinition.validateAndTransformInputValue(value.in);
expect(result.isValid).toBe(true);
if (result.isValid) {
expect(result.transformedValue).toEqual(value.out);
}
});
}
});
+2576 -3127
View File
File diff suppressed because it is too large. Load diff