Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make count queries publicly available for use #6608

Merged
merged 39 commits into from
Sep 28, 2022
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
b92e492
Mila/count update proto (#6482)
milaGGL Jul 28, 2022
598e948
compile protos and upload the generated json file (#6491)
milaGGL Jul 29, 2022
7f42490
Mila/count add api interface (#6499)
milaGGL Aug 4, 2022
981e6f1
create sample count test cases (#6506)
milaGGL Aug 4, 2022
bc61df7
Mila/count implement count query (#6528)
milaGGL Aug 24, 2022
c21304b
add test case for terminated firestore
milaGGL Aug 26, 2022
a3242ce
Mila/count add tests (#6566)
milaGGL Aug 30, 2022
6da4f4f
Mila/count export aggregate to api (#6575)
milaGGL Sep 1, 2022
fe871ce
change int32 to int64 (#6577)
milaGGL Sep 2, 2022
507e984
check client offline state and test (#6579)
milaGGL Sep 7, 2022
98189d5
Mila/count non lite api tests (#6581)
milaGGL Sep 8, 2022
0ebab69
Mila/count update api surface (#6589)
milaGGL Sep 13, 2022
4b273fe
remove DocumentFieldValue
milaGGL Sep 13, 2022
a9de825
Merge branch 'master' into mila/count
milaGGL Sep 13, 2022
576d202
fix lint error: Symbol not found for identifier
milaGGL Sep 13, 2022
dd693c6
format to pass lint check
milaGGL Sep 13, 2022
83fbc8d
remove the exports of aggregate query functions/types
milaGGL Sep 14, 2022
26a0bd2
Update aggregation.test.ts
milaGGL Sep 14, 2022
9cced28
skip count query test cases if not using emulator
milaGGL Sep 14, 2022
f490bfe
update the type of skipTestUnlessUsingEmulator
milaGGL Sep 14, 2022
078d581
roll back to any type for skipTestUnlessUsingEmulator
milaGGL Sep 15, 2022
a9ee57a
move aggregation test file into api_internal folder
milaGGL Sep 15, 2022
dbbc2f8
resolve comments
milaGGL Sep 15, 2022
c12d5a0
reformat to pass lint check
milaGGL Sep 15, 2022
74116a4
add changeset as requested by github
milaGGL Sep 15, 2022
de9dd9a
remove the changeset
milaGGL Sep 15, 2022
9438594
Merge branch 'master' into mila/count
milaGGL Sep 15, 2022
3a6cc27
add the changeset back
milaGGL Sep 15, 2022
2380959
export count qury
milaGGL Sep 16, 2022
597d1fb
add changeset
milaGGL Sep 16, 2022
03b9803
Merge branch 'master' into mila/count-export-count-quries
milaGGL Sep 21, 2022
e9759c7
Merge remote-tracking branch 'origin/master' into mila/count-export-c…
dconeybe Sep 23, 2022
e2cb6a1
update the firestore-client with datastore from client
milaGGL Sep 23, 2022
79670bc
fix circular dependency between count_query_runner.ts and lite-api/ag…
dconeybe Sep 26, 2022
c3675d4
fix comments
milaGGL Sep 26, 2022
bb87268
Fix document change leads to time travel across multiple tabs (#6619)
wu-hui Sep 26, 2022
2d55fc6
Merge remote-tracking branch 'origin/master' into mila/count-export-c…
dconeybe Sep 27, 2022
fa07ffa
update changeset
dconeybe Sep 27, 2022
88d0266
Firestore: Re-write API docs for COUNT API (#6632)
dconeybe Sep 27, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/wicked-tomatoes-grow.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@firebase/firestore': minor
hsubox76 marked this conversation as resolved.
Show resolved Hide resolved
---

Added `getCountFromServer()` (`getCount()` in the Lite SDK), which fetches the number of documents in the result set without actually downloading the documents.
40 changes: 40 additions & 0 deletions common/api-review/firestore-lite.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,38 @@ export type AddPrefixToKeys<Prefix extends string, T extends Record<string, unkn
[K in keyof T & string as `${Prefix}.${K}`]+?: T[K];
};

// @public
export class AggregateField<T> {
// (undocumented)
type: string;
}

// @public
export type AggregateFieldType = ReturnType<typeof count>;

// @public
export class AggregateQuerySnapshot<T extends AggregateSpec> {
data(): AggregateSpecData<T>;
// (undocumented)
readonly query: Query<unknown>;
// (undocumented)
readonly type = "AggregateQuerySnapshot";
}

// @public
export function aggregateQuerySnapshotEqual<T extends AggregateSpec>(left: AggregateQuerySnapshot<T>, right: AggregateQuerySnapshot<T>): boolean;

// @public
export interface AggregateSpec {
// (undocumented)
[field: string]: AggregateFieldType;
}

// @public
export type AggregateSpecData<T extends AggregateSpec> = {
[P in keyof T]: T[P] extends AggregateField<infer U> ? U : never;
};

// @public
export function arrayRemove(...elements: unknown[]): FieldValue;

Expand Down Expand Up @@ -63,6 +95,9 @@ export function connectFirestoreEmulator(firestore: Firestore, host: string, por
mockUserToken?: EmulatorMockTokenOptions | string;
}): void;

// @public
export function count(): AggregateField<number>;

// @public
export function deleteDoc(reference: DocumentReference<unknown>): Promise<void>;

Expand Down Expand Up @@ -169,6 +204,11 @@ export class GeoPoint {
};
}

// @public
export function getCount(query: Query<unknown>): Promise<AggregateQuerySnapshot<{
count: AggregateField<number>;
}>>;

// @public
export function getDoc<T>(reference: DocumentReference<T>): Promise<DocumentSnapshot<T>>;

Expand Down
40 changes: 40 additions & 0 deletions common/api-review/firestore.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,38 @@ export type AddPrefixToKeys<Prefix extends string, T extends Record<string, unkn
[K in keyof T & string as `${Prefix}.${K}`]+?: T[K];
};

// @public
export class AggregateField<T> {
// (undocumented)
type: string;
}

// @public
export type AggregateFieldType = ReturnType<typeof count>;

// @public
export class AggregateQuerySnapshot<T extends AggregateSpec> {
data(): AggregateSpecData<T>;
// (undocumented)
readonly query: Query<unknown>;
// (undocumented)
readonly type = "AggregateQuerySnapshot";
}

// @public
export function aggregateQuerySnapshotEqual<T extends AggregateSpec>(left: AggregateQuerySnapshot<T>, right: AggregateQuerySnapshot<T>): boolean;

// @public
export interface AggregateSpec {
// (undocumented)
[field: string]: AggregateFieldType;
}

// @public
export type AggregateSpecData<T extends AggregateSpec> = {
[P in keyof T]: T[P] extends AggregateField<infer U> ? U : never;
};

// @public
export function arrayRemove(...elements: unknown[]): FieldValue;

Expand Down Expand Up @@ -69,6 +101,9 @@ export function connectFirestoreEmulator(firestore: Firestore, host: string, por
mockUserToken?: EmulatorMockTokenOptions | string;
}): void;

// @public
export function count(): AggregateField<number>;

// @public
export function deleteDoc(reference: DocumentReference<unknown>): Promise<void>;

Expand Down Expand Up @@ -209,6 +244,11 @@ export class GeoPoint {
};
}

// @public
export function getCountFromServer(query: Query<unknown>): Promise<AggregateQuerySnapshot<{
count: AggregateField<number>;
}>>;

// @public
export function getDoc<T>(reference: DocumentReference<T>): Promise<DocumentSnapshot<T>>;

Expand Down
14 changes: 14 additions & 0 deletions packages/firestore/lite/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,20 @@
import { registerFirestore } from './register';
registerFirestore();

export {
aggregateQuerySnapshotEqual,
getCount
} from '../src/lite-api/aggregate';

export {
AggregateField,
AggregateFieldType,
AggregateSpec,
AggregateSpecData,
AggregateQuerySnapshot,
count
} from '../src/lite-api/aggregate_types';

export { FirestoreSettings as Settings } from '../src/lite-api/settings';

export {
Expand Down
14 changes: 14 additions & 0 deletions packages/firestore/src/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,20 @@
* limitations under the License.
*/

export {
aggregateQuerySnapshotEqual,
getCountFromServer
} from './api/aggregate';

export {
AggregateField,
AggregateFieldType,
AggregateSpec,
AggregateSpecData,
AggregateQuerySnapshot,
count
} from './lite-api/aggregate_types';

export { FieldPath, documentId } from './api/field_path';

export {
Expand Down
11 changes: 9 additions & 2 deletions packages/firestore/src/api/aggregate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,16 @@

import { Query } from '../api';
import { firestoreClientRunCountQuery } from '../core/firestore_client';
import { AggregateField, AggregateQuerySnapshot } from '../lite-api/aggregate';
import {
AggregateField,
AggregateQuerySnapshot
} from '../lite-api/aggregate_types';
import { cast } from '../util/input_validation';

import { ensureFirestoreConfigured, Firestore } from './database';
import { ExpUserDataWriter } from './reference_impl';

export { aggregateQuerySnapshotEqual } from '../lite-api/aggregate';

/**
* Executes the query and returns the results as a `AggregateQuerySnapshot` from the
Expand All @@ -35,5 +41,6 @@ export function getCountFromServer(
): Promise<AggregateQuerySnapshot<{ count: AggregateField<number> }>> {
const firestore = cast(query.firestore, Firestore);
const client = ensureFirestoreConfigured(firestore);
return firestoreClientRunCountQuery(client, query);
const userDataWriter = new ExpUserDataWriter(firestore);
return firestoreClientRunCountQuery(client, query, userDataWriter);
}
70 changes: 70 additions & 0 deletions packages/firestore/src/core/count_query_runner.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
/**
* @license
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

import { AbstractUserDataWriter, Query } from '../api';
import {
AggregateField,
AggregateQuerySnapshot
} from '../lite-api/aggregate_types';
import { Value } from '../protos/firestore_proto_api';
import { Datastore, invokeRunAggregationQueryRpc } from '../remote/datastore';
import { hardAssert } from '../util/assert';

/**
* CountQueryRunner encapsulates the logic needed to run the count aggregation
dconeybe marked this conversation as resolved.
Show resolved Hide resolved
* queries.
*/
export class CountQueryRunner {
constructor(
private readonly query: Query<unknown>,
private readonly datastore: Datastore,
private readonly userDataWriter: AbstractUserDataWriter
) {}

run(): Promise<AggregateQuerySnapshot<{ count: AggregateField<number> }>> {
return invokeRunAggregationQueryRpc(this.datastore, this.query._query).then(
result => {
hardAssert(
result[0] !== undefined,
'Aggregation fields are missing from result.'
);

const counts = Object.entries(result[0])
.filter(([key, value]) => key === 'count_alias')
.map(([key, value]) =>
this.userDataWriter.convertValue(value as Value)
);

const countValue = counts[0];

hardAssert(
typeof countValue === 'number',
'Count aggregate field value is not a number: ' + countValue
);

return Promise.resolve(
new AggregateQuerySnapshot<{ count: AggregateField<number> }>(
this.query,
{
count: countValue
}
)
);
}
);
}
}
21 changes: 14 additions & 7 deletions packages/firestore/src/core/firestore_client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,17 +17,17 @@

import { GetOptions } from '@firebase/firestore-types';

import {
AbstractUserDataWriter,
AggregateField,
AggregateQuerySnapshot
} from '../api';
import { LoadBundleTask } from '../api/bundle';
import {
CredentialChangeListener,
CredentialsProvider
} from '../api/credentials';
import { User } from '../auth/user';
import {
AggregateField,
AggregateQuerySnapshot,
getCount
} from '../lite-api/aggregate';
import { Query as LiteQuery } from '../lite-api/reference';
import { LocalStore } from '../local/local_store';
import {
Expand Down Expand Up @@ -68,6 +68,7 @@ import {
OfflineComponentProvider,
OnlineComponentProvider
} from './component_provider';
import { CountQueryRunner } from './count_query_runner';
import { DatabaseId, DatabaseInfo } from './database_info';
import {
addSnapshotsInSyncListener,
Expand Down Expand Up @@ -510,7 +511,8 @@ export function firestoreClientTransaction<T>(

export function firestoreClientRunCountQuery(
client: FirestoreClient,
query: LiteQuery<unknown>
query: LiteQuery<unknown>,
userDataWriter: AbstractUserDataWriter
): Promise<AggregateQuerySnapshot<{ count: AggregateField<number> }>> {
const deferred = new Deferred<
AggregateQuerySnapshot<{ count: AggregateField<number> }>
Expand All @@ -526,7 +528,12 @@ export function firestoreClientRunCountQuery(
)
);
} else {
const result = await getCount(query);
const datastore = await getDatastore(client);
const result = new CountQueryRunner(
query,
datastore,
userDataWriter
).run();
deferred.resolve(result);
}
} catch (e) {
Expand Down
Loading