diff --git a/geography/README.md b/geography/README.md index 54fc0145..fb05689f 100644 --- a/geography/README.md +++ b/geography/README.md @@ -97,7 +97,7 @@ Placeholder -- link to schema to be added later. | ---------------- | --------- | --- | --------------------------------------------------------------------------------------------- | | `name` | String | Required | Name of geography | | `description` | String | Optional | Detailed description of geography | -| `geography_type` | Sting | Optional | Type of geography, e.g. `municipal_boundary` or `council_district` or custom text. See [Geography Types](#geography-types). | +| `geography_type` | String | Optional | Type of geography, e.g. `municipal_boundary` or `council_district` or custom text. See [Geography Types](#geography-types). | | `geography_id` | UUID | Required | Unique ID of geography | | `geography_json` | UUID | Required | The GeoJSON that defines the geographical coordinates. | | `effective_date` | [timestamp][ts] | Optional | The date at which a Geography is considered "live". Must be at or after `published_date`. | diff --git a/metrics/core_metrics.md b/metrics/core_metrics.md index 4b0b2ee6..e061c423 100644 --- a/metrics/core_metrics.md +++ b/metrics/core_metrics.md @@ -41,7 +41,6 @@ The following represent the suggested MDS core metric dimensions: | provider_id | Transportation provider id issued by OMF and [tracked here](/providers.csv) | | geography_id | [MDS Geography](/geography) | | vehicle_type | [Vehicle Type](/agency#vehicle-type) defined by MDS | -| special_group_type | [Special Group Type](#special-group-type) defined by MDS | [Top][toc] diff --git a/provider/README.md b/provider/README.md index 77d604a7..ace02f6f 100644 --- a/provider/README.md +++ b/provider/README.md @@ -20,6 +20,11 @@ This specification contains a data standard for *mobility as a service* provider * [Routes](#routes) * [Status Changes][status] * [Status Changes - Query Parameters](#status-changes---query-parameters) +* [Reports](#reports) + * [Reports - Response](#reports---response) + * [Reports - Example](#reports---example) + * [Special Group Type](#special-group-type) + * [Data Redaction](#data-redaction) * [Realtime Data](#realtime-data) * [GBFS](#GBFS) * [Data Latency Requirements][data-latency] @@ -291,6 +296,118 @@ Without an `event_time` query parameter, `/status_changes` shall return a `400 B [Top][toc] + +## Reports + +Reports are information that providers can send back to agencies containing aggregated data that is not contained within other MDS endpoints, like counts of special groups of riders. + +The authenticated reports are monthly, historic flat files that may be pre-generated by the provider. + +### Reports - Response + +**Endpoint:** `/reports` +**Method:** `GET` +**[Beta feature][beta]:** Yes (as of 1.1.0) +**Schema:** TBD when out of beta +**`data` Filename:** monthly file named by year and month, e.g. `/reports/YYYY-MM.csv` +**`data` Payload:** monthly CSV files with the following structure: + +| Name | Type | Comments | +| ------------------ | ----------------------------------------- | ------------------------------------------------ | +| StartDate | date | Start date of the data row, ISO 8601 format, local timezone | +| Duration | string | Value is always `P1M` for monthly. Based on [ISO 8601 duration](https://en.wikipedia.org/wiki/ISO_8601#Durations) | +| Special Group Type | [Special Group Type](#special-group-type) | Type that applies to this row | +| Geography ID | [Geography](/geography) | ID that applies to this row. Includes all IDs in /geography. When there is no /geography then return `null` for this value and counts based on the entire operating area. | +| Vehicle Type | [Vehicle Type](/agency#vehicle-type) | Type that applies to this row | +| Trip Count | integer | Count of trips taken for this row | +| Rider Count | integer | Count of unique riders for this row | + +#### Data Notes + +Report contents include every combination of special group types, geography IDs, and vehicle types in operation for each month since the provider began operations in the jurisdiction. New files are added monthly in addition to the previous monthly historic files. + +Counts are calculated based the city's local time zone, and this time zone is returned within the `StartDate` value. For months where there is a Daylight Saving Time change, use the timezone that is in the majority of the month. + +All geography IDs included in the city published [Geography](/geography) API endpoint are included in the report results. In lieu of serving an API, this can alternately be a [flat file](/geography#file-format) created by the city and sent to the provider via link. If there is no `/geography` available, then counts are for the entire agency operating area, and `null` is returned for Geography ID. + +### Reports - Example + +For 3 months of provider operation in a city (September 2019 through November 2019) for 3 geographies, 2 vehicle types, and 1 special group. Timezone is Eastern Time in the US which is _-4_ from UTC before November 3, 2019, and _-5_ after. Values of `-1` represent [redacted data](#data-redaction) counts. + +**September 2019** `/reports/2019-09.csv` + +```csv +StartDate,Duration,Special Group Type,Geography ID,Vehicle Type,Trip Count,Rider Count +2019-09-01T00:00-04,P1M,all_riders,44428624-186b-4fc3-a7fb-124f487464a1,scooter,1302,983 +2019-09-01T00:00-04,P1M,low_income,44428624-186b-4fc3-a7fb-124f487464a1,scooter,201,104 +2019-09-01T00:00-04,P1M,all_riders,44428624-186b-4fc3-a7fb-124f487464a1,bicycle,530,200 +2019-09-01T00:00-04,P1M,low_income,44428624-186b-4fc3-a7fb-124f487464a1,bicycle,75,26 +2019-09-01T00:00-04,P1M,all_riders,03db06d0-3998-406a-92c7-25a83fc2784a,scooter,687,450 +2019-09-01T00:00-04,P1M,low_income,03db06d0-3998-406a-92c7-25a83fc2784a,scooter,98,45 +2019-09-01T00:00-04,P1M,all_riders,03db06d0-3998-406a-92c7-25a83fc2784a,bicycle,256,104 +2019-09-01T00:00-04,P1M,low_income,03db06d0-3998-406a-92c7-25a83fc2784a,bicycle,41,16 +2019-09-01T00:00-04,P1M,all_riders,8ad39dc3-005b-4348-9d61-c830c54c161b,scooter,201,140 +2019-09-01T00:00-04,P1M,low_income,8ad39dc3-005b-4348-9d61-c830c54c161b,scooter,35,21 +2019-09-01T00:00-04,P1M,all_riders,8ad39dc3-005b-4348-9d61-c830c54c161b,bicycle,103,39 +2019-09-01T00:00-04,P1M,low_income,8ad39dc3-005b-4348-9d61-c830c54c161b,bicycle,15,-1 +``` + +**October 2019** `/reports/2019-10.csv` + +```csv +StartDate,Duration,Special Group Type,Geography ID,Vehicle Type,Trip Count,Rider Count +2019-10-01T00:00-04,P1M,all_riders,44428624-186b-4fc3-a7fb-124f487464a1,scooter,1042,786 +2019-10-01T00:00-04,P1M,low_income,44428624-186b-4fc3-a7fb-124f487464a1,scooter,161,83 +2019-10-01T00:00-04,P1M,all_riders,44428624-186b-4fc3-a7fb-124f487464a1,bicycle,424,160 +2019-10-01T00:00-04,P1M,low_income,44428624-186b-4fc3-a7fb-124f487464a1,bicycle,60,0 +2019-10-01T00:00-04,P1M,all_riders,03db06d0-3998-406a-92c7-25a83fc2784a,scooter,550,360 +2019-10-01T00:00-04,P1M,low_income,03db06d0-3998-406a-92c7-25a83fc2784a,scooter,78,36 +2019-10-01T00:00-04,P1M,all_riders,03db06d0-3998-406a-92c7-25a83fc2784a,bicycle,205,83 +2019-10-01T00:00-04,P1M,low_income,03db06d0-3998-406a-92c7-25a83fc2784a,bicycle,33,13 +2019-10-01T00:00-04,P1M,all_riders,8ad39dc3-005b-4348-9d61-c830c54c161b,scooter,161,112 +2019-10-01T00:00-04,P1M,low_income,8ad39dc3-005b-4348-9d61-c830c54c161b,scooter,28,-1 +2019-10-01T00:00-04,P1M,all_riders,8ad39dc3-005b-4348-9d61-c830c54c161b,bicycle,82,31 +2019-10-01T00:00-04,P1M,low_income,8ad39dc3-005b-4348-9d61-c830c54c161b,bicycle,-1,0 +``` + +**November 2019** `/reports/2019-11.csv` + +```csv +StartDate,Duration,Special Group Type,Geography ID,Vehicle Type,Trip Count,Rider Count +2019-11-01T00:00-05,P1M,all_riders,44428624-186b-4fc3-a7fb-124f487464a1,scooter,834,629 +2019-11-01T00:00-05,P1M,low_income,44428624-186b-4fc3-a7fb-124f487464a1,scooter,129,66 +2019-11-01T00:00-05,P1M,all_riders,44428624-186b-4fc3-a7fb-124f487464a1,bicycle,339,128 +2019-11-01T00:00-05,P1M,low_income,44428624-186b-4fc3-a7fb-124f487464a1,bicycle,48,-1 +2019-11-01T00:00-05,P1M,all_riders,03db06d0-3998-406a-92c7-25a83fc2784a,scooter,440,288 +2019-11-01T00:00-05,P1M,low_income,03db06d0-3998-406a-92c7-25a83fc2784a,scooter,62,29 +2019-11-01T00:00-05,P1M,all_riders,03db06d0-3998-406a-92c7-25a83fc2784a,bicycle,164,66 +2019-11-01T00:00-05,P1M,low_income,03db06d0-3998-406a-92c7-25a83fc2784a,bicycle,26,0 +2019-11-01T00:00-05,P1M,all_riders,8ad39dc3-005b-4348-9d61-c830c54c161b,scooter,129,90 +2019-11-01T00:00-05,P1M,low_income,8ad39dc3-005b-4348-9d61-c830c54c161b,scooter,22,-1 +2019-11-01T00:00-05,P1M,all_riders,8ad39dc3-005b-4348-9d61-c830c54c161b,bicycle,-1,25 +2019-11-01T00:00-05,P1M,low_income,8ad39dc3-005b-4348-9d61-c830c54c161b,bicycle,0,0 +``` + +### Special Group Type + +Here are the possible values for the `special_group_type` dimension field: + +| Name | Description | +| ---------- | --------------------------------------------------------------------------------------------------------------------- | +| low_income | Trips where a low income discount is applied by the provider, e.g., a discount from a qualified provider equity plan. | +| all_riders | All riders from any group | + +Other special group types may be added in future MDS releases as relevant agency and provider use cases are identified. When additional special group types or metrics are proposed, a thorough review of utility and relevance in program oversight, evaluation, and policy development should be done by OMF Working Groups, as well as any privacy implications by the OMF Privacy Committee. + +### Data Redaction + +Some combinations of parameters may return a small count of trips, which could increase a privacy risk of re-identification. To correct for that, Reports does not return data below a certain count of results. This is called k-anonymity, and the threshold is set at a k-value of 10. + +If the report returns count values from 1 through 10, then a number of value `-1` is returned to represent redacted data. Counts of `0` are still returned as usual. This requirement is applied to both counts of trips and unique riders. + +[Top][toc] + + ## Realtime Data ### GBFS