RSG Phase 7 Development (ActivitySim#49)

* school escorting initial commit

* bundles created

* added pure escort tours after non_mand_sched

* created escorting tours & trips

* integrated escort tours and trips

* Differentiate examples between quality and validity of example models (ActivitySim#571)

* change examples that are no longer tied to an agency to fictional place names

* change name of full example also

* add back missing output dir

* restore test output dir also

* more empty dirs that got lost

* clean up docs

* example_mtc -> prototype_mtc

* Prototype MTC extended

* add all the ignored files

* add test output dirs

* remove superfluous example_test

* prototype_sf

* prototype_arc

* prototype_marin

* move dirs

* psrc

* semcog

* sandag_xborder

* placeholder_sandag

* placeholder_multiple_zone

* no more coveralls

* repair docs

* clean up example names

* black and isort (ActivitySim#578)

* black and isort

* stop emitting output dir

it fails later tests

* trace files in nested directories

because windows

* swap files for xborder model

* repair ARC MP

* downstream model integration

* print checksum even when not used

* add hashes for sandag_xborder_full

* fix dtype in university hack

* fix persons to match tours

* repair ARC

* initial commit of flexible tour/trip ids

* pycodestyle

* black formatting

* Bump version: 1.0.4 → 1.1.0

* adding frequency alternatives configs required for test system

* added additional unit tests

* added setup function for test configs dir

* formatting

* handling missing alts files in examples

* error catching around extension probs

* passing tests

* still passing tests around missing config files

* accounting for missing mtf spec in marin example

* nm tour scheduling passing

* updating stop freq in tours table

* added check for mand tour overlap with pe tours

* num chauffeurs and escortees

* fixed defaults not getting used if file missing

* merging canonical ids from flexible id work

* setting escorting tour and trip ids

* remove unneeded nmtf settings file

* black formatting

* excluding escort trips from trip scheduling

* fixing bug where mand tours not getting assigned correct id

* adding school_escorting to mp model list

* Added mwcog small area

* missed one edit

* adding school escort tour flavors as own category

* reformatting

* updating timetable windows with pure escort tours

* additional logging

* Update README.MD

* call as module

* github actions tests

* adding non-mand destination changes

* pre commit hooks

* pyproject toml

* limit numpy

* docs for mamba instead of conda

* ignore generated files

* add nbmake to test env

* fix dupe line

* fixing bad tdd merge

* repair test multiple zones for github actions

* publish develop docs

* fix docbuild env

* merging tdd alts to all tours

* adding ride share threshold to unvailability for pure escort time overlap

* cleanup

* Update .travis.yml

* fixed testing files

* fixed testing files (again)

* fixed test script again

* mins per time bin

* black formatting

* black formatting

* fixing reindex import bug

* fixing missed import

* replacing trips test table

* inserting default setting if no models in config for tests

* publish docs to branch name

* adding setup function to tests to set configs_dir injectable

* documentation

* updated testing scripts (note shorter travis script for now)

* fixed slash (windows vs. linux testing issue)

* added output folders

* updated travis script to run all tests, should pass

* docs cleaning

* docs re-style

* rebuild

* dynamic versioning docs

* version switcher

* blacken

* fix switcher url

* fix conf

* switcher update

* master to main

* deployment actions

* actions

* build wheel

* fix for testpypi

* blacken

* manual switcher

* branch docs service [makedocs]

* syntax [makedocs]

* travis depends

* checkout v3, fix versioning in docs

* only build develop docs once

* failsafe version

* documentation repairs

* python-simplified-semver

* front cards

* end testing w travis

* add mwcog test to gh-actions

* add mwcog to docs

* fixed origin bug and missing outbound trip to work

* point to data not copy it

* sort dependencies

* req sh 2.2.4

* account for variance across platforms in trip dest est

* copy bike skims for sandag test

* proper cleanup when trips that get removed if schedule failed

* changed failed trip scheduling option for example

* updating regress table

* blacken

* extending nmtf and stop frequency to demonstrate flexible ids

* param on rtol

* fix sandag_2 test files

* fix test file names

* allowing multiple escort tours in the same period

* sandag 3-zone example fix

* fixing expression for missing escort participants sandag 3_zone

* cleanup

* blacken

* Update trip_destination.py

applying valid primary origin default value

* Update trip_destination.py

blacken

* fixing bug for inbound destination for second escortee

* blacken

* Disaggregate Accessibilities (#5)

* initial commit with basic working proto pop gen

* fixed bad dependency range

* fixed vary_on method to vary all with mapped fields after

* added check if already dataframe

* added dataframe check if the table passed is already a dataframe

* working proto pop gen. still debugging mandatory work/school model

* check all table is df

* workplace runs, next testing school

* began setting up to get all logsums, not just chosen ones

* restructured to run as either one-off or model step

* revert to last merge. Removed dataframe check in iterate, no longer needed to modify this file

* extracted logsums, need to inject into pipeline or save output?

* working standalone or step model

* extractable accessibilities for fixed and nonmandatory

* extractable accessibilities for fixed and nonmandatory

* cleanup and add documentation

* resolved duplicates of old 'example' folders

* model uses write_tables functionality

* disaggregate accessibility runs as model step or standalone. Runs as subprocess

* working model test. table initialization not yet working

* added logsum short circuit to get logsums and avoid drawing unneccesary samples

* working model plus major cleanup

* working model plus major cleanup

* override sample size

* fixed trace output files

* skip_choice default to False

* fixed empty logsums bug

* cleanup redundent parameters

* fixed list index instead of set

* coordinated model run files

* added default skip to false

* added multiprocessing line

* began setup for multiprocessing

* fixed sampling problem. Pipeline for multi processing, NOT WORKING YET

* deleted run file, not needed anymore

* update release instructions

* auto-build docs on release

* remind to update switcher

* need checkout

* added helper functions for handling suffix args

* to enable multiprocessing, overhauled pipeline method to include proto_ tables rather than overwriting main tables

* working multiprocess, but requires debugging. fails on other models

* working multiprocess, but requires debugging. fails on other models

* cleanup of mp table registering

* fixed tracing and slicing issue

* removed old 'run' model

* setup example 2 zones

* minimum working 2 zone

* multizone basic working

* fixed buggy settings

* multiprocessing working, but empty final tables

* fixed blank output bug and cleaned up settings to use verbose table names (not just 'disaggregate_accessibility')

* cleanup run scripts a bit

* fixed missing base path for full run

* fixed missing path

* fixed typo

* fixed 1 per taz sampling

* fixing person merge bug and filtering workers

* fixed duplicate tours!

* updated settings for initialize disagg

* removed obsolete file

* cleaned up file and moved initialize to tables/...

* added find nearest node function

* added initialize disaggregate_accessibility to initialize_households

* moved initialize steps to 'tables/disaggregate_accessibility'

* updated settings to include initialize disaggregate accessibilities as part of initialize_households

* moved initialize disagg accessibilities to tables. Created working merge step using naive bayes and 'hard' join

* add mp back in

* PEP formatting revisions

* fixed logsums merge on households instead of persons

* fixed _accessibility suffix

* fixed conflict with persons_merged

* updated yaml to use simpler join method

* PEP formatting fixes

* refreshed example folder from develop

* added missing line at end of file

* black fixes

* black fixes to disaggregate accessibility changes

* fixed missing column pipeline error

* merged disagg accessibilities into mtc_extended and added doc section

* ran black on disaggregate_accessibility.py

* updated dependencies

* removing sklearn imports

* blacken

* add if none catch

* fixed None suffix default

* moved order of get_table(persons_merged) to avoid pulling prematurely in xborder

* tested and cleaned up rng channels

* setup injectable suffixes to allow add_size_table as model step not just function

* removed accessibility output from test

* re blacken py scripts

* fixed tracing typo

* added variable index name suffix to pass optionally

* pipeline housekeeping to cleanup any tables, traceables, or channels that were touched during disagg

* added multiprocess testing

* blacken updates

* updated test scripts to include MP, problem with vehicle model in mp (even without disagg acc?)

* added improved origin sampling, resolved issue with merging with sample <100%. Need to address random seed for origins

* added sci-kit learn to test depends

* fixed person merging error causing pytest fail, uses inject method to ensure consistency

* cleanup comments

* fixed pytest to include accessibility table in regress

* setup for mp test, but needs debugging

* 'blacken'

* 'blacken'

* cleanup example folder

* fixed pipeline NoneType bug for disagg accessibility table

* fixed pytest fail on mp, due to exept:True on mp_simulate

* 'blacken'

* added weighted k-means method

* created run script for multiple sampling scenarios

* blacken changes

* blacken

* fixed copy script

* blacken sampling script

* fixed n_zone when integer

* fixed typo

* update sampling script

* 'blacken'

* fixed replacement sample bug

* updated documentation

* more flexible scenario testing

* blacken

Co-authored-by: Nick Fournier <[email protected]>
Co-authored-by: Jeffrey Newman <[email protected]>
Co-authored-by: Nick Fournier <[email protected]>

* Shadow Pricing Enhancements (#7)

* updated scripts to include simulation-based shadow pricing

* blacken

* Updated shadow_pricing.yaml for mtc example

* code cleanup

* more cleanup

* documentation and passing tests

* passing tests

* passing tests

* updated doc on shadow pricing

* 2nd Update model doc on shadow pricing

* more doc update on shadow pricing

* fixing pandas future warning

* blacken

* bug in trying to access shadow price settings when not running shadow pricing

* limiting pandas version

* always updating choices

* testing removal of lognormal for hh vot

* putting hh vot back in

* updating to match sharrow test versions

* raw person table for buffer instead of injectable

* adding segmentation, output by iteration, and external worker removal

* formatting & documentation

Co-authored-by: aletzdy <[email protected]>

* updating mtc_extended test files

* no sp for non-work or school, better logging, weighting options

* sample TAZ only if available MAZ when shadow pricing

* adding missed weight column option

* cleaning up comments

Co-authored-by: Jeffrey Newman <[email protected]>
Co-authored-by: Andrew Rohne <[email protected]>
Co-authored-by: Nick Fournier <[email protected]>
Co-authored-by: Nick Fournier <[email protected]>
Co-authored-by: aletzdy <[email protected]>

.bumpversion.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 1.0.4
+current_version = 1.1.0
 commit = True
 tag = True
 parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(\.(?P<release>.*))?

.github/workflows/branch-docs.yml

@@ -0,0 +1,55 @@
+name: ActivitySim Branch Docs
+# This workflow is provided as a service for forks to build branch-specific documentation.
+
+on: push
+
+jobs:
+  docbuild:
+    if: "contains(github.event.head_commit.message, '[makedocs]') && (github.repository_owner != 'ActivitySim') && (github.ref_name != 'develop')"
+    # develop branch docs are built at the end of the core test workflow, regardless of repository owner or commit message flags
+    name: ubuntu-latest py3.9
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        shell: bash -l {0}
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 0  # get all tags, lets setuptools_scm do its thing
+      - name: Set up Python 3.9
+        uses: actions/setup-python@v2
+        with:
+          python-version: 3.9
+      - name: Install dependencies
+        uses: conda-incubator/setup-miniconda@v2
+        with:
+          miniforge-variant: Mambaforge
+          miniforge-version: latest
+          use-mamba: true
+          environment-file: conda-environments/docbuild.yml
+          python-version: 3.9
+          activate-environment: docbuild
+          auto-activate-base: false
+          auto-update-conda: false
+      - name: Install activitysim
+        run: |
+          python -m pip install .
+      - name: Conda checkup
+        run: |
+          conda info -a
+          conda list
+          echo REPOSITORY ${{ github.repository }}
+          echo REF ${{ github.ref }}
+          echo REF_NAME ${{ github.ref_name }}
+      - name: Build the docs
+        run: |
+          cd docs
+          make clean
+          make html
+      - name: Push to GitHub Pages
+        uses: peaceiris/[email protected]
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          # Token is created automatically by Github Actions, no other config needed
+          publish_dir: ./docs/_build/html
+          destination_dir: ${{ github.ref_name }}

.github/workflows/config-testpypi-version.py

@@ -0,0 +1,50 @@
+import argparse
+import copy
+import pathlib
+
+import tomli
+import tomli_w
+
+
+def split_path(path, sep="/"):
+    if isinstance(path, str):
+        return [part for part in path.split(sep) if part]
+    else:
+        return path
+
+
+def extract(mapping, path, sep="/"):
+    parts = split_path(path, sep=sep)
+    cur = mapping
+    for part in parts:
+        cur = cur[part]
+
+    return cur
+
+
+def update(mapping, path, value, sep="/"):
+    new = copy.deepcopy(mapping)
+
+    parts = split_path(path, sep=sep)
+    parent = extract(new, parts[:-1])
+    parent[parts[-1]] = value
+
+    return new
+
+
+parser = argparse.ArgumentParser()
+parser.add_argument("path", type=pathlib.Path)
+args = parser.parse_args()
+
+content = args.path.read_text()
+decoded = tomli.loads(content)
+with_local_scheme = update(
+    decoded, "tool.setuptools_scm.local_scheme", "no-local-version", sep="."
+)
+# work around a bug in setuptools / setuptools-scm
+with_setuptools_pin = copy.deepcopy(with_local_scheme)
+requires = extract(with_setuptools_pin, "build-system.requires", sep=".")
+requires[0] = "setuptools>=42,<60"
+
+new_content = tomli_w.dumps(with_setuptools_pin)
+args.path.write_text(new_content)