Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW] Code review for initial check-in #7

Merged
merged 115 commits into from
Sep 4, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
115 commits
Select commit Hold shift + click to select a range
1b716b1
init check-in (cpp/python)
zhangjianting Aug 14, 2019
ebe73bc
for code review
zhangjianting Aug 22, 2019
3c3295e
add paths for custring libs
zhangjianting Aug 23, 2019
1b49472
update on documentating python test/demo code
zhangjianting Aug 23, 2019
ae6fe28
Update README.md
zhangjianting Aug 23, 2019
378de35
Update README.md
zhangjianting Aug 23, 2019
675d719
Update README.md
zhangjianting Aug 23, 2019
6563b38
Update README.md
zhangjianting Aug 23, 2019
9d42ab2
Update README.md
zhangjianting Aug 23, 2019
9d5c761
Update cpp/include/cuspatial/hausdorff.hpp
zhangjianting Aug 23, 2019
73a2f05
Update cpp/include/cuspatial/hausdorff.hpp
zhangjianting Aug 23, 2019
5f97192
Update cpp/include/cuspatial/hausdorff.hpp
zhangjianting Aug 23, 2019
0eb5670
Update README.md
zhangjianting Aug 23, 2019
1c187a8
Update cpp/include/cuspatial/hausdorff.hpp
zhangjianting Aug 23, 2019
1022c8d
Update cpp/include/cuspatial/hausdorff.hpp
zhangjianting Aug 23, 2019
f0fc3a8
Update README.md
zhangjianting Aug 23, 2019
d9e7fca
Update README.md
zhangjianting Aug 23, 2019
4a67a78
Update README.md
zhangjianting Aug 23, 2019
1dc84b1
Update README.md
zhangjianting Aug 23, 2019
79b5b1f
Update README.md
zhangjianting Aug 23, 2019
cf6b51a
Update README.md
zhangjianting Aug 23, 2019
db4d6fc
Update README.md
zhangjianting Aug 23, 2019
afac815
Update README.md
zhangjianting Aug 23, 2019
111149c
Update cpp/include/cuspatial/coord_trans.hpp
zhangjianting Aug 23, 2019
0d5fd7d
Update cpp/include/cuspatial/coord_trans.hpp
zhangjianting Aug 23, 2019
6f3b2e2
Update cpp/include/cuspatial/coord_trans.hpp
zhangjianting Aug 23, 2019
67f3356
Update cpp/include/cuspatial/coord_trans.hpp
zhangjianting Aug 23, 2019
227d187
Update cpp/include/cuspatial/coord_trans.hpp
zhangjianting Aug 23, 2019
4d73d59
Update cpp/include/cuspatial/coord_trans.hpp
zhangjianting Aug 23, 2019
8dd7f5f
Update cpp/include/cuspatial/coord_trans.hpp
zhangjianting Aug 23, 2019
4bf2c66
Update cpp/include/cuspatial/coord_trans.hpp
zhangjianting Aug 23, 2019
fed0b48
Update cpp/include/cuspatial/coord_trans.hpp
zhangjianting Aug 23, 2019
f150a05
Update cpp/include/cuspatial/coord_trans.hpp
zhangjianting Aug 23, 2019
81cc626
Update cpp/include/cuspatial/cuspatial.h
zhangjianting Aug 23, 2019
c70cc57
Update cpp/include/cuspatial/cuspatial.h
zhangjianting Aug 23, 2019
05a3f9a
Update cpp/include/cuspatial/cuspatial.h
zhangjianting Aug 23, 2019
fc86a09
Update cpp/include/cuspatial/cuspatial.h
zhangjianting Aug 23, 2019
0e12651
Update cpp/include/cuspatial/cuspatial.h
zhangjianting Aug 23, 2019
0acd63a
Update cpp/include/cuspatial/cuspatial.h
zhangjianting Aug 23, 2019
ca85b2a
formatting
zhangjianting Aug 24, 2019
b3018bd
2nd round revision for code review
zhangjianting Aug 24, 2019
f1e4788
Update cpp/include/cuspatial/haversine.hpp
zhangjianting Aug 28, 2019
645479f
for 3rd round code review
zhangjianting Aug 28, 2019
3b5e989
Merge branch 'fea-initial-code' of github.com:zhangjianting/cuspatial…
zhangjianting Aug 28, 2019
b48b61c
Update cpp/include/cuspatial/trajectory.hpp
zhangjianting Aug 28, 2019
80faf5a
for 3rd round code review
zhangjianting Aug 28, 2019
645e606
Merge branch 'fea-initial-code' of github.com:zhangjianting/cuspatial…
zhangjianting Aug 28, 2019
28ac08a
Update cpp/include/cuspatial/soa_readers.hpp
zhangjianting Aug 28, 2019
1a880f9
for 3rd code review
zhangjianting Aug 28, 2019
0722277
Merge branch 'fea-initial-code' of github.com:zhangjianting/cuspatial…
zhangjianting Aug 28, 2019
b194258
Update cpp/include/cuspatial/soa_readers.hpp
zhangjianting Aug 28, 2019
ff18076
Add .gitignore
harrism Aug 28, 2019
9838cb3
Update version no. and make finding cudf more flexible
harrism Aug 28, 2019
8296a83
Merge branch 'fea-initial-code' of github.com:zhangjianting/cuspatial…
harrism Aug 28, 2019
02551e9
for 3rd round code review: making location3d and coord_2d templated
zhangjianting Aug 28, 2019
34cd979
Merge branch 'fea-initial-code' of github.com:zhangjianting/cuspatial…
zhangjianting Aug 28, 2019
5bd8913
Update cpp/include/cuspatial/soa_readers.hpp
zhangjianting Aug 28, 2019
e2b2de3
Update cpp/include/cuspatial/trajectory.hpp
zhangjianting Aug 28, 2019
ed23bfa
for 3rd round code review
zhangjianting Aug 28, 2019
20cc9b1
Merge branch 'fea-initial-code' of github.com:zhangjianting/cuspatial…
zhangjianting Aug 28, 2019
4b966bd
for 3rd round code review (continued)
zhangjianting Aug 29, 2019
71fcf3e
for 3rd round code review (amendment)
zhangjianting Aug 29, 2019
9809b9c
Update .gitignore cudf->cuspatial
harrism Aug 29, 2019
8646847
Fix URL in setup.py
harrism Aug 29, 2019
f625d93
for 3rd code review (amendment)
zhangjianting Aug 29, 2019
f1b7e2b
Merge branch 'fea-initial-code' of github.com:zhangjianting/cuspatial…
zhangjianting Aug 29, 2019
6da406a
adding typename
zhangjianting Aug 29, 2019
82d4ff4
Clean up and fix Cython bindings and trajectory APIs
harrism Aug 30, 2019
5982544
Merge branch 'fea-initial-code' of github.com:zhangjianting/cuspatial…
harrism Aug 30, 2019
4b729d9
Rename traj.p** to trajectory.p**
harrism Aug 30, 2019
2b7b8ed
Test cleanup and disabling failing tests for now.
harrism Aug 30, 2019
cdc4039
coord_to_traj --> derive_trajectories, plus cleanup
harrism Aug 30, 2019
2bd6371
Clean up README.md
harrism Aug 30, 2019
4f9c9ee
Clean up data README.md
harrism Aug 30, 2019
ce36d37
simplify pip_test for test on GPU only; add pip_compare to compare CP…
zhangjianting Aug 31, 2019
9b256bd
uint-->uint32_t;adding hausdorff_toy test; grouping tests
zhangjianting Aug 31, 2019
7b7bfb6
add hausdorff_compare test
zhangjianting Aug 31, 2019
d1c9db6
pip_test: comparing two rounds GPU results
zhangjianting Aug 31, 2019
a03f82a
Add haversine_toy test; remove uncessary header files
zhangjianting Sep 1, 2019
c71fc5b
remove unused code; remove __host__
zhangjianting Sep 1, 2019
2039056
Fix pip_test crash.
harrism Sep 2, 2019
c1cce54
Readme markdown fixes.
harrism Sep 2, 2019
9c395be
Merge branch 'fea-initial-code' of github.com:zhangjianting/cuspatial…
harrism Sep 2, 2019
a96d5b7
Fix merge error.
harrism Sep 2, 2019
13cbd63
cuspatial.h renamed to cuspatial/types.hpp
harrism Sep 2, 2019
2fc09b3
Header style and documentation cleanup
harrism Sep 2, 2019
02e4e0d
Point in polygon API cleanup
harrism Sep 2, 2019
99d894f
Reorganize / rename utilities
harrism Sep 2, 2019
cbf84fb
pip.hpp --> point_in_polygon.hpp
harrism Sep 2, 2019
fd7740b
Refactor query naming; cleanup
harrism Sep 2, 2019
c71c65c
More pip renaming
harrism Sep 2, 2019
99d5385
add subset_trajectory and its test on toy data
zhangjianting Sep 2, 2019
c489053
add two toy tests for derive_trajectories and trajectory_distance_and…
zhangjianting Sep 2, 2019
6058cbe
Eliminate global `using namespace ...` declarations.
harrism Sep 3, 2019
a8cc7ad
Merge branch 'fea-initial-code' of github.com:zhangjianting/cuspatial…
harrism Sep 3, 2019
e46e4c0
add three test (spaital_window, coordinate_transform, trajectory_spat…
zhangjianting Sep 3, 2019
dad03ce
trajectory cleanup
harrism Sep 3, 2019
c5724bf
Merge branch 'fea-initial-code' of github.com:zhangjianting/cuspatial…
harrism Sep 3, 2019
0a8c1f0
Fix forward declarations.
harrism Sep 3, 2019
63165f3
Remove more global using namespace decls
harrism Sep 3, 2019
3f5a931
Clean up polygon_soa_reader
harrism Sep 3, 2019
a137faf
clean-up: indentation, simplification
zhangjianting Sep 3, 2019
69cf890
code clean-up: indentation, using T* instead of thrust::device_ptr an…
zhangjianting Sep 3, 2019
7a3c36f
code clean-up: indentation, using T* instead of thrust::device_ptr an…
zhangjianting Sep 3, 2019
afc19b0
code clean-up: use anonymous namespace, remove output and timing code
zhangjianting Sep 3, 2019
d52a80b
code clean-up: use anonymous namespace, remove output and timing code
zhangjianting Sep 3, 2019
2e155bd
code clean-up: remove output, using anonymous namespace etc.
zhangjianting Sep 3, 2019
d20d7df
Remove C-style casts.
harrism Sep 3, 2019
452c56f
Merge branch 'fea-initial-code' of github.com:zhangjianting/cuspatial…
harrism Sep 3, 2019
ed953e9
code clean-up: remove debug output and timing code, using anonymous n…
zhangjianting Sep 3, 2019
c6608a1
Merge branch 'fea-initial-code' of https://github.com/zhangjianting/c…
zhangjianting Sep 3, 2019
60999ce
replace count_if with count
harrism Sep 4, 2019
7b472a0
Safely copy gdf_scalar values.
harrism Sep 4, 2019
4624e57
Changelog for PR #7
harrism Sep 4, 2019
07e68b9
Clearer window parameters for spatial_window_points
harrism Sep 4, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
139 changes: 139 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
## Common
__pycache__
*.py[cod]
*$py.class
*.a
*.o
*.so
*.dylib
.cache
.vscode
*.swp
*.pytest_cache
DartConfiguration.tcl
.DS_Store
*.manifest
*.spec

## Python build directories & artifacts
dist/
cudf.egg-info/
python/build
python/*/build
python/cuspatial/*/bindings/**/*.cpp
python/cuspatial/*/bindings/**/*.h
python/cuspatial/*/bindings/.nfs*
python/cuspatial/*.ipynb
python/cuspatial/.ipynb_checkpoints
.Python
env/
develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/

## Patching
*.diff
*.orig
*.rej

## C++ build directories & artifacts
CMakeFiles/
Debug
build/
cpp/build/
cpp/thirdparty/googletest/

## Eclipse IDE
.project
.cproject
.settings

## IntelliJ IDE
.idea/
.idea_modules/
*.iml
*.ipr
*.iws

## Doxygen
cpp/doxygen/html

#Java
target

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# pyenv
.python-version

# celery beat schedule file
celerybeat-schedule

# SageMath parsed files
*.sage.py

# dotenv
.env

# virtualenv
.venv
venv/
ENV/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
8 changes: 2 additions & 6 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,9 @@
# ___PROJECT___ 0.0.0 (DD Mon YYYY)
# cuSpatial 0.10.0 (Date TBD)

## New Features

- ...
- PR #7 Initial code

## Improvements

- ...

## Bug Fixes

- ...
180 changes: 104 additions & 76 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,76 +1,104 @@
# RAPIDS Standard Repo Template

This is repo is the standard RAPIDS repo with the following items to make all RAPIDS repos consistent:

- GitHub File Templates
- Issue templates
- PR template
- GitHub Repo Templates
- Issue/PR labels
- Project tracking and release board templates
- Files
- `CHANGELOG.md` skeleton
- `CONTRIBUTING.md` skeleton
- `LICENSE` file with Apache 2.0 License
- `README.md` skeleton


## Usage for new RAPIDS repos

1. Clone this repo
2. Find/replace all in the clone of `___PROJECT___` and replace with the name of the new library
3. Inspect files to make sure all replacements work and update text as needed
4. Customize issue/PR templates to fit the repo
5. Update `CHANGELOG.md` with next release version, see [changelog format](https://rapidsai.github.io/devdocs/docs/resources/changelog/) for more info
6. Add developer documentation to the end of the `CONTRIBUTING.md` that is project specific and useful for developers contributing to the project
- The goal here is to keep the `README.md` light, so the development/debugging information should go in `CONTRIBUTING.md`
7. Complete `README.md` with project description, quick start, install, and contribution information
8. Remove everything above the RAPIDS logo below from the `README.md`
9. Check `LICENSE` file is correct
10. Change git origin to point to new repo and push
11. Alert OPS team to copy labels and project boards to new repo

## Usage for existing RAPIDS repos

1. Follow the steps 1-9 above, but add the files to your existing repo and merge
2. Alert OPS team to copy labels and project boards to new repo

## Useful docs to review

- Issue triage & release planning
- [Issue triage process with GitHub projects](https://rapidsai.github.io/devdocs/docs/releases/triage/)
- [Release planning with GitHub projects](https://rapidsai.github.io/devdocs/docs/releases/planning/)
- Code release process
- [Hotfix process](https://rapidsai.github.io/devdocs/docs/releases/hotfix/)
- [Release process](https://rapidsai.github.io/devdocs/docs/releases/process/)
- Code contributions
- [Code contribution guide](https://rapidsai.github.io/devdocs/docs/contributing/code/)
- [Filing issues](https://rapidsai.github.io/devdocs/docs/contributing/issues/)
- [Filing PRs](https://rapidsai.github.io/devdocs/docs/contributing/prs/)
- [Code of conduct](https://rapidsai.github.io/devdocs/docs/resources/conduct/)
- Development process
- [Git branching and merging methodology](https://rapidsai.github.io/devdocs/docs/resources/git/)
- [Versions and tags](https://rapidsai.github.io/devdocs/docs/resources/versions/)
- [Changelog format](https://rapidsai.github.io/devdocs/docs/resources/changelog/)
- [Style guide](https://rapidsai.github.io/devdocs/docs/resources/style/)
- [Labels](https://rapidsai.github.io/devdocs/docs/maintainers/labels/)

---

# <div align="left"><img src="https://rapids.ai/assets/images/rapids_logo.png" width="90px"/>&nbsp;___PROJECT___</div>

The [RAPIDS](https://rapids.ai) ___PROJECT___ ..._insert project description_...

**NOTE:** For the latest stable [README.md](https://github.com/rapidsai/___PROJECT___/blob/master/README.md) ensure you are on the `master` branch.

## Quick Start

## Install ___PROJECT___

### Conda

### Pip

## Contributing Guide

Review the [CONTRIBUTING.md](https://github.com/rapidsai/___PROJECT___/blob/master/CONTRIBUTING.md) file for information on how to contribute code and issues to the project.
# cuSpatial
## GPU-Accelerated Spatial and Trajectory Data Management and Analytics Library

**NOTE:** cuSpatial depends on [cuDF](https://github.com/rapidsai/cudf) and
[RMM](https://github.com/rapidsai/rmm) from [RAPIDS](https://rapids.ai/).

## Implemented operations:
cuSpatial supports the following operations on spatial and trajectory data:
1. Spatial window query
2. Point-in-polygon test
3. Haversine distance
4. Hausdorff distance
5. Deriving trajectories from point location data
6. Computing distance/speed of trajectories
7. Computing spatial bounding boxes of trajectories

Future support is planned for the following operations.
1. Temporal window query
2. Temporal point query (year+month+day+hour+minute+second+millisecond)
3. Point-to-polyline nearest neighbor distance
4. Grid-based indexing for points and polygons
5. Quadtree-based indexing for large-scale point data
6. R-Tree-based indexing for Polygons/Polylines

## Compile / Install C++ backend
To compile and run cuSpatial, use the following steps.

## Install dependencies

Currently, building cuSpatial requires a source installation of cuDF. Install
cuDF by following the [instructions](https://github.com/rapidsai/cudf/blob/branch-0.10/CONTRIBUTING.md#script-to-build-cudf-from-source)

The rest of steps assume the environment variable `CUDF_HOME` points to the
root directory of your clone of the cuDF repo, and that the `cudf_dev` Anaconda
environment created in step 3 is active.

## Clone, build and install cuSpatial

1. export `CUSPATIAL_HOME=$(pwd)/cuspatial`
2. clone the cuSpatial repo

```
git clone https://github.com/rapidsai/cuspatial.git $CUSPATIAL_HOME
```

3. Compile and install C++ backend

```
cd $CUSPATIAL_HOME/cpp
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX
make # (or make -j [n])
make install
```

cuSpatial should now be installed at `$CONDA_PREFIX`. The cuspatial include path
is `$CONDA_PREFIX/include/cuspatial/` and the library path is
`$CONDA_PREFIX/lib/libcuspatial.so`.

4. Compile and install cuSpatial Python wrapper and run Python test code

```
cd $CUSPATIAL_HOME/python/cuspatial
python setup.py build_ext --inplace
python setup.py install
```

5. Run python test code <br>

First, add the cuSpatial Python API path to `PYTHONPATH` (there are tools under
tests subdir): `export PYTHONPATH=$CUSPATIAL_HOME/python/cuspatial`

Some tests using toy data can be run directly, e.g.,

```
python $CUSPATIAL_HOME/python/cuspatial/cuspatial/tests/pip2_test_soa_toy.py
```

However, many test code uses real data from an ITS (Intelligent Transportation
System) application. You will need to follow instructions at
[data/README.md](./data/README.md) to generate data for these test code.
Alternatively, you can download the preprocessed data ("locust.*",
"its_4326_roi.*", "itsroi.ply" and "its_camera_2.csv") from
[here](https://nvidia-my.sharepoint.com/:u:/p/jiantingz/EdHR7qlaRSVPtw46XYVR9sQBjCcnUHygCuPUC3Hf8gW73A?e=LCr9nK).
Extract the files and put them directly under $CUSPATIAL_HOME/data for quick
demos. A brief description of these data files and their semantic roles in the
ITS application can be found [here](doc/itsdata.md) TODO THIS IS MISSING

After data are downloaded and/or pre-processed, you can run the
[python test code](python/cuspatial/cuspatial/tests):

```
python $CUSPATIAL_HOME/python/cuspatial/cuspatial/tests/pip2_verify.py
python $CUSPATIAL_HOME/python/cuspatial/cuspatial/tests/traj2_test_soa3.py
python $CUSPATIAL_HOME/python/cuspatial/cuspatial/tests/stq_test_soa1.py
```

**NOTE:** Currently, cuSpatial supports reading point/polyine/polygon data using
Structure of Array (SoA) format (more readers are being developed).
Alternatively, python users can read any point/polyine/polygon data using
existing python packages, e.g., [Shapely](https://pypi.org/project/Shapely/),
to generate numpy arrays and feed them to
[cuSpatial python APIs](python/cuspatial/cuspatial/bindings).
Loading