[tables] Add flag to remove duplicate header rows #89
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
paulopaixaoamaral
force-pushed
the
remove-duplicate-header-rows
branch
from
6247868 to
d75001f
Compare
jstockwin
requested changes
May 22, 2020
There was a problem hiding this comment.
Thanks @paulopaixaoamaral - looks generally good to me.
I've added two suggestions. Also:
- A test where the element text (but not the font) matches, and also one where the font (but not the text) matches might be good, just to ensure these elements don't get removed. (I think I could e.g. remove font comparison from your function and still have tests pass).
- However, maybe it's cleaner to pull your inner function outside, since then you can test that in isolation?
| @@ -6,6 +6,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 | |||
|
|
|||
| ## [Unreleased] | |||
| ### Changed | |||
| - Added flag to `extract_simple_table` and `extract_table` functions to remove duplicate header rows. ([#89](https://github.com/jstockwin/py-pdf-parser/pull/89)) | |||
There was a problem hiding this comment.
I think this should probably be in a section called ### Added
py_pdf_parser/tables.py
Outdated
Comment on lines
468
to
471
| if (elem_1 is None or elem_2 is None) or ( | ||
| elem_2 is None and elem_1 is not None | ||
| ): | ||
| return False |
There was a problem hiding this comment.
Given the above check, we know they're not BOTH None, so I think this can be simplified to if elem1 is None or elem2 is None?
There was a problem hiding this comment.
Yeah I did that originally, but I thought it would be better to be explicit for readability purposes. But I guess you're right, it will be readable enough, I am going to change it 👍
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This will add a flag to the table extraction functions to enable the removal of rows which are duplicats of the header row.
Linked issues
Closes #76
Testing
Tests have been added to the table extraction tests to make sure the duplicate header rows are removed.
Checklist
docker-compose run --rm lint)CHANGELOG.md, following the format fromKeep a Changelog.