Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Support AsciiDoc and Markdown input format #168

Merged
merged 36 commits into from
Oct 23, 2024
Merged

feat: Support AsciiDoc and Markdown input format #168

merged 36 commits into from
Oct 23, 2024

Conversation

cau-git
Copy link
Contributor

@cau-git cau-git commented Oct 22, 2024 •
edited
Loading

Checklist:

  • Commit Message Formatting: Commit titles and messages follow guidelines in the
    conventional commits.
  • Documentation has been updated, if necessary.
  • Examples have been added, if necessary.
  • Tests have been added, if necessary.

PeterStaar-IBM and others added 27 commits October 17, 2024 19:58
@PeterStaar-IBM
updated the base-model and added the asciidoc_backend
1203353 
Signed-off-by: Peter Staar <[email protected]>
@PeterStaar-IBM
updated the asciidoc backend
c1d9241 
Signed-off-by: Peter Staar <[email protected]>
@cau-git @PeterStaar-IBM
Ensure all models work only on valid pages (#158)
63d3704 
Signed-off-by: Christoph Auer <[email protected]>
@dolfim-ibm @PeterStaar-IBM
ci: run ci also on forks (#160)
77fa1db 
---------

Signed-off-by: Michele Dolfi <[email protected]>
Signed-off-by: Michele Dolfi <[email protected]>
@vagenas @PeterStaar-IBM
fix: fix legacy doc ref (#162)
eb154a1 
Signed-off-by: Panos Vagenas <[email protected]>
@fadkeabhi @dolfim-ibm @PeterStaar-IBM
docs: typo fix (#155)
b6c0610 
* Docs: Typo fix

- Corrected spelling of invidual to automatic

Signed-off-by: ABHISHEK FADAKE <[email protected]>

* add synchronize event for forks

Signed-off-by: Michele Dolfi <[email protected]>

---------

Signed-off-by: ABHISHEK FADAKE <[email protected]>
Signed-off-by: Michele Dolfi <[email protected]>
Co-authored-by: Michele Dolfi <[email protected]>
@dolfim-ibm @PeterStaar-IBM
feat: add coverage_threshold to skip OCR for small images (#161)
006cfb4 
* feat: add coverage_threshold to skip OCR for small images

Signed-off-by: Michele Dolfi <[email protected]>

* filter individual boxes

Signed-off-by: Michele Dolfi <[email protected]>

* rename option

Signed-off-by: Michele Dolfi <[email protected]>

---------

Signed-off-by: Michele Dolfi <[email protected]>
@github-actions @PeterStaar-IBM
chore: bump version to 2.1.0 [skip ci]
c60c402
@PeterStaar-IBM
adding tests for asciidocs
1138cae 
Signed-off-by: Peter Staar <[email protected]>
@PeterStaar-IBM
first working asciidoc parser
5016dae 
Signed-off-by: Peter Staar <[email protected]>
@PeterStaar-IBM
reformatted the code
70b2ae3 
Signed-off-by: Peter Staar <[email protected]>
@PeterStaar-IBM
fixed the mypy
e60c525 
Signed-off-by: Peter Staar <[email protected]>
@PeterStaar-IBM
adding test_02.asciidoc
c23d049 
Signed-off-by: Peter Staar <[email protected]>
Drafting Markdown backend via Marko library
5986213 
Signed-off-by: Maksym Lysak <[email protected]>
work in progress on MD backend
1df89f7 
Signed-off-by: Maksym Lysak <[email protected]>
md_backend produces docling document with headers, paragraphs, lists
534b220 
Signed-off-by: Maksym Lysak <[email protected]>
Improvements in md parsing
bef429f 
Signed-off-by: Maksym Lysak <[email protected]>
Detecting and assembling tables in markdown in temporary buffers
fa2f8cf 
Signed-off-by: Maksym Lysak <[email protected]>
Added initial docling table support to md_backend
ba9beb6 
Signed-off-by: Maksym Lysak <[email protected]>
Cleaned code, improved logging for MD
dae3664 
Signed-off-by: Maksym Lysak <[email protected]>
Fixes MyPy requirements, and rest of pre-commit
1456a36 
Signed-off-by: Maksym Lysak <[email protected]>
Fixed example run_md, added origin info to md_backend
8c60dfa 
Signed-off-by: Maksym Lysak <[email protected]>
@PeterStaar-IBM
working on asciidocs, struggling with ImageRef
1c0a766 
Signed-off-by: Peter Staar <[email protected]>
@PeterStaar-IBM
able to parse the captions and image uri's
b04f14e 
Signed-off-by: Peter Staar <[email protected]>
@PeterStaar-IBM
fixed the mypy
bb3db07 
Signed-off-by: Peter Staar <[email protected]>
@cau-git
Merge branch 'dev/add-asciidocs-backend' of github.com:DS4SD/docling …
0bbd50f 
…into cau/backend-document-origin
@cau-git
Merge ASCIIDoc and Markdown backends in, fixes
789b29b 
Signed-off-by: Christoph Auer <[email protected]>
This was referenced Oct 22, 2024
Closed
Closed
@cau-git cau-git force-pushed the cau/backend-document-origin branch from 06c7a20 to b1a2af6 Compare October 22, 2024 12:11
cau-git and others added 8 commits October 22, 2024 14:34
@cau-git
Update to docling-core v2.1.0
578e30e 
Signed-off-by: Christoph Auer <[email protected]>
Fixes for MD Backend, to avoid duplicated text inserts into docling doc
47a4d31 
Signed-off-by: Maksym Lysak <[email protected]>
@cau-git
Fix styling
4fb803f 
Signed-off-by: Christoph Auer <[email protected]>
Added support for code blocks and fenced code in MD
186d71a 
Signed-off-by: Maksym Lysak <[email protected]>
cleaned prints
e8229fd 
Signed-off-by: Maksym Lysak <[email protected]>
Added proper processing of in-line textual elements for MD backend
0f81ffd 
Signed-off-by: Maksym Lysak <[email protected]>
Fixed issues with duplicated paragraphs and incorrect lists in pptx
82126e3 
Signed-off-by: Maksym Lysak <[email protected]>
Fixed issue with group ordeering in pptx backend, added gebug log int…
76d9041 
…o run with formats

Signed-off-by: Maksym Lysak <[email protected]>
@maxmnemonic maxmnemonic self-requested a review October 23, 2024 13:51
maxmnemonic
maxmnemonic approved these changes Oct 23, 2024
View reviewed changes
Copy link
Contributor

@maxmnemonic maxmnemonic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@cau-git cau-git marked this pull request as ready for review October 23, 2024 13:52
@cau-git cau-git merged commit 3023f18 into main Oct 23, 2024
7 checks passed
@cau-git cau-git deleted the cau/backend-document-origin branch October 23, 2024 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@maxmnemonic maxmnemonic maxmnemonic approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@cau-git @maxmnemonic @PeterStaar-IBM @dolfim-ibm @vagenas @fadkeabhi