Extract data reformat logic into a separate file

[difince]

Extract data reformat logic into a separate file - github_data.py

Available commands:

python ml-conversational-analytic-tool/featureVector.py  <raw_data_file> --words <words_file> --name <output_file_name>
python ml-conversational-analytic-tool/github_data.py  <raw_data_file> --name <output_file_name>
python ml-conversational-analytic-tool/commentAnalysis.py "Comment Message - Merry Christmas" --words <words_file>

--name is an optional parameter (supported by github_data.py and featureFector.py)
--words is an optional parameter (supported by featureVector.py and commentAnalysis.py)

Fixes: #4

Signed-off-by: Diana Atanasova [email protected]

[annajung]

thanks Diana, left a few comments - PTAL

[ericwb]

Can you split this PR into multiple? I see a lot of code refactoring mixed with other changes. Very large set of changes to review.

[annajung]

@ericwb if you filter out html files, it's easier to see the changes :)
I do agree that autogenerated docs are not helpful in reviewing PRs. We should look at how we can improve this process.

[tzstoyanov]

Any autogenerated files should be in a second commit in the PR.

[difince]

@ericwb, @tzstoyanov @annajung I have reverted all auto-generated HTML files, so now the PR contains 5 files.
I will later create an additional PR to add all HTML files

[tzstoyanov]

I will later create an additional PR to add all HTML files

I think it is better to have all related changes in the same PR, just add a second commit with the doc updates.

[difince]

@tzstoyanov I agree with you that it is better to have all related changes in the same PR, having it as a separate commit is a good approach, and less error prone. I just realized that code reviews could be done for a specific commit as well. So, no need to filter out *.html files.
But IMO it would be a good practice, all additional commits created to address review comments - in the end, to be squashed to the initial commit and thus keep the log history cleaner. But if we want to keep a commit (HTML related) separate from the squash, the reviewer won't be able to use GITHub "squash and merge" action. The squash need to be done by the PR author
WDYT?

[tzstoyanov]

But IMO it would be a good practice, all additional commits created to address review comments - in the end, to be squashed to the initial commit and thus keep the log history cleaner.

I think commits should not be squashed, as each commit is a separate logical change in the code. Keeping commits simplifies finding problems using git blame ... or git bisect ..., but may be I'm biased as this is the approach chosen in the kernel development workflow :)

[tzstoyanov]

lgtm

[difince]

@tzstoyanov sometime as a follow-up, we commit just a typo fix that just makes a noise I think. @annajung suggested that we could elaborate GitHub Actions for HTML auto-generated code.

[annajung]

thank you! lgtm

[ericwb]

Looks like some typos on filenames

[ericwb]

LGTM