bre vs por stopwords

[laygir]

Hello,

Could you confirm if bre is actually stopwords for Breton or is it mixed up with Portuguese (por)

https://github.com/fergiemcdowall/stopword/blob/main/src/stopwords_bre.js
https://github.com/fergiemcdowall/stopword/blob/main/src/stopwords_por.js

[eklem]

Hi, I'm not sure. It's taken from the stopwords-json repository. I checked the issues there, and found this comment:

The Breton list you have (br.json), is actually Brazilian Portuguese.

And I can see it removes just a few words from Breton text in the tests. So, then it seems we need to add another list or generate one? Any suggestions @laygir ?

[laygir]

Hey @eklem

Thank you for looking into it. The issue you mentioned confirms it.
Perhaps the words removed in the tests are just a coinciding intersection between the two language.

I'm no language expert here btw. but in this case since bre is actually Brazilian Portuguese, I'd do one or more of the following;

• Rename and document it as Brazilian Portuguese. Not sure if you strictly do language only or language-locale (pt-BR) is also acceptable.
• If above wouldn't work, then merge it with Portuguese after analyzing both lists and removing strictly pt-BR stopwords if any.
• Create a new Breton list with correct stopwords. I could only ask GPT to confirm the existing list and provide Breton stopwords. Here is the conversation.

[eklem]

The bre-code is added by this repository, but seems br was mistaken. It hasn't been fixed in the super-repository stopwords-iso/stopwords-iso, but is in the actual Breton stopword respository. I'll replace with that one. I won't do any merge on Brazilian Portugese and trusting ChatGPT with acutal output is maybe not so smart. But we have the generated output by Gene Diaz, so we're good to go. Thanks for opening the issue!

[eklem]

Published v3.1.4

• https://www.npmjs.com/package/stopword/v/3.1.4
• https://github.com/fergiemcdowall/stopword/releases/tag/v3.1.4