Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Grabing posts from Pleroma fails #45

Open
Eskuero opened this issue Aug 15, 2020 · 1 comment
Open

Grabing posts from Pleroma fails #45

Eskuero opened this issue Aug 15, 2020 · 1 comment

Comments

@Eskuero
Copy link

Eskuero commented Aug 15, 2020

  1. Clean run without any toots.db generated yet
  2. Bot account is following this accounts (raru.me are mastodon and the other two are pleroma instances)
  3. Running the bot:

Using config.json as configuration file
Downloading posts for user @[email protected], starting from 0
WebFingering...
Using standard mode
Downloading and saving posts............................................ Done!
Downloading posts for user @[email protected], starting from 0
WebFingering...
Using standard mode
Downloading and saving posts................................. Done!
Downloading posts for user @[email protected], starting from 0
WebFingering...
Using standard mode
Downloading and saving posts........... Done!
Downloading posts for user @[email protected], starting from 0
WebFingering...
Using standard mode
Traceback (most recent call last):
File "/app/main.py", line 215, in
j = r.json()
File "/usr/lib/python3.8/site-packages/requests/models.py", line 897, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/lib/python3.8/json/init.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

  1. As it can be seen on the fedi.absturztau.me fails to grab posts.
  2. Re-run the sync:

Using config.json as configuration file
Downloading posts for user @[email protected], starting from 40dd49ac-3b6e-4ea6-9f99-7415a57bed3c
WebFingering...
Using standard mode
Traceback (most recent call last):
File "/app/main.py", line 215, in
j = r.json()
File "/usr/lib/python3.8/site-packages/requests/models.py", line 897, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/lib/python3.8/json/init.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

  1. And now the other pleroma instance (kitsune.cafe) starts failing too. I tried waiting for half an hour in case it was a rate-limit but it still kept failing
@w3bb
Copy link

w3bb commented Jun 14, 2021

For anyone reading this issue, I've made an alternative scraper that isn't broken with Pleroma. It should work perfectly with your existing toots.db. Although make sure to do backups.

https://git.sr.ht/~welt/fuckpleroma

Install Go, and run go get -u spiderden.net/go/fuckpleroma

The binary should end up in your $GOPATH, which by default is ~/go/bin.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants