TOOLS: Flyscrape companions

[dynabler]

Below is a list of tools you can use in companion of Flyscrape. Got tools you use in companion of Flyscrape? Please comment!

ScrapeMate (Firefox Extension)

Instead of switching back and forth between your text editor and browser to copy & paste CSS selectors, you can use ScrapeMate extension, available for Firefox. ScrapeMate allows you to select all the selectors you need, copy & paste it in your script.js, eliminating the need to switch back and forth.

ScripMate GitHub
ScriptMate FireFox

./jq (Filter)

jq is a command-line JSON processor akin to sed, awk, grep, etc. for JSON data. It's allowing you to easily slice, filter, map, and transform structured data.

jq does nothing on its own, it cannot open files nor save files. It's an actual filter. How it works is as following:

cat script.json | jq '.[] | map({"description": .description})' > script-desc-only.json

Notice it being in-between commands that do work on their own.

TIP: make sure to get it right the first time with Flyscrape, as jq has a steep learning curve.