Merge branch 'develop'

AppCivico · May 5, 2021 · b6d8f70 · b6d8f70
2 parents dc990d9 + 39bb4a0
commit b6d8f70
Show file tree

Hide file tree

Showing 17 changed files with 1,007 additions and 491 deletions.
diff --git a/README.md b/README.md
@@ -26,127 +26,23 @@ TWITTER_CONSUMER_KEY="Your application consumer key"
 TWITTER_CONSUMER_SECRET="Your application consumer secret"
 TWITTER_ACCESS_TOKEN_KEY="Your application access token key, only for user authentication"
 TWITTER_ACCESS_TOKEN_SECRET="Your application access token secret, only for user authentication"
-```
-
-*Both User and App-only authentication are supported, for App-only, the Bearer token will be automatically requested*
-
-#### Start
-
-`npm start username`
-
-*or*
-
-`source/cli.js username`
-
-*`username` has to be replaced by the profile to analyze*
-
-#### Install bin locally on your system
-
-`npm link` *sudo might be necessary*
-
-*Then*
-
-`spottingbot username`
-
-### Module
-
-#### Call
-
-```js
-const spottingbot = require('spottingbot');
-
-spottingbot(username, index);
-```
-
-`username` is a string that contains the screen name of the Twitter profile to analyze.
-
-`twitter_config` is an object that contains Twitter credentials, both User and App-only authentication are supported, for App-only, the Bearer token will be automatically requested, the `twitter_config` object should be like this:
-
-```js
-{
-  consumer_key: "Your application consumer key",
-  consumer_secret: "Your application consumer secret",
-  access_token_key: "Your application access token key", // Only for User authentication
-  access_token_secret: "Your application access token secret" // Only for User authentication
-}
-```
-
-`index` is used to disable some indexes, it's an object that looks like this:
-```js
-{
-  user: true,
-  friend: true,
-  temporal: true,
-  network: true
-}
-```
-
-By default, or if omitted, everything is `true`.
-
-To disable only one index, it isn't necessary to add the others keys in the object, `{friend: false}`, will work.
-
-#### Return value
 
-*spottingbot* handles both *callback* style and *node promise* style
+PORT="The port that the app will attach itself"
 
-##### Callback
+USE_CACHE="Flag that will control the cache usage. Must be either 0 or 1.
+DEFAULT_CACHE_INTERVAL='The interval that the cache will be considered valid. Must be a valid Postgresql format, e.g: '30_days'."
 
-```js
-spottingbot(username, twitter_config, index, function(error, result) {
-  if (error) {
-    // Handle error
-    return;
-  }
-  // Do something with result
-})
-```
-
-##### Promise
-
-```js
-spottingbot(username, twitter_config, index)
-  .then(result => {
-    // Do something with result
-  })
-  .catch(error => {
-    // Handle error
-  })
-```
+DATABASE_HOST="Your Postgresql host"
+DATABASE_USER="Your database user"
+DATABASE_PASSWORD="Your database password"
+DATABASE_NAME="Your database password"
 
-##### Value
+BULK_API_KEY='Key that will be used to authorize calls to the bulk endpoint'
 
-The return value is an object that contains
+PUPPETER_SERVICE_ROOT_URL="Url to the service that will take screenshots from the Twitter profile"
+PUPPETER_SECRET_TOKEN="Token for the screenshot service"
 
-```js
-{
-  metadata: {
-    count: 1 // Always 1 for now
-  },
-  profiles: [
-     {
-       username: 'screen_name',
-       url: 'https://twitter.com/screen_name',
-       avatar: 'image link',
-       language_dependent: {
-         sentiment: {
-           value: 0.65
-         }
-       },
-       language_independent: {
-         friend: 0.19,
-         temporal: 0.37,
-         network: 0.95,
-         user: 0
-       },
-       bot_probability: {
-         all: 0.37
-       },
-       user_profile_language: 'en',
-     }
-  ]
-}
 ```
-
 ##### PSQL database
 
 Pegabot uses a cache system to avoid remaking an analysis on the same user.
@@ -168,6 +64,8 @@ DATABASE_NAME="pegabot"
 4. Install the npm module `sequelize-cli`
 5. Run migrations with `sequelize-cli db:migrate`
 
+## Starting the API
+You can run the command: `npm run dev` in order to start with nodemon and Babel. Or you can start with `npm run start`, which will build the application and run from a `/build` folder without Babel.
 
 ## Endpoints
 
@@ -176,14 +74,17 @@ Get analysis with only main indexes.
 
 **Method**: `GET`
 
-**Example**: `/botometer?profile=twitter&search_for=profile`
+**Example**: `curl --request GET --url 'http://127.0.0.1:5010/botometer?profile=twitter&search_for=profile&is_admin=true'`
 
 
 **Required parameters**
 - `profile` (STRING): user handle that will be analyzed;
 
 - `search_for` (STRING): only one value allowed `profile`
 
+**Optional parameters**
+- `is_admin` (BOOLEAN): used to identify requests that are coming from the Pegabot admin interface; (1|0 OR true|false)
+
 **Response**
 ```
 {
@@ -221,7 +122,7 @@ Get analysis with main indexes and subindexes.
 
 **Method**: `GET`
 
-**Example**: `/analyze?profile=twitter&search_for=profile`
+**Example**: `curl --request GET --url 'http://127.0.0.1:5010/analyze?profile=twitter&search_for=profile'`
 
 
 **Required parameters**
@@ -293,23 +194,36 @@ Get analysis with main indexes and subindexes.
 }
 ```
 
-### `/botometer-buld`
+### `/botometer-bulk`
 Bulk process for first endpoint.
 **The params for this request must be sent as JSON**
 
 **Method**: `GET`
-**Example**: `/botometer-bulk`
+
+**Example**:
+```
+curl --request GET \
+  --url http://127.0.0.1:5010/botometer-bulk \
+  --header 'Content-Type: application/json' \
+  --header 'X-API-KEY: $ENV{BULK_API_KEY}' \
+  --data '{
+	"is_admin": true,
+	"profiles": [
+		"twitter"
+	] 
+}'
+```
 
 **Required headers**
-- apiKey (string): Must be set on the `.env` file, with the `BULK_API_KEY` key;
+- `apiKey` (string): Must be set on the `.env` file, with the `BULK_API_KEY` name;
 
 **Required parameters**
 - `profiles` (STRING ARRAY): Array containing the profiles that will be analyzed.
   - **MAX ARRAY SIZE**: 50
 
 **Optional parameters**
 
-- `is_admin` (BOOLEAN): used to identify requests that are coming from the Pegabot admin interface;
+- `is_admin` (BOOLEAN): used to identify requests that are coming from the Pegabot admin interface; (1|0 OR true|false)
 - `twitter_api_consumer_key` (STRING): Consumer key that will be used to instantiate a Twitter API client;
   - **Must be sent alongside `twitter_api_consumer_secret`**
 - `twitter_api_consumer_secret` (STRING): Consumer key that will be used to instantiate a Twitter API client;
@@ -351,6 +265,128 @@ Bulk process for first endpoint.
 }
 ```
 
+
+## Using as a module
+*Both User and App-only authentication are supported, for App-only, the Bearer token will be automatically requested*
+#### Start
+
+`npm start username`
+
+*or*
+
+`source/cli.js username`
+
+*`username` has to be replaced by the profile to analyze*
+
+
+#### Install bin locally on your system
+
+`npm link` *sudo might be necessary*
+
+*Then*
+
+`spottingbot username`
+
+### Module
+
+#### Call
+
+```js
+const spottingbot = require('spottingbot');
+
+spottingbot(username, index);
+```
+
+`username` is a string that contains the screen name of the Twitter profile to analyze.
+
+`twitter_config` is an object that contains Twitter credentials, both User and App-only authentication are supported, for App-only, the Bearer token will be automatically requested, the `twitter_config` object should be like this:
+
+```js
+{
+  consumer_key: "Your application consumer key",
+  consumer_secret: "Your application consumer secret",
+  access_token_key: "Your application access token key", // Only for User authentication
+  access_token_secret: "Your application access token secret" // Only for User authentication
+}
+```
+
+`index` is used to disable some indexes, it's an object that looks like this:
+```js
+{
+  user: true,
+  friend: true,
+  temporal: true,
+  network: true
+}
+```
+
+By default, or if omitted, everything is `true`.
+
+To disable only one index, it isn't necessary to add the others keys in the object, `{friend: false}`, will work.
+
+#### Return value
+
+*spottingbot* handles both *callback* style and *node promise* style
+
+##### Callback
+
+```js
+spottingbot(username, twitter_config, index, function(error, result) {
+  if (error) {
+    // Handle error
+    return;
+  }
+  // Do something with result
+})
+```
+
+##### Promise
+
+```js
+spottingbot(username, twitter_config, index)
+  .then(result => {
+    // Do something with result
+  })
+  .catch(error => {
+    // Handle error
+  })
+```
+
+##### Value
+
+The return value is an object that contains
+
+```js
+{
+  metadata: {
+    count: 1 // Always 1 for now
+  },
+  profiles: [
+     {
+       username: 'screen_name',
+       url: 'https://twitter.com/screen_name',
+       avatar: 'image link',
+       language_dependent: {
+         sentiment: {
+           value: 0.65
+         }
+       },
+       language_independent: {
+         friend: 0.19,
+         temporal: 0.37,
+         network: 0.95,
+         user: 0
+       },
+       bot_probability: {
+         all: 0.37
+       },
+       user_profile_language: 'en',
+     }
+  ]
+}
+```
+
+
 **spottingbot is a project inspired by [Botometer](https://botometer.iuni.iu.edu/#!/), an [OSoMe](https://osome.iuni.iu.edu/) project.**
 
 **This project is part of the [PegaBot](http://www.pegabot.com.br) initiative.**

diff --git a/example.env b/example.env
@@ -23,5 +23,7 @@ DATABASE_USER="postgres"
 DATABASE_PASSWORD=""
 DATABASE_NAME="pegabot"
 
+BULK_API_KEY=''
+
 PUPPETER_SERVICE_ROOT_URL=""
 PUPPETER_SECRET_TOKEN=""