added module support 🔥

Author: achyuthjoism

README.md

@@ -1,54 +1,120 @@
-<h1>Tweeds - Twitter Scraping Tool</h1>
-<p align="center"><img src="https://media.tenor.com/Xrt-ty39PfEAAAAC/elon-musk-smoke.gif"/></p>
+# Tweeds - Twitter Scraping Tool
 
-Scrape tweets from any Twitter user profile. Twitter API alternative to scrape Twitter hashtags, threads, images, videos, statistics, 
-and Twitter history. Export data in JSON and CSV format. This project enables you to extract large amounts of data from Twitter. 
-It lets you do much more than the Twitter API, because it doesn't have rate limits and you don't even need to have a **Twitter account, a registered app, 
+<p align="center">
+<img src="https://media.tenor.com/Xrt-ty39PfEAAAAC/elon-musk-smoke.gif"/>
+</p>
+
+Scrape tweets from any Twitter user profile. Twitter API alternative to scrape Twitter hashtags, threads, images, videos, statistics,
+and Twitter history. Export data in JSON and CSV format. This project enables you to extract large amounts of data from Twitter.
+It lets you do much more than the Twitter API, because it doesn't have rate limits and you don't even need to have a **Twitter account, a registered app,
 or Twitter API key.**
 
 ### Do not forget to star this project.🌟 😍
 
 ## 💡 Features
-- No API Key required 
+
+- No API Key required
 - No Limit
 - No Authentication required
 - Get tweets from specific user
 - JSON and CSV export for further Data Analysis
 - And the best part it is OpenSource 😉
 
 ## ✔️ Requirements
+
 - Python >= 3.9
 
-## ⚙ Installation 
+## ⚙ Installation
+
 ```bash
 pip install tweeds
 ```
 
 ## 💃 Usage
-![Basic](https://files.catbox.moe/ph7txh.png)
 
-## 📙 Example
-Getting tweets from a specific user
-```Bash
-tweeds -u TheNameisYash --json yashtweets.json
-```
-Getting tweets from a search term or #hastag
 ```Bash
-tweeds -s "#Messi" --json messigoat.json --minLikes 10000 --limit 100
-```
-Getting tweets from a specific region (.ie Paris)
-```Bash
-tweeds -s "coffee sucks" --near Paris --limit 100
+achyuthjoism-MBP ~ %tweeds
+
+        ████████╗░██╗░░░░░░░██╗███████╗███████╗██████╗░░██████╗
+        ╚══██╔══╝░██║░░██╗░░██║██╔════╝██╔════╝██╔══██╗██╔════╝
+        ░░░██║░░░░╚██╗████╗██╔╝█████╗░░█████╗░░██║░░██║╚█████╗░
+        ░░░██║░░░░░████╔═████║░██╔══╝░░██╔══╝░░██║░░██║░╚═══██╗
+        ░░░██║░░░░░╚██╔╝░╚██╔╝░███████╗███████╗██████╔╝██████╔╝
+        ░░░╚═╝░░░░░░╚═╝░░░╚═╝░░╚══════╝╚══════╝╚═════╝░╚═════╝░
+
+        by Achyuth Jois M
+
+usage: tweeds [-h] [-u U] [-s S] [--since SINCE] [--until UNTIL] [--limit LIMIT] [--near NEAR] [--geocode GEOCODE]
+              [--year YEAR] [--today] [--verified] [--link LINK] [--videos] [--images] [--media] [--minLikes MINLIKES]
+              [--minRetweets MINRETWEETS] [--minReplies MINREPLIES] [--json JSON] [--csv CSV] [--silent]
+
+Scrape twitter user tweets
+
+options:
+  -h, --help            show this help message and exit
+  -u U                  Users Tweets you want to scrape.
+  -s S                  Search for Tweets containing this word or phrase.
+  --since SINCE         Filter Tweets sent since date (Example: "2017-12-27 20:30:15" or 2017-12-27).
+  --until UNTIL         Filter Tweets sent until date (Example: "2017-12-27 20:30:15" or 2017-12-27).
+  --limit LIMIT         Number of Tweets to pull
+  --near NEAR           Find tweets near a particular location
+  --geocode GEOCODE     Search for geocoded Tweets.
+  --year YEAR           Filter Tweets before specified year.
+  --today               Filter Tweets from today
+  --verified            Display Tweets only from verified users (Use with -s).
+  --links LINK          Exclude tweets containing one or more links.
+  --videos              Display only Tweets with videos.
+  --images              Display only Tweets with images.
+  --media               Display Tweets with only images or videos.
+  --minLikes MINLIKES   Minimun likes for the tweet
+  --minRetweets MINRETWEETS
+                        Minimun retweets for the tweet
+  --minReplies MINREPLIES
+                        Minimun replies for the tweet
+  --json JSON           File to write the JSON output to.
+  --csv CSV             To store the output in CSV
+  --silent              Dont print the tweets(Only works while taking an output!)[Type anything]
+
+achyuthjoism-MBP ~ %
+
 ```
-A some more simple examples to help you understand the basics:
 
-- `tweeds -u username` - Scrape all the Tweets of a *user* (doesn't include **retweets** but includes **replies**).
-- `tweeds -u username -s pineapple` - Scrape all Tweets from the *user*'s timeline containing _pineapple_.
-- `tweeds -s pineapple` - Collect every Tweet containing *pineapple* from everyone's Tweets.
+## 📙 Example
+
+### CLI Example
+
+Some simple examples to help you understand the basics:
+
+- `tweeds -u username` - Scrape all the Tweets of a _user_ (doesn't include **retweets** but includes **replies**).
+- `tweeds -u username -s pineapple` - Scrape all Tweets from the _user_'s timeline containing _pineapple_.
+- `tweeds -s pineapple` - Collect every Tweet containing _pineapple_ from everyone's Tweets.
 - `tweeds -u username --since "2015-12-20 20:30:15"` - Collect Tweets that were tweeted since 2015-12-20 20:30:15.
 - `tweeds -u username --since 2015-12-20` - Collect Tweets that were tweeted since 2015-12-20 00:00:00.
-- `tweeds -u username --csv file.csv` - Scrape Tweets and save as a csv file.
+- `tweeds -s "Rocking Star Yash" --verified` - Display Tweets by verified users that Tweeted about Rocking Star Yash.
+- `tweeds -geocode "48.880048,2.385939,1km" --csv paris.csv --limit 10` - Scrape Tweets from a radius of 1km around a place in Paris and export them to a csv file.
+- `tweeds -u username --images` - Scrape Tweets from a user containing only images.
+- `tweeds -u username --videos` - Scrape Tweets from a user containing only videos.
+- `tweeds -u username --media` - Scarape Tweets from a user containing both images and videos.
+- `tweeds -u username --links` - Scrape Tweets from a user which excludes links.
 - `tweeds -u username --json file.json` - Scrape Tweets and save as a json file.
+- `tweeds -u username --csv file.csv` - Scrape Tweets and save as a csv file.
+
+### Module Example
+
+Now Tweed can be used as a module and allows custom formatting.
+Example:
+
+```python
+import tweeds
+
+query = tweeds.Query()
+
+query.search = "Yash Boss"
+query.limit = 10
+query.verified = True
+
+tweeds.search(query)
+```
 
 **Have fun 🥰💞**
 

tweeds/__init__.py

@@ -0,0 +1,2 @@
+from tweeds.query import Query
+from tweeds.tweet_scrape import search

tweeds/classes/__init__.py

@@ -1,13 +1,17 @@
 import argparse as ap
 import sys
-import tweeds.classes.query as query
-import tweeds.tweet_scrape as ts
+from tweeds.query import Query
+from tweeds.tweet_scrape import search
 
 
 def banner():
     print("""	
-	▀█▀ █░█░█ █ ▀█▀ ▀█▀ █▀▀ █▀█   █▀█ █▀ █ █▄░█ ▀█▀
-	░█░ ▀▄▀▄▀ █ ░█░ ░█░ ██▄ █▀▄   █▄█ ▄█ █ █░▀█ ░█░
+        ████████╗░██╗░░░░░░░██╗███████╗███████╗██████╗░░██████╗
+        ╚══██╔══╝░██║░░██╗░░██║██╔════╝██╔════╝██╔══██╗██╔════╝
+        ░░░██║░░░░╚██╗████╗██╔╝█████╗░░█████╗░░██║░░██║╚█████╗░
+        ░░░██║░░░░░████╔═████║░██╔══╝░░██╔══╝░░██║░░██║░╚═══██╗
+        ░░░██║░░░░░╚██╔╝░╚██╔╝░███████╗███████╗██████╔╝██████╔╝
+        ░░░╚═╝░░░░░░╚═╝░░░╚═╝░░╚══════╝╚══════╝╚═════╝░╚═════╝░
 
         by Achyuth Jois M
     """)
@@ -17,7 +21,7 @@ def banner():
 
 
 def config(args: ap.Namespace):
-    c = query.Query()
+    c = Query()
     c.search = args.s
     c.username = args.u
     c.limit = args.limit
@@ -30,13 +34,20 @@ def config(args: ap.Namespace):
     c.minReplies = args.minReplies
     c.minRetweets = args.minRetweets
     c.silent = args.silent
+    c.verified = args.verified
+    c.geoCode = args.geocode
+    c.links = args.links
+    c.videos = args.videos
+    c.images = args.images
+    c.media = args.media
+    c.year = args.year
 
     return c
 
 
 def process_args(args: ap.Namespace):
     q = config(args)
-    ts.search(q)
+    search(q)
 
 
 def main():
@@ -57,6 +68,31 @@ def main():
     parse.add_argument(
         '--near', type=str, help="Find tweets near a particular location"
     )
+    parse.add_argument(
+        '--geocode', type=str, help="Search for geocoded Tweets."
+    )
+    parse.add_argument(
+        '--year', type=int, help="Filter Tweets before specified year."
+    )
+    parse.add_argument(
+        '--today', help="Filter Tweets from today", action='store_true'
+    )
+    parse.add_argument(
+        '--verified', help="Display Tweets only from verified users (Use with -s).", action='store_true'
+    )
+    parse.add_argument(
+        '--links', help="Exclude tweets containing one or more links.", action='store_true'
+    )
+    parse.add_argument(
+        "--videos", help="Display only Tweets with videos.", action='store_true'
+    )
+    parse.add_argument(
+        "--images", help="Display only Tweets with images.", action='store_true'
+    )
+    parse.add_argument(
+        "--media", help="Display Tweets with only images or videos.", action='store_true'
+    )
+
     parse.add_argument(
         '--minLikes', type=int, help="Minimun likes for the tweet"
     )
@@ -72,7 +108,7 @@ def main():
         '--csv', type=str, help="To store the output in CSV"
     )
     parse.add_argument(
-        '--silent', type=str, help="Don't print the tweets(Only works while taking an output!)[Type anything]"
+        '--silent', help="Don't print the tweets(Only works while taking an output!)[Type anything]", action='store_true'
     )
 
     args = parse.parse_args(args=None if sys.argv[1:] else ['--help'])

tweeds/classes/query.py

@@ -0,0 +1,32 @@
+import dataclasses
+from typing import Optional
+
+
+@dataclasses.dataclass
+class Query:
+    """
+        Class used to query in twitter
+    """
+    username: Optional[str] = None
+    search: Optional[str] = None
+    since: Optional[str] = None
+    until: Optional[str] = None
+    json: Optional[str] = None
+    limit: Optional[int] = None
+    near: Optional[str] = None
+    minLikes: Optional[int] = None
+    csv: Optional[str] = None
+    minReplies: Optional[int] = None
+    minRetweets: Optional[int] = None
+    silent: bool = False
+    verified: bool = False
+    geoCode: Optional[str] = None
+    links: bool = False
+    videos: bool = False
+    images: bool = False
+    media: bool = False
+    year: Optional[int] = None
+    today: bool = False
+
+    def __init__(self) -> None:
+        pass

tweeds/main.py

@@ -1,7 +1,8 @@
 import snscrape.modules.twitter as api
 import pandas as pd
 import json
-import tweeds.classes.query as query
+from tweeds.query import Query
+from datetime import date
 
 
 def make_json(data, jsonFilePath):
@@ -32,13 +33,19 @@ def toOBJ(tweet: api.Tweet) -> object:
     }
 
 
-def search(q: query.Query) -> None:
+def search(q: Query) -> None:
+    """Print tweets"""
     query = ""
 
     if q.search:
         query += f"{q.search} "
     if q.username:
         query += f"(from:{q.username}) "
+    if q.today:
+        today = date.today().strftime("%Y/%m/%d")
+        query += f"until:{today}"
+    if q.year:
+        query += f"until:{q.year}-01-01 "
     if q.until:
         query += f"until:{q.until} "
     if q.since:
@@ -51,7 +58,18 @@ def search(q: query.Query) -> None:
         query += f" min_retweets:{q.minRetweets} "
     if q.near:
         query += f"near:{q.near} "
-
+    if q.geoCode:
+        query += f"geocode:{q.geoCode} "
+    if q.verified:
+        query += f"filter:verified "
+    if q.media:
+        query += f"filter:media "
+    if q.videos and not q.media and not q.images:
+        query += f"filter:native_video "
+    if q.images and not q.media and not q.videos:
+        query += f"filter:images "
+    if q.links:
+        query += "-filter:links "
     jsonObj = {}
     csvObj = []