Skip to content

A Python scraper for real-time updates on upcoming space launches.

License

Notifications You must be signed in to change notification settings

Tanguy9862/Next-Launch-Scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

23 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

NextSpaceFlight Next Launch Scraper

NSF Next Launch Scraper is a Python module designed to scrape and export the latest upcoming space launch data from Next Spaceflight. It allows you to customize the environment (local, AWS, or GCP) and configure storage locations as needed, making it flexible for various use cases.


Features

  • Scrapes detailed information about the next space launch:
    • Date, Organization, Rocket, Mission Details, and more.
  • Supports local export (JSON file) and AWS S3, GCP Cloud Storage integration.
  • Easily configurable via environment variables (ENV).

Installation

Clone the repository and install the package:

git clone git+https://github.com/Tanguy9862/Next-Launch-Scraper.git
pip install -r requirements.txt

Usage

1. Setup Configuration

Create a .env file in the directory where you’ll run the scraper. Specify the environment:

  • ENV=local (default): Export to a local JSON file.
  • ENV=aws: Export to an S3 bucket (requires proper IAM permissions).

Example .env:

ENV=local

2. Run the Scraper

Import and call the main function:

from next_launch_scraper.scraper import scrape_next_launch_data

scrape_next_launch_data()
  • Local Mode: Exports data to a data/ folder in the current directory.
  • AWS/GCP Mode: Uploads the data to your specified S3/Cloud Storage bucket (requires IAM setup).

3. Example Integration

This scraper can be seamlessly integrated into pipelines. See Space-App for a practical example:

  • A Lambda function calls this scraper to update data in an S3 bucket.
  • The Space-App consumes the data for visualization.

AWS Integration

If using ENV=aws, ensure:

  1. Your AWS credentials are configured in your environment or via .aws/credentials.
  2. The Lambda function or local user has appropriate permissions:
    • s3:PutObject
    • s3:GetObject

About

A Python scraper for real-time updates on upcoming space launches.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages