Skip to content

Repository designed for scraping and automating the ingestion of oceanic drifter data. It automates the collection of data from multiple sources with different formats.

License

Notifications You must be signed in to change notification settings

NOAA-AOML/DrifterDataAutomation

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Current Phase: Phase 1 Overview This repository contains the initial development of automated data scraping tools designed for the DrifterDataAutomation project. The primary objective at this stage is to establish a robust framework for scraping data from manufacturer-specific spec sheets, which are pivotal for the accurate aggregation and analysis of oceanographic data equipment.

Current Progress We have initiated this project with the addition of two Python scripts: SIO_020.py and PG_102.py. These scripts are specifically tailored to scrape data from the Scripps Institute of Oceanography and Pacific Gyre, respectively. They serve as foundational components for our scraping toolkit. Comand to run: python SIO_020.py

Next Steps The immediate goal is to enhance these initial scrapers to refine their data extraction capabilities. Following this, the plan is to expand our suite of scrapers to cover all relevant manufacturer spec sheets.

A centralized script will be developed to orchestrate these scrapers, enabling the automated execution based on the specific manufacturer of the spec sheet in question. This approach will facilitate the simultaneous scraping from a multitude of spec sheets, significantly streamlining the data collection process.

Final Goal The ultimate aim for this phase is to automate the scraping of a wide array of manufacturer spec sheets through a single, unified command, thereby enhancing the efficiency and scalability of our data collection efforts in oceanographic research.

About

Repository designed for scraping and automating the ingestion of oceanic drifter data. It automates the collection of data from multiple sources with different formats.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%