Code

Feature/sdo ml v2 loader pt2#21

Code

Merged

mariusgiger merged 47 commits intomaini4Ds/sdo-cli:mainfrom feature/sdo-ml-v2-loader-pt2i4Ds/sdo-cli:feature/sdo-ml-v2-loader-pt2Copy head branch name to clipboardJun 11, 2022

+7,882-495

Commits

Commits on May 10, 2022

• adds sdo ml v2 data loader

  mariusgiger

  committed
  8d7a0ab

  Copy full SHA for 8d7a0ab

Commits on May 24, 2022

• adds notebook to show how to use SDO ML v2 data loader, WIP temporal sampling, adds alternatives for storage system

  mariusgiger

  committed
  fcbfa1c

  Copy full SHA for fcbfa1c

• fixes

  mariusgiger

  committed
  5bb1764

  Copy full SHA for 5bb1764

• preparations for training

  mariusgiger

  committed
  33d2bd4

  Copy full SHA for 33d2bd4

• fixes SDOMLDatasetV2 not allowed

  mariusgiger

  committed
  40b423b

  Copy full SHA for 40b423b

• fixes invalid option

  mariusgiger

  committed
  554187e

  Copy full SHA for 554187e

• fixes invalid option

  mariusgiger

  committed
  9004c8c

  Copy full SHA for 9004c8c

• try disabling log graph

  mariusgiger

  committed
  9246749

  Copy full SHA for 9246749

• readds transforms

  mariusgiger

  committed
  31e0ec3

  Copy full SHA for 31e0ec3

Commits on May 27, 2022

• adds temporal splitting for train/validation sets

  mariusgiger

  committed
  54ccaf2

  Copy full SHA for 54ccaf2

• several imporvements and option to filter by date without temporal downsampling

  mariusgiger

  committed
  01d8aee

  Copy full SHA for 01d8aee

• removes code used for debugging

  mariusgiger

  committed
  2ae87ac

  Copy full SHA for 2ae87ac

• adds chunk sampler to improve data loading efficiency

  mariusgiger

  committed
  5ff897a

  Copy full SHA for 5ff897a

• Merge pull request #23 from i4Ds/feature/chunk-sampler

  Show description for 6907fed
  mariusgiger

  authored
  6907fed

  Copy full SHA for 6907fed

• prepares training

  mariusgiger

  committed
  c7abfd7

  Copy full SHA for c7abfd7

• adds prefetch factor

  mariusgiger

  committed
  5845526

  Copy full SHA for 5845526

• increases patience for early stopping

  mariusgiger

  committed
  b8ddf04

  Copy full SHA for b8ddf04

Commits on May 29, 2022

• refactors config handling to be file-based

  mariusgiger

  committed
  37ea73f

  Copy full SHA for 37ea73f

• implements caching, adds run config for FHNW infrastructure

  mariusgiger

  committed
  8e8bc1a

  Copy full SHA for 8e8bc1a

• adjusts fhnw run config

  mariusgiger

  committed
  4b04c22

  Copy full SHA for 4b04c22

• expands home directory, fixes issue with lr

  mariusgiger

  committed
  3d576fc

  Copy full SHA for 3d576fc

• enables profiling

  mariusgiger

  committed
  bd19570

  Copy full SHA for bd19570

• use munchify to convert dict to object because of issues with serialization

  mariusgiger

  committed
  520c434

  Copy full SHA for 520c434

• fix issues with device code, logs config

  mariusgiger

  committed
  8c10a86

  Copy full SHA for 8c10a86

• log profile as file

  mariusgiger

  committed
  73f634d

  Copy full SHA for 73f634d

• adds log

  mariusgiger

  committed
  9122c45

  Copy full SHA for 9122c45

• adds log

  mariusgiger

  committed
  d0ba673

  Copy full SHA for d0ba673

• adds log

  mariusgiger

  committed
  578a3af

  Copy full SHA for 578a3af

• adds log

  mariusgiger

  committed
  fcfc3f4

  Copy full SHA for fcfc3f4

• adds more params

  mariusgiger

  committed
  2e85082

  Copy full SHA for 2e85082

• adds dirpath

  mariusgiger

  committed
  762841a

  Copy full SHA for 762841a

• removes stack trace again

  mariusgiger

  committed
  7256de1

  Copy full SHA for 7256de1

• adds pinned memory

  mariusgiger

  committed
  1e0f612

  Copy full SHA for 1e0f612

Commits on May 31, 2022

• adds utility for downloading goes fluxes

  mariusgiger

  committed
  911197d

  Copy full SHA for 911197d

• Merge pull request #22 from i4Ds/feature/temporal-split

  Show description for 40a7fce
  mariusgiger

  authored
  40a7fce

  Copy full SHA for 40a7fce

Commits on Jun 3, 2022

• adds note about leap seconds

  mariusgiger

  committed
  f054eac

  Copy full SHA for f054eac

Commits on Jun 4, 2022

• converts GOES handling to dask for more efficiency

  mariusgiger

  committed
  de1149b

  Copy full SHA for de1149b

• allows to filter the SDO ML v2 dataset by irradiance

  mariusgiger

  committed
  768b54a

  Copy full SHA for 768b54a

Commits on Jun 7, 2022

• adds GOES plotting

  mariusgiger

  committed
  fafa42e

  Copy full SHA for fafa42e

Commits on Jun 11, 2022

• adds illustrative example for working with GOES timeseries, adds filter by GOES quality

  mariusgiger

  committed
  ab3a0bc

  Copy full SHA for ab3a0bc

• Merge pull request #27 from i4Ds/feature/irradiance-filtering

  Show description for 0e3ccaf
  mariusgiger

  authored
  0e3ccaf

  Copy full SHA for 0e3ccaf

• Merge pull request #26 from i4Ds/feature/goes-as-dask

  Show description for ce20668
  mariusgiger

  authored
  ce20668

  Copy full SHA for ce20668

• Merge pull request #24 from i4Ds/feature/goes-ts

  Show description for e17e8b2
  mariusgiger

  authored
  e17e8b2

  Copy full SHA for e17e8b2

• Merge pull request #25 from i4Ds/feature/prepare-training

  Show description for 81f03eb
  mariusgiger

  authored
  81f03eb

  Copy full SHA for 81f03eb

• switches data dir

  mariusgiger

  committed
  e2fcc3c

  Copy full SHA for e2fcc3c

• Merge pull request #28 from i4Ds/bugfix/data-dir

  Show description for 155a913
  mariusgiger

  authored
  155a913

  Copy full SHA for 155a913

• Merge remote-tracking branch 'origin/main' into feature/sdo-ml-v2-loader-pt2

  mariusgiger

  committed
  fa52573

  Copy full SHA for fa52573