Skip to content

Commit

Permalink
Merge pull request #1135 from ie3-institute/ms/#1106-transfer-LoadPro…
Browse files Browse the repository at this point in the history
…file-parsing-code-from-SIMONA

Adding sources for load profile time series.
  • Loading branch information
danielfeismann authored Feb 26, 2025
2 parents feee56c + d8bec21 commit a631cc7
Show file tree
Hide file tree
Showing 16 changed files with 878 additions and 30 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Attributes `housingType` and `numberInhabitants` for `ThermalHouse`s [#1253](https://github.com/ie3-institute/PowerSystemDataModel/issues/1253)
- Added domestic hot water storage model [#1257](https://github.com/ie3-institute/PowerSystemDataModel/issues/1257)
- Validation for BDEW load profile values [#1243](https://github.com/ie3-institute/PowerSystemDataModel/issues/1243)
- Added load profiles sources [#1106](https://github.com/ie3-institute/PowerSystemDataModel/issues/1106)

### Fixed
- Removing opened `SwitchInput` during connectivity check [#1221](https://github.com/ie3-institute/PowerSystemDataModel/issues/1221)
Expand Down
27 changes: 25 additions & 2 deletions docs/readthedocs/io/csvfiles.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,11 +94,13 @@ Csv id coordinate sources can have two different ways to represent their coordin
* - Model
- File Name
* - individual time series
- *prefix_* its *_columnScheme_UUID_suffix*
- *prefix_* its *_columnScheme_UUID* *_suffix*
* - load profile input
- *prefix_* rts *_profileKey_UUID_suffix*
- *prefix_* lpts *_profileKey* *_suffix*
```

#### Individual Time Series

Let's spend a few more words on the individual time series:
Those files are meant to carry different types of content - one might give information about wholesale market prices,
the other is a record of power values provided by a real system.
Expand Down Expand Up @@ -154,6 +156,27 @@ The following keys are supported until now:
```


##### Load Profile Time Series

The following profiles are supported until now:
```{list-table}
:widths: auto
:class: wrapping
:header-rows: 1
* - Key
- Information
- Supported head line.
* - e.g.: H0
- BDEW standard load profiles ([source](https://www.bdew.de/energie/standardlastprofile-strom/))
- Permissible head line: ``SuSa,SuSu,SuWd,TrSa,TrSu,TrWd,WiSa,WiSu,WiWd,quarterHour``
* - random
- A random load proile based on: ``Kays - Agent-based simulation environment for improving the planning of distribution grids``
- Permissible head line: ``kSa,kSu,kWd,mySa,mySu,myWd,sigmaSa,sigmaSu,sigmaWd,quarterHour``
```

### Results

```{list-table}
Expand Down
5 changes: 5 additions & 0 deletions docs/readthedocs/models/input/additionaldata/timeseries.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,4 +55,9 @@ The following different values are available:
* - `WeatherValue`
- Combination of irradiance, temperature and wind information
* - `BdewLoadValues`
- Values for combination of seasons and day types
* - `RandomLoadValues`
- Parameters for a probability density function to draw random power consumptions
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
/*
* © 2024. TU Dortmund University,
* Institute of Energy Systems, Energy Efficiency and Energy Economics,
* Research group Distribution grid planning and operation
*/
package edu.ie3.datamodel.io.csv;

import edu.ie3.datamodel.io.naming.timeseries.LoadProfileMetaInformation;
import java.nio.file.Path;
import java.util.Objects;

public class CsvLoadProfileMetaInformation extends LoadProfileMetaInformation {
private final Path fullFilePath;

public CsvLoadProfileMetaInformation(String profile, Path fullFilePath) {
super(profile);
this.fullFilePath = fullFilePath;
}

public CsvLoadProfileMetaInformation(
LoadProfileMetaInformation metaInformation, Path fullFilePath) {
this(metaInformation.getProfile(), fullFilePath);
}

public Path getFullFilePath() {
return fullFilePath;
}

@Override
public boolean equals(Object o) {
if (this == o) return true;
if (!(o instanceof CsvLoadProfileMetaInformation that)) return false;
if (!super.equals(o)) return false;
return fullFilePath.equals(that.fullFilePath);
}

@Override
public int hashCode() {
return Objects.hash(super.hashCode(), fullFilePath);
}

@Override
public String toString() {
return "CsvLoadProfileMetaInformation{"
+ "uuid='"
+ getUuid()
+ '\''
+ ", profile='"
+ getProfile()
+ '\''
+ "fullFilePath="
+ fullFilePath
+ '}';
}
}
130 changes: 130 additions & 0 deletions src/main/java/edu/ie3/datamodel/io/source/LoadProfileSource.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
/*
* © 2024. TU Dortmund University,
* Institute of Energy Systems, Energy Efficiency and Energy Economics,
* Research group Distribution grid planning and operation
*/
package edu.ie3.datamodel.io.source;

import static edu.ie3.datamodel.models.profile.LoadProfile.RandomLoadProfile.RANDOM_LOAD_PROFILE;

import edu.ie3.datamodel.exceptions.FactoryException;
import edu.ie3.datamodel.exceptions.SourceException;
import edu.ie3.datamodel.io.csv.CsvLoadProfileMetaInformation;
import edu.ie3.datamodel.io.factory.timeseries.BdewLoadProfileFactory;
import edu.ie3.datamodel.io.factory.timeseries.LoadProfileData;
import edu.ie3.datamodel.io.factory.timeseries.LoadProfileFactory;
import edu.ie3.datamodel.io.factory.timeseries.RandomLoadProfileFactory;
import edu.ie3.datamodel.io.source.csv.CsvDataSource;
import edu.ie3.datamodel.io.source.csv.CsvLoadProfileSource;
import edu.ie3.datamodel.models.profile.BdewStandardLoadProfile;
import edu.ie3.datamodel.models.profile.LoadProfile;
import edu.ie3.datamodel.models.timeseries.repetitive.LoadProfileEntry;
import edu.ie3.datamodel.models.timeseries.repetitive.LoadProfileTimeSeries;
import edu.ie3.datamodel.models.timeseries.repetitive.RandomLoadProfileTimeSeries;
import edu.ie3.datamodel.models.value.PValue;
import edu.ie3.datamodel.models.value.Value;
import edu.ie3.datamodel.models.value.load.BdewLoadValues;
import edu.ie3.datamodel.models.value.load.LoadValues;
import edu.ie3.datamodel.models.value.load.RandomLoadValues;
import edu.ie3.datamodel.utils.Try;
import java.time.ZonedDateTime;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.function.Function;
import java.util.stream.Collectors;
import javax.measure.quantity.Energy;
import javax.measure.quantity.Power;
import tech.units.indriya.ComparableQuantity;

public abstract class LoadProfileSource<P extends LoadProfile, V extends LoadValues>
extends EntitySource {
protected final Class<V> entryClass;
protected final LoadProfileFactory<P, V> entryFactory;

protected LoadProfileSource(Class<V> entryClass, LoadProfileFactory<P, V> entryFactory) {
this.entryClass = entryClass;
this.entryFactory = entryFactory;
}

/**
* Build a list of type {@code E}, whereas the underlying {@link Value} does not need any
* additional information.
*
* @param fieldToValues Mapping from field id to values
* @return {@link Try} of simple time based value
*/
protected Try<LoadProfileEntry<V>, FactoryException> createEntries(
Map<String, String> fieldToValues) {
LoadProfileData<V> factoryData = new LoadProfileData<>(fieldToValues, entryClass);
return entryFactory.get(factoryData);
}

public abstract LoadProfileTimeSeries<V> getTimeSeries();

/**
* Method to return all time keys after a given timestamp.
*
* @param time given time
* @return a list of time keys
*/
public abstract List<ZonedDateTime> getTimeKeysAfter(ZonedDateTime time);

/**
* Method to get the value for a given time.
*
* @param time for which a value is needed
* @return an optional
* @throws SourceException if an exception occurred
*/
public abstract Optional<PValue> getValue(ZonedDateTime time) throws SourceException;

/** Returns the load profile of this source. */
public abstract P getLoadProfile();

/** Returns the maximal power value of the time series */
public abstract Optional<ComparableQuantity<Power>> getMaxPower();

/** Returns the load profile energy scaling for this load profile time series. */
public abstract Optional<ComparableQuantity<Energy>> getLoadProfileEnergyScaling();

/**
* Method to read in the build-in {@link BdewStandardLoadProfile}s.
*
* @return a map: load profile to load profile source
*/
public static Map<
BdewStandardLoadProfile, CsvLoadProfileSource<BdewStandardLoadProfile, BdewLoadValues>>
getBdewLoadProfiles() throws SourceException {
CsvDataSource buildInSource = getBuildInSource(LoadProfileSource.class, "/load");

BdewLoadProfileFactory factory = new BdewLoadProfileFactory();

return buildInSource
.getCsvLoadProfileMetaInformation(BdewStandardLoadProfile.values())
.values()
.stream()
.map(
metaInformation ->
new CsvLoadProfileSource<>(
buildInSource, metaInformation, BdewLoadValues.class, factory))
.collect(Collectors.toMap(CsvLoadProfileSource::getLoadProfile, Function.identity()));
}

/**
* Method to read in the build-in {@link RandomLoadProfileTimeSeries}.
*
* @return the random load profile source
*/
public static CsvLoadProfileSource<LoadProfile.RandomLoadProfile, RandomLoadValues>
getRandomLoadProfile() throws SourceException {
CsvDataSource buildInSource = getBuildInSource(LoadProfileSource.class, "/load");

CsvLoadProfileMetaInformation metaInformation =
buildInSource.getCsvLoadProfileMetaInformation(RANDOM_LOAD_PROFILE).values().stream()
.findAny()
.orElseThrow();
return new CsvLoadProfileSource<>(
buildInSource, metaInformation, RandomLoadValues.class, new RandomLoadProfileFactory());
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -7,26 +7,52 @@

import edu.ie3.datamodel.io.naming.timeseries.ColumnScheme;
import edu.ie3.datamodel.io.naming.timeseries.IndividualTimeSeriesMetaInformation;
import edu.ie3.datamodel.io.naming.timeseries.LoadProfileMetaInformation;
import edu.ie3.datamodel.models.profile.LoadProfile;
import java.util.Collections;
import java.util.Map;
import java.util.Optional;
import java.util.UUID;

/** Source for all available time series with their {@link UUID} and {@link ColumnScheme} */
public interface TimeSeriesMetaInformationSource {
public abstract class TimeSeriesMetaInformationSource {

protected Map<String, LoadProfileMetaInformation> loadProfileMetaInformation;

/**
* Get a mapping from time series {@link UUID} to its meta information {@link
* IndividualTimeSeriesMetaInformation}
*
* @return that mapping
*/
Map<UUID, IndividualTimeSeriesMetaInformation> getTimeSeriesMetaInformation();
public abstract Map<UUID, IndividualTimeSeriesMetaInformation> getTimeSeriesMetaInformation();

/**
* Get an option on the given time series meta information
*
* @param timeSeriesUuid Unique identifier of the time series in question
* @return An Option on the meta information
*/
Optional<IndividualTimeSeriesMetaInformation> getTimeSeriesMetaInformation(UUID timeSeriesUuid);
public abstract Optional<IndividualTimeSeriesMetaInformation> getTimeSeriesMetaInformation(
UUID timeSeriesUuid);

/**
* Gat a mapping from load profile to {@link LoadProfileMetaInformation}.
*
* @return that mapping
*/
public Map<String, LoadProfileMetaInformation> getLoadProfileMetaInformation() {
return Collections.unmodifiableMap(loadProfileMetaInformation);
}

/**
* Get an option on the given time series meta information
*
* @param loadProfile load profile of the time series in question
* @return An Option on the meta information
*/
public Optional<LoadProfileMetaInformation> getLoadProfileMetaInformation(
LoadProfile loadProfile) {
return Optional.ofNullable(loadProfileMetaInformation.get(loadProfile.getKey()));
}
}
42 changes: 36 additions & 6 deletions src/main/java/edu/ie3/datamodel/io/source/csv/CsvDataSource.java
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,15 @@
import edu.ie3.datamodel.exceptions.SourceException;
import edu.ie3.datamodel.io.connectors.CsvFileConnector;
import edu.ie3.datamodel.io.csv.CsvIndividualTimeSeriesMetaInformation;
import edu.ie3.datamodel.io.csv.CsvLoadProfileMetaInformation;
import edu.ie3.datamodel.io.naming.FileNamingStrategy;
import edu.ie3.datamodel.io.naming.TimeSeriesMetaInformation;
import edu.ie3.datamodel.io.naming.timeseries.ColumnScheme;
import edu.ie3.datamodel.io.naming.timeseries.IndividualTimeSeriesMetaInformation;
import edu.ie3.datamodel.io.naming.timeseries.LoadProfileMetaInformation;
import edu.ie3.datamodel.io.source.DataSource;
import edu.ie3.datamodel.models.Entity;
import edu.ie3.datamodel.models.profile.LoadProfile;
import edu.ie3.datamodel.utils.Try;
import edu.ie3.datamodel.utils.Try.Failure;
import edu.ie3.datamodel.utils.Try.Success;
Expand All @@ -25,6 +28,7 @@
import java.nio.file.Path;
import java.util.*;
import java.util.function.Function;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
import java.util.stream.Stream;
Expand Down Expand Up @@ -118,7 +122,8 @@ public FileNamingStrategy getNamingStrategy() {
*/
public Map<UUID, CsvIndividualTimeSeriesMetaInformation>
getCsvIndividualTimeSeriesMetaInformation(final ColumnScheme... columnSchemes) {
return getIndividualTimeSeriesFilePaths().parallelStream()
return getTimeSeriesFilePaths(fileNamingStrategy.getIndividualTimeSeriesPattern())
.parallelStream()
.map(
filePath -> {
/* Extract meta information from file path and enhance it with the file path itself */
Expand All @@ -136,13 +141,41 @@ public FileNamingStrategy getNamingStrategy() {
.collect(Collectors.toMap(TimeSeriesMetaInformation::getUuid, Function.identity()));
}

/**
* Receive the information for specific load profile time series. They are given back mapped to
* their uuid.
*
* @return A mapping from profile to the load profile time series meta information
*/
public Map<String, CsvLoadProfileMetaInformation> getCsvLoadProfileMetaInformation(
LoadProfile... profiles) {
return getTimeSeriesFilePaths(fileNamingStrategy.getLoadProfileTimeSeriesPattern())
.parallelStream()
.map(
filePath -> {
/* Extract meta information from file path and enhance it with the file path itself */
LoadProfileMetaInformation metaInformation =
fileNamingStrategy.loadProfileTimeSeriesMetaInformation(filePath.toString());
return new CsvLoadProfileMetaInformation(
metaInformation, FileNamingStrategy.removeFileNameEnding(filePath.getFileName()));
})
.filter(
metaInformation ->
profiles == null
|| profiles.length == 0
|| Stream.of(profiles)
.anyMatch(profile -> profile.getKey().equals(metaInformation.getProfile())))
.collect(Collectors.toMap(LoadProfileMetaInformation::getProfile, Function.identity()));
}

/**
* Returns a set of relative paths strings to time series files, with respect to the base folder
* path
*
* @param pattern for matching the time series
* @return A set of relative paths to time series files, with respect to the base folder path
*/
protected Set<Path> getIndividualTimeSeriesFilePaths() {
protected Set<Path> getTimeSeriesFilePaths(Pattern pattern) {
Path baseDirectory = connector.getBaseDirectory();
try (Stream<Path> pathStream = Files.walk(baseDirectory)) {
return pathStream
Expand All @@ -151,10 +184,7 @@ protected Set<Path> getIndividualTimeSeriesFilePaths() {
path -> {
Path withoutEnding =
Path.of(FileNamingStrategy.removeFileNameEnding(path.toString()));
return fileNamingStrategy
.getIndividualTimeSeriesPattern()
.matcher(withoutEnding.toString())
.matches();
return pattern.matcher(withoutEnding.toString()).matches();
})
.collect(Collectors.toSet());
} catch (IOException e) {
Expand Down
Loading

0 comments on commit a631cc7

Please sign in to comment.