Skip to content

Release 0.6.0

Latest
Latest
Compare
Choose a tag to compare
@steso-nommon steso-nommon released this 24 Jan 11:47
0.6.0
e3427fd

🐛 Fixes

  • Fix incorrect generation of Grid Ids for coordinates smaller than 10^7.

⬆️Features

SemanticCleaning

  • Event Cache: creates an auxiliar table with the first and last event of each device for each day. This allows fast loading for the DPS component.
  • Roaming out events: Outbound records are set to NO_ERROR unless they are different-location duplicates. Outbound records are any records where the first three numbers of PLMN do not match the country network MCC (given by configuration "[General] local_mcc").

DailyPerformanceScore (DPS):

  • Removed storage of DPS = 0 reducing the memory footprint of the Usual environment pipeline.
  • Storage of DPS as grid arrays reducing the storage needed by DPS.
  • Revert DPS indicator to tile level due to the DPS representation improvements.

NetworkCleaning

  • Made the cell_id CGI check optional by configuration.

Usual Environment:

  • 200m precision UE pipeline: Added functionality to calculate UE at a precision of 200m. This is enabled by configuration. By default the 100m precision will be used. This does not affect the resolution of final outputs of UE as they will be in 100m.

Final indicators at grid level: Added support to SpatialAggregation and Estimation components to generate final indicators at aggregated at 100m or 1Km grid tiles.

🚀 Performance improvements

General:

  • Changed the grid_id encoding from 20 Bytes to 8 Bytes granting a significant storage and performance increase in the Usual Environment pipeline.

DPS:

  • Performance improvement in the calculation of time slots.
  • Early removal of DPS = 0 values during the component processing.

MPS:

  • Performance improvement in the calculation of frequency, regularity mean and regularity deviation.

🎇General changes

  • Added [General] section to general_config.ini for storing pipeline wide values.
  • Added clear_destination_directory to DPS & MPS.
  • Added partition chunk processing of events to Usual environment pipeline components. This change affects DPS, MPS, LPS and UE-Labelling. This allows the user to choose how many user_id_modulo partitions are processed at the same time via partition_chunk_size in configuration. If this parameter is not given or it takes a value <= 0 it will use all partitions at the same time.

🛣️ Next Release

Our next release will focus on enhancing the pipeline with tourism use cases. This update aims to expand the repository's functionality, providing tailored solutions for tourism-related scenarios.

Assets 2
Loading