Skip to content

Latest commit

 

History

History
324 lines (219 loc) · 27.4 KB

fip-0100.md

File metadata and controls

324 lines (219 loc) · 27.4 KB
fip title author Discussions-to status type category Created
0100
Removing Batch Balancer, Replacing It With a Per-sector Fee and Removing Gas-limited Constraints
irene (@irenegia), AX (@AxCortesCubero), rvagg (@rvagg), molly (@momack2), kiran (@kkarrancsu)
Last Call
Technical
Core
2025-02-04

FIP-0100: Removing Batch Balancer, Replacing It With a Per-sector Fee and Removing Gas-limited Constraints

Simple Summary

This FIP proposes to:

  • Remove the batch fee from all PreCommit and ProveCommit methods to incentivize sector batching during the PreCommit step and proof aggregation during the ProveCommit step;
  • Introduce a per sector fee to replace the batch balancer cost structure with a more stable mechanism that supports the long-term economic sustainability of the Filecoin network;
  • Remove protocol constraints that are no longer necessary since the introduction of proper gas accounting.

Abstract

This FIP consists of three changes:

  • First, a straightforward way to reduce gas consumption is through sector batching and proof aggregation. Storage providers (SPs) should be incentivized to batch sectors at PreCommit and aggregate proofs at prove commit as much as possible. Currently, the efficiency of these processes is governed by the batch balancer mechanism, which is tied to the base fee and comes with several limitations. This FIP proposes the removal of the batch balancer (see details below) to enable full adoption of batching and aggregation, thereby enhancing scalability and onboarding growth.
  • Secondly, the batch balancer was introduced to address a misalignment in the Filecoin economy. Beyond blockchain execution, the Filecoin network provides SPs with a storage-auditing service through on-chain verification of proofs. While gas fees primarily account for execution costs, there is no dedicated system that properly captures the network’s storage-auditing value. The batch balancer partially addressed this gap, although attempted to achieve value capture through the gas mechanism which is primarily a means of constraining chain validation costs. With the removal of the batch balancer, a replacement mechanism is needed. This FIP proposes a better, more effective system to achieve that goal (see details below).
  • Finally, this FIP also includes the removal of gas-limited protocol constraints which are no longer needed. However, these modifications are not the primary focus of the proposal. Eliminating these constraints aims to simplify the protocol, and remove any unintentional network growth constraints.

Change Motivation

PreCommit batch fee removed

Due to the batch fee, currently PreCommit batching is only rational (ie, cost-effective) when the base fee exceeds 0.09 nanoFIL. Removing this fee will eliminate this obstacle, enabling more batching and therefore gas saving.

To have a rough estimate of the saving, we can compare two PreCommitSector messages posted by the same miner actor with little difference in the number of sectors when the message landed.

  • Msg 1: PreCommiSectorBatch2 for 1 sector : 16.7 M gas used
  • Msg 2: PreCommitSectorBatch2 for 4 sectors): 18.6 M/ 4 = 4.6 M per sector (3.6x smaller)

ProveCommit batch fee removed

There are two options for ProveCommitSector3: (1) "simple" batching as in PreCommit (ie, one message for ≥ 2 sectors) where there is a PoRep proof for each sector in the batched message, and (2) aggregating proofs of multiple sectors and post in one message.

Batching for ProveCommit is already rational, as the batch fee applies only to aggregated proofs (opposed to PreCommit, where the batch balancer applies to batching). On the other hand, aggregation is rational only when the base fee exceeds 0.065 nanoFIL due to the batch fee applied to aggregation. Removing this fee too will eliminate this obstacle, enabling more proof aggregation and therefore gas saving.

To have a rough estimate of the saving, we can compare these ProveCommitSector3 messages posted by the same miner actor with little difference in the number of sectors when the message landed1.

  • Msg 1: ProveCommitSectors3 for 1 sector: 267.5 M
  • Msg 2: ProveCommitSectors3 for 4 sectors, batched (no aggregation): 580 M/4 = 145 M per sector (1.85x smaller)
  • Msg 3: ProveCommitSectors3 for 4 sectors, aggregated: 517.4 M/4 = 129.3 M per sector (2x smaller).

The same logic applies to ProveCommitSectorsNI, where we have the same two options as in ProveCommitSector3.

Per-sector fee added

The batch balancer mechanism had several goals and effects. Among them, two key aspects were: (1) Creating a linear cost for onboarding storage in the presence of batching/aggregation and (2) Burning tokens to balance new token supply with network burn, ensuring value distribution among FIL holders and earners.

We propose a replacement mechanism that preserves these effects while adhering to the following design principles:

  • Simplify the system to improve predictability.
  • Preserve and align network value accrual.
  • Avoid additional onboarding rate limits.
  • Enable future optimizations in the network’s execution layer.

Proposal Details: We introduce a per-sector fee, whose value is proportional to a fixed fraction of the circulating supply and to the sector duration. To prevent this fee from becoming a significant upfront cost that could hinder onboarding, we propose daily payments instead of an upfront fee. Moreover, as a safety measure for the high-growth scenarios, we propose capping the daily payment at a fixed percentage of the daily expected per-sector block reward.

More in details:

  • At ProveCommit time, we compute the value dailyFee = k * CS(t), where k = 7.4 E-15 is a system constant and t is the sector activation epoch. The dailyFee value is stored but no payment is due at ProveCommit time.
  • Then, at the end of each deadline, we compute: dailyPayment = min (dailyFee, m * expected_day_reward), where dailyFee is a total for all sectors in the deadline, m= 0.5 is another system constant and expected_day_reward is the updated value for the daily expected per-sector block reward for all sectors in the deadline. The dailyPayment value is burnt. This happens for each sector assigned to the deadline and onboarded after this FIP is deployed. Note that the payment is due for sectors in any non-expiring state (active, faulty and in recovery).
  • If a sector is extended or updated, the dailyFee value is not changed but the cap will be computed considering the updated value of expected_day_reward (and therefore the dailyPayment is updated). The sector will keep paying dailyPayment for the extended duration.
  • When the sector expires or gets terminated, the sector is no longer counted in the dailyFee value for the deadline.

Gas-limited constraints

Various protocol constraints that were previously necessary when there was no gas applied on actor computation are no longer necessary since the introduction of proper gas accounting with FVM (see FIP-0032). This FIP proposes to remove the following constraints as they are no longer necessary in practice due to the gas mechanism being the primary constraint for computation:

  • PRE_COMMIT_SECTOR_BATCH_MAX_SIZE: The maximum number of sector PreCommitments in a single batch.
  • PROVE_REPLICA_UPDATES_MAX_SIZE: The maximum number of sector replica updates in a single batch.
  • DECLARATIONS_MAX: Maximum number of unique "declarations" in batch operations (i.e. terminations, faults, recoveries).

In most cases, users will be limited by the gas limit rather than these constraints (message execution failed: exit SysErrOutOfGas). Removing these constraints will simplify the protocol and remove any unintentional network growth constraints.

Specification

This FIP makes three main changes to the protocol:

  1. Removes the batch balancer fee mechanism from sector PreCommit and ProveCommit operations
  2. Removes certain gas-limited protocol constraints that are no longer necessary
  3. Introduces a new per-sector daily fee mechanism based on circulating supply and block rewards

Removal of gas-limited constraints

The following constants and their use will be removed from the storage miner actor:

  • PRE_COMMIT_SECTOR_BATCH_MAX_SIZE - used by PreCommitSectorBatch2
  • PROVE_REPLICA_UPDATES_MAX_SIZE - used by both ProveReplicaUpdates and ProveReplicaUpdates3
  • DECLARATIONS_MAX - used by TerminateSectors, DeclareFaults and DeclareFaultsRecovered

Removal of aggregate fee

Aggregate fee functionality and constants will be removed from the storage miner actor. This includes the following changes:

  • Remove the burning of aggregate_fee from PreCommitSectorBatch2.
  • Remove the burning of aggregate_fee from ProveCommitAggregate, ProveCommitSectorsNI, ProveCommitSectors3
  • Removal of aggregate_fee calculation methods and associated constants, including ESTIMATED_SINGLE_PROVE_COMMIT_GAS_USAGE and ESTIMATED_SINGLE_PRE_COMMIT_GAS_USAGE

Fee implementation

A daily fee value is calculated per-sector at time of onboarding. This value is stored as daily_fee in the SectorOnChainInfo and an aggregated daily_fee value is also stored in each Deadline. The fee is deducted from the miner actor's balance, processed by cron at the end of each deadline, where the fee is deducted in a similar manner to fault fees. Upon sector termination, the daily_fee recorded in the sector's Deadline is adjusted to remove the sector's fee.

The fee is calculated as a fixed fraction of the circulating supply, as per the formula above, at the time of sector activation. This value is obtained from the existing total_fil_circ_supply system call which is also used for the Initial Pledge (IP) calculation. Upon payment, the estimated block reward for a Deadlines power is calculated and used to apply a maximum cap, as per the formula above. Efficient application of the cap will require also adding a new power field to the Deadline as a sum of all the power its partitions.

Fee deduction will be handled in the existing handle_proving_deadline() function which is called from cron. It will need to make use of the ThisEpochReward method from the reward actor and CurrentTotalPower method of the power actor in order to calculate and apply the expected 24h block reward cap to the fee for the amount of power in the deadline.

The Deadline struct will be extended with live_power and daily_fee fields:

pub struct Deadline {
  // existing fields ...

  /// Power of not-yet-terminated sectors (incl faulty & unproven) assigned to this deadline.
  pub live_power: PowerPair,

  /// The total fee payable per day for this deadline, this represents a sum of the daily fee
  /// amounts for all sectors assigned to this deadline.
  pub daily_fee: TokenAmount,
}

The SectorOnChainInfo struct will be extended to include a new daily_fee field which will be used to store the daily fee for the sector. This value will be set at the time of sector activation and will be used to calculate the daily fee for the sector for the duration of its lifetime. This value will be zero for sectors onboarded prior to activation of this FIP.

At the time of this proposal, there are approximately 650M sectors on chain. The expense of migrating and additional bytes used to store zero values for old sectors is not insignificant. Therefore it is proposed to perform a lazy migration where existing SectorOnChainInfo objects are left alone and this new field is only added to new sectors, and sectors which are updated (for any reason). This means that reading SectorOnChainInfo blocks from state will support two forms: a 15 field version (old) and a 16 field version (new daily_fee field). The old version will instantiate with a zero value for daily_fee and this field will be written whenever a SectorOnChainInfo object is written to state, regardless of whether it contains a zero or non-zero value. i.e. reads will support 15 or 16 fields, writes will always write 16 fields.

A future migration may be proposed to unify the format, or introduce an explicit schema versioning mechanism, however this will likely be deferred until additional sector clean-up activities can take place to reduce the number of unused sectors on chain, thereby reducing the cost of migration.

pub struct SectorOnChainInfo {
  // existing fields ...

  /// The total fee payable per day for this sector. The value of this field is set at the time of
  /// sector activation and is used to calculate the daily fee for the sector for the duration of
  /// its lifetime.
  /// This field is not included in the serialised form of the struct prior to the activation of
  /// FIP-0100, and is added as the 16th element of the array after that point only for new sectors
  /// or sectors that are updated after that point. For old sectors, the value of this field will
  /// always be zero.
  pub daily_fee: TokenAmount,
}

Sector commitments: At each of ProveCommitSectors3, ProveCommitAggregate and ProveCommitSectorsNI, the daily_fee for each sector is calculated and recorded in SectorOnChainInfo. It is also added to the existing daily_fee total recorded in the Deadline the sector is assigned to.

Faults: have no impact on a sector or deadline's daily_fee value.

Early terminations: both automatic and manual, will result in a reduction of a Deadline's daily_fee value for the sectors being terminated.

Migration

At activation of this FIP, a new schema for the Deadline block type will be introduced, adding two new fields: live_power: PowerPair and daily_fee: TokenAmount. A migration will be performed on all deadlines, across all miner actors. As the majority of deadlines referenced on chain are the empty Deadline block, a CID for this singleton block may be used to speed up migration where the CID for the previous form's Deadline block is encountered. All daily_fee values will be set to zero at migration and the live_power will be calculated as a sum of the live_power in each of the Partitions assigned to the Deadline.

No migration is necessary for SectorOnChainInfo as the lazy migration strategy is employed.

Special handling for Calibration network

In order to properly test the implementation of this FIP, a long-standing problem with circulating supply calculation on Calibnet will need to be addressed. Circulating supply is calculated using an estimation method each epoch and supplied to the FVM, it uses these inputs (from the time of writing):

Metric Mainnet (FIL) Calibnet (FIL)
FilVested 484,847,134 323,483,605
FilMined 365,928,138 68,672,646
FilBurnt 40,327,377 32,117,860
FilLocked 140,975,297 3,893,484
InitialFilReserved 300,000,000 300,000,000
FilReserveBalance 282,933,381 869,278,271

The amount of reserved funds disbursed from f090 (FilReserveDisbursed) is then calculated as:

$$FilReserveDisbursed = InitialFilReserved - FilReserveBalance$$

For Mainnet:

$$FilReserveDisbursed = 300,000,000 - 282,933,381 = 17,066,619$$

For Calibnet:

$$FilReserveDisbursed = 300,000,000 - 869,278,271 = -569,278,271$$

Circulating supply (CS) is then calculated as:

$$CS = \max(0, FilVested + FilMined + FilReserveDisbursed - FilBurnt - FilLocked)$$

For Mainnet:

$$CS = \max(0, 484,847,134 + 365,928,138 + 17,066,619 - 40,327,377 - 140,975,297) = 686,539,216$$

For Calibnet:

$$CS = \max(0, 323,483,605 + 68,672,646 + (-569,278,271) - 32,117,860 - 3,893,484) = 0$$

Calibnet currently executes with a circulating supply of 0 each epoch, regardless of token movements.

Because this FIP uses a fixed portion of current circulating supply to calculate the fee, Calibnet fees will always be 0, which makes proper testing of the new functionality in live network conditions impossible.

To address this problem, this FIP will also adjust network parameters for Calibnet to ensure that the circulating supply is a reasonable and dynamic positive number. The core problem is the initial allocation of reserve funds which appears to have been set to 900M FIL. This was likely set to ensure a total supply of 2B FIL. Unfortunately, implementations use the same InitialFilReserved value as mainnet, 300M FIL.

Filecoin node implementations compatible with Calibnet will be required to set their InitialFilReserved for Calibnet (only) to 900,000,000 FIL from the activation epoch of this FIP (i.e. a value of 300,000,000 FIL must be used for epochs up until then, including when re-executing older tipsets). This will result in a more appropriate circulating supply calculation starting from the activation epoch of this FIP:

Metric Mainnet (FIL) Calibnet (FIL)
InitialFilReserved 300,000,000 900,000,000
FilReserveDisbursed 17,066,619 30,721,729
FilCirculating 686,539,216 386,866,636

Design Rationale

We aimed for a design that is simple to understand and model, and relatively simple to implement. With this in mind, we explored the following fee structures:

  • Upfront Payment vs. Ongoing Payment: The first offers better predictability, while the second reduces the risk of creating an entry barrier and a disincentive for sectors with longer durations. With this FIP, we chose a hybrid approach that captures the benefits of both by computing the fee value upfront for predictability, but allowing it to be paid daily over sector lifetime instead of as a lump sum upfront. Moreover, the daily payment option allows us to easily implement a security measure by capping the daily value at a fixed fraction of the per-sector block reward.

  • Flat Fee vs. Dynamic Fee: A fixed per-sector fee provides predictability and consistency, while a fluctuating fee based on network congestion, onboarding rate, or other variables better captures value when the network is actively providing a service. We chose a model where the fee is a fixed fraction of a quantity that adjusts to economic conditions. We evaluated the following economic indicators for determining the per-sector fee: Initial Pledge (IP), per-sector Block Reward (BR) and Circulating Supply (CS).

    • The IP-based fee was discarded to avoid complex system dependencies, as changes to IP in future FIPs could unintentionally impact the per-sector fee.
    • The BR-based fee was discarded because the fee-to-reward ratio did not adjust properly to network growth, leading to high fees even when onboarding demand was low. See here for more details.
    • CS was ultimately chosen because it normalizes fees relative to the Filecoin economy, ensuring both adaptability and predictability.

We reached this conclusion through simulation-based analysis, modeling the impact of fees under different network conditions. Specifically, we evaluated multiple input trajectories: 0.5×, 0.75×, 1×, and 2× the current onboarding, renewal, and FIL+ rates. Since both renewals and FIL+ are near 100%, we capped them at their current maximum (see Fig. 1).

Fig 1: Network state evolution for each input trajectory.

For each trajectory, we analyzed the expected fee and related metrics over time. Fig. 2 presents these results for the CS-based fee, incorporating a cap at 50% of expected_day_reward as a safety measure in high-growth scenarios. Notably, Fig. 2 confirms that the CS-based fee proposed in this FIP offers the following key advantages:

  • Stable Economic Reference: A fixed percentage of CS functions like a fixed USD fee if market cap remains stable, keeping costs predictable;
  • Adapts to Market Conditions: If Filecoin’s valuation rises, fees increase in FIL terms, scaling with the economy. If the market stagnates, fees stay economically consistent.

Figure 2: CS-based fee evolution as a function of various network metrics. Here we considered a daily payment that is a fixed % of CS-based and capped at 50% of `expected_day_reward` as a safety measure in high-growth scenarios.

Backwards Compatibility

This FIP modifies actor behavior and will require a new Filecoin network version.

Test Cases

builtin-actors

Many of the test scenarios outlined below already exist within the test suite for builtin-actors. Ensuring that all of the various cases, particularly edge-cases, are covered will be important to ensure that the new fee mechanism is correctly implemented and behaves as expected. In addition, any existing test scenarios that test sector lifecycles should be updated to include invariants that check the new fee mechanism: that it is correctly calculated, tallied in the partition, remains correct over time, is correctly deducted in handle_proving_deadline(), is correctly capped and is correctly adjusted for terminations.

  • Test that the batch balancer fee is removed from PreCommitSectorBatch2 and ProveCommitAggregate, ProveCommitSectorsNI, ProveCommitSectors3
  • Test that the gas-limited constraints are removed from the protocol
  • Test that the per-sector fee is correctly calculated and added to the respective partition's daily_fee value when using ProveCommitAggregate, ProveCommitSectorsNI, ProveCommitSectors3
    • Tests should cover different batching variations
  • Test that ProveReplicaUpdates and ProveReplicaUpdates3 do not have an impact on the fee mechanism
  • Test that the per-sector fee is correctly charged at the end of each deadline for all sectors in the active, faulty, and recovered states
    • Tests should cover different batching variations
  • Test that manual early terminations correctly adjust the deadline's daily_fee
  • Test that automatic early terminations correctly adjust the deadline's daily_fee
  • Test that sector expiration correctly adjusts the deadline's daily_fee
  • Test that the fee is correctly capped at the expected daily block reward at the time it is paid

Calibration network

Once Calibnet is upgraded, functional tests should be performed that cover:

  • Ensure that the circulating supply is a reasonable and dynamic positive number
  • Onboarding new sectors with PreCommitSectorBatch2 and ProveCommitAggregate, ProveCommitSectors3 (and ProveCommitSectorsNI if possible) and:
    • observing that the batch balancer fee is not applied
    • observing deadline state to ensure that the new fee mechanism is correctly implemented and behaves as expected
  • Observing that the per-sector fee is correctly charged at the end of each deadline for all sectors in the active, faulty, and recovered states
  • Observing that manual early terminations correctly adjust the deadline's daily_fee value

Security Considerations

This FIP does not affect underlying proofs or protocol security.

Incentive Considerations

Filecoin’s network revenue model is currently tied to the gas usage, and the most amount of gas is used by storage onboarding methods. This creates an incentive misalignment, where economic considerations lead to an incentive to not optimize the gas usage of onboarding methods, as this could hurt overall network revenue.

The Batch Balancer mechanism exemplifies this concept: batching and aggregating abilities were introduced to increase gas efficiency of onboarding methods, but the batch balancer mechanism restricted use of these abilities to when the network demand was high, to protect Filecoin’s economic interests.

This FIP fixes this incentive misalignment by disconnecting the network revenue from gas usage. The removal of the batch balancer allows for aggregation to occur at any level of demand regardless of network utilization rate (such as current levels of demand). We expect this to lead to a reduction of about ~30% in per-sector onboarding gas. This would in turn result in a drop in base fee, and therefore a drop in gas-based network revenue.

The explicit per-sector fee introduced by this FIP allows us to remove the batch balancer, and support other future gas usage optimizations, without adding macroeconomic risk.

The removal of batch balancer and associated reduction in gas fees are up-front improvements to SP costs designed to unblock and support faster data onboarding. This up-front cost-reduction is balanced by the per-sector fee that is charged gradually over the sector lifetime, starting at a small fraction of daily block rewards and adjusting dynamically based on network economic growth rate.

Product Considerations

From a product perspective, this FIP is expected to:

  • Decouple gas fees (blockspace costs) from service fees (storage-auditing costs): This separation creates a more resilient system, reducing sensitivity to base fee spikes and ensuring predictable and fair costs for SPs.
  • Simplify cost analysis and calculations for SPs: The network will provide a smoother user experience by clearly distinguishing between congestion control (managed via base fees) and payments for services unrelated to congestion.
  • Capture value for the storage-auditing service: By introducing a per-sector fee burn, the network can proportionally capture value based on storage demand.
  • Ensure economic stability amid future optimizations: Improvements in proof mechanisms or onboarding throughput will no longer risk destabilizing the network’s economic alignment. This ensures the Filecoin protocol can lower costs for SPs while still capturing value for the network.
  • Remove gas-limited constraints: Eliminating these unnecessary constraints simplifies the protocol and allows gas mechanisms to serve as the primary network constraint, making the system more adaptive and flexible to changing conditions.

Implementation

  1. Remove the batch balancer fee mechanism from sector PreCommit and ProveCommit operations (TBD)
  2. Remove gas-limited protocol constraints that are no longer necessary
  3. Implement per-sector daily fee mechanism based on circulating supply and block rewards (TBD)
  4. Fix VerifySeal gas bug
  5. Fix circulating supply calculation on Calibnet (TBD)

Copyright

Copyright and related rights waived via CC0.

Footnotes

  1. A bug is currently causing single proofs (VerifySeal) to be charged an incorrect amount of gas units. So we added the missing units (42M per proof) to msg1 and msg2 in order to provide the future correct estimates (the bug is planned to be fixed in nv25). See here ↩