Preload segments

[KKcorps]

This won't compile right now. Mostly meant to track progress.

[klsince]

the changes of batch loading segments look good to me.

and fyi, the methods to sync between loading and ready phases

1. the setReady() is supposed to be called at the end of preloadSegments()
2. I'm planning to call untilReady() in most segment mgmt methods on RealtimeTableDataManager, e.g. addSegment(), remoteSegment(), and also this addOrReplaceSegment(). So directly calling it would lead to deadlock. I'll think about either add a boolean on addOrReplaceSegment() or add a new method, w/o waiting on the condition.

// a feature flag to enable preloading; or simply _isReady = true, thus no blocking to handle helix state transitions
 @Override
  public void untilReady()
      throws InterruptedException {
    if (_isReady) { // volatile
      return;
    }
    _isReadyLock.lock();
    try {
      while (!_isReady) {
        _isReadyCon.await();
      }
    } finally {
      _isReadyLock.unlock();
    }
  }

  private void setIsReady() {
    _isReadyLock.lock();
    try {
      _isReady = true;
      _isReadyCon.signalAll();
    } finally {
      _isReadyLock.unlock();
    }
  }

[klsince]

I think the RealtimeTableDataManager.addSegment() should be called here to add segment. The addOrReplaceSegment() seems for the Offline table.

[KKcorps]

No, addOrReplaceSegment is the only method that's called by HelixInstanceDataManager iiuc.

Internally, it calls the addSegment and passed the required params such as segment metadata