prepare 6.4.1 release (#248)

* comments * add test for stats event * capture stream connection stats in diagnostic events * fix test * remove eventReportingDisabled from diagnostic event; only create diagnosticsManager if needed * revise tests to use new helper package * misc cleanup * use launchdarkly-js-test-helpers 1.0.0 * fix package reference * minor fixes to config validation messages + add comment * diagnostic eventsInQueue counter should be # of events at last flush * rename eventsInQueue to eventsInLastBatch * don't let user fall outside of last bucket in rollout * add unit tests for basic bucketing logic and edge case * avoid redundant property lookups * fix Redis client parameter to match TS declaration (but still support old incorrect parameter) * add event payload ID * remove mistakenly checked-in test code (note, this SDK key was only valid on staging) * add mention of singleton usage * update diagnostic event info for OS name, data store type, Node version * standardize linting * disallow window and document * fix null/undef checks * misc linting fixes * inlineUsersInEvents is not an unknown option * drop node-sha1 dependency * don't omit streamInits.failed when it's false * bump request dependency to get security patch; loosen some exact dependencies * remove request package; improve polling cache logic + add test * bump typescript version to fix build error in Node 6 * update @types/node to fix TypeScript check step * lint * make sure we keep polling regardless of whether we got new data * use launchdarkly-eventsource, make stream retry behavior consistent * stream retry delay option should be in seconds & should be included in diagnostics * minor test fix * fix: Throw an error on malformed user-supplied logger * don't call unref() on Redis client; ensure that database integration tests close the store * update Redis driver to major version 3 * add test case * allow redisOpts parameter to be omitted * add logger adapter shim + tests * minor cleanup and comments for ch74741 fix (logger wrapper) * fix proxy tunnel configuration and make sure it's used in streaming * change some string concatenation expressions to use interpolation * feat: upgrade winston (#189) * fix merge * remove support for indirect/patch and indirect/put (#182) * reuse same Promise and same event listeners for all waitForInitialization calls * better docs for waitForInitialization + misc doc cleanup (#184) * update js-eventsource to 1.3.1 for stream parsing bugfix (#185) * fix broken logger format (#186) * retroactively update changelog for bugfix in 5.13.2 release * allow get/getAll Redis queries to be queued if Redis client hasn't yet connected * set stream read timeout * adding the alias functionality (#190) * Removed the guides link * remove monkey-patching of setImmediate * Persist contextKind property during feature and custom event transformations (#194) * add inlineUsersInEvents option in TypeScript * Add support for seed to bucketUser * Add note for incorporating seed into evaluation * Send events when the evaluation is from an experiment * Use seed to evaluate. * Clean up test descriptions * Rename variable to be less confusing * Use ternary to eliminate mutation * Make return signature more consistent * Un-prettier the tests * redis lower bounds bump (#199) * update launchdarkly-js-test-helpers to fix TLS tests (#200) * update js-eventsource to remove vulnerability warning (#201) * add CI jobs for all compatible Node versions * CI fixes * more CI fixes * comment * use default value to simplify config * (6.0 - #1) stop saying we're compatible with Node <12 (#203) * add CI jobs for all compatible Node versions (#202) * (6.0 - #2) remove Redis integration (#204) * allow feature store to be specified as a factory (so it can get our logger) * (6.0 - #3) remove Winston (#205) * remove deprecated things for 6.0 (#206) * update node-cache to 5.x (drops old Node compat) * update semver to 7.x (drops old Node compat) * update uuid to 8.x (Node compat, perf improvements, bugfixes) * update dev dependencies * linter * replace lrucache package with lru-cache (#209) * make yaml dependency optional (#210) * update release metadata to include maintenance branch * remove package-lock.json (#211) * rm prerelease changelog * (big segments #1) add interfaces for big segments (#212) * (big segments #2) add all components for big segments except evaluation (#213) * (big segments #3) implement big segments in flag evaluation (#214) * (big segments #4) add standard test suite for big segment store tests + refactor feature store tests (#215) * move new interfaces to a module instead of a namespace (#216) * fix TS export of CachingStoreWrapper * use Releaser v2 config * fix overly specific test expectation that breaks in Node 17 * Initial work on FlagBuilder (#219) * Add TestData factory(with some dummy methods); Initial work on FlagBuilder * fixed indentation and linter errors; fixed an error in update; fixed incorrect test label * fixed typo in TestData store * converted boolean variation constants to be file variables instead of class variables Co-authored-by: charukiewicz <[email protected]> Co-authored-by: belevy <[email protected]> * implemented FlagRuleBuilder; added .build() methods to FlagBuilder/FlagRuleBuilder and changed tests to avoid using private interface * converted _targets to be Map instead of object literal; changed variationForBoolean to be a module-scoped function instead class-scoped * Implement stream processor(data source) interface for test data * Add TestData to index.js and write out the types for TestData and friends * added testdata documentation to index.d.ts; fix linter errors; changed flag default behavior to create boolean flag * Fix the interface file: reindented to 2 spaces, corrected definition of functions from properties to functions in interfaces; corrected issues in JSDoc comments * modify tests to fix capitalization and actually test the test datasource works as an LDClient updateProcessor. * Fix linter error on defaulted callback * explicitly enable JSDOM types in TypeScript build to avoid errors when jsdom is referenced for some reason * capitalize Big Segments in docs & logs * documentation comment fixes for TestData * pin TypeScript to 4.4.x * move TestData and FIleDataSource to integrations module * lint * rename types used by TestData for clarity (#229) * use varargs semantics for TestFlagBuilder.variations() and add it to the TS interface (#230) * don't ever use for...in (#232) * don't ever use for...in * add null guard * bump launchdarkly-eventsource dependency for sc-136154 fix * use TestData in our own tests (#231) * use TestData in our own tests * update TS interface * lint * typo * fix allFlagsState behavior regarding experimentation * lint * allow "secondary" to be referenced in clauses * don't throw an exception for non-string in semver comparison * correctly handle "client not ready" condition in allFlagsState * lint * Flags with a version of 0 reported as 'unknown' in summary events. (#239) * implement contract test service, not including big segments (#242) Co-authored-by: Eli Bishop <[email protected]> * Implement Application tags for the node SDK. (#241) * update js-eventsource to 1.4.4 for security fix * remove package-lock.json * adjust test expectation about error message to work in recent Node versions * Adds link to Relay Proxy docs * Update index.d.ts Co-authored-by: Eli Bishop <[email protected]> * ensure setTimeout task is cleared when polling is stopped * fix some flaky tests using async blocking logic * rm unused * simplify polling implementation using setInterval Co-authored-by: Eli Bishop <[email protected]> Co-authored-by: LaunchDarklyCI <[email protected]> Co-authored-by: Ben Woskow <[email protected]> Co-authored-by: Maxwell Gerber <[email protected]> Co-authored-by: Chris West <[email protected]> Co-authored-by: Ben Woskow <[email protected]> Co-authored-by: Mike Zorn <[email protected]> Co-authored-by: Robert J. Neal <[email protected]> Co-authored-by: Ben Levy <[email protected]> Co-authored-by: charukiewicz <[email protected]> Co-authored-by: belevy <[email protected]> Co-authored-by: charukiewicz <[email protected]> Co-authored-by: LaunchDarklyReleaseBot <[email protected]> Co-authored-by: Ryan Lamb <[email protected]> Co-authored-by: Ember Stevens <[email protected]> Co-authored-by: Ember Stevens <[email protected]>
launchdarkly · Apr 26, 2022 · 6f6ad33 · 6f6ad33
1 parent 1c20151
commit 6f6ad33
Show file tree

Hide file tree

Showing 8 changed files with 185 additions and 100 deletions.
diff --git a/index.d.ts b/index.d.ts
@@ -337,11 +337,12 @@ declare module 'launchdarkly-node-server-sdk' {
     streamInitialReconnectDelay?: number;
 
     /**
-     * Whether you are using the LaunchDarkly relay proxy in daemon mode.
+     * Whether you are using the LaunchDarkly Relay Proxy in daemon mode.
      *
      * In this configuration, the client will not connect to LaunchDarkly to get feature flags,
      * but will instead get feature state from a database (Redis or another supported feature
-     * store integration) that is populated by the relay. By default, this is false.
+     * store integration) that is populated by the Relay Proxy. By default, this is false.
+     * To learn more, read [Using daemon mode](https://docs.launchdarkly.com/home/relay-proxy/using#using-daemon-mode).
      */
     useLdd?: boolean;
 

diff --git a/polling.js b/polling.js
@@ -4,33 +4,31 @@ const dataKind = require('./versioned_data_kind');
 
 function PollingProcessor(config, requestor) {
   const processor = {},
-    featureStore = config.featureStore;
+    featureStore = config.featureStore,
+    intervalMs = config.pollInterval * 1000;
+
   let stopped = false;
 
+  let pollTask;
+
   function poll(maybeCallback) {
     const cb = maybeCallback || function () {};
 
     if (stopped) {
       return;
     }
 
-    const startTime = new Date().getTime();
     config.logger.debug('Polling LaunchDarkly for feature flag updates');
+
     requestor.requestAllData((err, respBody) => {
-      const elapsed = new Date().getTime() - startTime;
-      const sleepFor = Math.max(config.pollInterval * 1000 - elapsed, 0);
-      config.logger.debug('Elapsed: %d ms, sleeping for %d ms', elapsed, sleepFor);
       if (err) {
         if (err.status && !errors.isHttpErrorRecoverable(err.status)) {
           const message = messages.httpErrorMessage(err, 'polling request');
           config.logger.error(message);
           cb(new errors.LDPollingError(message));
+          processor.stop();
         } else {
           config.logger.warn(messages.httpErrorMessage(err, 'polling request', 'will retry'));
-          // Recursively call poll after the appropriate delay
-          setTimeout(() => {
-            poll(cb);
-          }, sleepFor);
         }
       } else {
         if (respBody) {
@@ -40,27 +38,26 @@ function PollingProcessor(config, requestor) {
           initData[dataKind.segments.namespace] = allData.segments;
           featureStore.init(initData, () => {
             cb();
-            // Recursively call poll after the appropriate delay
-            setTimeout(() => {
-              poll(cb);
-            }, sleepFor);
           });
-        } else {
-          // There wasn't an error but there wasn't any new data either, so just keep polling
-          setTimeout(() => {
-            poll(cb);
-          }, sleepFor);
         }
+        // There wasn't an error but there wasn't any new data either, so just keep polling
       }
     });
   }
 
   processor.start = cb => {
-    poll(cb);
+    if (!pollTask && !stopped) {
+      pollTask = setInterval(() => poll(cb), intervalMs);
+      // setInterval always waits for the delay before firing the first time, but we want to do an initial poll right away
+      poll(cb);
+    }
   };
 
   processor.stop = () => {
     stopped = true;
+    if (pollTask) {
+      clearInterval(pollTask);
+    }
   };
 
   processor.close = () => {

diff --git a/test/LDClient-tls-test.js b/test/LDClient-tls-test.js
@@ -2,12 +2,12 @@ import * as LDClient from '../index';
 
 import {
   AsyncQueue,
-  sleepAsync,
   TestHttpHandlers,
   TestHttpServer,
   withCloseable
 } from 'launchdarkly-js-test-helpers';
 import * as stubs from './stubs';
+import { failIfTimeout } from './test_helpers';
 
 describe('LDClient TLS configuration', () => {
   const sdkKey = 'secret';
@@ -36,18 +36,20 @@ describe('LDClient TLS configuration', () => {
     await withCloseable(TestHttpServer.startSecure, async server => {
       server.forMethodAndPath('get', '/sdk/latest-all', TestHttpHandlers.respondJson({}));
 
+      const logCapture = stubs.asyncLogCapture();
       const config = {
         baseUri: server.url,
         sendEvents: false,
         stream: false,
-        logger: stubs.stubLogger(),
+        logger: logCapture.logger,
         diagnosticOptOut: true,
       };
 
       await withCloseable(LDClient.init(sdkKey, config), async client => {
-        await sleepAsync(300); // the client won't signal an unrecoverable error, but it should log a message
-        expect(config.logger.warn.mock.calls.length).toEqual(2);
-        expect(config.logger.warn.mock.calls[1][0]).toMatch(/self.signed/);
+        const message1 = await failIfTimeout(logCapture.warn.take(), 1000);
+        expect(message1).toMatch(/only disable the streaming API/); // irrelevant message due to our use of polling mode
+        const message2 = await failIfTimeout(logCapture.warn.take(), 1000);
+        expect(message2).toMatch(/self.signed/);
       });
     });
   });

diff --git a/test/event_processor-test.js b/test/event_processor-test.js
@@ -1,6 +1,7 @@
 const { DiagnosticsManager, DiagnosticId } = require('../diagnostic_events');
 const EventProcessor = require('../event_processor');
-const { sleepAsync, TestHttpHandlers, TestHttpServer, withCloseable } = require('launchdarkly-js-test-helpers');
+const { TestHttpHandlers, TestHttpServer, withCloseable } = require('launchdarkly-js-test-helpers');
+const { failIfTimeout } = require('./test_helpers');
 
 describe('EventProcessor', () => {
 
@@ -656,8 +657,8 @@ describe('EventProcessor', () => {
       ep.sendEvent({ kind: 'identify', creationDate: 1000, user: user });
 
       // unfortunately we must wait for both the flush interval and the 1-second retry interval
-      await sleepAsync(1500);
-      expect(s.requestCount()).toEqual(2);
+      await failIfTimeout(s.nextRequest(), 500);
+      await failIfTimeout(s.nextRequest(), 1500);
     });
   }));
 

diff --git a/test/polling-test.js b/test/polling-test.js
@@ -1,8 +1,9 @@
 const InMemoryFeatureStore = require('../feature_store');
 const PollingProcessor = require('../polling');
 const dataKind = require('../versioned_data_kind');
-const { promisify, promisifySingle, sleepAsync } = require('launchdarkly-js-test-helpers');
+const { AsyncQueue, promisify, promisifySingle } = require('launchdarkly-js-test-helpers');
 const stubs = require('./stubs');
+const { failIfTimeout } = require('./test_helpers');
 
 describe('PollingProcessor', () => {
   const longInterval = 100000;
@@ -48,7 +49,8 @@ describe('PollingProcessor', () => {
     };
     processor = PollingProcessor(config, requestor);
 
-    await promisify(processor.start)(); // didn't throw -> success
+    const err = await new Promise(resolve => processor.start(resolve));
+    expect(err).not.toBe(expect.anything());
   });
 
   it('initializes feature store', async () => {
@@ -66,68 +68,90 @@ describe('PollingProcessor', () => {
   });
 
   it('polls repeatedly', async() => {
+    const calls = new AsyncQueue();
     const requestor = {
-      requestAllData: jest.fn(cb => cb(null, jsonData))
+      requestAllData: cb => {
+        calls.add();
+        cb(null, jsonData);
+      }
     };
-    config.pollInterval = 0.1;  // note, pollInterval is in seconds
+    config.pollInterval = 0.05;  // note, pollInterval is in seconds
     processor = PollingProcessor(config, requestor);
 
     processor.start(() => {});
-    await sleepAsync(500);
-
-    expect(requestor.requestAllData.mock.calls.length).toBeGreaterThanOrEqual(4);
+    const startTime = new Date().getTime();
+    for (let i = 0; i < 4; i++) {
+      await failIfTimeout(calls.take(), 500);
+    }
+    expect(new Date().getTime() - startTime).toBeLessThanOrEqual(500);
   });
 
   async function testRecoverableError(err) {
+    const calls = new AsyncQueue();
+    let count = 0;
     const requestor = {
-      requestAllData: jest.fn(cb => cb(err))
+      // The first two calls will return the error; the third will succeed.
+      requestAllData: cb => {
+        calls.add();
+        count++;
+        if (count > 2) {
+          cb(null, jsonData);
+        } else {
+          cb(err);
+        }
+      }
     };
-    config.pollInterval = 0.1;
+    config.pollInterval = 0.05;
     processor = PollingProcessor(config, requestor);
 
     let errReceived;
     processor.start(e => { errReceived = e; });
-    await sleepAsync(300);
 
-    expect(requestor.requestAllData.mock.calls.length).toBeGreaterThanOrEqual(2);
+    await failIfTimeout(calls.take(), 500);
+    await failIfTimeout(calls.take(), 500);
+    await failIfTimeout(calls.take(), 500);
+
     expect(config.logger.error).not.toHaveBeenCalled();
     expect(errReceived).toBeUndefined();
   }
 
-  function testRecoverableHttpError(status) {
+  it.each([400, 408, 429, 500, 503])(
+    'continues polling after error %d',
+    async (status) => {
+      const err = new Error('sorry');
+      err.status = status;
+      await testRecoverableError(err);
+    }
+  );
+
+  it('continues polling after I/O error', async () => await testRecoverableError(new Error('sorry')));
+
+  async function testUnrecoverableError(status) {
     const err = new Error('sorry');
     err.status = status;
-    it('continues polling after error ' + status, async () => await testRecoverableError(err));
-  }
 
-  testRecoverableHttpError(400);
-  testRecoverableHttpError(408);
-  testRecoverableHttpError(429);
-  testRecoverableHttpError(500);
-  testRecoverableHttpError(503);
+    const calls = new AsyncQueue();
+    const requestor = {
+      requestAllData: cb => {
+        calls.add();
+        cb(err);
+      }
+    };
+    config.pollInterval = 0.05;
+    processor = PollingProcessor(config, requestor);
 
-  it('continues polling after I/O error', async () => await testRecoverableError(new Error('sorry')));
+    const result = new AsyncQueue();
+    processor.start(e => result.add(e));
 
-  function testUnrecoverableHttpError(status) {
-    it('stops polling after error ' + status, async () => {
-      const err = new Error('sorry');
-      err.status = status;
-      const requestor = {
-        requestAllData: jest.fn(cb => cb(err))
-      };
-      config.pollInterval = 0.1;
-      processor = PollingProcessor(config, requestor);
-
-      let errReceived;
-      processor.start(e => { errReceived = e; });
-      await sleepAsync(300);
-
-      expect(requestor.requestAllData.mock.calls.length).toEqual(1);
-      expect(config.logger.error).toHaveBeenCalledTimes(1);
-      expect(errReceived).not.toBeUndefined();
-    });
-  }
+    const errReceived = await failIfTimeout(result.take(), 1000);
+    expect(errReceived.message).toMatch(new RegExp('error ' + status + '.*giving up permanently'));
 
-  testUnrecoverableHttpError(401);
-  testUnrecoverableHttpError(403);
+    expect(calls.length()).toEqual(1);
+    expect(config.logger.error).toHaveBeenCalledTimes(1);
+  }
+
+  it.each([401, 403])(
+    'stops polling after error %d',
+    testUnrecoverableError
+  );  
 });