Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delete rebuilds series index when series to be deleted are only found in cache #10503

Closed
hpbieker opened this issue Nov 20, 2018 · 3 comments
Closed
Assignees

Comments

@hpbieker
Copy link
Contributor

System info:
InfluxDB 1.7.0

Steps to reproduce:

  1. Start influxd with tsi1
  2. Run insert someMeasurement value=1 in the influx client to create a new series that only lives in the cache.
  3. Run delete from "someMeasurement" where time=0 in the influx client (which deletes nothing because I do not have any data with time=0)

Influx now decides to rebuild the series index files because the series "someMeasurement" does not exist in any tsm file so it is considered "deleted".

If I do a restart of InfluxDB between 2. and 3. it will not rebuild the series on the delete.

This issue is related to #9015, because it causes deletes to be very slow when the series only exist in cache. I get the profile below when running a query like delete from "someMeasurement" where time=0 in a loop.

image

Expected behavior:
The series index should only be rebuilt when we actually delete data.

Actual behavior:
The series index is rebuilt if it only exist in cache.

Additional info:

@hpbieker hpbieker changed the title Deletes rebuilds series that are only found in cache Delete rebuilds series that are only found in cache Nov 20, 2018
@hpbieker hpbieker changed the title Delete rebuilds series that are only found in cache Delete rebuilds series index when series to be deleted are only found in cache Nov 20, 2018
@hpbieker
Copy link
Contributor Author

hpbieker commented Dec 6, 2018

Did you have a chance to look at my PR @rbetts?

@dgnorton dgnorton added the 1.x label Jan 7, 2019
@stale
Copy link

stale bot commented Jul 24, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix label Jul 24, 2019
@stale
Copy link

stale bot commented Jul 31, 2019

This issue has been automatically closed because it has not had recent activity. Please reopen if this issue is still important to you. Thank you for your contributions.

@stale stale bot closed this as completed Jul 31, 2019
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue May 12, 2021
v1.8.4 [2021-01-27]
-------------------

### Bugfixes

-	[#19696](influxdata/influxdb#19697): fix(flux): add durations to Flux logging

v1.8.3 [2020-09-30]
-------------------

### Features

-	[#19187](influxdata/influxdb#19187): feat: Collect values written stats.
-	[#19611](influxdata/influxdb#19611): feat: Add -lponly flag to export sub-command.

### Bugfixes

-	[#19409](influxdata/influxdb#19409): chore: update uuid library from satori to gofrs.
-	[#19439](influxdata/influxdb#19439): fix(storage): ArrayFilterCursor truncation for multi-block data.
-	[#19460](influxdata/influxdb#19460): chore: Use latest version of influxql package.
-	[#19512](influxdata/influxdb#19512): chore: Quiet static analysis tools.
-	[#19592](influxdata/influxdb#19592): fix(services/storage): multi measurement queries return all applicable series.
-	[#19612](influxdata/influxdb#19612): fix: lock map before writes.

v1.8.2 [2020-08-13]
-------------------

### Bugfixes

-	[#19253](influxdata/influxdb#19253): fix(tsdb): Revert disable series id set cache size by default.

v1.8.1 [2020-07-08]
-------------------

### Features

-	[#18457](influxdata/influxdb#18457): feat(query): Parallelize field iterator planning.
-	[#18886](influxdata/influxdb#18886): feat(http): Allow user supplied HTTP headers.

### Bugfixes

-	[#17319](influxdata/influxdb#17319): fix(flux): buckets call no longer panics.
-	[#18212](influxdata/influxdb#18212): fix(tsdb): Defer closing of underlying SeriesIDSetIterators.
-	[#18286](influxdata/influxdb#18286): fix(tsdb): Disable series id set cache size by default.
-	[#18299](influxdata/influxdb#18299): refactor(http): Simplify Authorizer.
-	[#18694](influxdata/influxdb#18694): fix(tsi1): wait deleting epoch before dropping shard.
-	[#18687](influxdata/influxdb#18687): perf(tsi1): batch write tombstone entries when dropping/deleting.
-	[#18826](influxdata/influxdb#18826): fix: gracefully handle errors when creating snapshots.
-	[#18849](influxdata/influxdb#18849): chore(build): remove all of the go1.12 references from build.

v1.8.0 [2020-04-11]
-------------------

### Features

-	[#15952](influxdata/influxdb#15952): Add influx_inspect verify-tombstone tool.
-	[#16542](influxdata/influxdb#16542): Add offline series compaction to influx_inspect buildtsi.
-	[#16599](influxdata/influxdb#16599): Make influx CLI support custom HTTP endpoint.
-	[#16908](influxdata/influxdb#16908): Add support for InfluxDB 2.0 write API.
-	[#17621](influxdata/influxdb#17621): Update Flux to v0.65.0.
-	[#17188](influxdata/influxdb#17188): Enhance support for bound parameters.

### Bugfixes

-	[#10503](influxdata/influxdb#10503): Delete rebuilds series index when series to be deleted are only found in cache.
-	[#10504](https://github.com/influxdata/influxdb/issue/10504): Delete rebuilds series index when series to be deleted are outside timerange.
-	[#14485](influxdata/influxdb#14485): Parse Accept header correctly.
-	[#16524](influxdata/influxdb#16524): Upgrade compaction error log from `Info` to `Warn`.
-	[#16525](influxdata/influxdb#16525): Remove double increment of meta index.
-	[#16595](influxdata/influxdb#16595): Improve series cardinality limit for inmem index.
-	[#16606](influxdata/influxdb#16606): Ensure all block data returned.
-	[#16627](influxdata/influxdb#16627): Skip WriteSnapshot during backup if snapshotter is busy.
-	[#16709](influxdata/influxdb#16709): Reduce influxd and influx startup time if Flux isn't used.
-	[#16762](influxdata/influxdb#16762): Fix bugs in -compact-series-file.
-	[#16944](influxdata/influxdb#16944): Update to Go 1.13.8 and Go modules.
-	[#17032](influxdata/influxdb#17032): Fix a SIGSEGV when accessing tsi active log.
-	[#17656](influxdata/influxdb#17656): Verify precision in write requests.
-	[#17698](influxdata/influxdb#17698): Enable configuration of TLS 1.3.
jperkin added a commit to TritonDataCenter/pkgsrc that referenced this issue May 12, 2021
v1.8.4 [2021-01-27]
-------------------

### Bugfixes

-	[#19696](influxdata/influxdb#19697): fix(flux): add durations to Flux logging

v1.8.3 [2020-09-30]
-------------------

### Features

-	[#19187](influxdata/influxdb#19187): feat: Collect values written stats.
-	[#19611](influxdata/influxdb#19611): feat: Add -lponly flag to export sub-command.

### Bugfixes

-	[#19409](influxdata/influxdb#19409): chore: update uuid library from satori to gofrs.
-	[#19439](influxdata/influxdb#19439): fix(storage): ArrayFilterCursor truncation for multi-block data.
-	[#19460](influxdata/influxdb#19460): chore: Use latest version of influxql package.
-	[#19512](influxdata/influxdb#19512): chore: Quiet static analysis tools.
-	[#19592](influxdata/influxdb#19592): fix(services/storage): multi measurement queries return all applicable series.
-	[#19612](influxdata/influxdb#19612): fix: lock map before writes.

v1.8.2 [2020-08-13]
-------------------

### Bugfixes

-	[#19253](influxdata/influxdb#19253): fix(tsdb): Revert disable series id set cache size by default.

v1.8.1 [2020-07-08]
-------------------

### Features

-	[#18457](influxdata/influxdb#18457): feat(query): Parallelize field iterator planning.
-	[#18886](influxdata/influxdb#18886): feat(http): Allow user supplied HTTP headers.

### Bugfixes

-	[#17319](influxdata/influxdb#17319): fix(flux): buckets call no longer panics.
-	[#18212](influxdata/influxdb#18212): fix(tsdb): Defer closing of underlying SeriesIDSetIterators.
-	[#18286](influxdata/influxdb#18286): fix(tsdb): Disable series id set cache size by default.
-	[#18299](influxdata/influxdb#18299): refactor(http): Simplify Authorizer.
-	[#18694](influxdata/influxdb#18694): fix(tsi1): wait deleting epoch before dropping shard.
-	[#18687](influxdata/influxdb#18687): perf(tsi1): batch write tombstone entries when dropping/deleting.
-	[#18826](influxdata/influxdb#18826): fix: gracefully handle errors when creating snapshots.
-	[#18849](influxdata/influxdb#18849): chore(build): remove all of the go1.12 references from build.

v1.8.0 [2020-04-11]
-------------------

### Features

-	[#15952](influxdata/influxdb#15952): Add influx_inspect verify-tombstone tool.
-	[#16542](influxdata/influxdb#16542): Add offline series compaction to influx_inspect buildtsi.
-	[#16599](influxdata/influxdb#16599): Make influx CLI support custom HTTP endpoint.
-	[#16908](influxdata/influxdb#16908): Add support for InfluxDB 2.0 write API.
-	[#17621](influxdata/influxdb#17621): Update Flux to v0.65.0.
-	[#17188](influxdata/influxdb#17188): Enhance support for bound parameters.

### Bugfixes

-	[#10503](influxdata/influxdb#10503): Delete rebuilds series index when series to be deleted are only found in cache.
-	[#10504](https://github.com/influxdata/influxdb/issue/10504): Delete rebuilds series index when series to be deleted are outside timerange.
-	[#14485](influxdata/influxdb#14485): Parse Accept header correctly.
-	[#16524](influxdata/influxdb#16524): Upgrade compaction error log from `Info` to `Warn`.
-	[#16525](influxdata/influxdb#16525): Remove double increment of meta index.
-	[#16595](influxdata/influxdb#16595): Improve series cardinality limit for inmem index.
-	[#16606](influxdata/influxdb#16606): Ensure all block data returned.
-	[#16627](influxdata/influxdb#16627): Skip WriteSnapshot during backup if snapshotter is busy.
-	[#16709](influxdata/influxdb#16709): Reduce influxd and influx startup time if Flux isn't used.
-	[#16762](influxdata/influxdb#16762): Fix bugs in -compact-series-file.
-	[#16944](influxdata/influxdb#16944): Update to Go 1.13.8 and Go modules.
-	[#17032](influxdata/influxdb#17032): Fix a SIGSEGV when accessing tsi active log.
-	[#17656](influxdata/influxdb#17656): Verify precision in write requests.
-	[#17698](influxdata/influxdb#17698): Enable configuration of TLS 1.3.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants