ysql: preserve header upon history file truncation

[jaki]

Jira Link: DB-2106
ysqlsh history file can be specified by --variable=HISTFILE=/path/to/file. It seems that the default amount of entries (or lines) it keeps is 500. Once it goes past that, upon history file write, it truncates the file. However, it also truncates the header line _HiStOrY_V2_. This makes the file invalid next time it's read. This is possibly related to YB using libedit rather than libreadline (commit d372e88). I have not tested upstream postgres, but it should be tested with and without --with-libedit-preferred.

Repro:

First, copy 500 dull commands to the clipboard.

for _ in {1..500}; do echo ";"; done | myclipboardcommand

Then, start ysqlsh and paste the contents. End the session with control D.

./bin/ysqlsh --variable=HISTFILE=test_history.sql

I specifically make the commands very short because it makes it more reliable. In my testing, long commands may somehow mush up with adjacent ones.

Check the history file for 500 entries and the header:

wc test_history.sql

 501  501 1013 test_history.sql

head -2 test_history.sql

_HiStOrY_V2_
;

Now, add one more entry to the history by running any one command in ysqlsh (with the same HISTFILE variable). Upon closing the session, the history file is truncated. This can be noticed by tail -F test_history.sql while closing the ysqlsh session.

tail: test_history.sql: file truncated

Notice that the header is gone

head -2 test_history.sql

;
;

Try ysqlsh again, and the up arrow or ctrl R will not bring up the history. Interestingly, the current session's history will still be written.

select 1;

tail -2 test_history.sql

;
select\0401;

[Akshat1331]

Hi @jaki can contribute to this issue it is my first contribution so can you help me a little bit?

[jaki]

My suggestion is to first try to reproduce this on upstream PostgreSQL. As I mention, it should be tested with and without --with-libedit-preferred. We use PostgreSQL 11.2 with some imported patches, so you can test PostgreSQL 11.2.

If upstream PostgreSQL does not have the same issue, we can move forward with trying to fix it here.

[Akshat1331]

Thank you for your help.

…

On Thu, 2 Sep 2021, 23:11 Jason Kim, ***@***.***> wrote: My suggestion is to first try to reproduce this on upstream PostgreSQL. As I mention, it should be tested with and without --with-libedit-preferred. We use PostgreSQL 11.2 with some imported patches, so you can test PostgreSQL 11.2. If upstream PostgreSQL does not have the same issue, we can move forward with trying to fix it here. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#9861 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQLMPVQMZRH3NIWXF67CEZDT76ZMBANCNFSM5C6WHFUQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[PriyanshX1902]

@jaki is this still open? Care if i take it?

[jaki]

@PriyanshX1902, yes, it is still open. Anyone is free to work on it.