Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Final rows lost with batch mode of PendingQuery.cursor #150

Closed
MPvHarmelen opened this issue Feb 15, 2021 · 3 comments
Closed

Final rows lost with batch mode of PendingQuery.cursor #150

MPvHarmelen opened this issue Feb 15, 2021 · 3 comments
Labels
bug Something isn't working

Comments

@MPvHarmelen
Copy link

If the number of results isn't divisible by the batch size, the last (and incomplete) batch is never processed:

const p = require("postgres");
const q = p("postgresql://postgres:example@localhost/demo");

// this works and prints all 9 rows
console.log("a:")
const a = q`select * from "aircrafts"`.cursor(console.log)

setTimeout(() => {
  // this only prinst the first 5 rows
  console.log("b:")
  const b = q`select * from "aircrafts"`.cursor(5, console.log);
}, 1000);

setTimeout(() => {
  // this only prinst the first 8 rows
  console.log("c:")
  const c = q`select * from "aircrafts"`.cursor(2, console.log);
}, 2000);

setTimeout(q.end, 3000)
Setting up
# docker-compose.yml
# Use postgres/example user/password credentials
version: '3.1'

services:

  db:
    image: postgres:9
    restart: always
    environment:
      POSTGRES_PASSWORD: example
    volumes:
      - ${PWD}/data:/var/lib/postgresql/data
    ports:
      - 5432:5432
  • Start postgres
mkdir -p ~/postgres/data
# copy above docker-compose.yml to ~/postgres/docker-compose.yml
cd ~/postgres;
docker-compose up -d;
  • Add a database called demo to postgres
docker-bash
# select the postgres db container
# install some dependencies
apt update
apt install curl unzip -y
# download a test db dump
curl https://edu.postgrespro.com/demo-small-en.zip > demo.zip
unzip demo.zip
# restore the test db
PGPASSWORD=example psql --username=postgres  -f demo-small-en-20170815.sql
@Minigugus
Copy link
Contributor

Minigugus commented Feb 15, 2021

Seems to be related to #81. If it's the same problem, the last (and incomplete) batch should be available in c; it is returned by q`select * from "aircrafts"`.cursor(2, console.log); as the result of the promise without being passed to console.log.

@MPvHarmelen
Copy link
Author

Ah yes, they are most definitely caused by the same bug! Thanks for the workaround.

@porsager porsager added the bug Something isn't working label Mar 23, 2021
@naumf
Copy link

naumf commented May 18, 2021

Same happens if batch size is greater than the result count.
With the above example that has 9 resulting rows, if batch size is 10 or greater, no rows are returned.
@Minigugus's workaround in #81 works with both (currently non-working) use-cases.
@porsager Do you plan to release a fix with his solution, or maybe you have another idea?

EDIT: @porsager Just created the PR #184 to address this issue. Please have a look at it when you have a spare time.

p.s. Tested with v2.0.0-beta.5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants