Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker: Video record extract last session from json response #2602

Merged
merged 2 commits into from
Jan 21, 2025

Conversation

VietND96
Copy link
Member

@VietND96 VietND96 commented Jan 21, 2025

User description

Thanks for contributing to the Docker-Selenium project!
A PR well described will help maintainers to quickly review and merge it

Before submitting your PR, please check our contributing guidelines, applied for this repository.
Avoid large PRs, help reviewers by making them as simple and short as possible.

Description

Motivation and Context

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • I have read the contributing document.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

PR Type

Bug fix, Enhancement


Description

  • Updated session ID extraction logic for video recording.

  • Adjusted retry time for GraphQL query script.

  • Improved handling of multiple session scenarios in video recording.


Changes walkthrough 📝

Relevant files
Enhancement
video.sh
Refine session ID extraction logic                                             

Video/video.sh

  • Updated session ID query to extract the last session.
  • Adjusted logic for standalone and non-standalone modes.
  • Improved handling of session extraction for multiple sessions.
  • +2/-2     
    Bug fix
    video_graphQLQuery.sh
    Adjust retry time for GraphQL queries                                       

    Video/video_graphQLQuery.sh

  • Reduced retry time for GraphQL queries from 3 to 2.
  • Simplified retry logic for session handling.
  • +1/-1     

    Need help?
  • Type /help how to ... in the comments thread for any question about Qodo Merge usage.
  • Check out the documentation for more information.
  • Copy link

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
    🧪 No relevant tests
    🔒 No security concerns identified
    ⚡ Recommended focus areas for review

    Session Selection

    The change to select the last session (using [-1] index) assumes the most recent session is always last in the array. This assumption should be validated to ensure proper session selection.

      JQ_SESSION_ID_QUERY=".value.nodes[-1]?.slots[-1]?.session?.sessionId"
      SE_NODE_PORT=${SE_NODE_PORT:-"4444"}
      NODE_STATUS_ENDPOINT="$(/opt/bin/video_gridUrl.sh)/status"
    else
      JQ_SESSION_ID_QUERY=".[-1]?.node?.slots | .[-1]?.session?.sessionId"

    Copy link

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Score
    Possible issue
    Add null check for empty arrays

    Consider adding null check before accessing array indices to prevent script failure
    when no sessions are available. The current implementation might fail if the JSON
    response is empty.

    Video/video.sh [25]

    -JQ_SESSION_ID_QUERY=".value.nodes[-1]?.slots[-1]?.session?.sessionId"
    +JQ_SESSION_ID_QUERY="if (.value.nodes | length > 0) then .value.nodes[-1]?.slots[-1]?.session?.sessionId else null end"
    • Apply this suggestion
    Suggestion importance[1-10]: 8

    Why: The suggestion addresses a potential runtime error by adding proper null checks for empty arrays, which is crucial for script stability and error prevention in production environments.

    8
    General
    Make retry timing configurable

    The retry time reduction from 3 to 2 seconds might be too aggressive for slower
    networks or high-load scenarios. Consider making this configurable via an
    environment variable.

    Video/video_graphQLQuery.sh [4]

    -retry_time=2
    +retry_time=${SE_GRAPHQL_RETRY_TIME:-2}
    • Apply this suggestion
    Suggestion importance[1-10]: 6

    Why: Making the retry time configurable via environment variable improves flexibility and allows for runtime adjustments based on network conditions, though it's not addressing a critical issue.

    6

    Copy link

    qodo-merge-pro bot commented Jan 21, 2025

    CI Feedback 🧐

    (Feedback updated until commit 6fc20a4)

    A test triggered by this PR failed. Here is an AI-generated analysis of the failure:

    Action: Test Selenium Grid on Docker / Test Docker Selenium (test_node_relay, false, false, false, ubuntu-24.04-arm, true, true)

    Failed stage: Set up containerd image store feature [❌]

    Failed test name: setup_dev_env

    Failure summary:

    The action failed during the setup of the development environment due to Docker service issues:

  • Docker service failed to start and restart multiple times with timeout errors
  • The systemd service timed out repeatedly (service_start_timeout=25000ms)
  • The make command 'setup_dev_env' failed after 3 retry attempts due to inability to set up containerd
    image store
  • Root cause appears to be systemd service activation failures preventing Docker from properly
    initializing

  • Relevant error logs:
    1:  ##[group]Operating System
    2:  Ubuntu
    ...
    
    159:  �[36;1mfi�[0m
    160:  �[36;1m�[0m
    161:  �[36;1m# Option: Remove large packages�[0m
    162:  �[36;1m# REF: https://github.com/apache/flink/blob/master/tools/azure-pipelines/free_disk_space.sh�[0m
    163:  �[36;1m�[0m
    164:  �[36;1mif [[ false == 'true' ]]; then�[0m
    165:  �[36;1m  BEFORE=$(getAvailableSpace)�[0m
    166:  �[36;1m  �[0m
    167:  �[36;1m  sudo apt-get remove -y '^aspnetcore-.*' || echo "::warning::The command [sudo apt-get remove -y '^aspnetcore-.*'] failed to complete successfully. Proceeding..."�[0m
    168:  �[36;1m  sudo apt-get remove -y '^dotnet-.*' --fix-missing || echo "::warning::The command [sudo apt-get remove -y '^dotnet-.*' --fix-missing] failed to complete successfully. Proceeding..."�[0m
    169:  �[36;1m  sudo apt-get remove -y '^llvm-.*' --fix-missing || echo "::warning::The command [sudo apt-get remove -y '^llvm-.*' --fix-missing] failed to complete successfully. Proceeding..."�[0m
    170:  �[36;1m  sudo apt-get remove -y 'php.*' --fix-missing || echo "::warning::The command [sudo apt-get remove -y 'php.*' --fix-missing] failed to complete successfully. Proceeding..."�[0m
    171:  �[36;1m  sudo apt-get remove -y '^mongodb-.*' --fix-missing || echo "::warning::The command [sudo apt-get remove -y '^mongodb-.*' --fix-missing] failed to complete successfully. Proceeding..."�[0m
    172:  �[36;1m  sudo apt-get remove -y '^mysql-.*' --fix-missing || echo "::warning::The command [sudo apt-get remove -y '^mysql-.*' --fix-missing] failed to complete successfully. Proceeding..."�[0m
    173:  �[36;1m  sudo apt-get remove -y azure-cli google-chrome-stable firefox powershell mono-devel libgl1-mesa-dri --fix-missing || echo "::warning::The command [sudo apt-get remove -y azure-cli google-chrome-stable firefox powershell mono-devel libgl1-mesa-dri --fix-missing] failed to complete successfully. Proceeding..."�[0m
    174:  �[36;1m  sudo apt-get remove -y google-cloud-sdk --fix-missing || echo "::debug::The command [sudo apt-get remove -y google-cloud-sdk --fix-missing] failed to complete successfully. Proceeding..."�[0m
    175:  �[36;1m  sudo apt-get remove -y google-cloud-cli --fix-missing || echo "::debug::The command [sudo apt-get remove -y google-cloud-cli --fix-missing] failed to complete successfully. Proceeding..."�[0m
    176:  �[36;1m  sudo apt-get autoremove -y || echo "::warning::The command [sudo apt-get autoremove -y] failed to complete successfully. Proceeding..."�[0m
    177:  �[36;1m  sudo apt-get clean || echo "::warning::The command [sudo apt-get clean] failed to complete successfully. Proceeding..."�[0m
    ...
    
    416:  shfmt -l -w -d ${PWD}/*.sh ${PWD}/**/*.sh $PWD/**.sh $PWD/**/generate_** $PWD/**/wrap_* ; \
    417:  git diff --stat --exit-code ; \
    418:  EXIT_CODE=$? ; \
    419:  if [ $EXIT_CODE -ne 0 ]; then \
    420:  echo "Some shell scripts are not formatted. Please run 'make format_shell_scripts' to format and update them." ; \
    421:  exit $EXIT_CODE ; \
    422:  fi ; \
    423:  exit $EXIT_CODE
    424:  Error: Timeout was reached
    ...
    
    454:  (Reading database ... 90%
    455:  (Reading database ... 95%
    456:  (Reading database ... 100%
    457:  (Reading database ... 233178 files and directories currently installed.)
    458:  Preparing to unpack .../shfmt_3.8.0-1_arm64.deb ...
    459:  Unpacking shfmt (3.8.0-1) ...
    460:  Setting up shfmt (3.8.0-1) ...
    461:  Processing triggers for man-db (2.12.0-4build2) ...
    462:  Error: Timeout was reached
    ...
    
    464:  with:
    465:  timeout_minutes: 10
    466:  max_attempts: 3
    467:  command: make setup_dev_env
    468:  
    469:  retry_wait_seconds: 10
    470:  polling_interval_seconds: 1
    471:  warning_on_retry: true
    472:  continue_on_error: false
    473:  ##[endgroup]
    474:  ./tests/charts/make/chart_setup_env.sh ; \
    475:  make set_containerd_image_store
    476:  Set ENV variables
    477:  Installing Docker for AMD64 / ARM64
    478:  Error: Timeout was reached
    ...
    
    521:  Unpacking libcurl4t64:arm64 (8.5.0-2ubuntu10.6) over (8.5.0-2ubuntu10.5) ...
    522:  Preparing to unpack .../libcurl3t64-gnutls_8.5.0-2ubuntu10.6_arm64.deb ...
    523:  Unpacking libcurl3t64-gnutls:arm64 (8.5.0-2ubuntu10.6) over (8.5.0-2ubuntu10.5) ...
    524:  Setting up libcurl4t64:arm64 (8.5.0-2ubuntu10.6) ...
    525:  Setting up libcurl3t64-gnutls:arm64 (8.5.0-2ubuntu10.6) ...
    526:  Setting up curl (8.5.0-2ubuntu10.6) ...
    527:  Processing triggers for man-db (2.12.0-4build2) ...
    528:  Processing triggers for libc-bin (2.39-0ubuntu8.3) ...
    529:  Error: Timeout was reached
    530:  Error: Timeout was reached
    ...
    
    566:  (Reading database ... 100%
    567:  (Reading database ... 233183 files and directories currently installed.)
    568:  Preparing to unpack .../docker-ce-cli_5%3a27.5.0-1~ubuntu.24.04~noble_arm64.deb ...
    569:  Unpacking docker-ce-cli (5:27.5.0-1~ubuntu.24.04~noble) over (5:26.1.3-1~ubuntu.24.04~noble) ...
    570:  Preparing to unpack .../docker-ce_5%3a27.5.0-1~ubuntu.24.04~noble_arm64.deb ...
    571:  Unpacking docker-ce (5:27.5.0-1~ubuntu.24.04~noble) over (5:26.1.3-1~ubuntu.24.04~noble) ...
    572:  Setting up docker-ce-cli (5:27.5.0-1~ubuntu.24.04~noble) ...
    573:  Setting up docker-ce (5:27.5.0-1~ubuntu.24.04~noble) ...
    574:  Reload daemon failed: Failed to activate service 'org.freedesktop.systemd1': timed out (service_start_timeout=25000ms)
    575:  �[0;1;31mFailed to retrieve unit state: Failed to activate service 'org.freedesktop.systemd1': timed out (service_start_timeout=25000ms)�[0m
    576:  invoke-rc.d: could not determine current runlevel
    577:  Failed to get properties: Connection timed out
    578:  Reload daemon failed: Failed to activate service 'org.freedesktop.systemd1': timed out (service_start_timeout=25000ms)
    579:  Failed to get unit file state for docker.service: Connection timed out
    580:  Failed to retrieve unit state: Connection timed out
    581:  docker.service is a disabled or a static unit not running, not starting it.
    582:  Failed to get unit file state for docker.socket: Connection timed out
    583:  Failed to retrieve unit state: Connection timed out
    584:  docker.socket is a disabled or a static unit not running, not starting it.
    585:  Processing triggers for man-db (2.12.0-4build2) ...
    586:  Error: Timeout was reached
    ...
    
    632:  Preparing to unpack .../docker-compose-plugin_2.32.4-1~ubuntu.24.04~noble_arm64.deb ...
    633:  Unpacking docker-compose-plugin (2.32.4-1~ubuntu.24.04~noble) ...
    634:  Selecting previously unselected package qemu-user-static.
    635:  Preparing to unpack .../qemu-user-static_1%3a8.2.2+ds-0ubuntu1.4_arm64.deb ...
    636:  Unpacking qemu-user-static (1:8.2.2+ds-0ubuntu1.4) ...
    637:  Setting up qemu-user-static (1:8.2.2+ds-0ubuntu1.4) ...
    638:  Setting up docker-buildx-plugin (0.19.3-1~ubuntu.24.04~noble) ...
    639:  Setting up containerd.io (1.7.25-1) ...
    640:  Reload daemon failed: Failed to activate service 'org.freedesktop.systemd1': timed out (service_start_timeout=25000ms)
    641:  Failed to get unit file state for containerd.service: Connection timed out
    642:  Failed to retrieve unit state: Failed to activate service 'org.freedesktop.systemd1': timed out (service_start_timeout=25000ms)
    643:  containerd.service is a disabled or a static unit not running, not starting it.
    644:  Setting up docker-compose-plugin (2.32.4-1~ubuntu.24.04~noble) ...
    645:  Processing triggers for man-db (2.12.0-4build2) ...
    646:  Processing triggers for systemd (255.4-1ubuntu8.4) ...
    647:  �[0;1;31mFailed to restart systemd-binfmt.service: Connection timed out�[0m
    648:  �[0;1;31mSee system logs and 'systemctl status systemd-binfmt.service' for details.�[0m
    649:  Error: Timeout was reached
    ...
    
    695:  24752f2a9b6d: Pull complete
    696:  aa73ede2d57b: Verifying Checksum
    697:  aa73ede2d57b: Download complete
    698:  1bb83505e71f: Pull complete
    699:  b9922d81f8dd: Pull complete
    700:  aa73ede2d57b: Pull complete
    701:  Digest: sha256:9511e9363ce42fd18b9448151b1e0f6323eed396c5279295080375e01cc1b626
    702:  Status: Downloaded newer image for aptman/qus:latest
    703:  docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: unable to apply cgroup configuration: unable to start unit "docker-53899d92bd191e988b071528746ad977f8f6ffa54bd8050fd05339786a79d69b.scope" (properties [{Name:Description Value:"libcontainer container 53899d92bd191e988b071528746ad977f8f6ffa54bd8050fd05339786a79d69b"} {Name:Slice Value:"system.slice"} {Name:Delegate Value:true} {Name:PIDs Value:@au [7790]} {Name:MemoryAccounting Value:true} {Name:CPUAccounting Value:true} {Name:IOAccounting Value:true} {Name:TasksAccounting Value:true} {Name:DefaultDependencies Value:false}]): Failed to activate service 'org.freedesktop.systemd1': timed out (service_start_timeout=25000ms): unknown.
    704:  There is step failed with exit status 125
    705:  make[1]: Entering directory '/home/runner/work/docker-selenium/docker-selenium'
    706:  sudo mkdir -p /etc/docker
    707:  sudo mv /etc/docker/daemon.json /etc/docker/daemon.json.bak || true
    708:  mv: cannot stat '/etc/docker/daemon.json': No such file or directory
    709:  echo "{\"features\":{\"containerd-snapshotter\": true, \"containerd\": true}, \"experimental\": true}" | sudo tee /etc/docker/daemon.json
    710:  {"features":{"containerd-snapshotter": true, "containerd": true}, "experimental": true}
    711:  sudo systemctl restart docker
    712:  Failed to restart docker.service: Failed to activate service 'org.freedesktop.systemd1': timed out (service_start_timeout=25000ms)
    713:  See system logs and 'systemctl status docker.service' for details.
    714:  make[1]: *** [Makefile:67: set_containerd_image_store] Error 1
    715:  make[1]: Leaving directory '/home/runner/work/docker-selenium/docker-selenium'
    716:  make: *** [Makefile:60: setup_dev_env] Error 2
    717:  ##[warning]Attempt 1 failed. Reason: Child_process exited with error code 2
    718:  ./tests/charts/make/chart_setup_env.sh ; \
    719:  make set_containerd_image_store
    720:  Set ENV variables
    721:  Installing Docker for AMD64 / ARM64
    722:  Error: Timeout was reached
    723:  Reading package lists...
    724:  Building dependency tree...
    725:  Reading state information...
    726:  ca-certificates is already the newest version (20240203).
    727:  curl is already the newest version (8.5.0-2ubuntu10.6).
    728:  wget is already the newest version (1.21.4-1ubuntu4.1).
    729:  jq is already the newest version (1.7.1-3build1).
    730:  0 upgraded, 0 newly installed, 0 to remove and 145 not upgraded.
    731:  Error: Timeout was reached
    ...
    
    766:  GitCommit:        57f17b0a6295a39009d861b89e3b3b87b005ca27
    767:  runc:
    768:  Version:          1.2.4
    769:  GitCommit:        v1.2.4-0-g6c52b3f
    770:  docker-init:
    771:  Version:          0.19.0
    772:  GitCommit:        de40ad0
    773:  github.com/docker/buildx v0.19.3 48d6a39
    774:  docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: unable to apply cgroup configuration: unable to start unit "docker-82379f4fa4e56868c9afe5ef6be898eda8c156ef9d02563dc630279527cd4adc.scope" (properties [{Name:Description Value:"libcontainer container 82379f4fa4e56868c9afe5ef6be898eda8c156ef9d02563dc630279527cd4adc"} {Name:Slice Value:"system.slice"} {Name:Delegate Value:true} {Name:PIDs Value:@au [8606]} {Name:MemoryAccounting Value:true} {Name:CPUAccounting Value:true} {Name:IOAccounting Value:true} {Name:TasksAccounting Value:true} {Name:DefaultDependencies Value:false}]): Failed to activate service 'org.freedesktop.systemd1': timed out (service_start_timeout=25000ms): unknown.
    775:  There is step failed with exit status 125
    776:  make[1]: Entering directory '/home/runner/work/docker-selenium/docker-selenium'
    777:  sudo mkdir -p /etc/docker
    778:  sudo mv /etc/docker/daemon.json /etc/docker/daemon.json.bak || true
    779:  echo "{\"features\":{\"containerd-snapshotter\": true, \"containerd\": true}, \"experimental\": true}" | sudo tee /etc/docker/daemon.json
    780:  {"features":{"containerd-snapshotter": true, "containerd": true}, "experimental": true}
    781:  sudo systemctl restart docker
    782:  Failed to restart docker.service: Connection timed out
    783:  See system logs and 'systemctl status docker.service' for details.
    784:  make[1]: *** [Makefile:67: set_containerd_image_store] Error 1
    785:  make[1]: Leaving directory '/home/runner/work/docker-selenium/docker-selenium'
    786:  make: *** [Makefile:60: setup_dev_env] Error 2
    787:  ##[warning]Attempt 2 failed. Reason: Child_process exited with error code 2
    788:  ./tests/charts/make/chart_setup_env.sh ; \
    789:  make set_containerd_image_store
    790:  Set ENV variables
    791:  Installing Docker for AMD64 / ARM64
    792:  Error: Timeout was reached
    793:  Reading package lists...
    794:  Building dependency tree...
    795:  Reading state information...
    796:  ca-certificates is already the newest version (20240203).
    797:  curl is already the newest version (8.5.0-2ubuntu10.6).
    798:  wget is already the newest version (1.21.4-1ubuntu4.1).
    799:  jq is already the newest version (1.7.1-3build1).
    800:  0 upgraded, 0 newly installed, 0 to remove and 145 not upgraded.
    801:  Error: Timeout was reached
    ...
    
    836:  GitCommit:        57f17b0a6295a39009d861b89e3b3b87b005ca27
    837:  runc:
    838:  Version:          1.2.4
    839:  GitCommit:        v1.2.4-0-g6c52b3f
    840:  docker-init:
    841:  Version:          0.19.0
    842:  GitCommit:        de40ad0
    843:  github.com/docker/buildx v0.19.3 48d6a39
    844:  docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: unable to apply cgroup configuration: unable to start unit "docker-0a903ac75dc17af01a167adafa3c32fbabe9a728a8ff76c76c2b708de114ff43.scope" (properties [{Name:Description Value:"libcontainer container 0a903ac75dc17af01a167adafa3c32fbabe9a728a8ff76c76c2b708de114ff43"} {Name:Slice Value:"system.slice"} {Name:Delegate Value:true} {Name:PIDs Value:@au [9416]} {Name:MemoryAccounting Value:true} {Name:CPUAccounting Value:true} {Name:IOAccounting Value:true} {Name:TasksAccounting Value:true} {Name:DefaultDependencies Value:false}]): Failed to activate service 'org.freedesktop.systemd1': timed out (service_start_timeout=25000ms): unknown.
    845:  There is step failed with exit status 125
    846:  make[1]: Entering directory '/home/runner/work/docker-selenium/docker-selenium'
    847:  sudo mkdir -p /etc/docker
    848:  sudo mv /etc/docker/daemon.json /etc/docker/daemon.json.bak || true
    849:  echo "{\"features\":{\"containerd-snapshotter\": true, \"containerd\": true}, \"experimental\": true}" | sudo tee /etc/docker/daemon.json
    850:  {"features":{"containerd-snapshotter": true, "containerd": true}, "experimental": true}
    851:  sudo systemctl restart docker
    852:  Failed to restart docker.service: Connection timed out
    853:  See system logs and 'systemctl status docker.service' for details.
    854:  make[1]: *** [Makefile:67: set_containerd_image_store] Error 1
    855:  make[1]: Leaving directory '/home/runner/work/docker-selenium/docker-selenium'
    856:  make: *** [Makefile:60: setup_dev_env] Error 2
    857:  ##[error]Final attempt failed. Child_process exited with error code 2
    

    @VietND96 VietND96 force-pushed the record-multiple-sessions branch from cc4aaa7 to 6fc20a4 Compare January 21, 2025 05:11
    @VietND96 VietND96 merged commit c84cc85 into trunk Jan 21, 2025
    27 checks passed
    @VietND96 VietND96 deleted the record-multiple-sessions branch January 21, 2025 06:14
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    1 participant