Black png returned when using a vnc session #117

durack1 · 2017-01-19T23:38:28Z

@danlipsa @doutriaux1 I just ran the demo script BlueMarble_huss_satellite_view.html and received the appropriate graphic on the active VNC screen. What was saved to the png is below, and is certainly not what I was expecting:

This is certainly not the expected behavior. What additional information do you require to better replicate the issue:

In [1]: import cdat_info
In [2]: cdat_info.version()
Out[2]: ['', 'v2.8']

durack1 · 2017-01-19T23:48:38Z

@danlipsa @doutriaux1 this is even better, when I change the:

canvas = vcs.init(size=2)

to

canvas = vcs.init(size=2,bg=True)

I then get:

danlipsa · 2017-01-20T15:06:55Z

@durack1 Do you have the huss_Amon_HadGEM2-AO_historical_r1i1p1_186001-200512.nc file?
@aashish24 @doutriaux1 We should add the examples to the test suite, maybe as a separate section to be able to run them automatically.
I think the black PNG is what we have seen before - VNC does not support OpenGL. Our suggestion was to run VNC through Virtual GL.

doutriaux1 · 2017-01-20T15:16:02Z

Dan with recent updates my ubunutu seems to seg fault instantly via x forwarding. RH6 has been sending black png for a while now via x forwarding. @durack1 use the nox version, I know it sucks but that one works.

durack1 · 2017-01-20T19:21:49Z

@danlipsa this file is 187MB, it's got all the timesteps 186101 to 200512. I've pulled out a single timestep in the linked file (102KB) let me know if this isn't suitable:
huss_Amon_HadGEM2-AO_historical_r1i1p1_186001.nc.zip

How do I run VNC through Virtual GL?

durack1 · 2017-01-20T19:24:54Z

@doutriaux1 just confirming to install the *-nox I need the following conda command right?

conda create -n uvcdat280-nox -c conda-forge -c uvcdat uvcdat vcs-nox

Is that right? How about the vtk-cdat-nox? Or is that magically also grabbed as a dependency of vcs-nox?

durack1 · 2017-01-20T19:35:57Z

@doutriaux1 is it right that both the vcs AND the vcs-nox versions (along with their dependencies) are being installed? Seems like this is going to cause me a lot of problems..?

[duro@ocean 090605_FLR2_sptp]$ conda create -n uvcdat280nox -c conda-forge -c uvcdat uvcdat vcs-nox
Fetching package metadata .............                                                                   
Solving package specifications: .                                                                         

Package plan for installation in environment /export/durack1/anaconda2/envs/uvcdat280nox:

The following NEW packages will be INSTALLED:

... 
    dv3d:                               2.8-py27_2                        uvcdat     
    dv3d-nox:                           2.8-py27_2                        uvcdat     
...
    vcs:                                2.8-py27_2                        uvcdat
    vcs-nox:                            2.8-py27_2                        uvcdat
    vcsaddons:                          2.8-py27_2                        uvcdat
    vcsaddons-nox:                      2.8-py27_2                        uvcdat
    vistrails:                          2.8-2                             uvcdat
    vtk-cdat:                           7.1.0.2.8-py27_2                  uvcdat
    vtk-cdat-nox:                       7.1.0.2.8-py27_2                  uvcdat
...

doutriaux1 · 2017-01-20T19:57:29Z

@durack1 you want: conda create -n uvcdat280nox -c conda-forge -c uvcdat uvcdat-nox

durack1 · 2017-01-20T19:57:53Z

@doutriaux1 perfect, I'll give that a spin

durack1 · 2017-01-20T20:06:00Z

@doutriaux1 unfortunately no luck, I successfully installed uvcdat-nox which only had the appropriate *-nox versions and this is still the output:

doutriaux1 · 2017-01-20T20:16:05Z

@danlipsa this might be bad then! @durack1 try with a slightly more recent vcs
conda create -n uvcdat280nox -c conda-forge -c uvcdat/label/nightly -c uvcdat uvcdat-nox

durack1 · 2017-01-20T20:34:10Z

@doutriaux1 ok so installed the nightly uvcdat-nox (after fixing spaces in the command) and now I get:

(uvcdatNightlyNox) duro@ocean:[170118_PaperPlots_input4MIPs]:[2307]> python make_paperplots.py
Traceback (most recent call last):
  File "make_paperplots.py", line 9, in <module>
    import vcs
  File "/export/duro/anaconda2/envs/uvcdatNightlyNox/lib/python2.7/site-packages/vcs/__init__.py", line 57, in <module>
    from utils import *  # noqa
  File "/export/duro/anaconda2/envs/uvcdatNightlyNox/lib/python2.7/site-packages/vcs/utils.py", line 6, in <module>
    import boxfill
  File "/export/duro/anaconda2/envs/uvcdatNightlyNox/lib/python2.7/site-packages/vcs/boxfill.py", line 25, in <module>
    import VCS_validation_functions
  File "/export/duro/anaconda2/envs/uvcdatNightlyNox/lib/python2.7/site-packages/vcs/VCS_validation_functions.py", line 5, in <module>
    import genutil
  File "/export/duro/anaconda2/envs/uvcdatNightlyNox/lib/python2.7/site-packages/genutil/__init__.py", line 5, in <module>
    from grower import grower
  File "/export/duro/anaconda2/envs/uvcdatNightlyNox/lib/python2.7/site-packages/genutil/grower.py", line 4, in <module>
    import cdms2 as cdms
  File "/export/duro/anaconda2/envs/uvcdatNightlyNox/lib/python2.7/site-packages/cdms2/__init__.py", line 33, in <module>
    from dataset import createDataset, openDataset, useNetcdf3, \
  File "/export/duro/anaconda2/envs/uvcdatNightlyNox/lib/python2.7/site-packages/cdms2/dataset.py", line 6, in <module>
    import Cdunif
ImportError: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /export/duro/anaconda2/envs/uvcdatNightlyNox/lib/python2.7/site-packages/cdms2/Cdunif.so)

doutriaux1 · 2017-01-20T20:36:16Z

ok... so let's try this one on the bleeding edge:
conda create -n uvcdat280nox -c conda-forge -c uvcdat/label/patterns -c uvcdat uvcdat-nox

durack1 · 2017-01-20T20:39:18Z

@doutriaux1 the only patterns sourced package is vtk-cdat-nox should I proceed?

doutriaux1 · 2017-01-20T20:41:06Z

yes, that's all you need.

durack1 · 2017-01-20T20:44:08Z

@doutriaux1 no good, basically same output as above

doutriaux1 · 2017-01-20T20:45:57Z

@danlipsa that's a problem. @durack1 ok then you need to set your LD_LIBRARY_PATH unfortunately

export LD_LIBRARY_PATH=/export/duro/anaconda2/envs/uvcdatNightlyNox/lib/

let me know.

durack1 · 2017-01-20T20:48:20Z

@doutriaux1 doesn't help, same output:

(uvcdatPatternsNox) duro@ocean:[170118_PaperPlots_input4MIPs]:[2308]> python make_paperplots.py
(uvcdatPatternsNox) duro@ocean:[170118_PaperPlots_input4MIPs]:[2308]> export LD_LIBRARY_PATH=/export/duro/anaconda2/envs/uvcdatPatternsNox/lib/
(uvcdatPatternsNox) duro@ocean:[170118_PaperPlots_input4MIPs]:[2309]> python make_paperplots.py

doutriaux1 · 2017-01-20T20:55:00Z

hum... can you post:

ls /export/duro/anaconda2/envs/uvcdatPatternsNox/lib
ls /export/duro/anaconda2/lib

doutriaux1 · 2017-01-20T23:45:06Z

works for me on oceanonly

(PATT)doutriaux1@oceanonly:[~]:[1709]> python
Python 2.7.11 |Continuum Analytics, Inc.| (default, Dec  6 2015, 18:08:32) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> x.plot(s)
<vcs.displayplot.Dp object at 0x7f99c80b8050>
>>> 
(PATT)doutriaux1@oceanonly:[~]:[1709]> which python
/usr/local/anaconda2/envs/PATT/bin/python
(PATT)doutriaux1@oceanonly:[~]:[1707]> conda create -n PATT -c conda-forge -c uvcdat/label/patterns -c uvcdat uvcdat-nox

durack1 · 2017-01-20T23:54:11Z

@doutriaux1 it doesn't work, you don't get

Rather you get

If you're lucky

durack1 · 2017-01-21T00:01:53Z

@doutriaux1 ok, so if I run uvcdatPatternsNox with bg=False then I get:

doutriaux1 · 2017-01-21T00:10:16Z

ok that makes sense (sort of) @danlipsa we should look into why bg=True freaks out on mesa builds. But we could also always turn bg to True on mesa builds

doutriaux1 · 2017-01-21T00:10:24Z

thanks @durack1

durack1 · 2017-01-21T00:10:40Z

@doutriaux1 I can also confirm a similar (but not identically sized output file) png is generated with uvcdat280nox and bg=False

And why not, here 'tis:

durack1 · 2017-01-21T00:15:33Z

@danlipsa how do I go about running VNC through Virtual GL? While I can get graphics to write out to a file with these *-nox builds, I can't see anything interactively and that's a pretty horrible user experience

danlipsa · 2017-01-23T15:00:24Z

@durack1 Here are the instructions I followed to setup vnc/virtual GL on my ubuntu and connect from my mac. I used the latest version for vnc/Virtual GL rather the version specified in the instructions. I think you'll need root access on the linux machine.

I mostly followed instructions from
http://www.paraview.org/ParaView3/Doc/Nightly/www/js-doc/index.html#!/guide/virtualgl_turbovnc_howto

with help from:
https://cdn.rawgit.com/VirtualGL/virtualgl/2.5.1/doc/index.html

Make sure you install the 64 bit versions for virtualgl and turbovnc. Otherwise they won't work! I know because I did this first! :-(

When I start the vnc server I run
/opt/TurboVNC/bin/vncserver -geometry 1440x900 -xstartup ~/.vnc/xstartup

You can do this from ssh and pass the geometry for your own client.

Where xtartup contains:
#!/bin/sh
unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS
startxfce4 &
#gnome-session&

[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources

On the mac client I run:
open vnc://galati:5901

When running an OpenGL application instead of:

# ./app

type

# vglrun app

danlipsa · 2017-01-23T15:33:55Z

@doutriaux1 Do you install the gcc 4.8.4 package when installing cdat? This should take care of the missing symbol and will remove the need for LD_LIBRARY_PATH.

doutriaux1 · 2017-01-23T15:35:56Z

@danlipsa I think we're good now, @durack1 was able to run the nox version so I think we do not need the LD_LIBRARY_PATH thing.

doutriaux1 · 2017-06-19T15:39:02Z

@sankhesh I sudoed it. Anyhow I'm trying now now with vgl 2.4 which my yum was able to find and install. More soon

doutriaux1 · 2017-06-19T19:17:11Z

@sankhesh I get the server up and running
theres is no glxgears in the bin directory
but the turboserver starts but runnnig vglrun leads to this:

doutriaux1@crunchy:[~]:[15502]> export DISPLAY=crunchy.llnl.gov:3
doutriaux1@crunchy:[~]:[15503]> vglrun python
[VGL] NOTICE: Automatically setting VGL_CLIENT environment variable to
[VGL]    128.15.245.181, the IP address of your SSH client.
Python 2.7.13 | packaged by conda-forge | (default, May  2 2017, 12:48:11) 
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
CDMS system error: No such file or directory
CDMS I/O error: Opening file /usr/local/anaconda2/envs/2.10/share/uvcdat/sample_data/clt.nc
Traceback (most recent call last):
  File "/export/doutriaux1/.pythonrc", line 32, in <module>
    f=cdms2.open(fnm)
  File "/usr/local/anaconda2/envs/2.10/lib/python2.7/site-packages/cdms2/dataset.py", line 357, in openDataset
    return CdmsFile(path, mode, mpiBarrier=CdMpi)
  File "/usr/local/anaconda2/envs/2.10/lib/python2.7/site-packages/cdms2/dataset.py", line 1090, in __init__
    raise CDMSError('Cannot open file %s (%s)' % (path, err))
cdms2.error.CDMSError: Cannot open file /usr/local/anaconda2/envs/2.10/share/uvcdat/sample_data/clt.nc (No error)
>>> import vcs
>>> x=vcs.init()
>>> x.plot([1,2,3,4,3,2,1,2,3,4])
No protocol specified
[VGL] ERROR: Could not open display :0.

doutriaux1 · 2017-06-19T19:18:29Z

Also the follwing hangs and never renders anything

doutriaux1@crunchy:[~]:[15503]> vglrun xeyes
[VGL] NOTICE: Automatically setting VGL_CLIENT environment variable to
[VGL]    128.15.245.181, the IP address of your SSH client.

doutriaux1 · 2017-06-19T19:20:15Z

@sankhesh do I need to connect to crunchy via ssh or via another vnc client the instructions are not clear about this.

sankhesh · 2017-06-19T19:27:01Z

@sankhesh do I need to connect to crunchy via ssh or via another vnc client the instructions are not clear about this.

I don't think it matters how you're connected to the server. As long as you have a VNC server session running, you should have a valid X environment.

sankhesh · 2017-06-19T19:29:08Z

Also the follwing hangs and never renders anything

doutriaux1@crunchy:[~]:[15503]> vglrun xeyes
[VGL] NOTICE: Automatically setting VGL_CLIENT environment variable to
[VGL] 128.15.245.181, the IP address of your SSH client.

I think it might be rendering something in the VNC display. Try connecting via VNC to see if you see the xeyes window.

sankhesh · 2017-06-19T19:30:51Z

I'm not sure why the DISPLAY environment variable is not supplied to python. Perhaps try setting it within python using os.environ

doutriaux1 · 2017-06-19T19:41:58Z

@sankhesh it's a total failure on crunchy. Please try.
Instructions might need update. Here's what I did.

started vglserver on crunchy
download turnbvnc client on mac
connect to crunchy from mac via trubovnc
if I run vglrun python I get the bad display :0 error
If I run the regular python it seg fault on opening of display...

Please try from crunchy

@danlipsa can you please try on crunchy as well?

Thanks.

doutriaux1 · 2017-06-19T19:44:14Z

@sankhesh whilerunning vglrun python the os.environ["DISPLAY"] return correctly crunchy:3.
But I do get bad display error...

doutriaux1 · 2017-06-19T19:46:00Z

Also tried via regular ssh to crunchy

sankhesh · 2017-06-19T20:17:05Z

@doutriaux1 I think your setup should work. Try connecting via a VNC client and run xeyes to see if a window shows up.

doutriaux1 · 2017-06-20T12:51:18Z

@sankhesh both
vglrun xeyes
and
xeyes

work fine in my turbovnc client

but

doutriaux1@crunchy:[~]:[15936]> vglrun glxgears
[VGL] NOTICE: Automatically setting VGL_CLIENT environment variable to
[VGL]    128.15.245.181, the IP address of your SSH client.
No protocol specified
[VGL] ERROR: Could not open display :0.
doutriaux1@crunchy:[~]:[15937]> glxgears 
Error: couldn't get an RGB, Double-buffered visual

danlipsa · 2017-06-20T12:56:34Z

@doutriaux1 I get the same message if I run vglrun from an ssh connected to the server. You have to run vglrun only from turbovnc connected to your server.

danlipsa · 2017-06-20T12:57:20Z

@doutriaux1 This is my message:

~]$ vglrun glxgears
[VGL] NOTICE: Automatically setting the DISPLAY environment variable to
[VGL]    10.9.0.8:0.0, based on the IP address of your SSH client.
Error: couldn't open display 10.9.0.8:0.0
[~]$

sankhesh · 2017-06-20T13:54:20Z

@sankhesh both
vglrun xeyes
and
xeyes

work fine in my turbovnc client

Awesome 👍 👏
It means you have the setup working.

but

[VGL] ERROR: Could not open display :0.

Could you please share the output of the following two commands (when connected via TurboVNC client):

/opt/VirtualGL/bin/glxinfo -display :3.0

and

vglrun -sp /opt/VirtualGL/bin/glxspheres

doutriaux1 · 2017-06-20T14:00:42Z

@danlipsa I am getting this while connected via turbovnc

doutriaux1 · 2017-06-21T16:49:45Z

@sankhesh do you have an account on crunchy? I would like for you to try it. Because It's definitely not working for me from my mac.

sankhesh · 2017-06-21T17:46:25Z

@doutriaux1 I don't. How do I get one?

durack1 · 2017-06-21T17:59:28Z

@hoang1 can you help out here?

doutriaux1 · 2017-06-21T18:11:46Z

@durack1 @sankhesh is taken care of. No worries.

sankhesh · 2017-06-22T03:28:40Z

@doutriaux1 Could you please try the setup on a local desktop (Ubuntu or Red Hat)?

doutriaux1 · 2017-06-22T03:30:19Z

@sankhesh both my linux boxes are dead right now... I'll see if I can try the setup from a vm/docker

doutriaux1 · 2017-06-22T03:31:24Z

@sankhesh I'm going to look into this as well: https://github.com/yantis/docker-virtualgl

doutriaux1 · 2019-04-18T17:46:08Z

https://github.com/CDAT/cdat/wiki/Remote-server-setup-for-VNC @scottwittenburg please review and let me know if it helps.

scottwittenburg · 2019-04-18T17:47:55Z

Btw the link @danlipsa provided from the paraviewweb documention (high above), is broken now, but that documentation has moved here:

https://kitware.github.io/paraviewweb/docs/virtualgl_turbovnc_howto.html

durack1 added the bug label Jan 19, 2017

danlipsa mentioned this issue Jan 23, 2017

bg=True freaks out on OSMesa builds #119

Closed

doutriaux1 modified the milestones: 3.0, post 3.0 Mar 29, 2018

doutriaux1 modified the milestones: 8.1, 8.2 Mar 27, 2019

doutriaux1 assigned scottwittenburg Apr 18, 2019

Black png returned when using a vnc session #117

Black png returned when using a vnc session #117

Comments

durack1 commented Jan 19, 2017

durack1 commented Jan 19, 2017

danlipsa commented Jan 20, 2017 • edited Loading

doutriaux1 commented Jan 20, 2017

durack1 commented Jan 20, 2017

durack1 commented Jan 20, 2017 • edited Loading

durack1 commented Jan 20, 2017 • edited Loading

doutriaux1 commented Jan 20, 2017

durack1 commented Jan 20, 2017

durack1 commented Jan 20, 2017

doutriaux1 commented Jan 20, 2017

durack1 commented Jan 20, 2017 • edited Loading

doutriaux1 commented Jan 20, 2017 • edited Loading

durack1 commented Jan 20, 2017

doutriaux1 commented Jan 20, 2017

durack1 commented Jan 20, 2017

doutriaux1 commented Jan 20, 2017

durack1 commented Jan 20, 2017

doutriaux1 commented Jan 20, 2017

doutriaux1 commented Jan 20, 2017

durack1 commented Jan 20, 2017 • edited Loading

durack1 commented Jan 21, 2017

doutriaux1 commented Jan 21, 2017

doutriaux1 commented Jan 21, 2017

durack1 commented Jan 21, 2017 • edited Loading

durack1 commented Jan 21, 2017 • edited Loading

danlipsa commented Jan 23, 2017 • edited Loading

danlipsa commented Jan 23, 2017

doutriaux1 commented Jan 23, 2017

doutriaux1 commented Jun 19, 2017

doutriaux1 commented Jun 19, 2017

doutriaux1 commented Jun 19, 2017

doutriaux1 commented Jun 19, 2017

sankhesh commented Jun 19, 2017

sankhesh commented Jun 19, 2017

sankhesh commented Jun 19, 2017

doutriaux1 commented Jun 19, 2017

doutriaux1 commented Jun 19, 2017

doutriaux1 commented Jun 19, 2017

sankhesh commented Jun 19, 2017

doutriaux1 commented Jun 20, 2017

danlipsa commented Jun 20, 2017 • edited Loading

danlipsa commented Jun 20, 2017

sankhesh commented Jun 20, 2017

doutriaux1 commented Jun 20, 2017

doutriaux1 commented Jun 21, 2017

sankhesh commented Jun 21, 2017

durack1 commented Jun 21, 2017

doutriaux1 commented Jun 21, 2017

sankhesh commented Jun 22, 2017

doutriaux1 commented Jun 22, 2017

doutriaux1 commented Jun 22, 2017

doutriaux1 commented Apr 18, 2019

scottwittenburg commented Apr 18, 2019

danlipsa commented Jan 20, 2017 •

edited

Loading

durack1 commented Jan 20, 2017 •

edited

Loading

durack1 commented Jan 20, 2017 •

edited

Loading

durack1 commented Jan 20, 2017 •

edited

Loading

doutriaux1 commented Jan 20, 2017 •

edited

Loading

durack1 commented Jan 20, 2017 •

edited

Loading

durack1 commented Jan 21, 2017 •

edited

Loading

durack1 commented Jan 21, 2017 •

edited

Loading

danlipsa commented Jan 23, 2017 •

edited

Loading

danlipsa commented Jun 20, 2017 •

edited

Loading