-
Notifications
You must be signed in to change notification settings - Fork 83
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Docker debugging and sub-debugger with random port #107
Comments
Looks like this is a duplicate (or related): I'm a bit confused how some folks on the web say they got this working (without SSH). |
Hi @jcavalieri . Did you found any solution to that? I'm stuck on the same problem. |
Hi @ivanprado, |
I have the same problem. |
I haven't solved it. RubyMine states to have a solution but I couldn't get it to work. I think what RubyMine is doing is adding their own container one's docker environment. I haven't heard from the maintainers. @valich, I see you are the most active recently on this repo. Any thoughts on this? -John |
We'll recheck how the workaround with using docker container as SSH remote interpreter work (have you tried it?). Generally debugging multiprocess stuff in Docker may be cumbersome indeed due to automatic port generation for subprocesses. We are planning to do some refactoring to use single port for the whole debug session, but I doubt we'll make it to 2018.1. Right now I suppose one should either try to set up SSH connection, or we can make some fork with workarounds for specifying fixed ports. |
OK we discussed and found quite a simple solution of moving to a single-port scheme without much refactoring (at least on IDE side) so we'll try to implement it. In short, debuggee sets up a proxy to communicate with IDE via multiple connections (opposed to multiple connections from different debuggee processes). Subprocesses will communicate with the proxy with the request to forward the data to the outer client (IDE). The mapping (inner process <-> outer connection) will be also used to deliver responses (commands) back to subprocesses. |
That's great to hear @valich! |
@valich, that sounds great! Any update on this? I used patch for rdebug-ide where I could provide fixed set of ports that were opened for subprocesses, but it really is PITA - it requires opening these ports and adds complexity. Running this in some proxy mode sounds like a good idea - will it work for existing IDEs without requiring any changes? Also any idea when some initial version could be expected? Happy to give it a try. |
@noizex I think it just entered the state "in progress". I think it should be ready to try in a couple of weeks, no earlier. |
I was able to get this working over an ssh connection to a remote server (I have not tried this on a Docker container) by doing the following:
Note: At first I tried using |
@valich any update on this ticket? |
Also very interested in this, could help with some testing if needed, or anything else - just shout :) |
Hi @aaronblenkush , Hi @valich, did the proposed solution get any movement? |
I'm interested in a solution too. I've the same problem and can't find a solution otherwise using single cluster on Puma configuration. |
To work around this issue I overrode the find_free_port method in the # initializers/ruby-debug-ide.rb
# In non-dev and rake tasks the Ruby Debug IDE gem
# is not loaded so nothing to override.
if defined?(Debugger::MultiProcess)
Debugger::MultiProcess.module_eval do
class << self
def find_free_port(host)
server = TCPServer.open(host, 58438)
port = server.addr[1]
server.close
port
end
end
end
end In my Docker-Compose file I added the port: web:
ports:
- "58438:58438"
<Other ports, such as 8080 for Unicorn> Now the debugger no longer hangs when spawning the second process. It used to hang after the second "Fast Debugger" line but now it works as expected.
In development I only use one process so I just hard coded the port but you could setup a range if you wanted too. Details: I'm supporting an older application that previously used Thin as it's development webserver but Unicorn in production. Part of the application is multi-threaded which would hang the Thin webserver since it's single threaded by default. I decided to use Unicorn in development to be consistent with production and when I did the switch I ran into the above issue when debugging. There was no issues when running the application using Unicorn without debugging (i.e. clicking the green arrow). Ubuntu 18.04 LTS Note : The find_free_port method is moved in the 0.7 beta of Ruby Debug IDE to Feedback is welcome,. Edit (Dec 11/18): Added check that the Debugger::MultiProcess is defined. |
@mrbiggred how is this supposed to work if the initializers are loaded only when If anyone can help me, this is the ouput for a
|
@vmassuchetto if I understand your question correctly you are asking why does overriding the As for your If I misunderstood your questions please let me know. Finally, it is possible my workaround has a bug and/or only works for my particular setup. |
@mrbiggred Sorry if I was not clear. I'm trying to attach RubyMine to a running I just copied the As you can see in our output, rails didn't get called, so |
Got it working by using a rails launch script without bundler ( |
I think this wouldn't be too difficult to add a new command line option like |
@jcavalieri @vmassuchetto @mrbiggred There is a prototype of the debugger that uses only one port for all processes. It would be very helpful if you tried it. #164 |
Hi @ViugiNick , I like what you are doing in your branch. Have you created an PR for this? I'm wondering what the maintainers would think of your solution. |
Ha! @ViugiNick I see that you are a maintainer! I'll try out your branch and give you feedback. |
@ViugiNick what is the easiest to test your fix? I'm assuming the following updates to my Gemfile would work:
Will this replace the existing ruby-debug-ide gem since they will have the same version number? Is it better to manually delete the previous version first? Thank you. |
@mrbiggred Yes, it's better to delete the old one manually |
@ViugiNick I tested out your prototype. For my situation, it unfortunately isn't working any better. I'm not sure where I would go to offer any further debugging info about why the connection is failing with your prototype. I've made this change to my gemfile:
which brought in commit 593eef1 of your branch into my gemfile.lock. My situation:
I did some snooping in wireshark without using Docker, to see if there was some port being used other than the ports listed in the commands I'm running. I saw 1 or 2 ephemeral ports being used every time (which I haven't explicitly configured anywhere). That led me #73 and then here. OS - Mac 10.13 My docker-compose file looks like this: debug:
command: sh -c 'rm -rf ./tmp && bundle exec rdebug-ide --host 0.0.0.0 --port 1234 --dispatcher-port 26162 -- bin/rails s -b 0.0.0.0'
ports:
- 3000:3000
- 1234:1234
- 26162:26162 Thanks for working to fix the issue! |
@locofocos Thanks for the feedback, I'll try to fix it ASAP. In fact, this prototype still needs two ports (not more) |
@ViugiNick hm. good question.
Here is my part of docker-composer.yml I had add "26162-26168" but still have error. May be I'm doing it wrong. Do not have a lot of experience with it. |
@ViugiNick I have a slightly different issue
|
@peter-grainger seems like your dispatcher is not running, the error indicates that it failed to open connection to a given
|
@noizex I'm running in a docker container and need to attach to IP |
I guess putting |
I’ll fork and put some logging in. Maybe I can figure out what I’m not doing from that
|
@ViugiNick With your branch I can now remote debug specs from RubyMine. Thanks! |
@ViugiNick I tried your branch on a different project and it works. # Gemfile
gem 'ruby-debug-ide', git: 'https://github.com/ViugiNick/ruby-debug-ide.git', branch: 'so_reuseport'
gem 'debase' Ubunutu: 18.04 LTS Haven't had a chance to try it on my other project yet. Thank you and I hope your PR (#164) gets merged soon. |
As similarly proposed in ruby-debug/ruby-debug-ide#107 This hack is necessary as long as ruby-debug/ruby-debug-ide#164 is not merged and released because: * We cannot easily modify the Gemfile because it is not in the same repository. So we need to use `gem install` in the Dockerfile to do it * The newest RubyMine requires a prerelease version of ruby-debug-ide * The fix with SO_REUSEPORT is only available on a specific branch of a specific fork, which is not up-to-date with the prerelease version * We cannot use the monkey-patch in an initializer as proposed in the issue, because wagon:test doesn't run Rails and thus doesn't consider the initializers The combination of these three aspects renders it impossible to easily install a version that includes the fix. So instead, we simply overwrite the relevant code inside the bundle with something that works for our case for now. As soon as the PR is merged and released, we can revert to installing the official, unaltered version into our docker container.
As similarly proposed in ruby-debug/ruby-debug-ide#107 This hack is necessary as long as ruby-debug/ruby-debug-ide#164 is not merged and released because: * We cannot easily modify the Gemfile because it is not in the same repository. So we need to use `gem install` in the Dockerfile to do it * The newest RubyMine requires a prerelease version of ruby-debug-ide * The fix with SO_REUSEPORT is only available on a specific branch of a specific fork, which is not up-to-date with the prerelease version * We cannot use the monkey-patch in an initializer as proposed in the issue, because wagon:test doesn't run Rails and thus doesn't consider the initializers The combination of these three aspects renders it impossible to easily install a version that includes the fix. So instead, we simply overwrite the relevant code inside the bundle with something that works for our case for now. As soon as the PR is merged and released, we can revert to installing the official, unaltered version into our docker container.
I was using @ViugiNick I've created a new fix that uses a port range instead of a single port like my previous workaround. Currently the ports are hard code but if this fix works I can investigate making the ports configurable. Please try our my fix and let me know what you think. If the feedback is positive I will create a PR. To try it out first update your Gemfile: gem 'ruby-debug-ide', git: 'https://github.com/corgibytes/ruby-debug-ide', branch: 'feature-add-fixed-port-range' Then update your Docker Compose file to allow the port range 58430-58450: services:
web:
<Other setup>
ports:
- "3000:3000"
- "1234:1234"
- "58430-58450:58430-58450" Then remove the old Ruby Debug IDE gem and install the new one:
Now when click debug in RubyMine it and shouldn't stop at This fix is from version 7.2 of the Ruby Debug IDE. Feedback would be much appreciated. Thank you. |
@mrbiggred In my gemfile:
During the bundle install I see it reference your git repo. In my docker-compose.yml:
I bring my docker containers up via docker-compose up. I ssh into the container with the puma server and execute:
The server hangs there forever. I assume it is waiting for a remote debugger to attach? So I try to attach: I still get the dreaded, "Failed to find free socket port...". I will hit ctrl-c to motivate puma to keep coming up. Is there any other data from me you need? I am very interested in your solution so willing to help. Thanks. |
@cshupp1 thank you for trying my fix and your feedback. I've never tried the steps you outlined to run the RubyMine debugger. What I do is:
docker-compose build
docker-compose up web I should be able to connect to my Docker container via a web browser. If I can't then I need to debug you Docker setup.
Let me know if the above works for you. If you need to run the ruby-debug-ide via the command line let me know and I can dig into your issue some more. I'm also curious if running the ruby-debug-ide via the command line instead of through the RubyMine IDE is something other people do. |
@cshupp1 you can try to refresh the gems that RubyMine sees in your Docker file by clicking the down arrow overtop of the gems. Hopefully after clicking the down arrow will see the Rails gems. Also try removing the line that starts the ruby-debug-ide from your # Remove the below line
rdebug-ide --host 0.0.0.0 --port 1234 --dispatcher-port 26162 --bin/rails server --binding=0.0.0.0 |
@mrbiggred This RubyMine approach is new to me, I was expecting my remote debugger approach as I outlined previously. To get your approach to work I did a docker-compose down to allow RubyMine to manage it. Then I clicked the checkbox to resynch as you advised. With the image down the scan completed and Rails was found. I removed the following line:
To allow the debug configuration to manage it as advised. Then I click the debug button and voila! Of course, with this approach the application bombs out in initialization (it can't find the reddis service). If I have the rdebug line replaced with:
And docker-compose up myself in the command line everything seems to work. It would be interesting to know why my original approach fails. Thanks for your help! |
@mrbiggred
My docker-compose file's web service (named dev) looks like this
My docker dev setup is running on MacBook. |
The redis problem was on my end. As far as I can tell your solution works flawlessly! Thanks! |
@dhanaraj0 thank you for trying my fix the the feedback. I think you are experiencing the issue #186 (Error Raised when Debugging a Multi-Process app in Docker on a Mac). Based on the error:
Try the fix suggested in the issue and let me know if fixes the Connection refused error. I plan to deal with the Multi-Process Docker on Mac issue after I finish testing this fix. |
@mrbiggred If anyone is facing issues on Mac, we need to apply the fix for #186 and #107 |
@mrbiggred Just want to post that I have your patch working, through docker, with Visual Studio. I want to document it here for everyone. This is the approach I wanted to take earlier with RubyMine (that doesn't work there) as that approach would have been more IDE agnostic. From a docker exec console I run:
At this point, it just hangs around waiting for my debugger to attach. I create a launch.json as follows:
By default, the key remoteWorkspaceRoot is generated as:
That doesn't work even though, in my case, "/srv/vets-api/src" is "${workspaceRoot}" 🤷♂️ And in the docker shell we can see the rspec take off, breaks being hit along the way! |
This is my solution. In .env set WEB_CONCURRENCY to 0. or in puma.rb set the number of workers also to 0: The option provided by @antmonnier is bad since it makes the debugger useless, it does not intercept adding/removal of breakpoints which is a crucial option. |
but for some, me included, the only reason to use puma is the need for multi workers. |
Hi,
I'm trying to debug rails on a docker container using RubyMine. For lack of a better term, I do have an initial successful hand shake. However, the debugger then creates a "sub-debugger" on a different port. This port seems to be random, therefore, I can't pre-configure to have that port shared. Since the IDE doesn't respond to the sub-debugger port being open, then my rails site doesn't start.
Note the port 37922. I'm having trouble finding documentation on the sub-debugger. Do you have any suggestions on how to get around this?
Other info, I'm not using "Docker for Mac" I installed docker tools via brew and and my default docker machine is using the parallels driver.
-John
The text was updated successfully, but these errors were encountered: