How To Remote Start Server Service With Python

I aid your development teams become smarter, efficient, and solve complex problems for mission critical systems

Remote Debugging Guide for Python

Published May 16, 2019 Concluding updated May 17, 2019

Ever have a tough bug in production that doesn't testify up on your machine fifty-fifty later on having parity with pretty much everything? Or perhaps you lot don't have much of a choice, just to work directly on the server? Practise you cease up using impress or a logger and just haphazardly plopping those in in different places hoping to see what's going on?

I'll present to you remote debugging. Remote debugging allows yous to interactively debug lawmaking that's not on your machine line-by-line. Information technology'due south as others accept described a "God-transport" into debugging tough problems. Yous'll become to the root of the problem much quicker with it and it's extremely useful when you need to reverse-engineer a projection.

Nosotros'll exist using a python package to have a common API to work with.

Requirements

Visual Studio Code or Pycharm Professional person
Either ptvsd or pydevd_pycharm dependency installed

Installing

pip install git+ssh://git@github.com/2upmedia/debugger_helper

How it works

There'due south 5 parts to remote debugging with Python:

the server can communicate with your workstation
you take the corresponding IDE parcel installed on the server
your IDE is configured for remote debugging correctly
you run the remote debugging configuration in your IDE at the right
fourth dimension
y'all accept the same code that'due south on the server on your workstation

In that location'southward a debug package for each IDE that serves every bit the debugger on the
server. Information technology communicates dorsum and along between the server and the IDE.

You need to make sure that the server can communicate with your
workstation. This is an essential part of remote debugging because
there are TCP packets sent back and forth between the server and the
IDE. Those packets communicate things like where the breakpoints are set
and information about the current stack and variables. This is
essentially how all remote debuggers piece of work.

The easiest way to let a TCP line-of-communication that bypasses any
firewall issues is by port forwarding. This is possible with ssh or
putty.

With ssh just issue ssh -R LOCAL_PORT:IP:REMOTE_PORT user@host for a
reverse port forwards and ssh -L LOCAL_PORT:IP:REMOTE_PORT user@host
for a local port forrard. Typically you'd employ 127.0.0.1 for the IP and
the same port numbers, for instance,
ssh -L 9000:127.0.0.one:9000 user@host. A local port forward means that
a socket is allocated and listening on your local machine and forwards
the TCP connexion to the specified remote port on the server. A opposite
port forward means that a socket is allocated and listening on your
remote machine and forwards the TCP connection to the specified local
port on your machine. Visual Studio Lawmaking uses local port forwarding
while PyCharm needs reverse port forwarding.

To ensure that the server can communicate with your figurer you could
run the following commands on the server:

              $ openssl s_client -connect 127.0.0.1:9000 connect_to 127.0.0.1 port 9000: failed. Connected(00000004)

If you have telnet installed you could also try...

              $ telnet 127.0.0.1 9000 Trying 127.0.0.1... Continued to localhost.

What yous're looking for is Continued for openssl. For telnet, if information technology
doesn't exit telnet and doesn't get an error after some time, then that
means information technology'southward connected.

In terms of the setup, that ways that for PyCharm you need to run the
remote IDE configuration first before running your python script while
for Visual Studio Code you run information technology later on your python script.

The remote debugger IDE settings define the host, port, and path
mappings. The path mapping maps the folder path from the server to the
binder path on your local machine so that the debugger tin selection upward the
right file.

There are three environment variables that the debugger_helper module
uses:

START_DEBUGGER
DEBUGGER_HOST (default 127.0.0.1)
DEBUGGER_PORT (default 9000)

The START_DEBUGGER environs variable should be ready to any non-empty
value such as one right before starting the python script you'd like to
debug.

If everything is prepare upward correctly the debugger should immediately stop
at your breakpoint. If this is not the case, double bank check every step.

Setting up Visual Studio Code

install ptvsd on the server
make sure pydevd-pycharm isn't installed as information technology conflicts with
ptvsd
add together a local port frontwards from your machine to the server via ssh (or
putty)
add together the following to somewhere in the top-most part of your python
project. For instance, if it's a Flask app with an app.py file you
might place it right at the top after your imports.

                              import                ptvsd debugger_helper.attach_vscode(lambda                host, port: ptvsd.enable_attach(address=(host, port), redirect_output=Truthful))

go to the debug panel, add configuration, click on Python, then
Remote Attach, ready the host to 127.0.0.1, port to 9000 (or the
port to friction match the port forwarding and the DEBUGGER_PORT
environment variable). Yous should run across those values in launch.json.
for remoteRoot, set it to the absolute path of the folder
containing your python script on the server. For example, maybe
it's residing in /www/pythonapp/. You'd use that for remoteRoot.
set a breakpoint where you'd similar the debugger to stop
Run the python script $ START_DEBUGGER=1 python app.py and waiting
until it says it'southward set to connect to the debugger
Run the Remote Attach configuration in Visual Studio Lawmaking.

Setting up Pycharm

install pydevd_pycharm on the server
add a reverse port forward from the server to your machine via ssh
(or putty)
add a Run Configuration for Python Remote Debug. Set the host to
127.0.0.one and port 9000. Save information technology.
in the configuration and path mapping field add a mapping for the
accented path of the project root to the absolute path of the same
project root, but on the server
add the following to somewhere in the acme-most function of your python
project. For instance, if information technology'south a Flask app with an app.py file yous
might place it right at the top after your imports.

                              import                pydevd_pycharm debugger_helper.attach_pycharm(lambda                host, port: pydevd_pycharm.settrace(host, port=port, stdoutToServer=Truthful, stderrToServer=Truthful))

set a breakpoint where you'd like the debugger to stop. Yous may fix
pydevd_pycharm.settrace(..., append=False) if you'd like to avoid
the debugger from stopping on the line that settrace is on.
Run the Remote Debug configuration in PyCharm.
Run the python script $ START_DEBUGGER=ane python app.py

Restart server on file changes with watchgod

One of the challenges with remote debugging is that by default if yous brand lawmaking changes, they will not exist reflected immediately on the server. If yous're non aware of that y'all'll be debugging an older version of your code and seeing results that don't correlate to the changes that you made.

What you demand to exercise is to restart the python procedure when a python file is updated. We tin do this automatically with watchgod.

Then lets go through that.

install watchgod
create a second script that will telephone call your main python script that turns on the debugger. It should look something similar this:

                              # debug.py                import                subprocess                                  def                  main                  ():                subprocess.phone call(['python',                'app.py'])                # that'due south the same every bit the shell command ``$ python app.py``

Run the python script with watchgod $ START_DEBUGGER=ane watchgod debug.chief
Now in your IDE, upload the file

Just plow on the debugger for certain routes in Flask (only supported with PyCharm)

Mayhap you want to turn on the debugger only when you'd like to?

Hither'southward an instance of how to do that in Flask. Unfortunately, the IDE packages don't allow disabling the debugger once it's been turned on so we're going to have to brute-force it by killing the python process. That means that you'll need to manually get-go it again or y'all could use something similar gunicorn that'll spawn a new procedure after it sees that the old process gets killed.

Follow along with me.

create a function that'll check a query parameter and then trigger
the debugger to beginning
discover the call_immediately=Truthful statement in attach_pycharm.
That allows you to trigger the debugger based on your gear up of rules.
call this function in the torso of the route

                              from                flask                import                Flask, asking  is_debugger_enabled =                False                                  def                  attach_debugger                  ():                global                is_debugger_enabled                if                request.args.get('START_DEBUGGER',                ''):         debugger_helper.attach_pycharm(lambda                host, port: pydevd_pycharm.settrace(host, port=port, stdoutToServer=Truthful, stderrToServer=Truthful, suspend=Faux), call_immediately=True)         is_debugger_enabled =                True                elif                is_debugger_enabled:         impress('Trying to disable debugger that was enabled. Killing process to start a fresh ane.')         sys.exit(ane)                @app.road("/")                                  def                  howdy                  ():                attach_debugger()     message =                "How-do-you-do Worlddd!"                render                bulletin

follow the instructions for Setting up Pycharm above
call the URL in the browser with the query parameter
START_DEBUGGER appended to it. http://IP:5000/?START_DEBUGGER=1

Savor the taste of remote debugging

At present that y'all've learned how to exercise remote debugging, get ahead and get your hands dirty. One time you've used it, yous'll be request yourself why you didn't use information technology earlier.

Happy debugging!

Bask this mail service? Give Jorge Colon a like if it'south helpful.

I help your evolution teams become smarter, efficient, and solve complex bug for mission critical systems

**Mentees** Y'all're frustrated. You've spent hours trying to figure it out and it doesn't make sense. My specialty is debugging difficult issues. I've helped dozens of people on other platforms including startup founders to not just...

Discover and read more posts from Jorge Colon