Verified Commit 06793904 authored by Renato Alves's avatar Renato Alves 🌱
Browse files

Merge remote-tracking branch 'py2020/master'

parents c545e2b5 6290819b
MANIFEST
build
dist
_build
docs/man/*.gz
docs/source/api/generated
docs/source/config/options
docs/source/config/shortcuts/*.csv
docs/source/interactive/magics-generated.txt
docs/gh-pages
jupyter_notebook/notebook/static/mathjax
jupyter_notebook/static/style/*.map
*.py[co]
__pycache__
*.egg-info
*~
*.bak
*ipynb_checkpoints*
.tox
.DS_Store
\#*#
.#*
.cache
.coverage
*.swp
.vscode
.pytest_cache
.python-version
venv*/
.idea/
......@@ -2,16 +2,22 @@
The EMBL Python User Group (EPUG) is a group of people at EMBL Heidelberg who share an interest in the Python programming language.
You can contact the group by sending email to [python@embl.de](mailto:python@embl.de).
That mailing list is managed by [Toby Hodges](mailto:toby.hodges@embl.de) -
That mailing list is managed by [Renato Alves](mailto:renato.alves@embl.de) -
to join EPUG, please send him an email.
Meetings usually take place fortnightly, on Tuesdays at 16:00:
see the full schedule [here](https://docs.google.com/spreadsheets/d/1cMDumZ8GdhsTaiWzJl1qHDfi192sl4P1KJJhtuk0ZkE/edit?usp=sharing).
## Past Sessions
## Adding material to this repository
2019-04-30: [`dataclasses`](meetings/2019/2019-04-30-dataclasses/)
Material for each session should be added in a new folder,
with a name describing the main topic covered in that session.
Material from multiple sessions on a similar topic can be combined
together in a single folder.
2019-05-28: [`python cinema: How to think about data visualisation with Jake VanderPlas`](meetings/2019/2019-05-28-data-visualisation)
2019-06-11: [`python closures`](meetings/2019/2019-06-11-python-closures)
Material can be added to this repository by merge request.
To gain push access to the repository,
and for guidance on creating your merge request,
please contact one of the EPUG organisers:
- Wasiu Akanni
- Renato Alves
# Debugging in Python
## Some concepts
* *Breakpoint* - location in a file where the debugger should pause
* *Continue* - resume execution until next breakpoint or unexpected condition
* *Step over* - execute the current line and pause at the beginning of the next
* *Step into* - if the current line is a function call, enter the function and execute from the first line of the function
* *Step out* - execute the current function till the end and pause as soon as the function returns
* *Backtrace* - the execution path up to the location where the debugger is paused
## `pdb` cheat sheet
* `args` - show arguments from surrounding function
* `b` & `break` - set or list breakpoints
* `display` - show tracked variables
* `n` & `next` - execute instruction in next line (step over)
* `s` & `step` - execute next possible instruction (step into)
* `r` & `return` - execute until end of current function (step out)
* `p` & `pp` - print result of expression
* `ll` & `longlist` - show the source code/context of current paused state
* `bt` & `w` & `where` - show the execution path up to the current location
## launching pdb
Add `import pdb ; pdb.set_trace()` to your code and you will get access to `pdb` on the spot.
Alternatively run `python -mpdb script.py` from the command-line.
## VSCode / VSCodium
Install [Python extension](https://marketplace.visualstudio.com/items?itemName=ms-python.python) by Microsoft to have Python goodies and debugging powers.
## Remote debugging & development
The [remote development pack](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack) is pretty awesome. You might find the [Remote - SSH](https://aka.ms/vscode-remote/download/ssh) option very useful, together with the [Remote - Containers](https://aka.ms/vscode-remote/download/containers) if you work with a lot with docker and similar solutions.
### Reaching the location where the code is running
This requires some familiarity with network concepts and `SSH`.
You will need to setup port-forwarding (aka tunneling) using `SSH`.
You might also need to `ProxyJump` a few computers to get to the location where the code is running.
A simple, "one hop" `SSH` tunnel can be achieved with `ssh -L localhost:5678:localhost:5678 login.cluster.embl.de`.
For more about SSH tunnels try [these docs](https://www.ssh.com/ssh/tunneling/example) and [how to use `ProxyJump`](https://en.wikibooks.org/wiki/OpenSSH/Cookbook/Proxies_and_Jump_Hosts#Passing_Through_One_or_More_Gateways_Using_ProxyJump).
### debugpy
`debugpy` is [pretty awesome](https://github.com/microsoft/debugpy) but needs to connect via the network. You'll need to use the `SSH` tunnel setup above.
\ No newline at end of file
import os
import glob
def split(path):
return path.rsplit("/", 1)
def get_path(filename):
head, tail = split(filename)
return os.path.abspath(head)
def main(hostname):
extension = "one/**/*.py"
files = glob.glob(extension, recursive=True)
path = get_path(files[0])
print(">>>>", path, "<<<< on", hostname)
if __name__ == "__main__":
hostname = os.uname().nodename
if hostname == "login.cluster.embl.de":
import debugpy
print("debugpy loaded and waiting for connection")
debugpy.listen(12125)
debugpy.wait_for_client()
print("debugpy session active")
main(hostname)
\ No newline at end of file
../../../../../debugging.py
\ No newline at end of file
class MyExceptionValue(ValueError):
def __init__(self, error_code):
self.error_code = error_code
def __str__(self):
return f"The error number was {self.error_code}"
def __repr__(self):
return f"<Exception>: The error number was {self.error_code}"
def func():
raise MyExceptionValue(404)
%% Cell type:markdown id: tags:
# Defensive Programming
Defensive programming describes concepts and approaches we can use to ensure that...
* Code does what we think it does (i.e. is free of bugs)
* Our programs fail when they should...
* ... and do so with informative error messages (throwback to last session)
%% Cell type:code id: tags:
``` python
```
# Logging
## Usage
python intro_logging.py
or
python -m intro_logging
and with the current configuration you should see the output being saved to a
file `my.log` as well as printed to the console.
Edit `logging.conf` and re-run `intro_logging.py` to see how logged messages
get sent to different destinations depending on the order of loggers, their
level, propagate status and qualname.
## Useful links
* [logging HOWTO](https://docs.python.org/howto/logging.html) - from beginner to advanced
* [logging cookbook](https://docs.python.org/howto/logging-cookbook.html)
* [`logging`](https://docs.python.org/library/logging.html)
* [`logging.config`](https://docs.python.org/library/logging.config.html) - the many ways to configure the logging module
* [`logging.handlers`](https://docs.python.org/library/logging.handlers.html) - different ways to handle logging output
# -*- coding: utf-8 -*-
import logging
import logging.config
# Using a basic configuration
# logging.basicConfig(format="%(asctime)s %(levelname)-8s %(pathname)s: %(message)s",
# level=logging.DEBUG)
#
# or a configuration file to assist and allow customization
logging.config.fileConfig("logging.conf")
# more on configuring logging at:
# https://docs.python.org/howto/logging.html#configuring-logging
# Logs can be sent to different destinations, not just the console output
# You could send to a log server, a file, a different process, via email...
#
# For different handlers see:
# https://docs.python.org/library/logging.handlers.html
# The name passed to getLogger() can be used in logging.conf's qualname
# attribute to target the output of a specific logger
LOG = logging.getLogger(__name__)
def main():
# Logging uses an implicit severity level
# See: https://docs.python.org/library/logging.html#logging-levels
# CRITICAL
# ERROR
# WARNING
# INFO
# DEBUG
LOG.critical("HAAA!!! (running around with hands in the air!)")
LOG.error("OOPS!!")
# variables can be passed using %s as placeholder
variable = ("Bob", "is", "here")
LOG.warning("WARN: %s %s %s!!", *variable)
LOG.info("This is info")
LOG.debug("More verbosity")
other()
def other():
try:
1 / 0
except ZeroDivisionError:
LOG.exception("We shouldn't get zeros here")
LOG.critical("But life goes on")
if __name__ == "__main__":
main()
# Here you need at least one logger, one handler and one formatter
[loggers]
# The order defined here and in other sections is important
# messages flow from right to left
keys=root,file_log
[handlers]
keys=console,file
[formatters]
keys=my_formatter
[formatter_my_formatter]
# For additional formatting keywords see:
# https://docs.python.org/library/logging.html#logrecord-attributes
# The negative number next to the keywords defines white space padding
format=%(asctime)s %(levelname)-8s %(funcName)-5s %(pathname)s: %(message)s
[handler_console]
# For different types of handlers see:
# https://docs.python.org/library/logging.handlers.html
class=StreamHandler
level=DEBUG
formatter=my_formatter
# The order of arguments here corresponds to the arguments of StreamHandler
args=(sys.stdout,)
[handler_file]
# For different types of handlers see:
# https://docs.python.org/library/logging.handlers.html
class=FileHandler
level=DEBUG
formatter=my_formatter
# The order of arguments here corresponds to the arguments of FileHandler
# Use 'a' to preserve the logfile across executions and 'w' to empty the logfile
args=("my.log", 'w',)
[logger_root]
level=DEBUG
handlers=console
[logger_file_log]
level=DEBUG
handlers=file
# if propagate=0 messages are used and consumed by the current logger
# if propagate=1 they are used and passed to the next logger
propagate=1
# qualname restricts the current logger to only handle output of one target.
# The target name is specified in logging.getLogger("target")
qualname=__main__
# -*- coding: utf-8 -*-
print(__name__)
# vim: ai sts=4 et sw=4
This diff is collapsed.
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment