Python, the programming language, gained lot’s of popularity only in the past decade. In particular for big data applications, machine learning and data science the language is almost without alternative. But also for tool development or web applications backends, Python has huge adoption. Reasons are it’s huge ecosystem and a friendly, constructive community. Despite it’s newer competitors it has been around for 30 years. One of the most appreciated benefits is the steep learning curve, that allows virtually everyone to understand Python code.
Dropbox has an interview with Guido van Rossum, who published the first version of the language in 1989. The conversation revolves around the purpose of code and how python helps improve cooperation and productivity.
“You primarily write your code to communicate with other coders, and, to a lesser extent, to impose your will on the computer.”
When Python3 came out in 2009, it was already heavily debated. Python3 would be incompatible with previous versions of the popular language, but fix many drawbacks. While the vision was clear and the community initially planned to move forward much quicker. The demand for having a 2.x branch was so huge, however, that the community decided to extend support for 2.7 until the end of 2019. Stack Overflow took a look on why the path took so long.
Type Annotation is a feature that allows Python to maintain it’s dynamic typing and enable option static typing in the same code base. With the arrival of Python 3.5, the language implemented PEP 484, that describes a syntax to annotate code with type hints. Dropbox took a journey to leverage this option on 4 million lines of code for better quality. Here are their experiences.
Dropbox is a big user of Python. It’s our most widely used language both for backend services and the desktop client app (we are also heavy users of Go, TypeScript, and Rust).
It’s time for the last beta release of Python 3.8. Go find it at: https://www.python.org/downloads/release/python-380b4/ This release is the last of four planned beta release previews. Beta release previews are intended to give the wider community the opportunity to test new features and bug fixes and to prepare their projects to support the new feature release.
Jason Haley wrote a brief tutorial to get the Pythonista started with Kubernetes. Worth reading if you are new to the topic.
So, you know you want to run your application in Kubernetes but don’t know where to start. Or maybe you’re getting started but still don’t know what you don’t know. In this blog you’ll walk through how to containerize an application and get it running in Kubernetes.This walk-through assumes you are a developer or at least comfortable with the command line (preferably bash shell).
Celery is a distributed task execution environment for Python. While the emphasis is on distributed in this software, the concept of having workers allows for settings beyond the individual task. While the first rule of optimisation is “don’t”, sharing database connections is a low hanging fruit in most cases. And this can be configured per worker with Celery provided signals. To create a database connection for individual worker instances, leverage these signals to create the connection when the worker starts.
This can be achieved leveraging the worker_process_init signal, and the corresponding worker_process_shutdown signal to clean up when the worker shuts down.
The code should obviously be picked up at worker start, hence the tasks.py file will be a good location to keep these settings.
from celery.signals import worker_process_init
from celery.signals import worker_process_shutdown
app = Celery('tasks', broker=CELERY_BROKER_URL)
db = None
log.debug('Initializing database connection for worker.')
db = sqlite3.connect("urls.sqlite")
log.debug('Closing database connectionn for worker.')
The example above opens a connection to a sqlite3 database, which in itself has other issues, but is only meant as an example. This connection is established for each individual worker at startup.