Python Multiprocessing And AsyncIO
Catch-all for synchronous/asynchronous process frameworks and in-process concurrent programming - crunchers.
Python Multiprocessing Module
No just about breaking the Python GIL lock ... it's an entire process framework with high level abstractions.
And no external dependencies !
https://docs.python.org/2/library/multiprocessing.html
http://pymotw.com/2/multiprocessing/
Async IO
Python Async IO Resources - Curated content from around the net to get started with asyncio
https://bitbucket.org/mrdon/asyncio-mongo
An asynchronous Python driver for the Mongo database, based on Python's asyncio. This project is based on TxMongo? ... still in the very early alpha stage and shouldn't be used for production.
Python 3 Async IO
A reason to move up to Python 3.
https://docs.python.org/3/library/asyncio.html
New in version 3.4 ...
The asyncio package has been included in the standard library on a provisional basis. Backwards incompatible changes (up to and including removal of the module) may occur if deemed necessary by the core developers ...
This module provides infrastructure for writing single-threaded concurrent code using coroutines, multiplexing I/O access over sockets and other resources, running network clients and servers, and other related primitives ...
Pluggable event loop with various system-specific implementations;
Transport and protocol abstractions (similar to those in Twisted);
Concrete support for TCP, UDP, SSL, subprocess pipes, delayed calls, and others (some may be system-dependent);
Future class that mimics the one in the concurrent.futures module, but adapted for use with the event loop;
Coroutines and tasks based on yield from (PEP 380), to help write concurrent code in a sequential fashion;
Cancellation support for Futures and coroutines;
Synchronization primitives for use between coroutines in a single thread, mimicking those in the threading module;
Interface for passing work off to a threadpool, for times when you absolutely, positively have to use a library that makes blocking I/O calls.
See: PythonTwisted, WebSocket
Greenlets
https://pypi.python.org/pypi/greenlet
The greenlet package is a spin-off of Stackless, a version of CPython that supports micro-threads called “tasklets”. Tasklets run pseudo-concurrently (typically in a single or a few OS-level threads) and are synchronized with data exchanges on “channels”.
Problematic on AppleOSX, unfortunately ... see AppleOSX#lnstallingGreenlet
Gevent
http://learn-gevent-socketio.readthedocs.org/en/latest/index.html
Gevent is the use of simple, sequential programming in python to achieve scalability provided by asynchronous IO and lightweight multi-threading (as opposed to the callback-style of programming using Twisted’s Deferred).
It is built on top of libevent/libev (for asynchronous I/O) and greenlets (lightweight cooperative multi-threading).
http://learn-gevent-socketio.readthedocs.org/en/latest/greenlets.html
What are greenlets?
Greenlets are lightweight thread-like structures that are scheduled and managed inside the process. They are references to the part of the stack that is used by the thread. Compared to POSIX threads (pthreads), there is no stack allocated up front and there is only as much stack as is actually used by the greenlet
In python, we implement greenlets via the gevent package and we implement pthreads via python’s built-in threading module.
Both green threads (greenlets) and POSIX threads (pthreads) are mechanisms to support multithreaded execution of programs ...
gevent’s monkey patch
A monkey patch is a way to extend or modify the run-time code of dynamic languages without altering the original source code. Monkey patching as a programming technique is very powerful but can result in hard-to-debug code in the wrong hands.
Hey, I resent that insinuation !
... so in other words, monkey patching can be a problem, as the name implies.
Twisted
See PythonTwisted
http://pymotw.com/2/multiprocessing/basics.html
