wiki:Notes/PythonTwisted

Python Twisted

It's a beast, but it's usable. FTP ?, SMTP ?, what else ?

https://pypi.python.org/pypi/Twisted

Twisted implements a variety of networking and communication protocols and exposes them all as method-calls on your Python objects. Client and server implementations are provided for various standard protocols, including:

  • HTTP (twisted.web)
  • IMAP, POP, SMTP (twisted.mail)
  • DNS (twisted.names)
  • TLS (core)
  • SSH,
  • Telnet (twisted.conch)
  • IRC, XMPP, OSCAR (twisted.words)
  • Ethernet, IP, TUN/TAP (twisted.pair)
  • NMEA (twisted.positioning)

Note is still Python 2.7 ... maybe, download for .whl is still 2.7, not sure what that means.

Recent developments -> https://twistedmatrix.com/trac/milestone/Python-3.x ... getting there apparently.

Also see PythonServers#Wokkel extensions to Twisted servers ...

List of Twisted projects on GitHub - https://github.com/twisted

Twisted 15.0.0 ...

Twisted supports integration of the Win32, Tk, GTK+ and GTK+ 2 event loops with its main event loop.

There is experimental support for Mac OS X and wxPython event loop integration, which you use at your peril.

Not very encouraging, but not as bad as it sounds ... see AppleOSX#InstallingTwisted

https://twistedmatrix.com/trac/

http://twistedmatrix.com/trac/wiki#MoreProtocols

Twisted also supports many common network protocols, including SMTP, POP3, IMAP, SSHv2, and DNS.

http://stackoverflow.com/questions/tagged/twisted

Projects Using Twisted

http://twistedmatrix.com/trac/wiki/ProjectsUsingTwisted

Limited Documentation for Twisted

http://twisted.readthedocs.org/en/latest/index.html

http://twistedmatrix.com/trac/wiki

http://twistedmatrix.com/trac/wiki/Documentation

http://twistedmatrix.com/documents/current/

http://twistedmatrix.com/documents/current/web/howto/

http://twistedmatrix.com/documents/current/api/moduleIndex.html

https://twistedmatrix.com/documents/current/core/howto/index.html

https://twistedmatrix.com/documents/current/core/howto/servers.html

https://twistedmatrix.com/documents/current/core/howto/clients.html

http://www.amazon.com/exec/obidos/ASIN/1449326110/jpcalsjou-20

http://krondo.com/?page_id=1327 - Twisted Introduction

This multi-part series introduces Asynchronous Programming and the Twisted networking framework.

http://www.aosabook.org/en/twisted.html

Simple Examples

IMAP4 Client

Twisted includes a sophisticated IMAP4 client library.

import sys

from twisted.internet import protocol, defer, endpoints, task
from twisted.mail import imap4
from twisted.python import failure

@defer.inlineCallbacks
def main(reactor, username="alice", password="secret",
         strport="ssl:host=example.com:port=993"):
    endpoint = endpoints.clientFromString(reactor, strport)
    factory = protocol.Factory()
    factory.protocol = imap4.IMAP4Client
    try:
        client = yield endpoint.connect(factory)
        yield client.login(username, password)
        yield client.select('INBOX')
        info = yield client.fetchEnvelope(imap4.MessageSet(1))
        print 'First message subject:', info[1]['ENVELOPE'][1]
    except:
        print "IMAP4 client interaction failed"
        failure.Failure().printTraceback()

# This API requires Twisted 12.3 or later, or a trunk checkout:
task.react(main, sys.argv[1:])

Publish/Subscribe Server

Here's a simple publish/subscribe server, where clients see all messages posted by other clients:

from twisted.internet import reactor, protocol, endpoints
from twisted.protocols import basic

class PubProtocol(basic.LineReceiver):
    def __init__(self, factory):
        self.factory = factory

    def connectionMade(self):
        self.factory.clients.add(self)

    def connectionLost(self, reason):
        self.factory.clients.remove(self)

    def lineReceived(self, line):
        for c in self.factory.clients:
            c.sendLine("<{}> {}".format(self.transport.getHost(), line))

class PubFactory(protocol.Factory):
    def __init__(self):
        self.clients = set()

    def buildProtocol(self, addr):
        return PubProtocol(self)

endpoints.serverFromString(reactor, "tcp:1025").listen(PubFactory())
reactor.run()

Proxy Server

Need something lighter and more dumbed down than enterprise-ready Nginx.

Found this example, but ...

# Copyright (c) Twisted Matrix Laboratories.
# See LICENSE for details.

"""
This example demonstrates a very simple HTTP proxy.

Usage:
    $ python proxy.py

Then configure your web browser to use localhost:8080 as a proxy, and visit a
URL (This is not a SOCKS proxy). When browsing in this configuration, this
example will proxy connections from the browser to the server indicated by URLs
which are visited.

See also logging-proxy.py for a proxy with additional features.
"""

from twisted.web import proxy, http
from twisted.internet import reactor

class ProxyFactory(http.HTTPFactory):
    def buildProtocol(self, addr):
        return proxy.Proxy()

reactor.listenTCP(8080, ProxyFactory())
reactor.run()

Plugins

txMongo

https://pypi.python.org/pypi/txmongo

txmongo is a Python/Twisted driver for MongoDB that implements the wire protocol on non-blocking sockets. The API derives from the original pymongo.

Latest release 2015-01-23, tar.gz size 34KB

https://github.com/fiorix/txmongo

Requires Twisted, pymongo

In practice, pymongo dependencies other than pymongo.error are limited in the connection module and are very thin at that. parse_uri and auth._auth_key used once each... the rest is a reimplementation using Twisted callbacks.

http://txmongo.readthedocs.org/en/latest/

http://stackoverflow.com/questions/25557521/what-exactly-is-wrong-with-using-pymongo-in-twisted

In short: pymongo calls are blocking, when ran they freeze the twisted engine until the call returns, which is randomly destructive to twisted internal state because it's the opposite of what twisted is designed for.

Instead you should look for a twisted comparable driver such as tx-mongo. tx-mongo has a smaller audience then pymongo ...

Tutorials

http://krondo.com/an-introduction-to-asynchronous-programming-and-twisted/

This multi-part series introduces Asynchronous Programming and the Twisted networking framework ...

Also See

PythonTwistdServer

PythonMongoDB

Search wiki for 'twisted'

Last modified 6 months ago Last modified on 10/18/2016 07:14:02 PM