wiki:Notes/PythonTwistdServer

Twisted Servers

Twisted out of the box.

Seems to be much more mature than a year or two ago ... despite many warnings and caveats ... how usable are they ?

Needs work.

Twistd Options

Usage: twistd [options]

Options:

      --savestats      save the Stats object rather than the text output of the
                       profiler.
  -o, --no_save        do not save state on shutdown
  -e, --encrypted      The specified tap/aos file is encrypted.
  -n, --nodaemon       don't daemonize, don't use default umask of 0077
      --originalname   Don't try to change the process name
      --syslog         Log to syslog, not to file
      --euid           Set only effective user-id rather than real user-id.
                       (This option has no effect unless the server is running
                       as root, in which case it means not to shed all
                       privileges after binding ports, retaining the option to
                       regain privileges in cases such as spawning processes.
                       Use with caution.)
  -l, --logfile=       log to a specified file, - for stdout
      --logger=        A fully-qualified name to a log observer factory to use
                       for the initial log observer. Takes precedence over
                       --logfile and --syslog (when available).
  -p, --profile=       Run in profile mode, dumping results to specified file
      --profiler=      Name of the profiler to use (profile, cprofile, hotshot).
                       [default: hotshot]
  -f, --file=          read the given .tap file [default: twistd.tap]
  -y, --python=        read an application from within a Python file (implies
                       -o)
  -s, --source=        Read an application from a .tas file (AOT format).
  -d, --rundir=        Change to a supplied directory before running [default:
                       .]
      --prefix=        use the given prefix when syslogging [default: twisted]
      --pidfile=       Name of the pidfile [default: twistd.pid]
      --chroot=        Chroot to a supplied directory before running
  -u, --uid=           The uid to run as.
  -g, --gid=           The gid to run as.
      --umask=         The (octal) file creation mask to apply.
      --help-reactors  Display a list of possibly available reactor names.
      --version        Print version information and exit.
      --spew           Print an insanely verbose log of everything that happens.
                       Useful when debugging freezes or locks in complex code.
  -b, --debug          Run the application in the Python Debugger (implies
                       nodaemon), sending SIGUSR2 will drop into debugger
  -r, --reactor=       Which reactor to use (see --help-reactors for a list of
                       possibilities)
      --help           Display this help and exit.

twistd reads a twisted.application.service.Application out of a file and runs it.

Commands:

    conch            A Conch SSH service.
    dns              A domain name server.
    ftp              An FTP server.
    inetd            An inetd(8) replacement.
    mail             An email service
    manhole          An interactive remote debugger service accessible via
                     telnet and ssh and providing syntax coloring and basic line
                     editing functionality.
    manhole-old      An interactive remote debugger service.
    news             A news server.
    portforward      A simple port-forwarder.
    procmon          A process watchdog / supervisor
    socks            A SOCKSv4 proxy service.
    telnet           A simple, telnet-based remote debugging service.
    web              A general-purpose web server which can serve from a
                     filesystem or application resource.
    words            A modern words server
    xmpp-router      An XMPP Router server

Twistd FTP

Usage: twistd [options] ftp [options].

WARNING: This FTP server is probably INSECURE do not use it.

Options:

  -p, --port=            set the port number [default: 2121]
  -r, --root=            define the root of the ftp-site. [default:
                         /usr/local/ftp]
      --userAnonymous=   Name of the anonymous user. [default: anonymous]
      --help             Display this help and exit.
      --help-auth-type=  Show help for a particular authentication type.
      --auth=            Specify an authentication method for the server.
      --password-file=   Specify a file containing username:password login info
                         for authenticated connections. (DEPRECATED; see
                         --help-auth instead)
      --version          Display Twisted version and exit.
      --help-auth        Show all authentication methods available.

Twistd DNS

Twisted Names is both a domain name server as well as a client resolver library.

Twisted Names comes with an "out of the box" nameserver that can read most BIND-syntax zone files as well as a simple Python-based configuration format.

Twisted Names can act as an authoritative server, perform zone transfers from a master to act as a secondary, act as a caching nameserver, or any combination of these.

Twisted Names' client resolver library provides functions to query for all commonly-used record types as well as a replacement for the blocking gethostbyname() function provided by the Python stdlib socket module.

sage: twistd [options] dns [options]

Options:

  -c, --cache         Enable record caching
  -r, --recursive     Perform recursive lookups
  -v, --verbose       Increment verbosity level
  -i, --interface=    The interface to which to bind [default: ]
  -p, --port=         The port on which to listen [default: 53]
      --resolv-conf=  Override location of resolv.conf (implies --recursive)
      --hosts-file=   Perform lookups with a hosts file
      --help          Display this help and exit.
      --version       Display Twisted version and exit.
      --secondary=    Act as secondary for the specified domain, performing zone
                      transfers from the specified IP (IP/domain)
      --pyzone=       Specify the filename of a Python syntax zone definition
      --bindzone=     Specify the filename of a BIND9 syntax zone definition

Twistd Mail

An SMTP / POP3 email server plugin for twistd.

Twisted Mail contains high-level, efficient protocol implementations for both clients and servers of SMTP, POP3, and IMAP4.

Additionally, it contains an "out of the box" combination SMTP/POP3 virtual-hosting mail server.

Also included is a read/write Maildir implementation and a basic Mail Exchange calculator.

Examples:

  1. SMTP and POP server

twistd mail --maildirdbmdomain=example.com=/tmp/example.com --user=joe=password

Starts an SMTP server that only accepts emails to joe@… and saves them to /tmp/example.com.

Also starts a POP mail server which will allow a client to log in using username: joe@… and password: password and collect any email that has been saved in /tmp/example.com.

  1. SMTP relay

twistd mail --relay=/tmp/mail_queue

Starts an SMTP server that accepts emails to any email address and relays them to an appropriate remote SMTP server. Queued emails will be temporarily stored in /tmp/mail_queue.

Usage: twistd [options] mail [options]

Options:

  -E, --esmtp                 Use RFC 1425/1869 SMTP extensions
      --disable-anonymous     Disallow non-authenticated SMTP connections
      --no-pop3               Disable the default POP3 server.
      --no-smtp               Disable the default SMTP server.
  -S, --pop3s=                Port to start the POP3-over-SSL server on (0 to
                              disable). DEPRECATED: use '--pop3
                              ssl:port:privateKey=pkey.pem:certKey=cert.pem'
                              [default: 0]
  -c, --certificate=          Certificate file to use for SSL connections.
                              DEPRECATED: use '--pop3
                              ssl:port:privateKey=pkey.pem:certKey=cert.pem'
  -R, --relay=                Relay messages according to their envelope 'To',
                              using the given path as a queue directory.
  -H, --hostname=             The hostname by which to identify this server.
  -D, --default               Make the most recently specified domain the
                              default domain.
  -A, --aliases=              Specify an aliases(5) file to use for this domain
      --help-auth-type=       Show help for a particular authentication type.
  -s, --smtp=                 Add an smtp port listener on the specified
                              endpoint. You can listen on multiple ports by
                              specifying multiple --smtp options For backwards
                              compatibility, a bare TCP port number can be
                              specified, but this is deprecated. [SSL Example:
                              ssl:8465:privateKey=mycert.pem] [default:
                              tcp:8025]
  -d, --maildirdbmdomain=     Generate an SMTP/POP3 virtual domain. This option
                              requires an argument of the form 'NAME=PATH' where
                              NAME is the DNS Domain Name for which email will
                              be accepted and where PATH is a the filesystem
                              path to a Maildir folder. [Example:
                              'example.com=/tmp/example.com']
      --auth=                 Specify an authentication method for the server.
  -p, --pop3=                 Add a pop3 port listener on the specified
                              endpoint. You can listen on multiple ports by
                              specifying multiple --pop3 options. For backwards
                              compatibility, a bare TCP port number can be
                              specified, but this is deprecated. [SSL Example:
                              ssl:8995:privateKey=mycert.pem] [default:
                              tcp:8110]
      --version               Display Twisted version and exit.
  -u, --user=                 add a user/password to the last specified domains
      --help-auth             Show all authentication methods available.
  -b, --bounce-to-postmaster  undelivered mails are sent to the postmaster
      --help                  Display this help and exit.

Twistd News

Twisted News provides a very basic NNTP server, as well as an NNTP client protocol implementation.

Two messages storage systems are supported: the DB-API 2.0 backend stores and indexes messages in any compatible SQL database; the Twisted dirdbm backend uses serialized Python objects stored directly on the filesystem for message storage.

Twisted News also has very rudimentary support for moderated groups.

Usage: twistd [options] news [options]

Options:

  -p, --port=          Listen port [default: 119]
  -i, --interface=     Interface to which to bind [default: ]
  -d, --datadir=       Root data storage path [default: news.db]
  -m, --mailhost=      Host of SMTP server to use [default: localhost]
      --group=         The name of a newsgroup to carry.
      --help           Display this help and exit.
      --server=        The address of a Usenet server to pass messages to and
                       receive messages from.
      --moderator=     The email of the moderator for the most recently passed
                       group.
      --version        Display Twisted version and exit.
      --subscription=  A newsgroup to list as a recommended subscription.

Twistd XMPP Router

Usage: twistd [options] xmpp-router [options]

Options:

  -v, --verbose  Log traffic
      --port=    Port components connect to [default:
                 tcp:5347:interface=127.0.0.1]
      --secret=  Router secret [default: secret]
      --version  Display Twisted version and exit.
      --help     Display this help and exit.

Twistd Port Forwarder

Port Forwarder.

Usage: twistd [options] portforward [options]

Options:

  -p, --port=       Set the port number. [default: 6666]
  -h, --host=       Set the host. [default: localhost]
  -d, --dest_port=  Set the destination port. [default: 6665]
      --version     Display Twisted version and exit.
      --help        Display this help and exit.

Twistd Process Montior

procmon runs processes, monitors their progress, and restarts them when they die.

procmon will not attempt to restart a process that appears to die instantly; with each "instant" death (less than 1 second, by default), it will delay approximately twice as long before restarting it. A successful run will reset the counter.

twistd procmon sleep 10

Usage: twistd [options] procmon [procmon options] commandline

Options:

  -t, --threshold=        How long a process has to live before the death is
                          considered instant, in seconds. [default: 1]
  -k, --killtime=         How long a process being killed has to get its affairs
                          in order before it gets killed with an unmaskable
                          signal. [default: 5]
  -m, --minrestartdelay=  The minimum time (in seconds) to wait before
                          attempting to restart a process [default: 1]
  -M, --maxrestartdelay=  The maximum time (in seconds) to wait before
                          attempting to restart a process [default: 3600]
      --version           Display Twisted version and exit.
      --help              Display this help and exit.

Twistd Web

This starts a webserver. If you specify no arguments, it will be a demo webserver that has the Test class from twisted.web.demo in it.

Usage: twistd [options] web [web options]

Options:

      --personal          Instead of generating a webserver, generate a
                          ResourcePublisher which listens on the port given by
                          --port, or ~/.twistd-web-pb if --port is not
                          specified.
  -n, --notracebacks      Do not display tracebacks in broken web pages.
                          Displaying tracebacks to users may be security risk!
  -p, --port=             strports description of the port to start the server
                          on.
  -l, --logfile=          Path to web CLF (Combined Log Format) log file.
      --https=            Port to listen on for Secure HTTP.
  -c, --certificate=      SSL certificate to use for HTTPS. [default:
                          server.pem]
  -k, --privkey=          SSL certificate to use for HTTPS. [default:
                          server.pem]
  -i, --index=            Add the name of a file used to check for directory
                          indexes. [default: index, index.html]
      --wsgi=             The FQPN of a WSGI application object to serve as the
                          root resource of the webserver.
      --help              Display this help and exit.
      --class=            Create a Resource subclass with a zero-argument
                          constructor.
      --version           Display Twisted version and exit.
      --ignore-ext=       Specify an extension to ignore. These will be
                          processed in order.
  -u, --user              Makes a server with ~/public_html and ~/.twistd-web-pb
                          support for users.
      --allow-ignore-ext  Specify whether or not a request for 'foo' should
                          return 'foo.ext'
      --path=             <path> is either a specific file or a directory to be
                          set as the root of the web server. Use this if you
                          have a directory full of HTML, cgi, epy, or rpy files
                          or any other files that you want to be served up raw.
      --processor=        `ext=class' where `class' is added as a Processor for
                          files ending with `ext'.
  -m, --mime-type=        Specify the default mime-type for static files.
      --resource-script=  An .rpy file to be used as the root resource of the
                          webserver.

Twistd Words

Twisted Words contains implementations of many Instant Messaging protocols, including IRC, Jabber, MSN, OSCAR (AIM & ICQ), TOC (AOL), and some functionality for creating bots, inter-protocol gateways, and a client application for many of the protocols.

In support of Jabber, Twisted Words also contains X-ish, a library for processing XML with Twisted and Python, with support for a Pythonic DOM and an XPath-like toolkit.

Usage: twistd [options] words [options]

Options:

      --hostname=        Name of this server; purely an informative [default:
                         hcmint]
      --irc-port=        strports description of the port to bind for the irc
                         server
      --pb-port=         strports description of the port to bind for the pb
                         server
      --group=           Specify a group which should exist
      --help             Display this help and exit.
      --passwd=          Name of a passwd-style file. (This is for
                         backwards-compatibility only; you should use the --auth
                         command instead.)
      --help-auth-type=  Show help for a particular authentication type.
      --auth=            Specify an authentication method for the server.
      --version          Display Twisted version and exit.
      --help-auth        Show all authentication methods available.

Twistd Conch

Will run on Windows as SSH Server ??? Seems to, but ... See MicrosoftWindows#TwistedOnWindows.

Makes a Conch SSH server. If no authentication methods are specified, the default authentication methods are UNIX passwords, SSH public keys, and PAM if it is available. If --auth options are passed, only the measures specified will be used.

Usage: twistd [options] conch [-i <interface>] [-p <port>] [-d <dir>]

Options:

  -i, --interface=       local interface to which we listen [default: ]
  -p, --port=            Port on which to listen [default: tcp:22]
  -d, --data=            directory to look for host keys in [default: /etc]
      --moduli=          directory to look for moduli in (if different from
                         --data)
      --help-auth-type=  Show help for a particular authentication type.
      --version          Display Twisted version and exit.
      --help             Display this help and exit.
      --auth=            Specify an authentication method for the server.
      --help-auth        Show all authentication methods available.

Conch is an SSHv2 implementation written in Python.

SSH is a protocol designed to allow remote access to shells and commands, but it is generic enough to allow everything from TCP forwarding to generic filesystem access.

Since conch is written in Python, it interfaces well with other Python projects, such as Imagination.

Conch also includes a implementations of the telnet and vt102 protocols, as well as support for rudamentary line editing behaviors.

A new implementation of Twisted's Manhole 'telnet client' application is also included in conch, featuring server-side input history and interactive syntax coloring.

Ok, but very little documentation.

http://as.ynchrono.us/2011/03/twisted-conch-in-60-seconds.html

... This series will focus on using Twisted Conch as an SSH library, and the first articles will cover writing custom SSH servers, with clients covered later ...

First in series ...

http://as.ynchrono.us/search/label/conch

Also See

PythonTwisted

MicrosoftWindows#TwistedOnWindows

Last modified 16 months ago Last modified on 01/05/2016 03:23:33 PM