wiki:Notes/PythonToolSets

Python Tool Sets

Collections of Python packages, generally supported by a 'community of usage'.

Packages tend to have better integrated tools, plus a well thought-out framework for integration.

So far, I have recognized three communities, scientific, gaming and music, pretty much in that order. I'm explicitly excluding the extra-huge 'web developer' community, deserving a page of its own, see PythonFramework.

Scientific Community

Large university community, often funded, if not generously funded ...

SciPy

Contains Numpy, Scipy, Matplotlib, IPython, SymPy and 'Pandas' or Panda3D.

http://www.scipy.org/index.html

http://docs.scipy.org/doc/scipy/reference/tutorial/general.html

SciPy is a collection of mathematical algorithms and convenience functions built on the Numpy extension of Python. It adds significant power to the interactive Python session by providing the user with high-level commands and classes for manipulating and visualizing data.

With SciPy an interactive Python session becomes a data-processing and system-prototyping environment rivaling sytems such as MATLAB, IDL, Octave, R-Lab, and SciLab.

http://www.scipy-lectures.org/

http://www.scipy-lectures.org/intro/intro.html

http://www.scipy-lectures.org/intro/intro.html#the-interactive-workflow-ipython-and-a-text-editor

http://www.scipy-lectures.org/packages/scikit-learn/index.html

http://scikit-learn.org/stable/

Machine Learning in Python

Simple and efficient tools for data mining and data analysis

Accessible to everybody, and reusable in various contexts

Built on NumPy, SciPy, and matplotlib

Anaconda

Big Data / Big Computing bundle. Free version with 300 python packages and a commercial version with tools and support ( 10K/10 seats/yr ! ).

https://www.continuum.io/

http://docs.continuum.io/anaconda/index

Anaconda is an easy-to-install, free package manager, environment manager, Python distribution, and collection of over 150 open source packages with free community support.

https://anaconda.org/

Excellent and well-groomed list of python packages: http://docs.continuum.io/anaconda/pkg-docs.

PythonBottle is not on there ... unsurprisingly.

PythonXY

https://python-xy.github.io/

Python(x,y) is a free scientific and engineering development software for numerical computations, data analysis and data visualization based on Python programming language, Qt graphical user interfaces and Spyder interactive scientific development environment.

https://python-xy.github.io/downloads.html

It is recommended to uninstall any other Python distribution before installing Python(x,y) ...

Other Scienific Packages

Based on #SciPy.

Cloud ...

https://www.enthought.com/products/canopy/

Scientific and Analytic Python Deployment with Integrated Analysis Environment

Enthought Canopy is a comprehensive Python analysis environment that provides easy installation of the core scientific analytic and scientific Python packages, creating a robust platform you can explore, develop, and visualize on.

In addition to its pre-built, tested Python distribution, Enthought Canopy has valuable tools for iterative data analysis, visualization and application development including:

One-Click Python Package Deployment with a Graphical Package Manager

Code Editor with IPython Notebook Support

Interactive Graphical Python Code Debugger

Integrated IPython Prompt

Convenient Documentation Browser

Python for Excel with PyXLL (add-on)

Integration with the Intel MKL and Microsoft Python Tools for Visual Studio

https://store.enthought.com/downloads/#default

... Want to Get Even More From Canopy? See our subscription options to unlock additional features such as: ... [ long list ]

More Scientific Tools

SymPy

http://www.sympy.org/en/index.html

SymPy is a Python library for symbolic mathematics. It aims to become a full-featured computer algebra system (CAS) while keeping the code as simple as possible in order to be comprehensible and easily extensible.

https://github.com/sympy/sympy/

https://github.com/sympy/sympy/wiki

Pandas

Pandas ( Python Data Analysis Library ) -> http://pandas.pydata.org/

pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language.

Not to be confused with Panda3D game engine.

Neural Networks

Also see https://en.wikipedia.org/wiki/Neural_network_software - several good up-to-date NN implementations, often big, often with dependencies to tools listed above.

Deep Mind

Worth looking in to, when have time ...

https://en.wikipedia.org/wiki/DeepMind

DeepMind Technologies Limited is a British artificial intelligence company founded in September 2010.

Acquired by Google in 2014, the company has created a neural network that learns how to play video games in a fashion similar to that of humans, as well as a Neural Turing machine,or a neural network that may be able to access an external memory like a conventional Turing machine, resulting in a computer that mimics the short-term memory of the human brain.

https://deepmind.com/

https://deepmind.com/blog/open-sourcing-sonnet/

It’s now nearly a year since DeepMind made the decision to switch the entire research organisation to using TensorFlow (TF). It’s proven to be a good choice - many of our models learn significantly faster, and the built-in features for distributed training have hugely simplified our code.

https://github.com/deepmind/sonnet

Sonnet is a library built on top of TensorFlow for building complex neural networks.

https://en.wikipedia.org/wiki/TensorFlow

TensorFlow is an open-source software library for machine learning across a range of tasks. It is a symbolic math library, and also used as a system for building and training neural networks to detect and decipher patterns and correlations, analogous to human learning and reasoning.

https://www.tensorflow.org/

TensorFlow™ is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API.

https://github.com/tensorflow

https://github.com/tensorflow/tensorflow

TensorFlow is an open source software library for numerical computation using data flow graphs. The graph nodes represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) that flow between them. This flexible architecture lets you deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device without rewriting code.

https://github.com/tensorflow/models/tree/master/research/inception#getting-started

#TODO

Gaming Community

https://wiki.python.org/moin/PythonGameLibraries

Panda3D

Sort of overlaps several other areas and communities, such as simulations and gaming, even audio to some extent.

http://www.panda3d.org/

Panda3D is a game engine, a framework for 3D rendering and game development for Python and C++ programs.

A sweet little package, though not so little ( 40-80 MB ). Almost a reason to use Python ?.

https://en.wikipedia.org/wiki/Panda3D

Blender

Blender is excellent too, also with great Python integration ... an 'embarras de richesse'.

https://en.wikipedia.org/wiki/Blender_Game_Engine

Since 1995, huge community.

https://www.blender.org/

Recommends quad CPU and 8 GB.

PyGame

http://www.pygame.org/wiki/about

pygame (the library) is a Free and Open Source python programming language library for making multimedia applications like games built on top of the excellent SDL library. Like SDL, pygame is highly portable and runs on nearly every platform and operating system. Millions of people have downloaded pygame itself, which is a whole lot of bits flying across the interwebs.

pygame.org (the website) welcomes all python game, art, music, sound, video and multimedia projects.

https://en.wikipedia.org/wiki/Pygame

http://pyopengl.sourceforge.net/

PyOpenGL is the most common cross platform Python binding to OpenGL and related APIs.

http://pygame.org/hifi.html

Minecraft

https://en.wikipedia.org/wiki/Minecraft

... In September 2014, Microsoft announced a deal to buy Mojang and the Minecraft intellectual property for US$2.5 billion ...

minecraft-pi

I used it on a Raspberry Pi 2. It runs and it was was OK. However, there seems to be a 'lesson' lurking in the bushes.

https://projects.raspberrypi.org/en/projects/getting-started-with-minecraft-pi

Last update 2013.

https://minecraft.gamepedia.com/Pi_Edition

... The original team has stopped supporting it and starting with 0.9 MCPE became a lot more ambitious technically, which also means that it became a lot harder to strip down to run on a Raspberry Pi. Actually, we finished removing all Pi related code to reduce complexity in 2015.

Maybe someday we'll revisit it but I wouldn't hold my breath ...

... On the other hand, Win10 runs on the Pi so you can run the Win10 version there! ... I guess it's something.

Actually, I don't doubt what the developers are saying. It must have been a 'bear' to maintain both.

Yet note that there have been persistent source availability and licensing/commercialization issues dogging Minecraft since its beginning. It may be more interesting as a lesson about what can happen in the real world from source/license issues.

python-minecraft

Educational, runs on Ubuntu. Integration with IPython

https://en.wikipedia.org/wiki/CoderDojo

CoderDojo? is a global volunteer-led community of free programming clubs for young people between 7 and 17. The movement is a grassroots organisation with individual clubs (called "Dojos") acting independently.

https://www.coderdojotc.org/

https://github.com/CoderDojoTC/python-minecraft

http://coderdojotc.readthedocs.io/projects/python-minecraft/en/latest/contents.html

Music Community

Smaller, overlaps with gaming community.

https://wiki.python.org/moin/PythonInMusic

Deployment

Compilers, C++ APIs, etc.

All the following are fairly complex tools. In fact, check out SWIG first. It's more 'standard' and may be enough for simple 'Hello C' apps.

http://www.swig.org/tutorial.html

http://www.swig.org/Doc1.3/Python.html

https://github.com/swig/swig/tree/master/Examples/python

Boost Python

https://grey.colorado.edu/emergent/index.php/Boost.Python

Boost.Python, a C++ library which enables seamless interoperability between C++ and the Python programming language ...

http://www.boost.org/

Boost provides free peer-reviewed portable C++ source libraries.

We emphasize libraries that work well with the C++ Standard Library. Boost libraries are intended to be widely useful, and usable across a broad spectrum of applications.

http://www.boost.org/doc/libs/1_63_0/libs/python/doc/html/index.html

https://en.wikibooks.org/wiki/Python_Programming/Extending_with_C%2B%2B

This page describes Boost.Python. Before the emergence of Cython, it was the most comfortable way of writing C++ extension modules.

Hmmm ... probably O.K. for a specialized RaspberryPi 'always on' server that needs dynamic 'signally' behavior, potentially can build for Windows but not a general purpose solution.

Note Nov 2017: I'm seeing more Boost Python apps floating around these days. It's mostly compuational, but 'they say' that the GUI features are good.

Projects Using Boost.Python -> http://www.boost.org/doc/libs/1_57_0/libs/python/doc/projects.html

Cython

It wasn't very active a few years ago and now it is. Mature, support Python 3.

http://cython.org/

Cython is an optimising static compiler for both the Python programming language and the extended Cython programming language (based on Pyrex). It makes writing C extensions for Python as easy as Python itself.

Cython gives you the combined power of Python and C to let you:

write Python code that calls back and forth from and to C or C++ code natively at any point.

easily tune readable Python code into plain C performance by adding static type declarations.

use combined source code level debugging to find bugs in your Python, Cython and C code ...

The Cython language is a superset of the Python language that additionally supports calling C functions and declaring C types on variables and class attributes. This allows the compiler to generate very efficient C code from Cython code.

It can be a bit more complicated, but if you need a compiler ...

Also See

PythonTools

QTApps#Emergent - neural net simulator

Last modified 5 weeks ago Last modified on 11/07/2017 11:58:44 AM