[Fab-user] Status update for 1.3

[Jeff Forcier]

Hi all,

An update on 1.3.


## Parallel execution

Parallel execution landed in master about a week ago and should be in
decent shape for common use cases.

It uses a multiprocessing approach, so you should parallelize tasks
which don't need to communicate with one another or aggregate results
-- or put in your own IPC. Future feature releases may include some
IPC support if common patterns require it.

We've ensured that stdout/err during parallel runs isn't garbled
character-by-character, but it still multiplexes line-by-line, which
may make multiline output hard to read. Logging support is coming in
1.4 which will help re: complex debugging/postmortems.


## execute()

A long time coming: there's a new API function, execute(), which
encapsulates the core execution logic (it's now dogfooded in 'fab'
itself.) Give it a task or task name and it acts just like calling
"fab <task>", re: honoring @hosts/@roles, specifying host or role
overrides at runtime, etc.

Creating this involved a moderate amount of refactoring, but any bugs
found will be patched quickly (and it did involve adding a number of
new tests.)

A warning: since one call of execute() may run your task multiple
times, using it inside other tasks which themselves run on multiple
hosts is going to lead to lots of duplicate calls. You probably don't
want to do that :) It's mostly useful for "meta" tasks with no host
list of their own, or use via library code.


## Dependency changes

Unfortunately, to support parallelism we need a not-yet-released
patched version of the latest Paramiko -- you will not be able to use
Fabric 1.3 with any currently-released Paramiko (including 1.7.7.x.)
Furthermore, Paramiko 1.7+ requires a slightly newer PyCrypto (2.1+
instead of the 1.9+ from before.)

(We also need 'multiprocessing', but this is lazily imported, and is
part of the stdlib in Python 2.6+, so it won't be listed in our
setup.py.)

I recognize that requiring a more recent Paramiko/PyCrypto may be
troublesome for some, but we've never had a perfectly painless install
process for users with mitigating circumstances -- and I do not feel
the changes are severe enough to justify complicating the installation
process further.


## Dev docs

If you want to review the docs for these unreleased features, you can check out:

    http://docs.fabfile.org/en/latest/changelog.html

which reflects the latest master branch. I don't usually link to these
because I prefer to just release instead, but see the next section.

There's also a roadmap page now, which is **not set in stone or
exhaustive** but usually reflects my intended direction at any given
time:

    http://docs.fabfile.org/en/latest/roadmap.html


## Scheduling

Because we need this not-yet-released Paramiko with the abovementioned
patches, I cannot release Fabric until those patches are out on PyPI.
I am in talks with Paramiko's maintainer right now about the best way
to expedite that process, and will keep you posted about the results.

My hope is it won't take more than another week to work out something
acceptable.


Thanks,
Jeff

-- 
Jeff Forcier
Unix sysadmin; Python/Ruby engineer
http://bitprophet.org