On Fri, Apr 3, 2009 at 10:34 PM, Paul Baumgart <
address@hidden> wrote:
> Ok, fair enough. :-)
>
> How's this then:
>
> @operation
> def rsync_project(remotedir, exclude=[], delete=False, extra_opts='',
> **kwargs):
> """
> Uploads the current project directory using rsync.
> By using rsync, only changes since the last upload are actually sent
> over
> the wire, rather than the whole directory like when using
> upload_project.
>
> Requires the rsync command-line utility to be available both on the
> local
> and the remote machine.
>
> Parameters are:
> remotedir: the directory on the remote machine to which to
> rsync the current project. The project directory
> becomes a subdirectory of the remotedir.
> exclude (optional): values passed to rsync's --exclude option.
> If the parameter object is iterable, (that is,
> it
> defines an __iter__ method), each of its
> elements is
> passed to a separate --exclude option.
> Otherwise,
> the object is passed as a string to a single
> --exclude option.
> See the rsync manpage for details on specifying
> filter rule arguments for --exclude.
> delete (optional): True or False, whether to delete remote files
> that
> don't exist locally. Defaults to False.
> extra_opts (optional): Additional command-line options to set for
> rsync.
>
> The rsync command is built from the options as follows:
> rsync [--delete] [--exclude exclude[0][, --exclude[1][, ...]]] \\
> -pthrvz [extra_opts] ../<project dir>
> <fab_user>@<host>:<remotedir>
> """
> username = ENV.get('fab_user')
>
> if not hasattr(exclude, '__iter__'):
> exclude = [exclude]
>
> exclude_opts = ' --exclude "%s"' * len(exclude)
> exclusions = tuple([str(s).replace('"', '\\\\"') for s in exclude])
>
> options_map = {
> "delete" : '--delete' if delete else '',
> "exclude" : exclude_opts % exclusions,
> "extra" : extra_opts
> }
> options = "%(delete)s%(exclude)s -pthrvz %(extra)s" % options_map
> cwd = '../' + os.getcwd().split(os.sep)[-1]
> userhost = "$(fab_user)@$(fab_host)"
> rdir = _lazy_format(remotedir, ENV)
>
> cmd = "rsync %s %s %s:%s" % (options, cwd, userhost, rdir)
> local_per_host(cmd, **kwargs)
>
>
> Besides reducing the length of lines, I also made it more pythonic by using
> duck typing to determine how to handle the exclude parameter. So now you can
> pass integers, complex numbers, or anything else with a __str__ method and
> it will work.
>
> I also cleaned up the documentation a bit and made some parts of it more
> precise.
>
> Paul
>
>
>
>
>
> On Fri, Apr 3, 2009 at 10:50 AM, Christian Vest Hansen
> <
address@hidden> wrote:
>>
>> Good idea. But as you may have guessed, I'm not fond of long lines :)
>>
>> On Fri, Apr 3, 2009 at 5:21 AM, Paul Baumgart <
address@hidden> wrote:
>> > Hi Christian,
>> >
>> > Thanks, your version looks a lot cleaner.
>> >
>> > I would like to add one thing, because I think it makes the exclude
>> > parameter much more useful:
>> >
>> > @operation
>> > def rsync_project(remotepath, exclude=False, delete=False,
>> > extra_opts='',
>> > **kwargs):
>> > """
>> > Uploads the current project directory using rsync.
>> > By using rsync, only changes since last upload are actually sent over
>> > the wire, rather than the whole directory like using upload_project.
>> >
>> > Requires the rsync command-line utility to be available both on the
>> > local
>> > and the remote machine.
>> >
>> > Parameters are:
>> > remotepath: the path on the remote machine to which to
>> > rsync
>> > the
>> > current project
>> > exclude (optional): values passed to rsync's --exclude option.
>> > If the parameter is iterable, each of its
>> > elements
>> > is passed to a separate --exclude argument.
>> > If
>> > the
>> > parameter is a string, that string is the
>> > value
>> > passed to the single --exclude argument.
>> > See the rsync manpage for details on using
>> > the
>> > --exclude option.
>> > delete (optional): True or False, whether to delete remote files
>> > that
>> > don't exist locally.
>> > extra_opts (optional): Additional command-line options to set for
>> > rsync.
>> >
>> > The rsync command is built from the options as follows:
>> > rsync [--delete] [--exclude exclude] -pthrvz [extra_opts] \\
>> > <project dir> <fab_user>@<host>:<remotepath>
>> > """
>> > username = ENV.get('fab_user')
>> >
>> > if isinstance(exclude, basestring):
>> > exclude = [exclude]
>> >
>> > options_map = {
>> > "delete" : '--delete' if delete else '',
>> > "exclude" : exclude and ' '.join(['--exclude "%s"' %
>> > opt.replace('"',
>> > '\\\\"') for opt in exclude]) or '',
>> > "extra" : extra_opts
>> > }
>> > options = "%(delete)s %(exclude)s -pthrvz %(extra)s" % options_map
>> > cwd = '../' + os.getcwd().split(os.sep)[-1]
>> > userhost = "$(fab_user)@$(fab_host)"
>> > rpath = _lazy_format(remotepath, ENV)
>> >
>> > cmd = "rsync %s %s %s:%s" % (options, cwd, userhost, rpath)
>> > local_per_host(cmd, **kwargs)
>> >
>> > So, instead of taking just a string, it can take either a string or a
>> > list,
>> > and multiple --exclude options are generated from a list. A string
>> > parameter
>> > is treated just as it is currently.
>> >
>> > Paul
>> >
>> > On Thu, Apr 2, 2009 at 12:38 PM, Christian Vest Hansen
>> > <
address@hidden> wrote:
>> >> Ok. I implemented this change, but I modified it a little bit.
>> >>
>> >> On Thu, Apr 2, 2009 at 9:17 PM, Christian Vest Hansen
>> >> <
address@hidden> wrote:
>> >>> On Sun, Mar 22, 2009 at 12:51 AM, Paul Baumgart <
address@hidden>
>> >>> wrote:
>> >>>> A different topic this time:
>> >>>>
>> >>>> The current upload_project() function is a neat feature, but I have
>> >>>> two issues with it:
>> >>>>
>> >>>> 1) It doesn't allow me to exclude certain files/directories (like
>> >>>> .git* for example)
>> >>>> 2) It uploads the entire project every time, which can be irritating
>> >>>> if the change is small but the project is large.
>> >>>>
>> >>>> So, I made this function, which uses rsync to only upload the
>> >>>> differences between the local and remote copies.
>> >>>>
>> >>>> I would like to get feedback on it, primarily as to whether this
>> >>>> would
>> >>>> be useful in the mainline Fabric code, and if not, if there is any
>> >>>> way
>> >>>> to make it be useful.
>> >>>>
>> >>>> @operation
>> >>>> @connects
>> >>>> def rsync_project(host, client, env, remotepath, exclude=False,
>> >>>> delete=False, extra_opts='', **kwargs):
>> >>>> """
>> >>>> Uploads the current project directory using rsync, so only changes
>> >>>> are
>> >>>> uploaded rather than the whole directory like using
>> >>>> upload_project.
>> >>>>
>> >>>> Requires the rsync command-line utility to be available both on
>> >>>> the
>> >>>> local
>> >>>> and the remote machine.
>> >>>>
>> >>>> Parameters are:
>> >>>> remotepath: the path on the remote machine to which to
>> >>>> rsync the
>> >>>> current project
>> >>>> exclude (optional): the string passed to rsync's --exclude
>> >>>> option.
>> >>>> See the rsync manpage for details.
>> >>>> delete (optional): True or False, whether to delete remote
>> >>>> files
>> >>>> that
>> >>>> don't exist locally.
>> >>>> extra_opts (optional): Additional command-line options to set
>> >>>> for
>> >>>> rsync.
>> >>>>
>> >>>> The rsync command is built from the options as follows:
>> >>>> rsync [--delete] [--exclude exclude] -pthrvz [extra_opts]
>> >>>> <project dir> <fab_user>@<host>:<remotepath>
>> >>>> """
>> >>>>
>> >>>> remotepath = _lazy_format(remotepath, env)
>> >>>>
>> >>>> username = ENV.get('fab_user')
>> >>>> username = username + '@' if username else username
>> >>>>
>> >>>> cwd_name = '../' + os.getcwd().split(os.sep)[-1]
>> >>>
>> >>> This line right here. Why do you go through all this trouble to build
>> >>> a relative CWD?
>> >>
>> >> Elementary, dear Watson. To get a prettier output when the assembled
>> >> command is printed to the console.
>> >>
>> >>>
>> >>>
>> >>>>
>> >>>> delete_opt = '--delete' if delete else ''
>> >>>>
>> >>>> exclude_opt = '--exclude' if exclude else ''
>> >>>> exclude = '"' + exclude.strip('"') + '"' if exclude else ''
>> >>>>
>> >>>> return local('rsync %(delete_opt)s %(exclude_opt)s %(exclude)s
>> >>>> -pthrvz %(extra_opts)s %(cwd_name)s
>> >>>> %(username)s%(host)s:%(remotepath)s'
>> >>>> % locals(), **kwargs) == 0
>> >>>>
>> >>>>
>> >>>> Note that it requires adding a line to the bottom of local():
>> >>>>
>> >>>> return retcode
>> >>>>
>> >>>> Regards,
>> >>>> Paul
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> Fab-user mailing list
>> >>>>
address@hidden
>> >>>>
http://lists.nongnu.org/mailman/listinfo/fab-user
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Venlig hilsen / Kind regards,
>> >>> Christian Vest Hansen.
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Venlig hilsen / Kind regards,
>> >> Christian Vest Hansen.
>> >>
>> >
>> >
>>
>>
>>
>> --
>> Venlig hilsen / Kind regards,
>> Christian Vest Hansen.
>
>