# # # rename "wiki/DaggyFixes.moin" # to "wiki/DaggyFixes.mdwn" # # patch "wiki/DaggyFixes.mdwn" # from [58b4395f8be0f8ccee41f3c37b691082ea6df17f] # to [3a2753bb5c10e14638917f47d68f4c41408c5d9e] # ============================================================ --- wiki/DaggyFixes.moin 58b4395f8be0f8ccee41f3c37b691082ea6df17f +++ wiki/DaggyFixes.mdwn 3a2753bb5c10e14638917f47d68f4c41408c5d9e @@ -1,191 +1,354 @@ -All software has bugs, and not all changes that you commit to a source tree are entirely good. -Therefore, some commits can be considered "development" (new features), and others can be considered "bugfixes" (redoing or sometimes undoing previous changes). -It can often be advantageous to separate the two: it is common practice to try and avoid mixing new code and bugfixes together in the same commit, often as a matter of project policy. This is because the fix can be important on its own, such as for applying critical bugfixes to stable releases without carrying along other unrelated changes. +[[!toc levels=2]] -Monotone, and other similar version control tools that utilise a DAG-structured revision history, offer developers an additional option to handle fixes in a way not possible (or at least, not convenient) in other tools with more linear history structures. +All software has bugs, and not all changes that you commit to a source +tree are entirely good. -This page describes a BestPractices workflow pattern that uses the DAG structure, and the capability to easily diverge and re-merge from arbitrary past revisions, for advantage in handling fixes. It's an application of the same principles used by the ZipperMerge pattern, but for the scenario where you want to make the same change to two or more branches that must remain separate otherwise, rather than merge two branches together. +Therefore, some commits can be considered "development" (new +features), and others can be considered "bugfixes" (redoing or +sometimes undoing previous changes). -= Scenario = +It can often be advantageous to separate the two: it is common +practice to try and avoid mixing new code and bugfixes together in the +same commit, often as a matter of project policy. This is because the +fix can be important on its own, such as for applying critical +bugfixes to stable releases without carrying along other unrelated +changes. -You have a project with a number of revisions and an existing graph that looks like this: {{{ - A - | - B - / \ - C D - \ / - E -}}} +Monotone, and other similar version control tools that utilise a +DAG-structured revision history, offer developers an additional option +to handle fixes in a way not possible (or at least, not convenient) in +other tools with more linear history structures. -It is later discovered that revision `B` introduced a bug, and there's a series of children -`B->..->E` that have inherited the bug from `B`. (There could, of course, be many many intermediate revisions between `B` and `E`). +This page describes a [[BestPractices]] workflow pattern that uses the DAG +structure, and the capability to easily diverge and re-merge from +arbitrary past revisions, for advantage in handling fixes. It's an +application of the same principles used by the [[ZipperMerge]] pattern, +but for the scenario where you want to make the same change to two or +more branches that must remain separate otherwise, rather than merge +two branches together. -In monotone, we have two useful choices for how (or, more specifically, ''where'') to '''commit''' the fix: +# Scenario +You have a project with a number of revisions and an existing graph +that looks like this: + + A + | + B + / \ + C D + \ / + E + +[[graph src="A -> B -> C -> E; B -> D -> E;" ]] + +It is later discovered that revision `B` introduced a bug, and there's +a series of children `B->..->E` that have inherited the bug from `B`. +(There could, of course, be many many intermediate revisions between +`B` and `E`). + +In monotone, we have two useful choices for how (or, more +specifically, *where*) to `commit` the fix: + * as a child of the current head, `E->F1`. - * as a direct child of the revision that introduced the problem, `B->F2` (and then '''merge''' the two heads `E` and `F2` to produce `M`). + * as a direct child of the revision that introduced the problem, + `B->F2` (and then `merge` the two heads `E` and `F2` to produce + `M`). -Both produce a new head with the bug fixed, but they leave behind a different ancestry graph, and that difference can be important and useful. +Both produce a new head with the bug fixed, but they leave behind a +different ancestry graph, and that difference can be important and +useful. -== Fix F1: the old way == +## Fix F1: the old way -The bug is fixed close to the point where development happened to be up to ''when it was found'':{{{ - A - | - B - / \ - C D - \ / - E - | - F1 -}}} +The bug is fixed close to the point where development happened to be +up to *when it was found*: -This is the common established practice in -traditional sequential VCSs, in part because in those systems, branching and merging aren't lightweight enough to be worthwhile using for a small fix. -You might annotate the fix with some -mention of `B` in the changelog as the bad revision being backed out or fixed, but -that's usually as far as it goes. + A + | + B + / \ + C D + \ / + E + | + F1 -There's nothing really wrong with this (after all, it has worked for most software developers for a long time), but it represents a lost opportunity to use the power of monotone to full advantage. +This is the common established practice in traditional sequential +VCSs, in part because in those systems, branching and merging aren't +lightweight enough to be worthwhile using for a small fix. -== Fix F2: the daggy way == +You might annotate the fix with some mention of `B` in the changelog +as the bad revision being backed out or fixed, but that's usually as +far as it goes. -The bug is fixed close to the point ''where it was introduced'', and then '''merge'''d back:{{{ - A - | - B --+ - / \ | - C D F2 - \ / | - E | - \ / - M -}}} +There's nothing really wrong with this (after all, it has worked for +most software developers for a long time), but it represents a lost +opportunity to use the power of monotone to full advantage. -There are good reasons to prefer this practice. This different graph structure gives you a strong and direct representation of the ''logical'' (rather than ''chronological'') relationship between bug, fix, and intermediate development: +## Fix F2: the daggy way +The bug is fixed close to the point *where it was introduced*, and +then `merge`d back: + + A + | + B --+ + / \ | + C D F2 + \ / | + E | + \ / + M + +There are good reasons to prefer this practice. This different graph +structure gives you a strong and direct representation of the +*logical* (rather than *chrono-logical*) relationship between bug, +fix, and intermediate development: + * You can see easily, in a tool like monotone-viz, the revisions containing - the bug: all the revisions in the parallel path(s) spanned by `B->F2->M`. + the bug: all the revisions in the parallel path(s) spanned by + `B->F2->M`. - * The fix is separated from other development as a direct part of the ancestry structure. Therefore, you can use the in-built merging tools of monotone, that understand how to work with this structure, to make sure the fix goes to other places that need it without confusion from other unrelated changes. + * The fix is separated from other development as a direct part of the + ancestry structure. Therefore, you can use the in-built merging tools + of monotone, that understand how to work with this structure, to make + sure the fix goes to other places that need it without confusion from + other unrelated changes. -This reverses the common practice of "backporting" fixes to older code (more on this below). Instead, the fix is developed against that older code, and then ported forward as part of the '''merge''' to the head. -This may also make the root cause of the error more apparent and thus easier to learn from, to avoid similar mistakes in future. +This reverses the common practice of "backporting" fixes to older code +(more on this below). Instead, the fix is developed against that +older code, and then ported forward as part of the `merge` to the +head. +This may also make the root cause of the error more apparent and thus +easier to learn from, to avoid similar mistakes in future. + There are some variations on this theme that may be useful to consider: - * Where the bug introduced in `B` represents the ''entirety'' of the change `A->B`, you may want `F2` to completely undo the change. - For this specific case, monotone has the '''disapprove''' command, which will internally create a new revision with the reverse diff as a child of `B` (this command name is unfortunate, because it is ''not'' symmetrical with '''approve'''). - * In some rare cases where you want to redo `B` completely differently from scratch, you might want to '''commit''' that as a child of `A` (and sibling of `B`) instead. You will then have to resolve any conflicts between the two alternate changes when you '''merge'''. This kind of pattern is most relevant for more fundamental changes than simple fixes, perhaps where the line of later development eventually leads to the conclusion that `B` was a bad idea rather than a simple bug. This will usually involve an explicit named branch and several revisions for the parallel development path. You might even experiment with several variations before picking a winner. - * For most cases, you just want to go back to `B` and finish the original change properly. It's too late to fix the revisions that already inherited the incomplete or incorrect change, but you can '''merge''' the rest of the later development with the fix easily. + * Where the bug introduced in `B` represents the *entirety* of the + change `A->B`, you may want `F2` to completely undo the change. -= Daggy release management = + For this specific case, monotone has the `disapprove` command, + which will internally create a new revision with the reverse diff as a + child of `B` (this command name is unfortunate, because it is *not* + symmetrical with `approve`). -Many projects create release branches to track critical fixes to stable code while more active development goes on elsewhere. The important thing about a release branch, in this context, is that you want to separate fixes and development: + * In some rare cases where you want to redo `B` completely + differently from scratch, you might want to `commit` that as a + child of `A` (and sibling of `B`) instead. You will then have to + resolve any conflicts between the two alternate changes when you + `merge`. - * unlike a development branch, you don't intend to ever merge a release branch back with - mainline (where it would eventually pick up the fix). + This kind of pattern is most relevant for more fundamental changes + than simple fixes, perhaps where the line of later + development eventually leads to the conclusion that `B` was a bad idea + rather than a simple bug. This will usually involve an explicit named + branch and several revisions for the parallel development path. You + might even experiment with several variations before picking a + winner. - * nor do you want to introduce arbitrary new developments from mainline to a release branch, wholesale. + * For most cases, you just want to go back to `B` and finish the + original change properly. It's too late to fix the revisions that + already inherited the incomplete or incorrect change, but you can + `merge` the rest of the later development with the fix easily. -Lets suppose, further to our example, that you had started a new release branch -from `D`, midway along the span containing the bug:{{{ - A - | - B -----+ - / \ | - C D F2 - \ / \ : - E R1 - : \ - R2 -}}} +# Daggy release management -You can immediately see that your release branch `D->R1->R2` has inherited the bug, and is therefore also going to need the fix. +Many projects create release branches to track critical fixes to +stable code while more active development goes on elsewhere. The +important thing about a release branch, in this context, is that you +want to separate fixes and development: -Even better, because `F2` and `R2` have a common ancestor `B` (which introduced the bug), and `F2` contains ''only'' the fix for the bug, approving `F2` to that branch (and merging, as before) does ''exactly'' the right thing, producing a fixed revision `RF` on the branch:{{{ - A - | - B ------+ - / \ | - C D F2 - \ / \ : \ - E R1 | - : \ | - R2 | - \ | - RF -}}} + * unlike a development branch, you don't intend to ever merge a + release branch back with mainline (where it would eventually pick + up the fix). -Remember that monotone branch memberships are determined by certs, and branches need not be fully connected. After the '''approve''' of `F2` but before the '''merge''' that produced `RF`, the release branch contained disconnected revisions: `D->R1->R2` and `F2`. These are still multiple heads of the branch, and a '''merge''' can succeed because of the common ancestor `B` even though `B` is not a member of the branch. + * nor do you want to introduce arbitrary new developments from + mainline to a release branch, wholesale. -This represents the true place of the fix in the graph and in both branches, without -introducing any of the other changes added in `C`, `E`, or any other point past `D` where the branch diverged. -If we had committed the fix the old way as `F1` instead, and tried to use '''approve''' like this, merging the release branch would pull in all the changes from `B` to `E` as well, just like a full '''propagate''' would. +Lets suppose, further to our example, that you had started a new +release branch from `D`, midway along the span containing the bug: -Although not drawn here, you can just as readily see that any other branches which diverged above `B` don't need a fix, because the bug was introduced later. -If some of those were development branches that had '''propagate'''d from mainline to them, and some of those '''propagate'''s included `B`, then those branches would also need the fix. -This knowledge is the benefit of identifying `B` as the source of the bug, which you might have needed to do anyway regardless of where the fix was applied. -Making sure to apply the daggy fix at `F2` means that you automatically get the same benefit for knowledge of what has also inherited the fix. + A + | + B -----+ + / \ | + C D F2 + \ / \ : + E R1 + : \ + R2 -In general, any revision with `B` as an ancestor and without `F2` as an ancestor, has inherited the bug but not the fix. +You can immediately see that your release branch `D->R1->R2` has +inherited the bug, and is therefore also going to need the fix. -Remember that in distributed development with monotone, we might not yet have learned about revisions descended from `B`; in fact we might ''never'' learn of someone else's private branch derived from somewhere below B. We still want to publish the fix close to `B` so that they can then '''approve''' `F2` for their private branch and inherit the fix as well as the bug. +Even better, because `F2` and `R2` have a common ancestor `B` (which +introduced the bug), and `F2` contains ''only'' the fix for the bug, +approving `F2` to that branch (and merging, as before) does +''exactly'' the right thing, producing a fixed revision `RF` on the +branch: -If you were to add a new test and testresult cert along with the fix in `F2`, it becomes even -clearer where your fix has gone - especially if your tool can colour the -graph according to this ancestry or the testresult. + A + | + B ------+ + / \ | + C D F2 + \ / \ : \ + E R1 | + : \ | + R2 | + \ | + RF -= Plucking and CherryPicking = +Remember that monotone branch memberships are determined by certs, and +branches need not be fully connected: -In many projects, the need to apply fixes to older revisions (especially releases) is addressed by applying patches containing the fix, sometimes adjusting or backporting for the older code. -These fixes are taken from individual changes mixed with development along the mainline, and need to be separated out. -This is often called CherryPicking, and is a frequent feature request for monotone. -It's clear that many such requests are motivated by these kinds of concerns, especially from developers accustomed to these practices and familiar with support for similar features in other tools. +* After the `approve` of `F2` + but before the `merge` that produced `RF`, the release branch + contained disconnected revisions: `D->R1->R2` and `F2`. +* `R2` and `F2` are still multiple heads of the branch, and a `merge` can + succeed because of the common ancestor `B` even though `B` is not a + member of the branch. -Monotone now has a new command, '''pluck''', that can be used to pull the set of changes between two (arbitrary) revisions "over there" into the current workspace, taking into consideration other changes (like file renames) that have happened between "there" and "here". -This allows isolated changes to jump across parts of the revision graph, without making full ancestry connections in the same way that '''propagate''' and similar commands would. +`F2` represents the true place of the fix in the graph and in both +branches, without introducing any of the other changes added in `C`, +`E`, or any other point past `D` where the branch diverged. +If we had committed the fix the old way as `F1` instead, and tried to +use `approve` like this, merging the release branch would pull in all +the changes from `B` to `E` as well, just like a full `propagate` +would. -Doing Daggy fixes, as outlined above, will minimise the need to cherry-pick fixes instead. Daggy fixes mean using rather than losing the true origin and relationship between bugs and fixes in the ancestry graph. -However, it's worth looking at some valid and recommended uses for this functionality, even in a project using Daggy Fixes. +Although not drawn here, you can just as readily see that any other +branches which diverged above `B` don't need a fix, because the bug +was introduced later. +If some of those were development branches that had `propagate`d from +mainline to them, and some of those `propagate`s included `B`, +then those branches would also need the fix. +This knowledge is the benefit of identifying `B` as the source of the +bug, which you might have needed to do anyway regardless of where the +fix was applied. +Making sure to apply the daggy fix at `F2` means that you +automatically get the same benefit for knowledge of what has also +inherited the fix. -== Plucking and backporting fixes == +In general, any revision with `B` as an ancestor and without `F2` as +an ancestor, has inherited the bug but not the fix. -Doing daggy fixes all the time isn't for everyone. It's not always so easy to develop a fix directly against the revision where the bug was introduced. Perhaps the bug wasn't discovered until some other more recent code used it in ways that exposed the bug; it would be hard to debug and find the fix without this other code around. Or perhaps the importance or scope of the fix simply hadn't been realised at the time. +Remember that in distributed development with monotone, we might not +yet have learned about revisions descended from `B`; in fact we might +*never* learn of someone else's private branch derived from +somewhere below B. We still want to publish the fix close to `B` so +that they can then `approve` `F2` for their private branch and +inherit the fix as well as the bug. -So, continuing our scenario, assume the fix had been '''commit'''ted directly as a new head, `E->F1`, as in the first example. +If you were to add a new test and testresult cert along with the fix +in `F2`, it becomes even clearer where your fix has gone - especially +if your tool can colour the graph according to this ancestry or the +testresult. -Now, we've found that the bug originated in `B`, and realised that the fix will be needed on the release branch as well. -Or perhaps the bug was also found on the release branch, and traced back to the common ancestor `B` from there. -Either way, we can use the new '''pluck''' command to help us. Once again, we have a couple of useful choices about where to '''commit''' the '''pluck'''ed change: +# Plucking and CherryPicking - * we could '''pluck''' the `F1` change straight across to the release branch, producing `R2->RF` as a simple fix '''commit''', much like was done on the mainline head, with none of the structural benefits of daggy fixes. +In many projects, the need to apply fixes to older revisions +(especially releases) is addressed by applying patches containing the +fix, sometimes adjusting or backporting for the older code. +These fixes are taken from individual changes mixed with development +along the mainline, and need to be separated out. +This is often called CherryPicking, and is a frequent feature request +for monotone. +It's clear that many such requests are motivated by these kinds of +concerns, especially from developers accustomed to these practices and +familiar with support for similar features in other tools. +Monotone's `pluck` can be used to pull the +set of changes between two (arbitrary) revisions "over there" into the +current workspace, taking into consideration other changes (like file +renames) that have happened between "there" and "here". +This allows isolated changes to jump across parts of the revision +graph, without making full ancestry connections in the same way that +`propagate` and similar commands would. + +Doing Daggy fixes, as outlined above, will minimise the need to +cherry-pick fixes instead. Daggy fixes mean using rather than losing +the true origin and relationship between bugs and fixes in the +ancestry graph. +However, it's worth looking at some valid and recommended uses for +this functionality, even in a project using Daggy Fixes. + +## Plucking and backporting fixes + +Doing daggy fixes all the time isn't for everyone. It's not always so +easy to develop a fix directly against the revision where the bug was +introduced. Perhaps the bug wasn't discovered until some other more +recent code used it in ways that exposed the bug; it would be hard to +debug and find the fix without this other code around. Or perhaps the +importance or scope of the fix simply hadn't been realised at the +time. + +So, continuing our scenario, assume the fix had been `commit`ted +directly as a new head, `E->F1`, as in the first example. + +Now, we've found that the bug originated in `B`, and realised that the +fix will be needed on the release branch as well. +Or perhaps the bug was also found on the release branch, and traced +back to the common ancestor `B` from there. +Either way, we can use the `pluck` command to help us. Once again, we +have a couple of useful choices about where to `commit` the +`pluck`ed change: + + * we could `pluck` the `F1` change straight across to the release + branch, producing `R2->RF` as a simple fix `commit`, much like was + done on the mainline head, with none of the structural benefits of + daggy fixes. + * or, now that we know better, it might be nice to go back up to `B` and - create the 'proper' daggy fix `F2`, gaining the advantages outlined above for any other branches and descendants of `B`. + create the 'proper' daggy fix `F2`, gaining the advantages outlined + above for any other branches and descendants of `B`. -Creating `F2` is easy with '''pluck''': {{{ - $ mtn co ... -r B . - $ mtn pluck -r E -r F1 - $ mtn commit -}}} +Creating `F2` is easy with `pluck`: -For simple fixes, this new `F2` will '''merge''' cleanly with `F1` on mainline, and with `R2` on the release branch as before. + $ mtn co ... -r B . + $ mtn pluck -r E -r F1 + $ mtn commit -For more intricate fixes, this step corresponds to backporting the fix from mainline to the point of origin of the bug. -This makes it easier to '''approve''' and then '''merge''' that base fix forward to other branches (which may have diverged since) as a separate change, rather than attempting to adjust the change for both development paths in the one step. +For simple fixes, this new `F2` will `merge` cleanly with `F1` on +mainline, and with `R2` on the release branch as before. -== Plucking development == +For more intricate fixes, this step corresponds to backporting the fix +from mainline to the point of origin of the bug. +This makes it easier to `approve` and then `merge` that base fix +forward to other branches (which may have diverged since) as a +separate change, rather than attempting to adjust the change for both +development paths in the one step. -Attractive new features are developed on the mainline, and sometimes it's desirable to add these new features to a stable branch, once it's clear they are good. -However, we only want to bring the specific changes related to this feature onto the release branch, and not all of the other unrelated changes that would be carried along with it if we did a '''propagate'''. -If you can identify the list of changes that correspond to the feature, you can '''checkout''' a workspace on the stable branch, and '''pluck''' each of these changes into it before committing. However, doing so carries with it the responsibility to ensure that you find ''all'' of the relevant changes -- including future changes that fix bugs in the code you just '''pluck'''ed. Because you've bypassed recording the full origin of the changes in the DAG history, daggy fixes won't work: you have to '''pluck''' fixes for '''pluck'''ed features. Even then, daggy fixes are preferred: at least they're easier to see because they're kept near the source revisions the features were originally '''pluck'''ed from. +## Plucking development +Attractive new features are developed on the mainline, and sometimes +it's desirable to add these new features to a stable branch, once it's +clear they are good. +However, we only want to bring the specific changes related to this +feature onto the release branch, and not all of the other unrelated +changes that would be carried along with it if we did a +`propagate`. + +If you can identify the list of changes that correspond to the +feature, you can `checkout` a workspace on the stable branch, and +`pluck` each of these changes into it before committing. However, +doing so carries with it the responsibility to ensure that you find +**all** of the relevant changes -- including future changes that fix +bugs in the code you just `pluck`ed. Because you've bypassed +recording the full origin of the changes in the DAG history, daggy +fixes won't work: you have to `pluck` fixes for `pluck`ed +features. Even then, daggy fixes are preferred: at least they're +easier to see because they're kept near the source revisions the +features were originally `pluck`ed from. + Another example is where you've done a whole pile of development in a private branch, but for one reason or another you really can't or don't +see the need to publish the whole gory history and internal working +details for a push into a more public repository. +You `pluck` the entire span of the private development branch onto the +mainline, producing a single summarised change that rolls up all the +intermediate work and rework and rerework, showing only the +consolidated result. -see the need to publish the whole gory history and internal working details for a -push into a more public repository. -You '''pluck''' the entire span of the private development branch onto the mainline, producing a single summarised change that rolls up all the intermediate work and rework and rerework, showing only the consolidated result.