Difference between revisions of "Bazaar for Emacs Contributors"

From WikEmacs
Jump to: navigation, search
Line 5: Line 5:
This is a quick-start guide to using [[Bazaar]] for Emacs development.
This is a quick-start guide to using [[Bazaar]] for Emacs development.
This guide presents a simple Bazaar workflow oriented toward the "regular contributor," whether a "core" committer, a maintainer of a well-defined package (/e.g./, a major mode), or simply someone who enjoys occasionally hacking on Emacs and then contributing those improvements to the community.  The emphasis is on /necessary/ and /simple/, not on "general" nor on maximal efficiency.  However, the ways presented here are intended to be straightforward, effective, reasonably efficient, and relatively simple.
In the workflows described below, a quick "bzr status" is inserted before each commit.  
In the workflows described below, a quick "bzr status" is inserted before each commit.  
=== CVS-like workflow ===
The workflow documented on this page takes advantage of the fact that Bzr is a decentralized version control system.  It is especially useful when you are working on large changes to Emacs.
If you just need a quick and simple workflow, see BzrQuickStartForEmacsDevs.  That page describes a simple CVS-like workflow, which allows you to contribute changes to Emacs as quickly as you can, with a minimum amount of extra effort.  One advantage of that workflow is that you can use VC-dir without complications.
Later, when you have a chance, come back here and learn the workflow described on this page.  This workflow hews more closely to what experienced Bazaar users are accustomed to, which will make it easier for you to get support online.  One disadvantage of this workflow is that VC-dir support is still somewhat lacking.

Revision as of 09:21, 27 March 2012

Fork of Emacswiki article as on 13:42, 27 March 2012 (EEST).



This is a quick-start guide to using Bazaar for Emacs development.

In the workflows described below, a quick "bzr status" is inserted before each commit.



If you find the terminology in this guide confusing, read Bazaar for CVS Users and come back. In particular, keep in mind that the main unit of currency in Bzr is the /branch/, and that /repository/ in Bzr does not mean the same thing as in CVS. In Bzr, a repository is just a place to store multiple branches efficiently; you're as likely to have a repository locally as remotely. In Bzr, you send and receive changes to/from an upstream /branch/, and whether that branch is in a repository upstream is irrelevant. The mainline branch of the public Emacs repository will be called the /master branch/, /master/ for short, or /upstream master branch/ for emphasis. (The mainline branch is the one containing the changes targeted for future public releases.)

The command examples are generally given in Unix command-line style, although we did try to use forms that would work on MS-Windows as well. If you use Windows or some other system, please feel free to add examples appropriate for that system.


Getting Started


What Version of Bazaar to Use

As Emacs needs some of the improvements provided by 1.17 release, this quick-start guide assumes you use GNU Bazaar 1.17 or higher to develop Emacs. Use of version 2.0 or higher is /strongly/ recommended because a large number of performance improvements have been made.


Where is Emacs?

You can browse just the top level of the Emacs repository (to see the branches) here: http://bzr.savannah.gnu.org/r/emacs/ .

To browse the entire repository there is a web front-end called "loggerhead" which may eventually be installed at
http://bzr.savannah.gnu.org/lh/emacs/ (see sr #107142). For now you can see it at Launchpad, however note this is a mirror of the trunk only.

To make a local branch or branches based on upstream Emacs, start at #InitializingYourBzrRepo and follow instructions from there. (This is the Bazaar equivalent of `cvs checkout`.)


Getting help on Bazaar

Bazaar doesn't provide extensive Unix-style man pages, or Info. Instead, use ##bzr help##, and see #OtherResources for more documentation.


The Bazaar Work Routine

Overview of the Bazaar work routine:

  1. Tell Bazaar your name and email address; these will be used as defaults to identify your commits. (Usage: bzr whoami "Frank Chu <fchu@example.com>")
  2. Create a bzr shared repository locally, to hold all your branches.
  3. Create your first local branch: a mirror of the upstream project trunk (it will be "bound to" the upstream trunk; more on that below)
  4. Maybe create other branches for specific purposes (branches are cheap).
  5. Do trivial, single-commit work in your local trunk mirror and commit to upstream from there; do your non-trivial work in dedicated branches, committing locally as you go (more about that below).
  6. From time to time, update to get upstream newness into your mirror branch (this is like cvs update), and then merge the newness into your dev/task branches from there (also sort of like cvs update).
  7. When a task is finished (a local branch's changes are complete), then either
    1. merge them into the master branch (if you're a maintainer and have the necessary access rights --- this is analogous to committing in CVS), /or/
    2. package up your changes and send them as a "bundle" to the project mailing list (analogous to posting a patch).
  8. Receive feedback.
  9. Lather, rinse, repeat until the change is accepted.
  10. If you did all this on a local feature branch, you can remove the branch now. The change history will live upstream.

Now let's look at the specific bzr commands. Throughout the examples, we'll assume you have some specific directory where you keep your sources. For concreteness, imagine you have an environment variable:

     export DEVHOME=/home/jrandom/projects       # season to taste

Of course some people like to organize things differently. However, there are very good reasons to keep all of your Emacs branches in one place when using Bazaar, so we recommend you use this kind of organization until you are familiar with Bazaar.


Personalizing Bazaar

Unlike centralized VCSes where you are identified by an account in the repository, in Bazaar you can create branches in many places, and you may be sending changes to many places (including, but not limited to, the upstream master branch). So Bazaar stores your identity locally, and includes it as part of every revision you commit. This information is your name and email address. To configure Bazaar to use them, do

     bzr whoami "J. Random Hacker <jrandom@gnu.org.invalid>"

When you are integrating somebody else's work, you will want to be identified as the committer (so you can handle technical issues with the commit), but not as the author. In that case you can use the ##--author## option to ##bzr commit## to specify the author.


Initializing Your Local Bazaar Repository

This section describes how to set things up if you plan on making contributions to Emacs (whether or not you're one of the Emacs maintainers). You can also use this way just to track the Emacs sources, even if you never contribute changes.

For the general public, Emacs's public master trunk lives at ##bzr://bzr.savannah.gnu.org/emacs/trunk##. But committers should use an ##bzr+ssh## URL instead: ##bzr+ssh://USERNAME@bzr.savannah.gnu.org/emacs/trunk##.

First, initialize a shared repository in which to store all your branches:

     cd $DEVHOME
     bzr init-repo emacs/

(If you are using Bazaar version older than 2.0.0, you will need to use ##bzr init-repo --2a emacs/##.) In Bazaar, a /repository/ contains the history of one or more branches. It's possible to create a "standalone" branch, which contains its own repository under the ##./.bzr## directory. But here we are using a /shared repository/, which is very important for performance: because all branches created under ##$DEVHOME/emacs## will share history storage. Not only does this save a lot of space, but it speeds up branch creation immensely because the history does not need to be copied each time.

[:InitializeTrunk] Now, create a branch called ##trunk## whose main role is to be a mirror of the mainline, though you may also use it for making quick, one-off changes:

     cd emacs/
     bzr branch bzr+ssh://USERNAME@bzr.savannah.gnu.org/emacs/trunk trunk

(Here USERNAME stands for your Savannah user name. Use ##bzr://bzr.savannah.gnu.org/emacs/trunk## instead if you are not a committer and just expect read-only access. You could also use ##bzr branch lp:emacs trunk##, which might be faster, depending on your network connection.)

This first ##bzr branch## operation in a new repository may take many minutes --- people have reported as little as 5 and as many as 60 minutes. (If your network connection is extremely slow, using the "##nosmart+##" option to the protocol makes it faster, for this one-time command. That is, use ##nosmart+bzr://bzr.savannah.gnu.org/emacs/trunk##.) At the end of the long wait, the repository will contain the whole history of the trunk, and the ##trunk## directory will contain the working files -- that is, an Emacs tree.

Next, we'll further safeguard that mirror by making it a "two-way" mirror. We do this by "binding" the local trunk to the upstream master:

     cd trunk/
     echo "public_branch = bzr+ssh://USERNAME@bzr.savannah.gnu.org/emacs/trunk" >> .bzr/branch/branch.conf
     bzr bind bzr+ssh://USERNAME@bzr.savannah.gnu.org/emacs/trunk
     cd ..

(Again, use ##bzr://bzr.savannah.gnu.org/emacs/trunk## as the URL, in both instances above, if you are not a committer and just expect read-only access. Also, note that on MS-Windows you need either to remove the quotes around the arguments to the ##echo## command or use a ported GNU ##echo##. Finally, do //not// use "##nosmart+##" in any bzr command but the initial ##bzr branch##, because the "smart" operation is much faster for routine work. Even the initial ##bzr branch## command should be faster without "##nosmart+##, so use it only if the initial branch command takes forever.)

If there are other upstream branches you'd like to mirror, you can set them up in the same way. In your local repository, you can efficiently branch them from the upstream branch, because very little history copying will be needed (as long as they themselves were branched from the upstream master).

//Important:// all your branches should be sibling directories under ##$DEVHOME/emacs/## -- this way they will all share history data, which makes Bazaar operations, including ##bzr branch##, orders of magnitude faster.


Doing Quick Fixes

You can do quick, self-contained fixes (one-line patches and the like) directly in ##trunk## if you want:

     cd $DEVHOME/emacs/trunk
     bzr update                  # get in sync with the upstream master
     <<<make your edit>>>
     bzr status                  # (optional) sanity check -- lists changed and unknown files
     bzr commit -m "Fix silly typo in README."

Because ##trunk## is "bound" to upstream, this commit will automatically push the changes upstream.

    1. bzr## will automatically identify you as the committer and author of these changes (see #PersonalizingBzr). If you are integrating another developer's work, you can use ##--author 'J. Other Hacker <jhacker@gnu.org.invalid>'## to identify JO Hacker as the author of the change.

//Note:// For larger tasks, such as a complicated bug fix or new feature that might require multiple commits and rounds of feedback, we always recommend creating a "task branch" dedicated to that fix or feature -- see #FeatureBranch.


Another Way: Using a Dedicated Branch for Quick Fixes

Some people don't like doing development work in ##trunk## at all, they prefer it to remain a pristine mirror of the upstream trunk at all times. Instead, they keep a special branch for quick changes, and keep reusing that branch. Like trunk, it is bound to upstream, so that commits will be sent upstream automatically. To create such a branch:

     cd $DEVHOME/emacs
     bzr branch trunk/ quickfixes/
     cd quickfixes
     echo "public_branch = bzr+ssh://USERNAME@bzr.savannah.gnu.org/emacs/trunk" >> .bzr/branch/branch.conf
     bzr bind bzr+ssh://USERNAME@bzr.savannah.gnu.org/emacs/trunk
     cd ..

(Those instructions are subject to the same caveats as given earlier when binding the ##trunk## branch.)

This will create a directory ##quickfixes##, populated with the Emacs source tree, and bind it to upstream. Creating ##quickfixes## should be much faster than the first ##bzr branch##, because this is all happening inside a shared repository. In a Bazaar shared repository, all the branches share history, which means less space used and less time spent downloading history and file content; this makes Bazaar branches much more lightweight than CVS branches.

However, it still takes time to check out the source tree for a new branch (which we just did), and bootstrapping a new tree takes an annoyingly long time. That is why reusing the same branch for small changes makes sense. After you have bootstrapped a branch once, the subsequent update, build, and commit steps of the update-edit-build-test-commit cycle are each very fast, as long as you continue working in the same branch.

Once you have created and bootstrapped the ##quickfixes## branch, the routine work in it is as shown above for the trunk:

     cd $DEVHOME/emacs/quickfixes
     bzr update                  # get in sync with the upstream master
     <<<make your edit>>>
     bzr status                  # (optional) sanity check -- lists changed and unknown files
     bzr commit -m "Fix silly typo in README."

Because ##quickfixes## is "bound" to upstream, this commit will automatically push the changes upstream.

[:InitializeBranch] [:FeatureBranch] [:RegularDevelopmentBranch]

Workflow for Regular Development

Creating a Task Branch

When you are working on a task that touches many files, or requires a lot of thought and several stages, or for any reason may take a fair amount of time, it's a good idea to work on a separate branch to keep the changes from interfering with concurrent work (including your own!), and to keep the history separate from other work. This makes review easier, too. So first create a new branch:

     cd $DEVHOME/emacs
     bzr branch trunk/ TASKNAME/
     cd TASKNAME

...where "TASKNAME" could be, e.g., the bug tracker ticket number of the bug you're trying to fix, or just a short, descriptive name for the task.

Working in a Task Branch

At this point, you're ready to do work. You edit the source files, build and test as needed. When you've accomplished a subtask, it's a good idea to stop and commit your work to the branch. Here, you often have a larger ChangeLog than can comfortably be entered on the command line with ##-m##. In this case, you put the log message in a file we'll call ##log-message.txt##. Then

     bzr status                  # (optional) sanity check -- lists changed and unknown files
     bzr commit -F log-message.txt

You can also do ##bzr commit -m "blah blah blah"##, if your log message is short enough to fit on one line. Remember, these commits are local: they affect only the task branch, since it is not bound to upstream. The commits don't affect the upstream master branch, and they don't even affect your local trunk mirror. No one else will see the commits until you send them upstream.

(Note: the Emacs project will continue to keep versioned ChangeLog files. See this mail for more.)

If the task isn't done yet, keep working. Edit yet more source files, until you reach a good stopping point:

     bzr status                              # (optional) sanity check -- lists changed and unknown files
     bzr commit -F another-log-message.txt

etc. You can commit any time you reach a good checkpoint. We recommend committing much more frequently than you might with CVS or Subversion.

From time to time you may wish to refresh the local trunk mirror, by updating to get changes from upstream:

     pushd $DEVHOME/trunk/
     bzr update

And after refreshing the mirror, you'll want to get those changes into your task branch, by merging them:

     # Back in the TASKNAME branch
     bzr merge
     bzr status                  # (recommended) sanity check -- lists changed and unknown files
     bzr commit -m "Merge from mainline."

The reason you use "##merge##" instead of "##update##" in the task branch is that your task branch is an independent branch (not bound to anything else) and has its own local commits. In other words, it has diverged (a bit) from the upstream master, and so any changes from upstream have to be merged with your local changes.

The merge may fail due to an upstream change that conflicts with your branch. You'll need to fix the problem (looking for conflict markers and editing, just as in CVS). When you save the files which had conflict markers, Emacs will automatically invoke "bzr resolve" for you, to tell Bazaar that conflicts were resolved. If for some reason you don't use Emacs to fix the conflicts, you will need to do it manually:

     bzr resolve file-you-changed ...      # Mark these conflicts as resolved.
                                           # Bazaar keeps track of unresolved conflicts,
                                           # and won't let you commit until all are resolved.

Once you have achieved a clean merge, commit it:

     bzr status                  # (recommended) sanity check -- lists changed and unknown files
     bzr commit -m "Merge from mainline."

Note that you will need this ##commit## command even if there were no conflicts detected during the merge.

Once you have completed the task, you'll want to send it for review or maybe even put it into upstream. You can send for review either by publishing your branch (see #PublishBranch) or sending a bundle (see #SendABundle), and if you are a committer you can merge to upstream yourself (see #MergeToUpstream).


Publishing a Task Branch

A common practice in Bazaar is to publish your entire branch for review (most of the branch consists of stuff from upstream, and Bazaar easily shows the reviewers the differences -- that is, your changes). Posting a branch is sort of like posting a patch, but carries more information, such as the commit messages.

You can publish your branch anywhere that can host Bazaar branches. One convenient place is Launchpad.net. Once you have a Launchpad account, publishing is just:

     cd $DEVHOME/emacs/TASKNAME
     bzr push lp:~USERNAME/emacs/TASKNAME

After that, the branch will be visible at


and others can obtain it via Bazaar by

     bzr branch lp:~USERNAME/emacs/TASKNAME

(The "USERNAME" in that last command is your username, not the reviewer's, of course.)

As you make new changes to your branch, you can repeatedly push to the same location to publish them. The ##--remember## flag to ##bzr push## may be handy, if you don't want to keep typing the same destination URL.

If you are a committer, you could also push the branch to Savannah by a similar command, and people can pull it from there for review. Or if you have your own server somewhere, it's fine to post branches there too. It doesn't really matter. Wherever it is, as long as you can provide a URL to it, other developers can get the data.


Sending a Bundle for Review

Another way to post changes for review is to create a "bundle", which is a file containing your change, in a special "merge directive" format that bzr understands but that also contains a human-readable diff:

     bzr send -o PATCHNAME.txt

where ##PATCHNAME## is some brief mnemonic name for the change. Send a mail to the Emacs Devel list describing the change; attach ##PATCHNAME.txt## to that mail (please use MIME type ##text/plain##). If the reviewers ask for further tweaks, repeat the cycle. Edit the files, build and test as necessary, and

     bzr status                  # (optional) sanity check -- lists changed and unknown files
     bzr commit -m "Address reviewer comments."

Now generate a new bundle:

     bzr send -o PATCHNAME-v2.txt

and send it off.

When revising your change, even though nobody else knows about your first commit yet, it is simplest to just make more commits. (It's possible to "uncommit" in Bazaar, but it's not straightforward, and there's no compelling reason to do it if you're just revising a change.) Although in the ##quickfixes## branch a plain ##bzr log## will show you all your changes, when you do plain ##bzr log## after your changes have been merged in ##trunk##, you will see only a merge commit log, which describes the whole series of commits for this task. It will typically look something like "Merge: fix bla bla bla (closes Bug #1)."

[:PushToUpstream] [:MergeToUpstream]

Merging Into the Upstream Master

If you are a committer, you can merge to the upstream master directly. To do this, first update your mirror:

     cd $DEVHOME/emacs/trunk/
     bzr update
     bzr merge ../TASKNAME

and then commit

     bzr status                  # (recommended) sanity check -- lists changed and unknown files
     bzr commit -m "Merge: fix bla bla bla (closes Bug #1)."

which automatically merges all your new commits to the upstream master, because the mirror is set up as bound branch. In this process, updating your ##trunk## with ##bzr update## is not really optional. There is a race condition here, just as in CVS: if your mirror is not up-to-date because another developer has committed since you last updated, your commit will fail. In this case you should abort the merge

     bzr revert

do another ##bzr update##, merge from the ##TASKNAME## branch again, and commit. After you successfully commit, the changes will appear grouped as a single merge in ##bzr log## on the mainline, but anyone can use ##bzr log -n0## to see the sub-commits (or "nested commits") within that merge.

It might occur to you to save some effort by just doing ##bzr push## directly to the upstream master from inside the ##TASKNAME## branch:

     cd $DEVHOME/emacs/TASKNAME
     bzr push bzr+ssh://USERNAME@bzr.savannah.gnu.org/emacs/trunk/

//Do not do this// --- it can cause history to be displayed in a strange way in the upstream master, any mirrors or branches of it, and your own branch later. Search for the word "hidden" in this mail for more details.


After You've Finished Your Task

When you're done with your work and it's all been sent upstream, you can delete your task branch...

     cd ..
     rm -rf TASKNAME/

...and update your mirror to receive your own changes:

     cd trunk/
     bzr update
     cd ..

Note that you will still be able to see the entire history of your branch in the trunk by using ##bzr log -n0##, even though you've deleted the local copy.

When you decide to work on the next task, create a new task branch:

     cd $DEVHOME/emacs
     bzr branch trunk/ NEW_TASKNAME/

...and you know what to do from here.



If you've read this far, you should have a basic understanding of Bazaar, and how develop Emacs using Bazaar. Below are some more resources that you or people you work with might find useful.


Resources for Casual Developers

Ideally, one-time contributors, beta testers, and anyone else needing to follow bleeding-edge Emacs would use the same workflow as described above for regular contributors. However, in case that workflow feels like too much trouble, then BzrForEmacsCasualDevs offers a couple of slightly simpler setups.

Even if you use one of those simpler setups at first, we still encourage you to migrate to the #RegularContributors workflow as soon as you can -- not because you necessarily plan to become a regular contributor, but because then your setup will be the same as what most other people are using, so they can better help you if you have questions.


Other Resources

  • Bazaar home site
  • Bzr Users Guide
  • BzrQuickStartForEmacsDevs provides an alternate, more CVS-like, simpler workflow.
  • Contributing to Free Software Projects Made Easy An excellent tutorial on using the Bazaar Explorer GUI. Warning: the workflow advocated there is both more complex in some ways and slightly different from the one recommended in /Bazaar for Emacs Devs/.
  • Emacs as commit message editor Tips for setting up Emacs as Bazaar commit editor.
  • Bazaar for Git users A pretty complete overview of the differences between Git and Bazaar. Contains a couple of glossaries of VCS terms, which are probably useful to those who don't know git, too.
  • Bazaar for CVS users A whirlwind introduction to the features and command-line UI of Bazaar. The workflow described is very similar to that of BzrQuickStartForEmacsDevs, and the latter document may be easier to understand because it concentrates on introducing the workflow rather than the wider field of distributed version control.
  • For diehard lovers of git, JohnWiegley reports that git-bzr, a bi-directional gateway, works well.