Python

How can we turn Emacs as full featured Python IDE ? Many packages are explained below. You don't have to install all of them one by one, see the all-in-one solutions, like Elpy, and starter kits configuration layers.

Default modes

There are a number of python modes for Emacs. fgallina's python.el is the python.el included in Emacs 24.2 and onwards.

All-in-one solutions

emacs-for-python

emacs-for-python is a bundle of the above modes (and more), and it's an easy way to turn Emacs into a Python IDE.

emacs-for-python at github

Elpy

Elpy is a collection of elisp packages for Python too.

https://github.com/jorgenschaefer/elpy/wiki/Features

It permits, among others, to:

• refactor code (using rope),
• send a region or a buffer to a python interpreter,
• run unit tests, run the test the cursor is in, with a test runner of choice (django, pytest, …),

and of course

• on-the-fly checks with flymake,
• code completion with company-mode and either rope or jedi,
• on the fly code helper with eldoc (and rope or jedi),
• code navigation,
• change of virtual environment inside Emacs (pyvenv), etc

EMP: EMacs Python

The goal of emp is to take all the best development components available for Python, and combine it conveniently under Emacs. It builds on top of elpy, and provides even more bindings.

https://kootenpv.github.io/2016-01-01-emacs-python

Emacs-bootstrap

Emacs-bootstrap is a website where you can select a few programming languages, some options, and have it generate a full .emacs.d folder for you.

Refactoring

rope & ropemacs

Presentation

rope is a library for refactoring and manipulating Python code, pymacs is an interface between emacs lisp and python, and ropemacs is an Emacs interface to the rope library which uses rope and pymacs.

If you do some search and replace of code objects in your code and find it sometimes tricky, and/or you need to do it in more than one file, then you should consider a good tool of refactoring.

But Rope can do more than this simple example, it can:

• Rename anything
• Extract method/local variable
• Change method signature
• Perform cross-project refactorings
• Support Mercurial, GIT, Darcs and SVN in refactorings

Rope can also help IDE's with:

• Auto-completion
• Finding definition location
• Getting pydoc
• Finding occurrences
• Organizing imports (removing unused and duplicate imports and sorting them)
• Generating python elements

Installation

Note: see also emacs-traad below that has an easy installation process.

With el-get

So an easy way is to use an el-get recipe: M-x el-get-install ropemacs. It installs and build Pymacs, rope and ropemacs. The difference from the solution using pip is that it setups the Emacs side of Pymacs correctly.

Manually

You can follow the instructions as given here ,if you don't want to use marmalade you can use the following instructions .

   sudo pip install rope ropemacs

As of February 2014 you can't install Pymacs via pip, therefore:

   git clone http://github.com/pinard/pymacs
   cd pymacs
   make check
   sudo make install

make check ensures the prerequisites. sudo make install puts Pymacs module in Python local modules path, usually /usr/local/lib/python2.7/dist-packages/. People generally frown upon make install, because if you lose the Makefile, it can become hard to cleanly uninstall. But you shouldn't fear it, as pip uses the same directory, so pip uninstall pymacs will work.

After you do the make install of pymacs do not delete the folder.Copy the file pymacs.el to ~/.emacs.d/site-lisp/

Now add path of pyamcs.el to your ~/.emacs file,using this code:

   (setq emacs-config-path "~/.emacs.d/") 
   (setq base-lisp-path "~/.emacs.d/site-lisp/")
   (setq site-lisp-path (concat emacs-config-path "/site-lisp"))
   (defun add-path (p)
     (add-to-list 'load-path (concat base-lisp-path p)))
   (add-path "")
   (add-to-list 'load-path "~/.emacs.d")

You can now lazily load ropemacs or load it at runtime.For lazy evaluation add this to your .emacs file:

   (defun load-ropemacs ()
     "Load pymacs and ropemacs"
     (interactive)
     (require 'pymacs)
     (pymacs-load "ropemacs" "rope-")
     ;; Automatically save project python buffers before refactorings
     (setq ropemacs-confirm-saving 'nil)
   )
   (global-set-key "\C-xpl" 'load-ropemacs)

use c-xpl to load ropemacs.Read more about it here

Rope & traad

Traad is a client-server approach to using the rope Python refactory library. It involves two basic components:

• A HTTP server exposing the rope API via JSON, and
• Client libraries for talking to the server

Since rope is written in Python, any tool that wants to use rope needs to either embed Python, or it needs to find a way to communicate with a Python process running rope. The embedding approach is difficult if the target environment is not written in Python, and it also faces challenges when dealing with more than one Python version.

So traad aims to simplify communication with rope running in an independent process. HTTP communication and JSON data handling is well supported in many, many languages, so any environment that wants to use rope should be able to easily communicate with traad.

emacs-traad, easy installation and use

emacs-traad (simply traad in melpa) makes it very easy to install traad, and thus rope. Install traad from melpa and then M-x traad-install-server. Then start a connection with traad-open and enjoy. Furthermore it installs traad in its own virtualenv, so it won't clutter your current virtualenv, and you don't have to re-install it in every project.

usage

It is possible to remove a method's argument, normalize arguments, see the occurrences of a method, organize imports, some more python refactoring, and also see the history of traad changes.

Other Python "refactoring" tools

Projectile - replace in project

This is not specific to Python but is very helpful anyway. Projectile has a function projectile-replace which interactively offers to replace all occurrences of a term in all the files of the project. So it's handy to rename a method.

Iedit - replace in buffer

Iedit isn't specific to Python and isn't a "real" "intelligent" refactoring, but it helps editing similar portions of text in the current buffer, with highlighting and simultaneous editing, like multicursors.

Refactor with Elpy

See Elpy's documentation: http://elpy.readthedocs.io/en/latest/ide.html?highlight=django#refactoring

One can refactor code (with rope), format code, edit all the occurences of the symbol at point simultaneously, and fix imports.

Auto-Completion

Jedi

Jedi.el is a Python auto-completion package for Emacs. It aims at helping your Python coding in a non-destructive way. It also helps you to find information about Python objects, such as docstring, function arguments and code location.

Jedi is simple to install and it works out of the box.

See screenshots and get the full documentation : http://tkf.github.io/emacs-jedi/released/

Jedi's official page: http://jedi.jedidjah.ch/en/latest/

Installation

Install Jedi.el via el-get, Marmalade or MELPA (see install for more info) and add this to your Emacs configuration:

   (add-hook 'python-mode-hook 'jedi:setup)
   (setq jedi:setup-keys t)                      ; optional
   (setq jedi:complete-on-dot t)                 ; optional

or call M-x jedi:setup

Note: it's nice to use it in a python interpreter inside emacs :)

Anaconda

Anaconda-mode is a mode for code navigation, documentation lookup and completion for Python.

It runs on emacs 24.3 with python >= 2.6.

It provides:

• context-sensitive code completion for Python
• jump to definition
• find references
• view documentation
• virtualenv management

The package is available in MELPA. For more information, read its documentation.

Code Checker

Flymake

flymake is a on-the-fly syntax checker for Emacs. We can use it alongside with flyspell.

To run pep8, pylint, pyflakes and unit-tests (with nose), you can be interested in using flymake-python.

Don't forget about pylint too.

Indexing sources: ctags, cscope, pycscope

Indexing sources allows you to do neat things, like going to the definition of a function or finding which functions are calling another one.

etags, ctags

Etags (Exuberant Ctags) generates an index (or tag) file of language objects found in source files that allows these items to be quickly and easily located by a text editor or other utility. A tag signifies a language object for which an index entry is available (or, alternatively, the index entry created for that object). Etags is a multilingual implementation of ctags.

The primary use for the tags files is looking up class/method/function/constant/etc declaration/definitions. Cscope is more powerful (see below).

usage

Generate the tags with this command at the root of your project:

    find . -name "*.py" -print | etags -

it creates the file TAGS.

Note that projects like Projectile or Helm provide an integrated use of etags (finding one, re-generating the index, etc).

cscope

cscope is a much more powerful beast. While it operates on more or less the same principle (generating a file of useful metadata) it allows you do some fancier things like find all references to a symbol, see where a function is being invoked, etc.+ (you can find definitions as well).

It was originally designed for C and C++, but thanks to version 0.3 of pycscope, pythonistas can make use of it.

usage

The following commands should get you running:

   apt-get install cscope
   pip install pycscope
   # in project base dir:
   find . -name '*.py' > cscope.files
   cscope -R 

now install the xcscope emacs package with ELPA and require it:

   (require 'xcscope)

When done, you can either enable the mode with cscope-minor-mode (which will add a menu you are free to explore) or call some functions like M-x cscope-find-global-definition, M-x cscope-find-functions-calling-this-function, etc.

Interactivity with helm-cscope

You can do all that interactively with helm-cscope: when you are on symbol, just call M-x helm-cscope-find-global-definition (for example) and enjoy the nice interactive interface.

Running tests

Elpy has commands to run either all tests of the current buffer or only the test we are on. It supports launchers for Django (thus using "manage.py test"), pytest and nose. See http://elpy.readthedocs.io/en/latest/ide.html?highlight=django#testing

If you wish to modify the way it runs test, set the following variable:

(setq elpy-test-<django, nose, pytest>-runner-command '("./manage.py" "test"))

Debugging

Well, let's start with the debuggers documentation:

• pdb doc
• pdb++ has a few more commands
• ipdb to make use of ipython (aka jupyter)

Ipdb, ipython debugger

If you call ipdb, the execution will stop and give a nice ipython-looking prompt. Just add the line

   import ipdb; ipdb.set_trace()

Now you can use the usual commands: c to continue the execution, n to execute the next one, s to step into the next function call, l to list the source code, ! to execute a statement in the current context, etc.

Note: if you like IPython/Jupyter, look at the Emacs IPython Notebook.

Note: You can use yasnippet to quickly enter this sentence, or just a simple function:

(defun python-add-breakpoint ()
  "Add a break point"
  (interactive)
  (newline-and-indent)
  (insert "import ipdb; ipdb.set_trace()")
  (highlight-lines-matching-regexp "^[ ]*import ipdb; ipdb.set_trace()"))

You can use the following to highlight this line, and not forget it :

; Highlight the call to ipdb
; src http://pedrokroger.com/2010/07/configuring-emacs-as-a-python-ide-2/
(defun annotate-pdb ()
  (interactive)
  (highlight-lines-matching-regexp "import ipdb")
  (highlight-lines-matching-regexp "ipdb.set_trace()"))
(add-hook 'python-mode-hook 'annotate-pdb)

And just delete every breakpoints of the buffer:

(defun ipdb-cleanup ()
    (interactive)
    (save-excursion
      (replace-regexp ".*ipdb.set_trace().*\n" "" nil (point-min) (point-max))
      ;; (save-buffer)
      ))

pdb track

If you use emacs-for-python given above, you can track your source code while debugging with pdbtrack.

A tool to use in a non-emacs terminal would be pdbpp

Realgud, a front end to many debuggers to stay in the source window

See Realgud. The goal is to call the debugger from within emacs and manipulate it (continue, next commands, setting breakpoints, etc) within the source window.

Redefine the code being run in the debugger session

The tip in this article makes use of the pdb feature to execute code in the context of the current stack frame with the "!" command (which can actually be omitted. See the pdb doc) and it uses an emacs package to make it much easier to write code blocks for the (i)pdb prompt (à la org-mode code blocks).

M-x compile and breakpoints

You may run your script from a real terminal, from emacs with M-x shell or with M-x compile. If you like the latter you'll notice that it hangs if you set breakpoints inside your code. The function below checks if your code has breakpoints and if so, it gives you the hand for the debugger prompt:

; thanks to https://masteringemacs.org/article/compiling-running-scripts-emacs (slightly modified)
(defadvice compile
    (before ad-compile-smart activate)
  "Advises `compile' so it sets the argument COMINT to t if breakpoints are present in `python-mode' files"
  (when (derived-mode-p major-mode 'python-mode)
    (save-excursion
      (save-match-data (goto-char (point-min))
                       (if (search-forward "ipdb.set_trace()"
                                           (point-max)
                                           t)
                           (progn (ad-set-arg 1 t)
                                  (switch-to-buffer-other-window "*compilation*")))))))

Now just run M-x compile as usual.

Documentation

Insert Sphinx docstrings

The package sphinx-doc (on melpa) offers, with one shortuct, to insert or update Sphinx-like docstrings for the current method's arguments.

Eldoc: argument list in the echo area

Eldoc is a MinorMode which shows you, in the echo area, the argument list of the function call you are currently writing. It works out of the box for elisp editing and for modes that implement its support, such as python-mode. You just have to call eldoc-mode.

To always enable it:

   (add-hook 'python-mode-hook 'eldoc-mode)

helm-pydoc

helm-pydoc is a little utility that lets you interactively select a module you want read the doc and displays it in another buffer.

pydoc linkification

If you've ever used the emacs lisp documentation system, you may find that the pydoc is poorer in functionnalities:

• there is no direct link to code source
• there is no "back" button

John Kitchin's pydoc aims at fixing that with more enhancements, like a bit of source fontification. See more at his presentation and on github.

info format

Another way to read the python documentation is in the texinfo format. You'll need to install the package python-info on MELPA and go through the installation process.

Other tools

Virtual environments

Some tools allow to apply the changes of virtual environment activation inside emacs. They make M-x compile aware of the environment, etc.

• virtualenvwrapper emulates much of the functionnality of virtualenvwrapper, integrates well with M-x shell or eshell, is aware of hooks, has an automatic activation by project and integrates in the mode line.

• pyvenv sets the python path, uses virtualwrapper's hooks, ...

Pymacs

Pymacs is an Emacs extension that enables the use of Python alongside with Emacs Lisp.

Pymacs homepage

include import statements or remove unused

Ropemacs (see above) is a plugin for performing python refactorings in emacs. It uses rope library and pymacs. It has rope-auto-import, so if you write

   rmtree

and then execute M-x rope-auto-import,

   from shutil import rmtree

is inserted at the top of the file.

Unfortunately Ropemacs can not do non-relative imports, it can only create imports of the "from X import Y" variety (if you type shutil.rmtree it doesn't write import shutil).

pyimport can add import statements or remove the unused ones (with pyflakes).

Elpy too can import missing ones.

See also a snippet to add many imports at once.

Sort import statements

See the py-isort Melpa package to automatically sort import statements.

pip-requirements mode

pip-requirements (in MELPA) is a major mode for editing pip requirements files, with the following features:

• syntax highlighting
• Auto completion of package names from PyPI
• togglable comments

pip utilities

pippel (in melpa) provides a similar package menu as "package-list-packages", to list, install, upgrade, remove etc pip packages.

pip-utils (not in melpa) provides some functions with completion of packages and of the requirements files of the project:

• choose one requirements file of the project if many and install its packages,
• chooses the right virtual env,
• install a new package and add it into a requirements file (with interactive choice, needed when there is a "dev-requirements" for instance),
• check the current package version (with pip freeze),
• open the package page on pypi.

Code coverage with Python 3

There's a tool to highlight in real time the lines that are (not) covered by your unit tests. It's called pycoverage.el and it is in melpa.

See also

You'll be certainly interested in the following packages (that you will find on ELPA or MELPA):

Magit, a git interface

Emacs + git is magic : see magit

Code navigation: jump to function definitions

Get started with imenu and helm-imenu (see helm).

Then for cross-project code navigation see xcscope that answers questions like "show all methods that call this one".

See also grep, rgrep, occur, helm-swoop (interactive grep of the current buffer) and enhanced grep tools like ag and its related helm-ag.

Code navigation: move by indentation levels

See indent-tools (in MELPA) that allows to move and fire actions by levels of indentation, for code source like Python or Yaml. For example: go to the next line that has the same indentation level, comment/kill/fold/… the current indentation level, the current paragraph, etc.

Yasnippet, a template system

yasnippet

How to do code folding

See folding

other ELPA packages

If you're running Emacs with package support (usually Emacs 24 and later), check out what is available in ELPA:

M-x list-packages

and see flymake-pyhon-pyflakes, flymake-shell, abl-mode (a python TDD minor-mode), nose (easy python test-running in emacs), visual-regexp / visual-regexp-steroids, python-magic and more.