Making your own packages (and installing them!)

Making your own packages (and installing them!)#

Thus far, you have likely only pip installed other people’s packages. But did you know you can do that for your own code as well?

Throughout this summer, and your research career, you will write useful code (functions, classes) which you’ll use again and again. Instead of tracking down some old python file, copying that function, and pasting it into our new code each time (ouch), in this walkthrough, you’ll learn how to make your small collections of functions a python package, so that you can pip install it on your own computer and import it any time you like! This has several benefits.

no copying and pasting and tracking down old files
there’s now one definative versino of the function. When you change/improve it, you don’t have to do so multiple times
can simply import myfunc in your scripts
makes it easier to git-backup (next notebook) and distribute to others

Exercise 1: Make a folder

First, we need a folder to house not just this code, but also any software that we are downloading/creating and working with the code base directly. I recommend putting this in your home directory, or if you use Dropbox or similar, within that directory. Call it something like Software or git (since eventually this will be a git repo). It is very handy to keep all software of this kind in one place.

Once you’ve made your overall software folder, make a folder inside it. For the purposes of this walkthrough, I’m going to be calling our mini-package myutils, as a collection or assortment of utility functions useful to us. You can call yours whatever you like. Inside THAT, make another folder, with the same name (or you can make the outer one slightly different).

Expert move: Do this from the shell command line. You can even make both folders with one line! It would look like this:

mkdir ~/Software; mkdir ~/Software/MyUtils; mkdir ~/Software/MyUtils/myutils

Once we have our folder, we are ready to start structuring our package.

The innermost folder, the one I’ve named myutils, is where the code is going to live. Outside it, in MyUtils is where my setup file, readme, license, documentation if any, etc. will live.

In Python, every python file or folder of python files is implicitly a module. If we have two python files, a.py and b.py in the same directory, and file a.py contains a function foo, I could open file b.py with import foo and it would work.

To go the extra mile of installing a.py and b.py such that we could import foo anywhere, we’ll need to add a couple of things.

Exercise 2: Init

Make an empty file called __init__.py in your myutils (innermost) directory.

Expert Move: Do this from the shell/command line. You can do so in one line:

cd ~/Software/MyUtils/myutils; touch __init__.py

The __init__.py file we created above tells python and its installers that this is a package. As packages get more complicated, you will start adding things into your init files, but for the purposes of this exercise, you don’t need to! It can simply be an empty placeholder file.

We’re now ready to add some code! For this walkthough, I’ll use the implot() function you would have made at the end of the Astronomical Imaging tutorial.

Exercise 3: Add Implot

Create a file plotting.py in your myutils (innermost) folder. Open it. From the Astronomical Imaging walkthrough, grab your implot() function, and any library imports needed to make it work. Copy it into this file.

If you didn’t do that lab, here is a simple version you can use:

def implot(im,scale=1.0):
    m = np.mean(im)
    s = np.std(im)
    vmin=m-s*scale; vmax=m+s*scale
    fig,ax=plt.subplots(figsize=(15,10))
    ax.imshow(im,origin='lower',cmap='gray',vmin=vmin,vmax=vmax)
    return fig, ax

Great! Now we have a package with some useful code in it! Let’s try to install it. For that, we’ll need to add a simple setup.py function to our outer directory.

Exercise 4: Setup.py

In your MyUtils (outer) directory, create a file called setup.py. Inside, add the following:

import setuptools

setuptools.setup(
    name="myutils",
    version="0.1",
    author="Your Name",
    author_email="Your email",
    description="A small example Python package",
    packages=["myutils"]
)

Replace the strings with things relevant to you.

Believe it or not, we’re done! Packaging code can obviously get more complicated — in particular, you may want the added organizational benefit of subfolders inside myutils. This creates a bit more complexity, as these are considered sub-packages and need their own __init__ files, and need to be specified in the setup.py file.

Ultimately, though, the structure we’ve made here will serve you for a while! If you have new things to add to your utils that aren’t about plotting, make a new file next to plotting.py (like, e.g., analysis.py) and add your code there.

Exercise 5: Install!

We’re ready to install your code! I recommend installing in development mode, which tells your path to point to this location where you’ve been editing (rather than copying to the site-packages directory elsewhere). The advantage is that if you add/edit your package, you don’t need to reinstall it to access the changes, just restart any python kernels or interpreters and re-import (or use the reload module).

Navigate your shell to the outer MyUtils folder (the one with your setup.py) and type

pip install -e .

This should insall your package! Try opening a new shell, opening ipython, and typing

from myutils.plotting import implot

Did it work?