Object Oriented Programming

Overview

Teaching: 30 min
Exercises: 40 min

Questions

How can we use code to describe the structure of data?

How should the relationships between structures be described?

Objectives

Describe the core concepts that define the object oriented paradigm

Use classes to encapsulate data within a more complex program

Structure concepts within a program in terms of sets of behaviour

Identify different types of relationship between concepts within a program

Structure data within a program using these relationships

Introduction

Object oriented programming is a programming paradigm based on the concept of objects, which are data structures that contain (encapsulate) data and code. Data is encapsulated in the form of fields (attributes) of objects, while code is encapsulated in the form of procedures (methods) that manipulate objects’ attributes and define “behaviour” of objects. So, in object oriented programming, we first think about the data and the things that we’re modelling - and represent these by objects - rather than define the logic of the program, and code becomes a series of interactions between objects.

Structuring Data

One of the main difficulties we encounter when building more complex software is how to structure our data. So far, we’ve been processing data from a single source and with a simple tabular structure, but it would be useful to be able to combine data from a range of different sources and with more data than just an array of numbers.

data = pd.DataFrame(data = [[1., 2., 3.],
                            [4., 5., 6.]],
                    columns = list('abc'))

Using this data structure has the advantage of being able to use Pandas operations to process the data and Matplotlib to plot it, but often we need to have more structure than this. For example, we may need to attach more information about the object which light curve we analyse, such as cutout image of this source or its spectra.

In a way, we already encountered this problem when we needed to store light curves of a single object, but obtained in different bands. Our solution then was to use a dictionary where keys corresponded to the bands names, and values were the DataFrames with the measurements. Generally speaking, we can expand this solution by adding new elements with values of other data types. For example, we could write something like this:

lc = {}
...
lc['spectra'] = np.array([4,5,6]) 

Since Python distionaries can store elements of different types, there is nothing that would stop us from this. We can also make nested dictionaries, creating really complex data structures.

Then we can get lost in them.

Classes in Python

Using nested dictionaries and lists should work for some of the simpler cases where we need to handle structured data, but they get quite difficult to manage once the structure becomes a bit more complex. For this reason, in the object oriented paradigm, we use classes to help with managing this data and the operations we would want to perform on it. A class is a template (blueprint) for a structured piece of data, so when we create some data using a class, we can be certain that it has the same structure each time.

With our dictionaries we had in the examples befpre, we have no real guarantee that each dictionary has the same structure unless we check it manually. With a class, if an object is an instance of that class (i.e. it was made using that template), we know it will have the structure defined by that class. Different programming languages make slightly different guarantees about how strictly the structure will match, but in object oriented programming this is one of the core ideas - all objects derived from the same class must follow the same behaviour.

You may not have realised, but you should already be familiar with some of the classes that come bundled as part of Python, for example:

my_list = [1, 2, 3]
my_dict = {1: '1', 2: '2', 3: '3'}
my_set = {1, 2, 3}

print(type(my_list))
print(type(my_dict))
print(type(my_set))

<class 'list'>
<class 'dict'>
<class 'set'>

Lists, dictionaries and sets are a slightly special type of class, but they behave in much the same way as a class we might define ourselves:

They each hold some data (attributes or state).
They also provide some methods describing the behaviours of the data - what can the data do and what can we do to the data?

The behaviours we may have seen previously include:

Lists can be appended to
Lists can be indexed
Lists can be sliced
Key-value pairs can be added to dictionaries
The value at a key can be looked up in a dictionary
The union of two sets can be found (the set of values present in any of the sets)
The intersection of two sets can be found (the set of values present in all of the sets)

Encapsulating Data

Let’s start with a minimal example of a class representing a variable object.

class Variable:
    def __init__(self, obj_id):
        self.obj_id = obj_id
        self.lc = {
                   'mjd': np.array([]),
                   'mag': np.array([])
                  }

star_obs = Variable(obj_id)
print(star_obs.obj_id)

1405624461041897445

Here we’ve defined a class with one method: __init__. This method is the initialiser method, which is responsible for setting up the initial values and structure of the data inside a new instance of the class - this is very similar to constructors in other languages, so the term is often used in Python too. The __init__ method is called every time we create a new instance of the class, as in Variable(obj_id). The argument self refers to the instance on which we are calling the method and gets filled in automatically by Python - we do not need to provide a value for this when we call the method.

Data encapsulated within our Variable class includes the object’s id, and a light curve dictionary, that contains a numpy array with timestamps and a numpy array with magnitude measurements. In the initialiser method, we set an object’s id to the value provided, and create the numpy arrays for observations (initially empty). Such data is also referred to as the attributes of a class and holds the current state of an instance of the class. Attributes are typically hidden (encapsulated) internal object details ensuring that access to data is protected from unintended changes. They are manipulated internally by the class, which, in addition, can expose certain functionality as public behavior of the class to allow other objects to interact with this class’ instances.

Encapsulating Behaviour

In addition to representing a piece of structured data (e.g. an object that has an id and the lists with timestamps and magnitude observations), a class can also provide a set of functions, or methods, which describe the behaviours of the data encapsulated in the instances of that class. To define the behaviour of a class we add functions which operate on the data the class contains. These functions are the member functions or methods.

Methods on classes are the same as normal functions, except that they live inside a class and have an extra first parameter self. Using the name self is not strictly necessary, but is a very strong convention - it is extremely rare to see any other name chosen. When we call a method on an object, the value of self is automatically set to this object - hence the name. As we saw with the __init__ method previously, we do not need to explicitly provide a value for the self argument, this is done for us by Python.

Let’s add another method on our Variable class that adds observations to a Variable instance.

class Variable:
    """A Variable class"""
    def __init__(self, obj_id):
        self.obj_id = obj_id
        self.lc = {
                   'mjd': np.array([]),
                   'mag': np.array([])
                  }

    def add_observations(self, mjds, mags, mag_errs=None):
        """
        Adds observations to the light curve.
    
        Args:
          mjds: A vector of Modified Julian Dates (x values).
          mags: A vector of luminosities (y values).
        """
        self.lc['mjd'] = np.array(mjds)
        self.lc['mag'] = np.array(mags)
        if mag_errs is not None:
          self.lc['mag_errs'] = np.array(mag_errs)

        return


obj_id = lc_datasets['lsst']['objectId'].unique()[7]
b = 'g'
filt_band_obj = (lc_datasets['lsst']['objectId'] == obj_id) & (
        lc_datasets['lsst']['band'] == b
    )
obj_obs = lc_datasets['lsst'][filt_band_obj]
star = Variable(obj_id)
star.add_observations(obj_obs[time_col],obj_obs[mag_col])
print(star)
print(star.lc)

<__main__.Variable object at 0x7fc0fb40a750>
{'mjd': array([60559.2973682, 59791.3473572, 60559.2978172, 61017.0665232,
       60281.1630512, 59840.2103322, 60560.2654012
...

Note also how we used mag_errs=None in the parameter list of the add_observations method, then initialise it if the value is not None, i.e. if the user passed magnitude errors. This is one of the common ways to handle an optional argument in Python, so we’ll see this pattern quite a lot in real projects.

Class and Static Methods

Sometimes, the function we’re writing doesn’t need access to any data belonging to a particular object. For these situations, we can instead use a class method or a static method. Class methods have access to the class that they’re a part of, and can access data on that class - but do not belong to a specific instance of that class, whereas static methods have access to neither the class nor its instances.

By convention, class methods use cls as their first argument instead of self - this is how we access the class and its data, just like self allows us to access the instance and its data. Static methods have neither self nor cls so the arguments look like a typical free function. These are the only common exceptions to using self for a method’s first argument.

Both of these method types are created using decorators - for more information see the classmethod and staticmethod decorator sections of the Python documentation.

Dunder Methods

Why is the __init__ method not called init? There are a few special method names that we can use which Python will use to provide a few common behaviours, each of which begins and ends with a double-underscore, hence the name dunder method.

When writing your own Python classes, you’ll almost always want to write an __init__ method, but there are a few other common ones you might need sometimes. You may have noticed in the code above that the method print(star) returned <__main__.Patient object at 0x7fd7e61b73d0>, which is the string representation of the star object. We may want the print statement to display the object’s id instead. We can achieve this by overriding the __str__ method of our class.

...
    def __str__(self):
      return str(self.obj_id)


star = Variable(obj_id')
print(star)

1405624461041897445

These dunder methods are not usually called directly, but rather provide the implementation of some functionality we can use - we didn’t call star.__str__(), but it was called for us when we did print(star). Some we see quite commonly are:

__str__ - converts an object into its string representation, used when you call str(object) or print(object). Pay attention that we had to convert our self.obj_id to string in order for this method to work
__getitem__ - Accesses an object by key, this is how list[x] and dict[x] are implemented
__len__ - gets the length of an object when we use len(object) - usually the number of items it contains

There are many more described in the Python documentation, but it’s also worth experimenting with built in Python objects to see which methods provide which behaviour. For a more complete list of these special methods, see the Special Method Names section of the Python documentation.

Exercise: Useful Methods for the Variable Class

Add several methods to our class, that would provide the following functionality:

return the length of the lightcurve as the length of the Variable instance;
check that we are passing the suitable type of the data to our add_observations method, convert it into np.array and check that the length of all observational arrays is the same.

A hint: you may want to write several methods for the second task.

Solution

For the first task we can write our own __len__ dunder method:

class Variable:
...
def __len__(self):
    return len(self.lc["mjd"])

For the second task we may want to break it into several features and write a function for each of them:

class Variable:
...

  def convert_to_array(self,data):
       # Check if the data is iterable and convert it into np.array, otherwise raise an exception
       if not isinstance(data, np.ndarray):
           if isinstance(data, (list,tuple,pd.Series)):
               data = np.array(data)
           elif isinstance(data, (int, float)):
               data = np.array([data])
           else:
               raise ValueError("The data type of the input is incorrect!")
       return data

   def compare_len(self,arrs):
       # Check that all arrays are of the same length
       lens = [len(arr) for arr in arrs]
       if len(set(lens)) > 1: # set() turns an iterable into a set of unique values.
       # If the values are all the same, the set will contain only one element
           raise ValueError("Passed timestamps and mags or mag_errs arrays have different lengths!")
       return
       
  def add_observations(self, mjds, mags, mag_errs=None):
       """
       Adds observations to the light curve.

       Args:
         mjds: A vector of Modified Julian Dates (x values).
         mags: A vector of luminosities (y values).
         mag_errs: A vector of magnitude errors.
       """
       self.lc["mjd"] = self.convert_to_array(mjds)
       self.lc["mag"] = self.convert_to_array(mags)
       if mag_errs is not None:
           self.lc["mag_errs"] = self.convert_to_array(mag_errs)
       self.compare_len(self.lc.values())
       return        

Properties

The final special type of method we will introduce is a property. Properties are methods which behave like data - when we want to access them, we do not need to use brackets to call the method manually.

class Variable:
    ...

    @property
    def mean_mag(self):
        return np.mean(self.lc['mags'])
...
star = Variable(obj_id)
...
mean_mag = star.mean_mag
print(mean_mag)

18.03180312045771

You may recognise the @ syntax from episodes on parameterising unit tests and functional programming - property is another example of a decorator. In this case the property decorator is taking the last_observation function and modifying its behaviour, so it can be accessed as if it were a normal attribute. It is also possible to make your own decorators, but we won’t cover it here.

Relationships Between Classes

We now have a language construct for grouping data and behaviour related to a single conceptual object. The next step we need to take is to describe the relationships between the concepts in our code.

There are two fundamental types of relationship between objects which we need to be able to describe:

Ownership - x has a y - this is composition
Identity - x is a y - this is inheritance

Composition

In object oriented programming, we can make things components of other things.

We often use composition where we can say ‘x has a y’ - for example in our lcanalyzer project, we might want to say that a star has a multiband lightcurve, or that a lightcurve has a single-band lightcurve.

In the case of our example, we’re already saying that our variable star has a lightcurve, so we’re already using composition here. We’re currently implementing a single-band lightcurve as a dictionary with a known set of keys though, and in the previous examples we used a dictionary to store DataFrames with single-band observations to represent multi-band data. Nothing stops us from turning these dictionaries into proper classes. In fact, this is exactly what we should do. For our current class example, it will look like this:

class Lightcurve:
    """Class Lightcurve"""
    def __init__(self, mjds=None, mags=None, mag_errs = None):
        self.lc = {}
        if mjds is not None:
            self.lc = self.add_observations(mjds, mags, mag_errs)

    def add_observations(self, mjds, mags, mag_errs = None):
        self.lc["mjds"] = self.convert_to_array(mjds)
        self.lc["mags"] = self.convert_to_array(mags)
        if mag_errs is not None:
            self.lc["mag_errs"] = self.convert_to_array(mag_errs)
        self.compare_len(self.lc.values())
        return self.lc
    
    def convert_to_array(self,data):
...

class Variable:
    """A Variable class"""

    def __init__(self, obj_id):
        self.obj_id = obj_id
        self.mband_lc = {}
    
    def add_lc(self,band,mjds,mags,mag_errs=None):
        self.mband_lc[band] = Lightcurve(mjds,mags,mag_errs)
        return self.mband_lc
        
    def __str__(self):
        return str(self.obj_id)


star = Variable(obj_id)
star.add_lc(band = b,mjds = obj_obs[time_col], mags = obj_obs[mag_col])
star.mband_lc['g'].mean_mag

18.03180312045771

Now we’re using a composition of two custom classes to describe the relationship between two types of entity in the system that we’re modelling. The benefit of this approach is that we can create a new class called e.g. Asteroid, and it will require implementing its own analysis methods that will differ from those of the class Variable. The implementation of Asteroid class will be different, but it can still have the Lightcurve. Note that this is only one possible implementation; for example, we are still storing our light curve observations in a dictionary (self.lc = {}) to avoid rewriting our already existing functions, but in reality it is likely will be more practical to turn mjds and mags into separate variables.

Inheritance

The other type of relationship used in object oriented programming is inheritance. Inheritance is about data and behaviour shared by classes, because they have some shared identity - ‘x is a y’. If class X inherits from (is a) class Y, we say that Y is the superclass or parent class of X, or X is a subclass of Y.

Extending the previous example, we can recall that there are different types of variables. For example, we can have periodic variables, such as RR Lyrae, or transient ones, such as supernovae (or SNe for short). Periodic variables will need a method for determining their periods, while transient ones will benefit from implementing an algorithm for SNe classification. Instead of writing these two classes completely independently, we can make them both the subclasses of the class Variable.

To write our class in Python, we used the class keyword, the name of the class, and then a block of the functions that belong to it. If the class inherits from another class, we include the parent class name in brackets.

class RRLyrae(Variable):
    """A class for RR Lyrae stars."""
    def __init__(self, obj_id):
        super().__init__(obj_id)
        self.period = None

    def period_determination(self, period_range=(0.1,3)):
        """A function to determine the period"""
        self.period = 0.3
        return

rr_lyrae = RRLyrae(obj_id)
rr_lyrae.period_determination()
print(rr_lyrae.mband_lc)
print(rr_lyrae.period)

{}
0.3

In this example, Variable is a parent class (or superclass), and RRLyrae is a subclass.

There’s something else we need to add as well - Python doesn’t automatically call the __init__ method on the parent class if we provide a new __init__ for our subclass, so we’ll need to call it ourselves. This makes sure that everything that needs to be initialised on the parent class has been, before we need to use it. If we don’t define a new __init__ method for our subclass, Python will look for one on the parent class and use it automatically. This is true of all methods - if we call a method which doesn’t exist directly on our class, Python will search for it among the parent classes. The order in which it does this search is known as the method resolution order.

The line super().__init__(obj_id) gets the parent class, then calls the __init__ method, providing the obj_id variable that Variable.__init__ requires. This is quite a common pattern, particularly for __init__ methods, where we need to make sure an object is initialised as a valid X, before we can initialise it as a valid Y - e.g. a valid Variable must have an obj_id, before we can properly initialise a RRLyrae model with their data.

Composition vs Inheritance

When deciding how to implement a model of a particular system, you often have a choice of either composition or inheritance, where there is no obviously correct choice. For example, it’s not obvious whether a multi-messenger event (i.e. the one that has information coming with different carriers, such as electromagnetic waves and gravitational waves) is a light curve and is a chirp, or has a light curve and has a chirp.
class Observation:
    pass

class Lightcurve(Observation):
    pass

class Chirp(Observation):
    pass

class MultiMessengerEvent(Lightcurve, Chirp):
    # Multi-messenger event `is a` Lightcurve and `is a` Chirp
    pass
class Observation:
    pass

class Lightcurve(Observation):
    pass

class Chirp(Observation):
    pass

class MultiMessengerEvent(Observation):
    def __init__(self):
        # Multi-messenger event `has a` Lightcurve and `has a` Chirp
        self.lc = Lightcurve()
        self.chirp = Chirp()
Both of these would be perfectly valid models and would work for most purposes. However, unless there’s something about how you need to use the model which would benefit from using a model based on inheritance, it’s usually recommended to opt for composition over inheritance. This is a common design principle in the object oriented paradigm and is worth remembering, as it’s very common for people to overuse inheritance once they’ve been introduced to it.

For much more detail on this see the Python Design Patterns guide.

Multiple Inheritance

Multiple Inheritance is when a class inherits from more than one direct parent class. It exists in Python, but is often not present in other Object Oriented languages. Although this might seem useful, like in our inheritance-based model of the Multi-messenger event above, it’s best to avoid it unless you’re sure it’s the right thing to do, due to the complexity of the inheritance heirarchy. Often using multiple inheritance is a sign you should instead be using composition - again like the Multi-messenger event model above.

Key Points

Object oriented programming is a programming paradigm based on the concept of classes, which encapsulate data and code.

Classes allow us to organise data into distinct concepts.

By breaking down our data into classes, we can reason about the behaviour of parts of our data.

Relationships between concepts can be described using inheritance (is a) and composition (has a).

previous episode

Intermediate Python for Astronomical Software Development

next episode