Base Methods

We use the Tastypie. package as a baseline for our API, and have modeled our API around the idea of Model Resource, and Non Model Resources.

Model Resources provide basic CRUD functionality for our models. This means that the format that you use to POST, GET, or DELETE from one resource will be for the most part the same as any other.

Non Model Resources are in essence API calls that trigger the application to do something in the background. For instance, we call have a NonModelResource that triggers an aggregation refresh for a particular campaign.

Both the Model and the Non Model Resource have a number of options that you can put on the Meta class for instance to say which parameters are needed for at GET or a POST method. This means that the resources themselves are very lightweight, and that the majority of the logic and code that we have for the API is written in the BaseResources.

The Tastpie package is great for getting started on a CRUD api, but there is alot that happens in the background that makes debugging really difficult, so what we have done is overriden all of the methods that are executed when a request comes in so that we have control and insight into what happens when a request is happening.

../_images/api_inheritence.png

BaseResource

The BaseResource inherits from the Tastypie Resource Class.` and in this method we do two things.

  1. Override the Dispatch method, which controls what happens when a request of any kind comes in.
  2. Create a method called validate_filters, which makes sure that a GET request has the correct parameters.
BaseResource.dispatch(request_type, request, **kwargs)[source]

Overrides Tastypie and calls get_list for GET, obj_create for POST, an obj_delete for DELETE, obj_update for PATCH and get_detail when fetching an object when a primary key is requested (i.e. GET /<my_resource>/pk).

This method also gets the top_level_location_id for a user which will allow later on for the API to filter the locations that they can see based on what has been set for them in the user admin scren.

This location filtering is not used right now, but all of the ground work has been put into place for a developer to set up regional, provincial or country level access for specific users.

BaseResource.validate_filters(request)[source]

For a GET request, make sure that all required filters have been passed and throw an exceptin if there are any filters missing.

In order to control which parameters are requried in a GET request, simply add the GET_params_required attribute to the Meta class of a given resource.

BaseNonModelResource

This class is all about using the API to trigger the backend to do something.

This isn’t something you would use, but rather something you might hit from cron, in order to process some data on a schedule.

There is only one method here that matters and it is pre_process_data. If you want to create an api resource that does something in the backend, you can simply extend this class, then override the method below to do whatever you want.

After running this method, the api will return whatever is in the queryset atttribute of the meta class.

class rhizome.api.resources.base_non_model.BaseNonModelResource(api_name=None)[source]

This class only accepts GET requests, which are reouted through the get_list method.

The get list mehtod first checks to see if the class that is being executed has a method called pre_process_data. If it does have that method, then, it executes it.

After the method is executed, it returns a set of objects that come back from the get_object_list method.

This resource class is used for data manipulation and unlike the BaseModelResource is not meant to be a fully functioning CRUD interface.

So, if you want to create a resource in which you need to manipulate some data then return it to the application, override the pre_process_data method, do what you need to do, then return the results, or whatever you need to via get_object_list.

As an example, check out what happens when we hit the transform_upload api for a particular document

DocTransFormResource.pre_process_data(request)[source]
## when you upload a file, step one is getting data into source_submission
document.doc_transform()
## Step two is translating form source_submission into datapoints
document.refresh_master()
## step three, if the file is of type campaign, is aggregation
for each campaign in the document campaign.aggregate_and_calculate()

BaseModelResource

The BaseModelResource offers a full CRUD interface for the resources that extend it. The idea is that, with very lttle work you can create an interface for a django model by setting a few attributes in the Meta class.

In order to set up a model resource, you must at most set up two attributes in the meta class
  • resource_name : This is the namespace that we access when we interface with a resource, i.e. /api/v1/<resource_name>
  • object_class : This refers to the Model that you want to interface with. All of the CRUD applications for a resource will be executed against this model.

GET (by PK)

GET : /api/v1/campaign/1/

returns to us a single object.

{
  campaign_type_id: 1,
  created_at: "2016-05-04T02:31:29.227679",
  end_date: "2016-02-29",
  id: 1,
  name: "February 2016 - SNID - OPV ",
  start_date: "2016-02-02"
}

A GET request in which a primary key is passed is routed via the dispatch method to the obj_get method.

BaseModelResource.obj_get(bundle, **kwargs)[source]

Takes optional kwargs, which are used to narrow the query to find the instance.

Currently used to find one object from the url api/v1/resource/<pk>/

Try to find an object using the pk, if the resource is not found, or more than one object is found, throw an error, then throw an erro

GET (with query filters)

A get requests can be used to access an object via primary key, but it is also possible to send query filters and get in the response the objects that match.

For instance a query like:

GET : /api/v1/indicator/
PARAMS: {name"Polio Cases"}

or

GET: /api/v1/indicator/
PARAMS: {id__gt: 130}

Filters the indicator resource in accordance to the filters passed.

These filters are based directly off of the Django ORM Filters.`.

This is a bit of an over simplification of what happens when we filter, but it should give a good idea of how we take advantage of the djano ORM to return relevant data to the application.

query_filters = request.GET
model_class = self._meta.object_class
fields_to_return = self._meta.object_class

queryset = model_class.objects.filter(query_filters).values(fields_to_return)
# queryset = Location.objects.filter({name:'NYC'}).values(['id','name','parent_location__name'])
return queryset

this type of filtering is handled via the obj_get_list method

BaseModelResource.obj_get_list(bundle, **kwargs)[source]

Get the filters from the request

Query the database using the get_object_list method

Use the query_fields meta atribtue to tell the database what fields we want

Use the cleaned filters from the request to filter the relevant object.s

BaseModelResource.apply_filters(request, applicable_filters)[source]

An ORM-specific implementation of apply_filters. The default simply applies the applicable_filters as **kwargs, but should make it possible to do more advanced things.

BaseModelResource.get_object_list(request)[source]

Take the query fields from the request and return the relevant objects. By setting up the query fields using the Django join syntax, we can access related models, for instance, if we are searching a blog table, which has an author_id in it, we can use the query_fields atribute to get the authors name, by adding

query_fields = [‘title’, ‘author__name’]`` to the meta class.

That way, we can use the functionality of the ORM to keep our code simple even when we need for the database to execute one or more joins.

Note: Django queryset are lazy which means that we can pass them around and alter them with the filter syntax, see here <https://docs.djangoproject.com/en/1.10/topics/db/queries/#querysets-are-lazy>`_

POST

Certain models have non nullable fields, which means that you need to set up the required_fields_for_post parameter in the meta class, so that if the API does not receive this in the request, it can formulate a sensible error message, as opposed to returning the integrity error message from the database.

The POST requests are handled in the obj_create method and while you have the freedom to override this ( see api/v1/document ) to handle a particular use case, the obj_create uses the Meta attributes defined in the resource to validate and Create objects.

BaseModelResource.obj_create(bundle, **kwargs)[source]

A ORM-specific implementation of obj_create.

This also handles updates ( PUT ) requests by looking if the request has the ID in there, and if so, updating the resorce with the relevant data items.

PATCH

When updating a resource, submit a PATCH request with the fields you want to update as parameters.

PATCH /api/v1/indicator/123/
PARAMS {'name': 'a new name for indicator 123'}
BaseModelResource.obj_update(bundle, skip_errors=False, **kwargs)[source]

Get an object ID via the ``obj_get``method

Query the database using the object_class specified.

Update the object in the database].

DELETE

The API allows for traditional REST deletion protocol

DELETE /api/v1/custom_chart/123/
BaseModelResource.obj_delete(bundle, **kwargs)[source]

A ORM-specific implementation of obj_delete.

Get an object ID via the ``obj_get``method

Delete the object from the database using the django ORM

Note – the only place currenlty we DELETE is with charts and dashbaords. For other resources, for example indicators, we will need to be delete any data associated to that ID via a foreign key data.

That means, that if you want to delete an indicator, you will have to override this method in the IndicatorResource and make sure that all foreign keys are deleted as well.

It would alsobe a good idea to check here to see if the user is a superuser before executing that type of query.

We can also delete by using query parameters but only by using the ID. This is because we have some legacy code ( in manage system when we delete calculated_indicator_compoents and indicator_to_tag relations ) that deletes w

Moving forward, please use the DELETE /api/v1/resource/<pk>/ request format in deleteing resources.

Note, if you try to delete a list of objects with the obj_delete_list method, you will get an error saying that you

It is reasonable however to want to delete a list of resources all at once with a particular query filter, for instance you want to delete all tags from an indicator. If so, then the obj_delete_list method is where that would happen.

DELETE /api/v1/custom_chart/?id=123
BaseModelResource.obj_delete_list(bundle, **kwargs)[source]

A ORM-specific implementation of obj_delete_list.

GET Schema

Since we override Tastypie’s model resource In order to get the schema

For instance

/api/v1/indicator/schema

will return to you the information needed about each field of the resource as well as which methods can be applied

For more information on this, research Inspecting the Resource Schema.`

Get Locations From URL

The most important model used in the Base Model Resource is the get_locations_to_return_from_url method.

There are four main resources for which this method applies:

  • /api/v1/location
  • /api/v1/geo
  • /api/v1/date_datapoint
  • /api/v1/campaign_datapoint

These API methods all need to get locations in some way and thus all use this method to take particular query parameters in order to return the appropriate locations to the application.

The following parameters are available and filter locations as described.

  • location_id__in : If this parameter is passed, then these are the exact locations that will be received in the response. This type of filter is sent when you now that you want a specific list of location with no aggregation.
  • location_id & location_type : This means that you want all of the locations of a particular type, underneath the location given. So for instance, if you wanted to say, show me all districts in Afghanistan you would use these two parameters.
  • location_id & location_depth : This means that you want data underneath a particular location, at a particular level of depth.

If you pass location_id=<Afghanistan>&location_depth=0 that will give you Afghanistan only, but location_id=<Afghanistan>&location_depth=1 will give you the direct children ( Regions ) of Afghanistan and location_id=<Afghanistan>&location_depth=2 will give you the grandchildren ( provinces ). This is used in the “drill down” functionality so that if for instance you are looking at a map of a particular Indicator at Afghanistan level and the data is rendering for the Regions, when you drill down to a particular region, using this filter, the application will return the Provinces underneath the region you select. The custom chart functionality relies heavily on the location_depth parameter which gives sensible charts at all location levels

In addition to the functionality above which is used widely throughout the application, we also allow for an additional filter of locations based on particular indicator values. This could be used for instance, to filter a list of locations to those that are not under government control.

  • filter_indicator && filter_value

for more on this, take a look at: get_locations_from_filter_param

BaseModelResource.get_locations_to_return_from_url(request)[source]

This method is used in both the /geo and /datapoint endpoints. Based on the values parsed from the URL parameters find the locations needed to fulfill the request based on the four rules below.

If location_id__in requested.. we return exactly those ids for instance if you were doing data entry for 5 specific districts you would use the location_id__in param to fetch just those ids

TO DO – Check Location Permission so that the user can only see What they are permissioned to.