Building REST APIs with Preside

These are the notes from my PresideCon talk at CFCamp 2016.

You can download the slides here:

As there is not much in there and most of the talk was a demo, I will share the detailed notes from that demo here. You should be able to reproduce the whole project that I created by following the guide step by step.

Preside REST framework

The REST implementation of Preside gives you a quick and easy-to-use framework to build RESTful web services. It is inspired by the excellent http://taffy.io CFML REST framework by Adam Tuttle.

If you want to get a basic understanding of what REST can do for you, Adam's book http://restassuredbook.com is a highly recommended read.

Demo project notes

Enough on theory, let’s dive into Preside and coding.

Requirements

You should have the latest stable version of CommandBox installed. Get it here: https://www.ortussolutions.com/products/commandbox

The Preside commands for CommandBox are also required. Install them with the following command:

box install preside-commands

If you already had them installed before, you might need the --force option and install them again. This way you can make sure that you have the latest version.

Details on the Preside commands can be found here: https://www.forgebox.io/view/preside-commands

In addition you need to have either MariaDB or MySQL on your computer running.

Project setup

Start CommandBox, create a new project directory and cd into it. All commands after box, if not otherwise stated are within the CommandBox CLI. For example (assuming you have an existing projects directory):

box
cd projects
mkdir cfcamp2016/presidecon/restproject --cd

Hint: with the handy --cd option for mkdir you directly change to the just created directory.

preside new site

This pulls all available Preside skeletons from https://www.forgebox.io. For detailed descriptions of all available skeletons, have a look at the following list:

https://www.forgebox.io/type/preside-skeletons

So we pick the REST skeleton:

skeleton: rest

This downloads the skeleton code and uses CommandBox's Package Manager to install it along with depending packages. In this case the depending packages are the Preside core framework as well as four REST-specific extensions (more on that later).

You have to fill out some basic site-specific things that are automatically inserted for you at the end of the installation. Example:

site id: presidecon
admin path: admin
site name: presidecon
author: me

Now the project code has been setup.

open

This opens the folder in Finder/Explorer/whatever (depending on your OS). Add the whole folder to your Editor or IDE project (e.g. I am using Sublime Text).

The box.json file defines the project dependencies.

Hint: if you are going to version your project, just (git)ignore the external dependencies.

Sidenote:

It's easy to see the list of dependencies and whether they are outdated. Just do:

list
outdated

This way it's easy to update your dependencies.

Anyway, let's spin up the server:

preside start

Oh... just forgot we need to create an empty database. I for instance am heading over to mysqlworkbench and create the scheme presidecon2016_restproject.

As it's the first time the server is started, the DB info needs to be entered:

database: presidecon2016_restproject
user: ...
pw: ...
...

Just use the options how you configured your DB.

CommandBox spins up the server and you will see a default homepage in your browser. You will  see quicklinks to access the admin, reload the app and get the generated Swagger specification + UI (more on that later).

REST Endpoints in Preside applications

Head over to the project code and see how endpoints are implemented:
API Endpoints are ColdBox handlers with some special annotations and Preside knows how to deal with them.
There are two special variables (restRequest and restResponse) available.

Full documentation is here: https://docs.presidecms.com/devguides/restframework.html

The REST skeleton ships with a dummy hello-world endpoint which you find at

/application/handlers/rest-apis/sample/hello_world.cfc

In this case /sample is the APIs root URL, /hello/ is the URI and the HTTP verb GET is used (by convention via the function name). There are some additional Swagger specific annotations.

Swagger Spec and UI

From the homepage use the quick link to access the generated Swagger JSON spec. Maybe copy/paste and format the JSON string in your IDE to get a clearer picture of how the format looks like.

Swagger defines a common format to represent a REST API  that various tools / languages can use or interact with.
Bundled within the Preside Swagger extension is the Swagger UI tool.
It’s a pure HTML/JS client that consumes the JSON spec and is a nice little client to try out the API.
Access it from the homepage via a quick link as well.

Import sample data

As the current API doesn't do very much we need some more data - a bit more complex stuff.
Let’s get some better sample data:

Get back to your terminal/commandline:

install preside-ext-starwars

This installs a Preside extension that is able to import data from http://swapi.co.

Go back to the application's homepage and access the Preside admin.
As this is the first time, you have to setup a sysadmin account. Afterwards you can login using the username sysadmin and the entered password.

Some explanations regarding Preside admin: almost everything is customizable - in an easy way.
Here for instance we don’t really deal with sites or a site tree like in a regular installation. So we removed several options in the skeleton. You can do this within your Config.cfc. Head over to the code and look at:

/application/config/Config.cfc

We will change some code in this file later.

Within Preside admin we can also configure stuff via custom system settings.
Go to System > Settings > REST Swagger
Edit for example the title. Change it to "Awesome Star Wars API".

In the Preside admin there is a custom Swagger sidebar item. This has no real function but also gives you quick access to the generated Spec and the UI.
Click on the link to access the Swagger UI. We should see the just updated title there as the UI consumes the JSON specification.

Back to the sample data import:

Open the Preside admin developer terminal. This is a - for now - undocumented but very powerful and customizable feature.

Within the Preside admin you can open and close it with the keyboard shortcut `  - which is the back tick on an english keyboard (German keyboard: Shift + Ctrl +` - this is the accent left to the backspace).

Within this developer terminal do:

extension enable preside-ext-starwars
reload all

Then refresh your browser (Hint: from now on each time I'll ask you to reload the application a browser window refresh should be done as well).

Go to the Data manager within the Preside admin. There is a new Star Wars category along with several object types. Have a look at those. Tables for those objects have been automatically created during application reload. But no data is in there yet.

Go to System > Task manager and perform the Star Wars data import task.

This automatically imports a lot of data from swapi.co, which is an excellent resource, useful for a lot of testing and development stuff and FUN to code with.

Go back to the Data manager and have a look at the generated data for films, characters, etc. Open a single record to see the object references. Play around with the search, e.g. search for "falcon" in the starship list or "luke" in the character list.

If you are curious go to the star wars extension's code and have a look at the preside objects code, e.g. /application/extensions/preside-ext-starwars/preside-objects/starwars_film.cfc.

Scaffold REST endpoints

One of the extensions bundled with the REST skeleton provides the ability to scaffold REST endpoints for existing Preside objects. Open the Preside developer terminal again and do:

new restendpoint
object: starwars_film
rootpath: /starwars
uri: /films/

Let the command also generate the Swagger annotations and don't use an extension. We now have 2 generated handlers, one for the collection endpoint and one for a single record endpoint. Have a look at the code. The collection supports pagination and the single record endpoint will return a 404 in case the object was not found.

Let's scaffold two additional endpoints for starships and characters.

new restendpoint
object: starwars_starship
rootpath: /starwars
uri: /starships/

new restendpoint
object: starwars_character
rootpath: /starwars
uri: /characters/

reload all

Now let's examine the generated Swagger Spec again and have a look at the Swagger UI. Try out some of the endpoints there.

It could be fine as is, but maybe we need some code improvements:
Edit the REST handler endpoint film collection at /application/handlers/rest-apis/starwars/starwars_film_collection.cfc:

We want to get rid of the internal SWAPI_ID and sort the records:

var result = dao.selectData(
      maxRows=arguments.maxRows
    , startRow=arguments.startRow
    , selectFields=[ "title", "episode_id", "opening_crawl", "director", "producer", "release_date" ]
    , orderBy="release_date"
);

Reload the application again using reload all.

Sidenote: Instead of using reload all you can also add fwreinit=true as an URL parameter.

Check the result in the Swagger UI.

Multilingual REST requests

Now we may want content in multiple languages. To try this out - let’s do content translation into wookie language.

Open /application/config/Config.cfc and enable the features multilingual and restI18n:

settings.features.restI18n.enabled = true;
...
settings.features.multilingual.enabled = true;

Then reload your application.
Within the Preside Admin you now have 2 new entries beneath System > Settings.
Go to System > Settings > Content translations:

• enter english as default language (isocode=en)
• wookie as additional language (isocode=wk)

I know wk is not a real iso language code, but for the sake of the example we use that here.

Go to System > Settings > REST multilingual support and add X-LANGUAGE as a custom header to be evaluated.

Hint: Accept-Language is the default HTTP header that can be used (but you can disable that option).

Now go back to the data manager. Have a look at the lists, e.g. the films. It now shows a translation status for each record.

Not all fields are configured to be translatable. You can see how this is achieved by having a look at the Preside object definitions, e.g. /application/extensions/preside-ext-starwars/preside-objects/starwars_film.cfc (component and property annotations).

You could manually add wookie translation - if your speak wookie. Otherwise let’s get some translation data in automatically:

Go to System > Task Manager and execute the Task to translate to wookie.

Go back to the Data manager and have a look at the awesome translations. ;-)

Now to use this within our REST requests we head over to another tool that has more options than the Swagger UI tool.
I use Paw on Mac (https://paw.cloud) but there are dozens of others, e.g. the Chrome extension Boomerang. Ideally your tool of choice supports Swagger Spec import. In Paw there is an extension that is trivial to install. The import and usage is then like the following:

1. Open Paw
2. File > Import > URL
3. Open Film Collection endpoint
4. Disable pagination url params > invalid data in there by default - maybe a weird Paw option?.
5. Execute request
6. Check JSON response
7. Add header Accept-language: wk
8. perform the request again > the translated fields should be shown in Wookie now.
9. This also works with the custom header X-LANGUAGE

No code change was required for this one but now we can return data in multiple languages.

REST Security

As we may not want to have our REST API open to the whole world, we should add at least some basic security.

edit /application/config/Config.cfc and enable rest security:

settings.features.restSecurity.enabled = true;

Then reload your application.

In the Preside admin go to System > Settings > REST API Security (which is now available)
Beside enabling/disabling the feature programmatically, you can also do that within the admin itself, e.g. to temporarily disable security.

Simplest form of security:
One single API key, define it here.
There is also the ability to customize the HTTP header to be evaluated, defaults to X-API-KEY (the X-prefix is typically used for custom http headers).

Go to Paw (or your REST client of choice), reload an endpoint, see authentication error response.
Add X-API-KEY http header, use an invalid value > see error response again.
Use valid defined master key > now you should get a valid response again.

Let’s remove the single master API key again and use multiple labelled ones instead.
In the REST API Security Sys Settings enable the options for key management and to track access.
Now head over to the data manager where you will find a new object type for REST api keys.
Define 3 keys: presidecon, skywalker, vader.

Copy/paste the generated key for vader, enter it in Paw. This should work.
Do a couple of requests.

Now have a look at the keys in the data manager. We should see the last access timestamp as well as how often a single key was used for REST requests.

Just to test this out - edit the vader key and disable access. If you then do a request in the client again using the key, you should get an authentication error.

You could either manually disable a key, or maybe come up with something more sophisticated, e.g. automatically disable/re-enable it based on defined limits.

Next-level security:
Preside has a built-in frontend website user management.
Let’s enabled this in /application/config/Config.cfc:

settings.features.websiteUsers.enabled = true;

Reload your application.
Within the Preside Admin you now have a new system setting as well as a custom sidebar entry to manage your frontend website users.
(implementing a login UI would be up to you)
For now - and the sake of example, we just do everything in the Preside admin.

Within the Preside Admin go to Website Users > Users and define a new website user.

• login id = luke
• email = an own one
• displayname = Luke Skywalker
• set a password via the website users list view: r2d2
• edit website user again
• open tab REST API Access
• enable API access and copy/paste the generated REST API key

Enter the key in Paw and perform request. This should work okay. You can then check access tracking in website user form in the Preside admin.

If we don’t want to deal with API keys at all - we could use basicauth too. This is a common practice and considered safe - as long as it is used over SSL.

Enable it in System > Settings > REST API security
Note: the option is only visible and selectable if the websiteusers feature is programmatically enabled.

• Head over to Paw again
• Remove the api key > perform request > error shows 2 options for authentication
• add basic auth: luke/r2d2 > try it > should work
• change pw to c3p0 > try it > shows basic auth error

You can disable the whole security feature for the rest of the guide again if you want to make it simpler.
This can be done in Preside admin System > settings > REST API security > enabled=false.

Bonus: Mash-Up

swapi.co, the API we used to import the Star Wars data is already a REST API.
As a good developer we usually do not re-invent the wheel (building an API based on an API) but we could EXTEND it and add some value to it.
For instance our goal for now is: we need IMDb ratings for the films.

Therefore we need another custom Preside extension.

Open your OS terminal again (running box) and do:

install preside-ext-omdb

OMDb is is a free API to use to get all kinds of film information, with a lot of data from imdb and optionally also from rotten tomatoes. If you are curious more infos here: http://www.omdbapi.com. The extension wraps that API and more.

Within the Preside admin we need the developer terminal again to enable the extension.

extension list
extension enable preside-ext-omdb
reload all

Hint: you can enable/disable the extensions also within the code. Have a look at /application/extensions/extensions.json

Now we have another new system setting in Preside admin named OMDb movie import:

• define a movie to search for
• search string: "Star Wars: Episode" (without the quotation marks)
• no release year
• all options selected

Go to System > Task manager
there is a new task for the OMDb movie import.
Sidenote: as there are now different task groups those are shown

Run the omdb movie import task.
Go to data manager, which has a new category. Have a look at the movies. Obviously some pretty strange ones have been imported as well. ;-)

But now we have to mash things up, as they are now sitting in parallel to each other (starwars films and omdb movies).

A neat thing of Preside is that this is really easy to do:

Create a new preside object starwars_film in your own app /application/preside-objects/starwars_film.cfc with the following content:

component {
    property name="omdb_movie" relationship="many-to-one";
}

Preside detects objects by naming convention. Conceptually this is not an extension but all object properties are mashed together - from the extension as well as from your app.
(so you are able to change, overwrite, add or remove properties)
Hint regarding the load order: core, extensions in defined priority order, your app comes last.

For completeness we add the corresponding i18n file as well: /i18n/preside-objects/starwars_film.properties with content:

field.omdb_movie.title=OMDb movie

Now reload your app.

Go to Data manager > star wars > films and edit Episode 4: "A new hope"

There is now a new Property (at the bottom) to link the OMDb movie.
Search for "hope" in the selector field, add the movie and save
Do the same for Episode 5 (empire strikes back) and 6 (return of the jedi).

Obviously this is very manual - you may come up with some business logic that automatically wires the films based on title and release year. But for this demo manual editing will do the job as well.

Check Paw (or your REST client of choice) and request the films, nothing changed.
We need to adapt the REST endpoint as well.
Edit /application/handlers/rest-apis/starwars/starwars_film_collection.cfc and append 2 properties to the select list. Afterwards the selectData call should look like this:

var result = dao.selectData(
    maxRows=arguments.maxRows
  , startRow=arguments.startRow
  , selectFields=[ "title", "episode_id", "opening_crawl", "director", "producer", "release_date", "omdb_movie.imdb_rating", "omdb_movie.imdb_votes" ]
  , orderBy="release_date"
);

Hint: you can check the omdb_movie preside object in the extension which properties are available.
selectfields needs to be extended.

Sidenote: The whole SQL joining is done automatically by Preside for you.

Back to paw and perform the request again. You should now get the IMDb ratings and votes for at least the classic episodes.

That’s it for the quick demo.

General hints on Preside coding

Check the docs. But even more:
Check the Preside core code - always! (include it in you editor project or IDE and check how things are implemented there)
Preside core eats it’s own dog food for a lot of stuff (objects, pages, services, etc.).
Check code of available skeletons and extensions.
Your app as well as extensions and the core all share the same folder structures and concepts.
There is a lot of power and flexibility in the system. It is way more than a CMS and it’s a fun platform to use.

Feel free to contribute and definitely consider joining the Preside Slack.

Happy Coding!