Welcome to Trombone’s documentation!¶

Hello, World!¶

@todo

Build Prerequisites¶

To build Trombone you need

• the Cabal tool – a build system for Haskell programs, and
• a recent version of GHC (The Glasgow Haskell Compiler).

Both of these come bundled with the Haskell Platform, which is available for all major operating systems. This is the recommended installation strategy, unless you have more specific requirements.

$ cabal update

$ git clone https://github.com/johanneshilden/trombone.git

$ cd trombone
$ cabal sandbox init
$ cabal install --only-dependencies
$ cabal build

dist/build/trombone/trombone

Troubleshooting¶

Report bugs and other issues to github.com/johanneshilden/trombone/issues.

Running¶

To start the service

• on port 3010 (default),
• with the configuration file my.conf,
• connecting to the database my_database,

run the following command:

$ trombone -d my_database -r my.conf

Some commonly used flags are:

For a complete list of flags and switches, see Command Line Flags, or give the command trombone --help.

$ curl localhost:3010/ping

< HTTP/1.1 200
< Transfer-Encoding: chunked
< Content-Type: application/json; charset=utf-8
< Server: Trombone/0.8
{
   "status":true,
   "message":"Pong!"
}

Console app¶

Unix signal handlers¶

Trombone responds to SIGHUP by reloading all configuration data and restarting the service, and to SIGTERM by shutting down the server after completion of all pending requests.

kill -SIGHUP `ps -a | awk '/trombone/ {print $1}'`

Configuration data storage¶

As a fallback, the server will look for a database table called trombone_config in the event that a configuration file is not specified (i.e., the -r flag is omitted). This comes in handy if you cannot rely on persistent disk storage (e.g. on ephemeral file systems), or simply prefer to keep configuration data in the database.

CREATE TABLE IF NOT EXISTS trombone_config (
    id   serial                PRIMARY KEY,
    key  character varying(40) UNIQUE        NOT NULL,
    val  text                                NOT NULL
);

Placeholders¶

Placeholders are denoted by a double pair of surrounding curly-braces (akin to e.g., Handlebars.js). Trombone templates acknowledge three types of placeholder variables:

• JSON value {{placeholders}};
• Uri segment {{:variables}}; and
• DRY-block placeholders {{..}}.

POST /customer  <>  INSERT INTO customer (name, address, phone)
                    VALUES ( {{name}}, {{address}}, {{phone}} )

{
    "name": "OCP",
    "address": "Delta City",
    "phone": "555-MEGACORP"
}

INSERT INTO customer (name, address, phone)
VALUES ('OCP', 'Delta City', '555-MEGACORP')

GET customer/:id   ->   SELECT * FROM customer WHERE id = {{:id}}

Comments¶

Comments start with a single octothorpe (#) character and may appear at the end of a route definition;

GET photo       >>  SELECT * FROM photo   # Retreive all photos!

or stretch over an entire line;

# Return some specific photo.
GET photo/:id   ->  SELECT * FROM photo WHERE id = {{:id}}

Multi-line expressions¶

SQL routes are allowed to span across multiple lines, as long as each subsequent, non-empty line is indented with, at least, one blank space; as in the example below.

GET resource  >>

      select name,
             address,
             phone,
             shoe_size
      from customer
      order by id

This, however, is not valid:

GET resource  >>

select name,
       address,
       phone,
       shoe_size
from customer
order by id

Except from this “single-space” requirement, indentation does not matter. Hence, the following is a valid route description.

GET resource  >>  select name
                       , address
                       , phone
                       , shoe_size
                  from customer
                  order by
                    id

Types of Routes¶

Parameter hints¶

With joins, and more complex queries, the server can have a difficult time figuring out the attribute names to return, from looking at the template alone. In such cases, and in situations where more control is needed, it is therefore possible (and necessary) to specify the list of property names. This list should appear immediately before the query template, enclosed in parentheses.

GET /customer  >>

    (id, name, phone)

    SELECT a.a, a.b, a.c
    FROM customer
      AS a
    JOIN something
      AS b...

A similar syntax is available for INSERT statements, which can be used if the server is unable to infer the table name and sequence necessary to obtain the last inserted id.

POST /customer  <>  (tbl_name, sequence) INSERT INTO...

Special Considerations¶

SELECT * FROM customer WHERE customer.name LIKE '%{{q}}%'

SELECT * FROM customer WHERE customer.name LIKE {{q}}

{
   "q": "%ACME%"
}

DRY-block Notation¶

A common pattern is to have multiple database queries that are similar in one way or another.

GET customer/all        >>
   select id, name, phone, address from customer order by id

GET customer/:id        ->
   select id, name, phone, address from customer where id = {{:id}}

GET customer/area/:id   >>
   select id, name, phone, address from customer where area_id = {{:id}} order by id

To avoid repetition, an alternative DRY notation can be employed in cases such as this. The following is an equivalent route definition using a DRY-block.

DRY
     select id, name, phone, address from customer {{..}}      # base template
{
     GET customer/all       >>  order by id                          ;
     GET customer/:id       ->  where id = {{:id}}                   ;
     GET customer/area/:id  >>  where area_id = {{:id}} order by id
}

A DRY-block consists of a base template and a number of stubs, each with the segment of the statement unique to its corresponding route.

<method> <uri> <symbol> <stub>

Here are some important observations.

• The {{..}}-placeholder must appear in the base query to indicate where the stub should be inserted. The preprocessor looks at each item within the block, expands it by inserting the base query with the stub replaced for {{..}}.
• A semi-colon delimiter is required to separate the stubs within the block. (It may be omitted for the last item.)
• Each block item must be indented with at least one blank space. The opening and closing brackets should appear on their own lines (without indentation):

{
    GET /..
    GET /..
}

Pipelines¶

Pipelines can be declared in two different ways; either in a separate file or as inline definitions.

GET  /my-pipeline  |>
{
    "processors": [
    ],
    "connections": [
    ]
}

node.js¶

http://nodejs.org/

GET /stuff  <js>  node/demo1.js

// node/demo1.js

var response = {
    statusCode : 200,
    body       : 'Just saying "hello".'
};

console.log(JSON.stringify(response));

POST /oracle <js>  node/demo2.js

// node/demo2.js

var fs = require('fs');

function parseStdin() {
    var data = fs.readFileSync('/dev/stdin').toString();
    if (data) {
        return JSON.parse(data);
    } else {
        return null;
    }
};

// Parse request object
var obj = parseStdin();

// Do some heavy computation
obj.string = obj.string.replace(/\%1/, '42');

// Send response
var response = {
    statusCode : 200,
    body       : obj
};

console.log(JSON.stringify(response));

$ curl http://localhost:3010/oracle -d '{"string": "The answer is %1."}'
The answer is 42.

Static Objects¶

The {..} syntax enables for static JSON response objects to be embedded directly in the route description.

GET /stuff  {..}  {"status":"Ok.","response":[1,2,3,4]}

A possible use-case for this is to deliver machine readable documentation as part of a service (self-describing APIs), where clients automatically can determine their abilities against a communication endpoint using the OPTIONS HTTP method. See, e.g., http://zacstewart.com/2012/04/14/http-options-method.html for a discussion of this approach.

At the very least, services should be responding with a 200 and the Allow header. That’s just correct web server behavior. But there’s really no excuse for JSON APIs not to be returning a documentation object.

OPTIONS /photo  {..}  {"GET":{"description":"Retreive a list of all photos."},"POST":{"description":"Create a new photo."}}

The rationale for the OPTIONS method is outlined in RFC 2616, Section 9.2.

The OPTIONS method represents a request for information about the communication options available on the request/response chain identified by the Request-URI. This method allows the client to determine the options and/or requirements associated with a resource, or the capabilities of a server, without implying a resource action or initiating a resource retrieval.

OPTIONS /photo  {..}  {"<Allow>":"GET,POST,OPTIONS","GET":{"description":"Retreive a list of all photos."},"POST":{"description":"Create a new photo."}}

< HTTP/1.1 200
< Allow: 'GET,POST,OPTIONS'
< Content-Type: application/json; charset=utf-8
{"GET":{"description":"Retreive a list of all customers."},"POST":{"description":"Create a new customer."}}

Naming¶

Trombone makes two fairly idiomatic assumptions; namely that,

• database tables and columns follow the lowercase_separated_by_underscores naming convention, and that
• JSON objects use camelCase formatting.

Conversion between these two formats is usually implicit.

Array Actions¶

curl http://localhost:3010 --verbose -d '[{}, {}]'

curl http://localhost:3010 --verbose -d '[{"summary":"","name":""}, {"summary":"","name":""}, {"summary":"","name":""}]'

var obj = [
    {
        name: 'Object #1',
        summary: '...'
    },
    {
        name: 'Object #2',
        summary: '...'
    },
    {
        name: 'Object #3',
        summary: '...'
    }
];

Trombone.request({
    host     : 'http://localhost:3010',
    client   : 'demo',
    key      : 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
    type     : 'POST',
    resource : 'util',
    data     : obj,
    nonce    : Date.now()/10 | 0,
    success  : function() { alert('Ok.'); }
});

Response Codes¶

Error Codes¶

@todo

{
    "status"       : false,
    "error"        : "NOT_FOUND",
    "responseCode" : 404,
    "message"      : "Resource not found."
}

Notes about HTTP Methods¶

@todo

Security model¶

To establish the authenticity of a request, the server must perform a message integrity check, operating on a cryptographic primitive known as a HMAC (hash-based message authentication code). A MAC is attached to each request, in the form of an API-Access header. During dispatch, a subsequent code is computed from the request object using

• a token (secure key) associated with the client application,
• an incremental nonce (see below), and
• the request method together with the path info.

The result of this operation is compared with the original MAC attached to the request, in order to verify its authenticity.

The key is a random, 40-character long, hexadecimal string.

53d5864520d65aa0364a52ddbb116ca78e0df8dc

CREATE TABLE trombone_keys (
    id serial,
    client character varying(40) NOT NULL,
    key character varying(40) NOT NULL,
    nonce bigint NOT NULL
);

ALTER TABLE ONLY trombone_keys
    ADD CONSTRAINT trombone_keys PRIMARY KEY (id);

ALTER TABLE ONLY trombone_keys
    ADD CONSTRAINT unique_trombone_keys_client UNIQUE (client);

API-Access: <client_id>:<nonce>:<hash>

<client_id>:<method>:<uri>:<nonce>:<json_body>

Reference Implementations¶

CREATE DATABASE basic_auth_demo;

\c basic_auth_demo

CREATE TABLE IF NOT EXISTS utilities (
    id        serial PRIMARY KEY,
    name      character varying(255)       NOT NULL,
    summary   character varying(255)       NOT NULL
);

INSERT INTO utilities (name, summary) VALUES
    ('ls',   'list directory contents'),
    ('htop', 'interactive process viewer'),
    ('df',   'report file system disk usage'),
    ('pwd',  'print name of current/working directory'),
    ('awk',  'pattern scanning and text processing language');

CREATE TABLE IF NOT EXISTS trombone_config (
    id        serial PRIMARY KEY,
    key       character varying(40) UNIQUE NOT NULL,
    val       text                         NOT NULL
);

INSERT INTO trombone_config (key, val) VALUES
    ('routes', E'GET /utils >> SELECT * FROM utilities\nPOST /util <> INSERT INTO utilities (name, summary) VALUES ({{name}}, {{summary}})');

Create a file basic-keyman.conf:

host     = 'localhost'
port     =  5432
dbname   = 'basic_auth_demo'
user     = 'postgres'
password = 'postgres'

(Modify the file as required.)

$ ./keyman register demo -c basic-keyman.conf

Client registered:
demo: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Start the server

$ trombone -d basic_auth_demo -C

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

// auth-example.js

$(document).ready(function() {

    var render = function(obj) {
        $('#response').html('<pre>' + JSON.stringify(obj, null, 4) + '</pre>');
    };

    var onError = function(e) {
        render(JSON.parse(e.responseText));
    };

    var defaults = {
        host     : 'http://localhost:3010',
        key      : 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
        client   : 'demo',
        type     : 'GET',
        error    : onError
    };

    $('#insert-action').click(function() {

        var name    = $('#insert-title').val(),
            summary = $('#insert-description').val();

        if (!summary || !name) {
            $('#response').html('Please fill out both fields.');
            return;
        }

        var obj = {
            summary : summary,
            name    : name
        };

        Trombone.request($.extend({}, defaults, {
            data     : obj,
            nonce    : Date.now()/10 | 0,
            type     : 'POST',
            resource : 'util',
            success  : function() {
                $('#response').html('Ok.');
            }
        }));

    });

    $('#request-action').click(function() {

        Trombone.request($.extend({}, defaults, {
            nonce    : Date.now()/10 | 0,
            resource : 'utils',
            success  : render
        }));

    });
});

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="utf-8">
        <title>Trombone data access service example: Request authentication</title>
    </head>
    <body>

        <div>
            <a id="request-action" href="javascript:">Request some data</a>
        </div>
        <div>
            <div><input id="insert-title" type="text"></div>
            <div><textarea id="insert-description"></textarea></div>
            <div><a id="insert-action" href="javascript:">Insert some data</a></div>
        </div>
        <div id="response"></div>

        <script src="http://code.jquery.com/jquery-2.1.1.min.js"></script>
        <script src="http://crypto-js.googlecode.com/svn/tags/3.1.2/build/rollups/aes.js"></script>
        <script src="http://crypto-js.googlecode.com/svn/tags/3.1.2/build/rollups/hmac-sha1.js"></script>
        <script src="js/trombone.request.min.js"></script>
        <script src="js/auth-example.js"></script>
    </body>
</html>

Defaults¶

Many of these settings have sensible default values.

Available Components¶

• RabbitMQ
• CORS (cross-origin resource sharing)
• Logging
• Static File Serving

RabbitMQ¶

RabbitMQ is a a messaging system based on the Advanced Message Queuing Protocol – an emerging standard for multi-purpose, asynchronous message delivery. The AMQP middleware integrates Trombone with RabbitMQ and makes it possible for participating applications to receive notifications when server resources are modified.

POST customer/new:{"status":true,"id":49,"message":"Ok."}

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="utf-8">
        <title>Trombone/RabbitMQ over STOMP</title>
    </head>
    <body>

        <div id="notification"></div>

        <script type="text/javascript" src="js/sockjs.min.js"></script>
        <script type="text/javascript" src="js/stomp.min.js"></script>
        <script type="text/javascript">

            // See: http://www.rabbitmq.com/web-stomp.html
            var ws = new SockJS('http://127.0.0.1:55674/stomp'),
                client = Stomp.over(ws);

            // Heartbeats won't work with SockJS.
            client.heartbeat.outgoing = 0;
            client.heartbeat.incoming = 0;

            var onConnect = function() {
                client.subscribe('/exchange/trombone/api', function(msg) {
                    var div = document.getElementById('notification');
                    div.innerHTML += msg.body + '<br>';
                });
            };

            var onError = function() {
                console.log('Error connecting to RabbitMQ server.');
            };

            client.connect('guest', 'guest', onConnect, onError, '/');

        </script>
    </body>
</html>

CORS¶

The CORS component provisions Trombone with the ability to accept cross-domain requests. It implements the handshake and response headers mandated by CORS-compliant client applications, such as modern web browsers.

Logging¶

The logging format is similar to Apache’s log file output.

Static File Serving¶

Trombone can also act as a simple file server. Files located under the public/ directory or any of its subdirectories are HTTP accessible. E.g.,

public/image.png   <~>   http://localhost:3010/image.png

Console¶

@todo

Keyman¶

The keyman utility implements a simple CRUD interface, suitable for command line administration of client keys.

Usage:
  keyman list [--config=<file>]
  keyman (register|renew) <client> [<key>] [--config=<file>]
  keyman revoke <client> [--config=<file>]
  keyman --help

Options:
  -c --config=<file>  Path to database connection file.
  -? --help           Display this help.

The configuration file contains a list of parameters (identical to those described here.) used to establish a database connection. Note that the default location for this file is ~/.config/trombone/keyman.conf.

Sample keyman.conf file:

host     = 'localhost'
port     =  5432
dbname   = 'trombone'
user     = 'postgres'
password = 'postgres'

$ ./keyman list

generic            : 14ad0ef86bf392b38bad6009113c2a5a8a1d993a
batman             : 53d5864520d65aa0364a52ddbb116ca78e0df8dc
spock              : 78a302b6d3e0e37d2e37cf932955781900c46eca

$ ./keyman register my_application

Client registered:
my_application: 53d5864520d65aa0364a52ddbb116ca78e0df8dc

$ ./keyman list | grep my_application

my_application      : 53d5864520d65aa0364a52ddbb116ca78e0df8dc

$ ./keyman revoke unwanted_client

JavaScript libraries¶

trombone.request.js
trombone.request.min.js

@todo