This server's main purpose is to provide an inter-process comunication tool with a memory efficient persistent storage for messages. These messages usually are delayed job descriptions serialized in JSON that are published by the application server and later on retrieved by application workers.
Subscription uses [long polling](http://en.wikipedia.org/wiki/Push_technology#Long_polling) technique. When application worker subscribes to a queue which is empty at the moment, connection is kept open until a client publishes a message to this queue, or subscription timeout is reached. If there is already a message in the queue it is removed from the queue and returned to the worker.
To store messages Burlesque uses [Kyoto Cabinet](http://fallabs.com/kyotocabinet/), which is a powerful DIY database. Usage of Kyoto Cabinet is thoroughly described in the [storage](#storage) section of this document.
`-storage` argument defines the way data is stored in the database. You can read more on Kyoto Cabinet database types [here](http://fallabs.com/kyotocabinet/spex.html#tutorial_dbchart).
In order to use a persistent database, use the path to the database file (or directory) as the `-storage` argument value. File extension in the database path defines the type of the database created.
When the database type is defined, you can also add [tuning parameters](http://fallabs.com/kyotocabinet/spex.html#tips) to the `-storage` argument. Tuning parameters are separated by the `#` symbol, parameters' name and value are separated by the `=` symbol.
| `opts` | Additional options: `s`, `l` and `c` (can be specified together, e.g `lc`). `s` stands for "small" and reduces the width of record address from 6 bytes to 4 bytes. As a result, the footprint for each record is reduced from 16 bytes to 12 bytes. However, it limits the maximum size of the database file to 16GB. `l` stands for "linear" and changes the data structure of the collision chain of hash table from binary tree to linear linked list. `c` enables compression of the record values. If the value is bigger than 1KB compression is effective. |
| `rcomp` | Comparator used to compare key names. `lex` for the lexical comparator, `dec` for the decimal comparator, `lexdesc` for the lexical descending comparator, or `decdesc` for the decimal descending comparator |
| `zcomp` | Compression library: `zlib` for the [ZLIB raw](http://en.wikipedia.org/wiki/Zlib#Encapsulation) compressor, `def` for the ZLIB [deflate](http://en.wikipedia.org/wiki/DEFLATE) compressor, `gz` for the ZLIB [gzip](http://en.wikipedia.org/wiki/Gzip) compressor, `lzo` for the [LZO](http://en.wikipedia.org/wiki/Lempel%E2%80%93Ziv%E2%80%93Oberhumer) compressor, `lzma` for the [LZMA](http://en.wikipedia.org/wiki/Lempel%E2%80%93Ziv%E2%80%93Markov_chain_algorithm) compressor, or `arc` for the [Arcfour](http://en.wikipedia.org/wiki/RC4) cipher |
For production usage it is strongly recommended to choose a **persistent** database. Burlesque uses Kyoto Cabinet as a persistent hash-table, which means `HashDB` would be a smart choice.
If the average message size is expected to be more than 1KB then compression should be considered as an option. To enable compression you need to pass `opts` tuning parameter to the database path with value `c` (`#opts=c`) in it, you also need to define compression algorithm using the `zcomp` parameter (e.g `#zcomp=gz`).
You can define maximum memory limit; when the limit is reached new records are swapped to disk. Memory limit is defined by value of `msiz` parameter in bytes (e.g `#msiz=524288000`)
So, to use a persistent hash database with enabled compression and 512MB memory limit the `-storage` argument value should be `storage.kch#opts=c#zcomp=gz#msiz=524288000`.
If development database doesn't need to be persisted consider using `ProtoHashDB` (which locks the whole table), `StashDB` (locks record) or `CacheDB` (locks record using a mutex). By default `ProtoHashDB` is used.
This endpoint is used to publish messages to a queue. If there is a connection waiting to recieve a message from this queue, the message will be handed directly to the awaiting worker.
Publication can be done via both `GET` and `POST` methods. Both methods use `queue` argument to pass queue name. When using `GET` method pass message body with `msg` argument. To publish a message via `POST` method pass message body via request body instead of the `msg` argument.
This endpoint is used to try and fetch a message from one of the queues given. If at least one of these queues contains a message, this message will be removed from the queue and returned as a response body. The name of the queue where this message was taken from will be provided as a `Queue` response header.
Subscription is always done via `GET` method. To fetch a message from a queue use the name of the queue as the `queues` argument value. Multiple queue names could be passed separated with the comma character.
