The lxd-client package is a client library for the LXD daemon written in Haskell. It provides a high-level Haskell interface to communicate with the LXD daemon, which allows you to launch and configure VM-like containers, create images, manage networks and volumes, and many other things. This blog post explains how the servant libraries are used to create a non-trivial type-safe HTTP/WebSockets client and discusses the efforts involved.
What is LXD?
LXD is a container manager, which uses LXC under the hood. It offers a user experience similar to a virtual machine hypervisor but uses Linux containers to provide the isolation. LXD exposes a REST API over a local unix socket and over HTTPS, allowing any type of client to manage containers, images and other configuration objects. LXD’s home page provides more information and excellent tutorials.
Some of the more important features of LXD are listed below, directly taken from LXD’s home page:
- Image based, with a variety of Linux distributions published daily.
- Support for cross-host container and image transfer, including live migration.
- Advanced resource control for cpu, memory, network I/O, block I/O, disk usage and kernel resources.
- Device passthrough for USB, GPU, unix character and block devices, NICs, disks and paths.
- Network management
- Storage management with support for multiple storage backends, pools and volumes.
Building a Haskell client
The LXD daemon exposes a REST-like API that allows you to fully manage all LXD resources. A standard command line utility is provided to manage LXD daemons, but the lxd-client package allows you to go beyond the command line interface. Using the package, you can leverage the power of Haskell when orchestrating LXD containers, both on a local host and on remote hosts.
This blog post discusses how the lxd-client package leverages Servant to quickly build a type-safe low-level interface for the LXD API. This low-level interface is actually wrapped by a high-level interface. A code example showing off the final product can be found below. The high-level interface won’t be discussed any further, but the Haddock documentation provides an overview on how to start using the high-level interface.
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Control.Monad.IO.Class (liftIO)
import Network.LXD.Client.Commands
main :: IO ()
main = runWithLocalHost def $ do
liftIO $ putStrLn "Creating my-container"
lxcCreate . containerCreateRequest "my-container"
. ContainerSourceRemote
$ remoteImage imagesRemote "ubuntu/xenial/amd64"
liftIO $ putStrLn "Starting my-container"
lxcStart "my-container"
liftIO $ putStrLn "Stopping my-container"
lxcStop "my-container" False
liftIO $ putStrLn "Deleting my-container"
lxcDelete "my-container"Leveraging Servant to rapidly describe a large API.
The LXD REST API is quite big, yet well structured, and rather well documented on GitHub. The API exposes quite a lot of endpoints, of which some are listed below.
/1.0
+-- /1.0/certificates
| +-- /1.0/certificates/<fingerprint>
+-- /1.0/containers
| +-- /1.0/containers/<name>
| +-- /1.0/containers/<name>/exec
| +-- /1.0/containers/<name>/files
| +-- /1.0/containers/<name>/state
| +-- ...
+ -- /1.0/events
+ -- /1.0/images
| +-- /1.0/images/<fingerprint>
| | +-- /1.0/images/<fingerprint>/export
| | +-- /1.0/images/<fingerprint>/refresh
| + /1.0/images/aliases
| + -- /1.0/images/aliases/<name>
+ ...Servant is a type-level DSL for describing both server and client APIs using Haskell types. By specifying the API at the type-level, Servant takes a way a lot of the boilerplate you’d otherwise have to write manually. It handles encoding and dispatching requests, as well as receiving and properly decoding responses, using your custom JSON-enabled types. An excellent tutorial is provided by Servant itself.
LXD API responses
Let’s start by describing the response objects returned by the LXD API. Each endpoint replies with either a synchronous or an asynchronous response object. A synchronous response immediately returns the requested information, while an asynchronous response first starts an operation in the background and returns an operation ID, which can be used to track its progress.
We’ll define a data type GenericResponse to describe both response types, which happen to share a lot of fields. It has two type parameters: op describes the operation ID of the response, while a describes the actual data returned by the request.
-- | Generic LXD API response object.
data GenericResponse op a = Response {
responseType :: ResponseType
, status :: String
, statusCode :: StatusCode
, responseOperation :: op
, errorCode :: Int
, error :: String
, metadata :: a
} deriving (Show)
instance (FromJSON op, FromJSON a) => FromJSON (GenericResponse op a) where
parseJSON = withObject "Response" $ \v -> Response
<$> v .: "type" <*> v .: "status" <*> v .: "status_code"
<*> v .: "operation" <*> v .: "error_code" <*> v .: "error"
<*> v .: "metadata"A synchronous Response is a generic response without an operation ID and user-specified return data. An AsyncResponse is a generic response with an operation ID of type OperationId. Its return data contains more information about the operation, described by the BackgroundOperation data type.
-- | LXD API synchronous response object, without resulting operation.
type Response a = GenericResponse String a
-- | LXD API asynchronous response object, with resulting operation
type AsyncResponse a = GenericResponse OperationId (BackgroundOperation a)Our first endpoint
We almost have enough types to specify the /1.0/containers endpoint. This endpoint simply returns a list of existing containers, like this:
We declare a convenience type ContainerName that extracts the container name by newtype-wrapping a string.
-- | LXD container name.
newtype ContainerName = ContainerName String deriving (Eq, Show)
instance FromJSON ContainerName where
parseJSON = withText "ContainerName" $ \text ->
let prefix = "/1.0/containers/" in
case stripPrefix prefix (unpack text) of
Nothing -> fail $ "could not parse container name: no prefix " ++ prefix
Just name -> return $ ContainerName name
instance ToJSON ContainerName where toJSON (ContainerName name) = toJSON name
instance IsString ContainerName where fromString = ContainerName
instance ToHttpApiData ContainerName where toUrlPiece (ContainerName name) = pack nameWe can now describe our first endpoint using the Servant type-level DSL. Quickly adding a Container data type allows us to also query the /1.0/containers/<name> endpoint, which provides information about the specified container.
type API = "1.0" :> "containers" :> Get '[JSON] (Response [ContainerName])
:<|> "1.0" :> "containers" :> Capture "name" ContainerName :> Get '[JSON] (Response Container)Path components are separated by the :> operator, while constant symbols like "1.0" and "containers" specify fixed path components. Capture captures a variable path component, while Get describes the structure of the response. In our case the response content type is JSON, which should be deserialized in a synchronous Response object.
Other endpoints
As we have seen in the previous section, we only need to declare suitable data types and FromJSON and ToJSON instances to describe an API endpoint. For the LXD endpoint all data types are implemented in the Network.LXD.Client.Types module, while the full API is declared in the Network.LXD.Client.API module.
Following these links, you’ll see that a lot of types and a lot of Servant endpoint specifications are needed to describe the full LXD API. This task is quite repetitive, yet it is very robust against errors and allows you to quickly and more importantly correctly describe a large REST-like API.
Querying the Servant API.
The API type we declared earlier, successfully describes the LXD daemon API. But now, we also want to query it. Luckily, the Servant project also includes the servant-client library. This library can automatically generate regular Haskell functions from our API type.
api :: Proxy API
api = Proxy
containerNames :: ClientM (Response [ContainerName])
container :: ContainerName -> ClientM (Response Container)
containerNames :<|> container = client apiWe declare two functions containerNames and container, of which the type signatures closely resemble the Servant specification of the API endpoint. Their definition is provided by the client function provided by the servant-client library. It simply takes a proxy of our API type, and automatically provides an implementation for our functions. In reality the list of functions is of course quite a bit longer.
The containerNames and container functions, return a result in the ClientM monad. You can run these functions by using runClientM.
runClientM :: ClientM a -> ClientEnv -> IO (Either ServantError a)
runClientM = ...
main = do
response <- runClientM containerNames myEnv
print responseConnecting to the LXD instance.
Servant allows you to describe the API itself, but not how to connect to the API endpoint. As you can see in the previous example, the runClientM function takes a ClientEnv object, which takes a base URL and a HTTP connection Manager from the Network.HTTP.Client module.
Convenience functions exist to create managers for standard HTTP and HTTPS clients, but LXD uses either unix sockets or self-signed HTTPS with public key client authentication. This requires us to construct a custom Manager from low-level building blocks.
The Network.LXD.Client module can be used to construct these custom Managers to connect to local and remote LXD instances, providing the correct client certificates for authentication and verifying the self-signed LXD HTTPS certificates.
The localHostClient and remoteHostClient functions return appropriate ClientEnv objects. If you require a high level of control on how Servant-enabled client connections are established, the source of the Network.LXD.Client might be of interest to you.
Conclusion
Servant is an excellent tool to quickly build a robust client for a REST-like API. It consists of three phases:
- Implementing the necessary data types that describe the information content of the API.
- Describe the endpoints and declare function signatures using those data types and the Servant type-level DSL.
- Provide functions that allow Servant to actually connect to the API endpoints using a suitable transport layer protocol.
The result is a robust, easy-to-use and type-safe client, that can be used to safely interact with the API. Of course, many APIs will not only use plain HTTP requests for interaction. For example, connections to some LXD API endpoints are upgraded to a WebSockets connection. Servant is not capable of interacting with these endpoints, requiring custom application logic, but this is out of scope for this blog post.