Documentation July 28, 2020


This document uses the terms may, must, should as defined in RFC2119

Why you need MetaMate, and what it can do

  • MetaMate provides an API for your personal and other interesting data
  • Build applications that work with abstract types, your application will work independent of where the data comes from
  • Integrate data sources and share them with the world as Docker images


Clients, MetaMate, and Services make up the MetaMate ecosystem.


A Client connects to a MetaMate instance, and uses the data, the instance can access, to power applications. A Client may need to authenticate itself with the MetaMate instance.

Examples for Clients may be the following:

  • Web applications
  • Mobile applications
  • Command line interfaces


A MetaMate instance facilitates the communication between Clients, and Services with the following:


  • protects resources.
  • discovers, and looks up Services.
  • authenticates to upstream data sources via Services.
  • turns Client requests into Service requests, and routes the Service requests to Services.
  • merges, filters, and sorts the data, that Service responses return.


Services give MetaMate instances access to upstream data sources. A data source may be, more or less everything, that returns data:

  • APIs
  • Websites
  • Databases
  • Files
  • Blockchains


Deploy & discover services

Integrate data sources




MetaMate may provide HTTPS, and HTTP Basic authentication for Clients.


If the setup meets the following:

  • The metamate.yaml sets
  • The metamate.yaml sets host.https.autoCert.
  • The metamate.yaml sets host.https.certPath.
  • Let’s Encrypt can reach MetaMate on


  1. Obtains a SSL certificate.
  2. Stores the certificate at host.https.certPath.
  3. Uses the certificate to provide a secure HTTPS connection.


Authentication may happen on the following levels:

  • The Client may authenticate to MetaMate with HTTP Basic authentication credentials.
  • MetaMate may authenticate to Services with ServiceAccounts.

Client authentication

If the metamate.yaml sets host.basicAuth.user, and host.basicAuth.password, MetaMate adds an internal HTTP Basic authentication middleware. Therefore the Browser, and Client SDKs must authenticate to MetaMate with HTTP Basic authentication credentials.

Service authentication

If Service.endpoints.authenticateServiceAccount is set, MetaMate must authenticate to the Service with a ServiceAccount. Service.endpoints.authenticateServiceAccount.filter specifies what data the ServiceAccount must contain.

The metamate.yaml may set multiple serviceAccounts of the MetaMate internal type ServiceAccountAssignment.

The MetaMate internal type ServiceAccountAssignment holds the following fields:

  • serviceAccount of Type ServiceAccount
  • serviceId of Type ServiceId, that corresponds to the Service, that the ServiceAccount is designated for

The metamate.yaml must set a subset of the following Fields for every ServiceAccount:

  • username
  • email
  • password
  • clientId
  • clientSecret
  • host

If Service.endpoints.authenticateServiceAccount is set, the following happens before MetaMate sends a ServiceRequest to a Service:

  1. MetaMate finds the ServiceAccount, that corresponds to the Service
  2. If ServiceAccount.token is not yet set, or if ServiceAccount.token.expiresAt is in the past, the following happens:

    1. MetaMate sends a AuthenticateServiceAccountServiceRequest, that contains the ServiceAccount, to the Service
    2. The Service either obtains an authentication token with

      • the credentials, provided by the ServiceAccount
      • a Refresh Token, provided by ServiceAccount.token.refreshToken

      from the upstream data source. The token may be a Session Cookie, a JSON Web Token (JWT) or any other kind of token.

    3. The Service creates a Token, that

      • must set Token.value to the authentication token
      • may set Token.refreshToken
      • may set Token.expiresAt
    4. The Service sends a AuthenticateServiceAccountServiceResponse, that contains the Token, back to MetaMate

    5. MetaMate copies AuthenticateServiceAccountServiceResponse.output.token to ServiceAccount.token

  3. MetaMate injects the ServiceAccount into the ServiceRequest

  4. The Service authenticates requests with ServiceAccount.token.value to the upstream data source


If the metamate.yaml sets host.basicAuth, every authenticated User has access to all Endpoints of all Services.

If the metamate.yaml doesn’t set host.basicAuth, every User has access to all Endpoints of all Services.

If the metamate.yaml sets endpoints.config.on, MetaMate exposes the metamate.yaml at /metamate.yaml, and therefore all ServiceAccounts set in the metamate.yaml. endpoints.config.on should only be set for debugging in a local environment.


Clients, MetaMate, and Services use Meta Query Language (MQL) to communicate with one another.


The encoding capabilities of MQL are derived as the lowest common denominator of the encoding capabilities of the following serialization technologies:

Therefore MQL requests, and MQL responses seamlessly map onto the following transport technologies:

Special Types






A Union is a Type that must hold exactly one value. Every Union has a kind, that indicates, what value the Union holds.


MQL identifies Entities by ServiceIds, and GlobalIds.


The Type ServiceId holds the following fields:

  • serviceName, that contains the name of the Service, that returned the Entity
  • value, a distinct string value, that identifies the Entity within the Service. This value may be one of the following:
    • a row index, in the context of a Service, that integrates a database
    • a username, in the context of a Service, that integrates a social network

A Service must set a ServiceId for every Entity it returns. The Service must only set ServiceId.value. MetaMate sets ServiceId.serviceName for every ServiceId, that is returned by a Service, to


The Type GlobalId is a Union, and must hold one the following fields:

  • email
  • name
  • username
  • me
  • url

A Service may set one more GlobalIds for Entities it returns. A Client may request multiple Entities from multiple Services by providing a GlobalId.


X = Name of Endpoint

  1. Client sends a XClientRequest to MetaMate.
  2. MetaMate sends XServiceRequests to Services.
  3. Services send XServiceResponses back to MetaMate.
  4. MetaMate sends a XClientResponse back to Client.

1. Client sends a XClientRequest to MetaMate.

When MetaMate receives a XClientRequest, MetaMate looks for Services that meet the following 4 conditions:

  • Service.endpoints.X is set.
  • If XClientRequest.serviceFilter is set, the Service must pass the ServiceFilter.
  • If XClientRequest.mode.byServiceId or XClientRequest.mode.byServiceIdRelation is set, the Service must correspond to the ServiceId.
  • If Service.endpoints.X.requestFilter is set, the XServiceRequest, that corresponds to the XClientRequest, must pass the RequestFilter.

2. MetaMate sends XServiceRequests to Services.

MetaMate creates a XServiceRequest for every Service.

  • If XClientRequest.pages of type []ServicePage is set, and one of the ServicesPages correspond to the Service, MetaMate sets to


MQL provides multiple methods to interact with MetaMate. Requests, and Responses adhere to one of the following methods:

  • Get
  • Action


X = Name of Type

The Get method provides multiple modes to get Xs. The Client can request

  • the whole collection of Xs
  • one or more specific Xs by providing an identifier
  • Xs that relate to an identifier
  • Xs by searching for a term


A GetXsServiceResponse, and therefore also the GetXsClientResponse, may only return a paginated subset of the overall available Xs.

The Page union Type

A Page must set one of the following Fields:

  • index of Type IndexPage

    • IndexPage.value must set the page index, starting from 0
  • offset of Type OffsetPage

    • As defined in
    • OffsetPage.offset must set the offset
    • OffsetPage.limit must set the amount of Xs to return
  • cursor of Type CursorPage

Get methods modes

The GetXsClientRequest.mode must set one of the following Fields:

CollectionGetModeServices return all Xs, or a paginated subset of all Xs
ByServiceIdGetModethe Service, that corresponds to ServiceId, may return one X, that corresponds to the ServiceId
ByGlobalIdGetModeServices may return each one X, that corresponds to the GlobalId
RelatedToServiceIdGetModethe Service, that corresponds to, may return Xs, which relate to via relatedToServiceId.path
RelateToGlobalIdGetModeServices may return Xs, which relate to via relateToGlobalId.path
SearchGetModeServices may return Xs, which the Services finds for search.term

Get method specific Request, and Response field

The Client may set the following GetXsClientRequest specific Fields:

  • The pages field of Type []ServicePage
  • The relations field of Type GetRelations
  • The filter field of Type XFilter

MetaMate may set the following GetXsServiceRequest specific Fields:

  • The page field of Type Page

A Service may set the following GetXsServiceResponse specific Fields:

  • The pagination field of Type ServicePagination

MetaMate may set the following GetXsClientResponse specific Fields:

  • The pagination field of Type ClientPagination
comments powered by Disqus