Configuration¶

The app is configured by cloud_registry/config.yaml, the base config, which documents its values. To override some of them, write them to a config overlay file and list it in environment variable FOCA_CONFIG_OVERLAY (several files separated by :); FOCA merges overlays into the config on startup (see the FOCA deployment guide). The Helm chart uses the same mechanism: it renders its config value (in a values file per deployment) into an overlay. Overlays cannot change the address the image listens on: server.host and server.port apply to the development server only, not to Gunicorn (see Server).

Deployment configuration¶

The registry works with any OpenID Connect provider that issues access tokens as per RFC 9068 (JWTs). The identity provider, its claims, the groups mapped to roles and the roles of users on services are deployment configuration: the base config has no defaults for them, so each deployment sets them in an overlay (or in the chart's config value), otherwise the app refuses to start:

security.auth.issuers: the trusted token issuers (the iss claim of their tokens, e.g., https://login.example.org/oidc/); tokens of other issuers are rejected (see Authentication). Usually one; for several, see Several token issuers;
security.auth.audience: the client IDs of the apps whose tokens are accepted (the aud claim, e.g., of the registry's front end), unless security.auth.verify_audience is false, which accepts tokens for any audience (not recommended);
security.access_control.claim_groups: the claim with the groups of users (e.g., groups), needed for admins and roles of groups; add it to security.auth.userinfo_claims if the identity provider releases it via user info only, not in access tokens (as name, which the registry shows to owners of services);
security.access_control.group_mapping: the group(s) whose members are admins (values of the group claim, see Access control), or security.access_control.require_admin: false to run without admins.

For example:

security:
    auth:
        issuers:
            - https://login.example.org/oidc/
        audience:
            - registry-portal
        userinfo_claims:
            - groups
            - name
    access_control:
        claim_groups: groups
        group_mapping:
            registry-admins: admin

Entitlement-style groups¶

Some identity providers release groups as entitlements, i.e., URNs, in a multi-valued claim (here entitlement) other than groups, and only via user info:

security:
    auth:
        issuers:
            - https://idp.example.org/
        audience:
            - <client ID of the front end>
        userinfo_claims:
            - entitlement
            - name
            - email
            - email_verified
    access_control:
        claim_groups: entitlement
        group_mapping:
            "urn:example:group:admins#issuer.example.org": admin

Settings for specific identity providers belong to the documentation of the deployment, not here.

Notes:

audience lists the client IDs of the apps that call the registry, usually the front end's (the aud claim of its access tokens).
The group (the URN above is illustrative) is a value of the claim. Quote such keys in YAML, as they contain : and #.
Without the claim in userinfo_claims, no user gets the admin role or roles of groups; without email and email_verified, the email addresses of users who accept invitations are not stored (see Invitations).
Clients must request the scope that releases the claim when logging in (here the one for entitlement, besides openid and the scopes for name, email and email_verified), and the identity provider must allow it for the client.
The registry accepts only access tokens with the typ header at+jwt (security.auth.verify_token_type, enabled by default). Before going live, check that the identity provider issues such tokens to the front end, and that their aud matches audience: otherwise requests fail with 401 Unauthorized.
Roles of groups are granted with the URL-encoded group in the path (see Roles of groups).

Roles of users and groups on services are granted by owners and admins via the API, not configured (see Roles on services). The policies (security.access_control.policies, see FOCA's access control) say what each role may do. They are named, so that an overlay can change, add or remove single rules by name (FOCA's named entries), e.g., remove the rule that lets any authenticated user register services, so that only admins can:

security:
    access_control:
        policies:
            register-services: null

An overlay that gives the policies as a list replaces all rules of the base config instead; FOCA refuses to start if an overlay gives named rules after one that gives a list. Don't rename the roles owner and maintainer (security.access_control.resource_roles), as the API is written for them.

The API specification (api.specs) is named registry, too, so that an overlay can change it by name, e.g., to require login for reads.

FOCA's service clients (security.auth.service_clients, security.auth.on_behalf_of_header) are not supported; the app refuses to start with them (see Access control).

Several token issuers¶

Deployments can trust several token issuers, e.g., an institutional login besides a community one. Their users are then told apart by issuer: FOCA namespaces the identifiers of users (sub) by issuer, as <namespace>|<sub> (by default, the namespace is the issuer, e.g., https://login.example.org/oidc/|abc123), so equal identifiers of different issuers are different users, also as owners and maintainers of services and as the users who added service types. GET /me returns the namespaced identifier (userId), which owners use to grant roles (see Roles on services).

Issuers can be listed with settings of their own (FOCA's IssuerConfig), which replace the global ones for their tokens: namespace, audience, claim_groups and userinfo_claims, e.g.:

security:
    auth:
        issuers:
            # Trusted so far: keeps its users' identifiers bare, so that
            # their stored roles stay valid
            - issuer: https://login.example.org/oidc/
              namespace: ""
            # Added: its users are namespaced (`https://login.other.org/|<sub>`)
            - issuer: https://login.other.org/
              audience:
                  - registry-portal-other
              # Its groups don't count (e.g., for admins), as they could
              # collide with those of the first issuer
              claim_groups: ""
        audience:
            - registry-portal

Group claim values are not namespaced: groups of different issuers with the same name are the same group, also for the admin groups (security.access_control.group_mapping) and roles of groups. Only take groups from issuers that can't issue each other's groups, e.g., by setting claim_groups: "" for the others. To add an issuer to a running deployment, see Upgrading.

Local overrides¶

Docker Compose mounts config.dev.yaml, an overlay for local development that trusts a fictitious token issuer (https://login.example.org/oidc/; set your own in a local override), sets an example group claim (groups), accepts tokens for any audience and http service URLs, and requires no admins. To override more values locally, put them in config.local.yaml and mount it in a docker-compose.override.yaml (both are ignored by Git):

services:
    cloud-registry:
        environment:
            FOCA_CONFIG_OVERLAY: /etc/cloud-registry/config.dev.yaml:/etc/cloud-registry/config.local.yaml
        volumes:
            - ./config.local.yaml:/etc/cloud-registry/config.local.yaml:ro

Public URLs¶

Public URLs, e.g., in Location headers, are built from the public URL of the app as clients see it, FOCA's server.public_url, including any path prefix of a reverse proxy (e.g., https://registry.example.org/ or https://example.org/registry/). The app refuses to start without it. The Helm chart sets it to https://<hostName>/, or to its publicUrl value; other deployments must set it (the Docker Compose setup sets http://localhost:8080 in config.dev.yaml). It replaces custom.endpoints.service of earlier versions; see Upgrading.

Service URL schemes¶

Clients send requests to the registered services, often with access tokens, so service URLs (url) must use one of the schemes listed in custom.endpoints.services.url_schemes: https only by default; http must be allowed explicitly (e.g., [http, https]). Registrations (POST /services, PUT /services/{serviceId}) of services with other schemes are rejected with 400 Bad Request (code scheme-not-allowed, see Errors), naming the allowed schemes. Only http and https can be listed, and the list must not be empty. Links (documentationUrl, organization.url) can be HTTP(S) URLs either way, and contactUrl also a mailto: URL (see Registrations). Clients read the allowed schemes from the service info (see Registration rules). The Docker Compose setup allows http too, in config.dev.yaml, e.g., for services running locally; the Helm chart keeps the default (set config.custom.endpoints.services.url_schemes to change it).

On startup, the app warns of stored services whose URLs use other schemes (e.g., http URLs registered while http was allowed): they are still served, but can only be updated with an allowed scheme.

Service info import¶

The registry fetches the service info of services when users import it, from public addresses only:

custom:
    endpoints:
        services:
            service_info:
                # Networks to fetch from besides public addresses; none by default
                private_networks:
                    - 10.1.0.0/16
                # How often each user may have service info fetched per hour
                fetches_per_hour: 30

private_networks (CIDR notation, IPv4 or IPv6) is for services in private networks, e.g., in tests or inside an organization. Whatever it permits, users can reach through the registry, and tell from its responses whether something there serves service info, so keep it narrow, and never permit the addresses of cloud metadata (169.254.169.254) or of the registry's own infrastructure.

The registry connects directly: proxy settings of the environment (e.g., HTTPS_PROXY) don't apply. Above fetches_per_hour (a positive number; each user on their own, admins too; failed fetches count, as they also limit probing), users get 429 Too Many Requests. So do requests while two others of the same worker are fetching, so that slow addresses leave threads for other requests. These responses have Retry-After, which the default security.cors.expose_headers lets browser apps on other origins read: keep it if you set your own list. As a second layer, the Helm chart can restrict the app's connections.

Service URLs must use an allowed scheme, so http services can only be imported if http is allowed (with the chart's policy, also add port 80).

Service info sync¶

Services can be synced with their service info, which is on by default, every 24 hours:

custom:
    endpoints:
        services:
            service_info:
                # Whether services can be synced
                sync: true
                # Time between syncs of a service, in hours (1 to 8760)
                sync_interval_hours: 24

The interval must be at least an hour, so that the registry doesn't fetch from the registered services more often than their service info changes, and at most 8760 hours (365 days); the app refuses to start with other values.

Clients read both from the service info (serviceInfoSync of the registration rules), e.g., to show how often services are synced.

Without sync (sync: false), nothing is fetched periodically, PUT /services/{serviceId}/sync returns 409 Conflict, no service has a label, and URLs aren't locked; what was stored stays, and applies again when sync is turned back on, except for services whose URL changed meanwhile: their sync ends. Imports keep working, as users start them; to stop all fetches, block the app's connections.

How it runs: each worker runs a background thread, without a task queue, and whether or not requests come in. Once a minute (or each interval, if shorter), it syncs the services that are due, one after the other. Workers and replicas share the work through the database, without any setup: a service is claimed for five minutes in a single write, so that only one of them syncs it. With a single worker, this costs one indexed query per minute, besides the fetches.

On restarts, nothing is lost, as the times are stored: services that became due meanwhile are synced at the first check, and those of a worker that died when their claim expires. A shorter interval applies at once, on startup. A longer one applies after the next sync of each service. If the app is down for longer than two intervals, labels are out-of-sync until the next sync.

What reasons reveal: the reasons of syncs are served, and a registrant controls the DNS of their host name. By pointing it elsewhere after turning sync on, they can learn whether a name resolves to a public address and, for permitted networks, whether something answers there: slowly, as it takes one probe per service and interval. To limit who can read them, require login for reads and restrict the rules get-service-sync and list-syncs.

Service environments¶

The GA4GH specification describes the environment of a service (environment) as a free string, and suggests prod, test, dev and staging. So that filters by environment work (e.g., without Prod or production besides prod), registrations may only use the values listed in custom.endpoints.services.environments: these four by default. Registrations (POST /services, PUT /services/{serviceId}) with other values are rejected with 400 Bad Request (code invalid-value, see Errors), naming the allowed values (case-sensitive); services without environment are accepted. An empty list allows any value. For example, in a config overlay (with the Helm chart, in config), with an environment of the registry from the list, as the app requires (see below):

custom:
    endpoints:
        services:
            environments:
                - production
                - testing
service_info:
    service:
        environment: production

Clients that let users choose an environment (e.g., UIs) must offer the same values: they read them from the service info rather than configuring them (see Registration rules). Stored services with other values are served as they are, and only checked when they are next written; the app logs a warning on startup that names them.

The registry must describe itself with an allowed value, too: the app refuses to start if the environment of its service info (service_info.service.environment) isn't in the list, with an error that names both settings. An empty list allows any value, and a service info without environment (null in an overlay) always passes. The defaults fit each other (dev is in the default list), so this only concerns deployments that change one of the two: environments without dev, for example, needs service_info.service.environment set to one of the listed values, too.

Service types¶

custom.access_control.service_types sets who may add service types:

open (the default): any authenticated user, explicitly (POST /services/types) or by registering a service with a new type;
controlled: admins only; other users can only register services with registered types, so that the registry offers a curated list of types;
moderated: any authenticated user, but the types of users other than admins, and the services that have them, are pending until an admin approves them (see Pending approvals), so that admins curate the list without being asked for each type.

Users other than admins may only have added custom.access_control.max_unused_service_types types that no service has (default: 10; null for no limit), in modes open and moderated (in mode controlled, they can't add types at all): further types are rejected (409), whether added explicitly or by registering or changing services, until services have them or admins remove them (approved types count too). Types become unused when their services are deleted or change their type (except for versions that syncs leave behind), so the limit bounds how many types each user can add without services (users can still leave more types unused by registering services with new types and deleting them, one registration per type). Set 0 to let only admins add types: users other than admins can then only register services with types that are registered already, as in mode controlled. Concurrent requests of a user can exceed the limit by their number. Clients read the mode and the limit from the service info, rather than configuring them (see Registration rules), which gives the limit as 0 in mode controlled, whatever its value.

A version of a type that a sync leaves behind stops counting for the user the service counts for, if that user added it and no other service has it, so that services that release new versions keep syncing (versions that other users added still count for them). An owner can therefore leave types behind by changing the version in the service info of their service before each sync:

in mode open, one per sync: per service and interval (sync_interval_hours) for the syncs of the app, and up to fetches_per_hour (default: 30) per hour and user for syncs that users start (PUT /services/{serviceId}/sync);
in mode moderated, new versions from syncs are pending until an admin approves them, and pending ones still count;
in mode controlled, or with the limit 0, syncs don't add types.

The limit is per user, so accounts that anyone can create (e.g., at an open identity provider) can still add types, and pending types in mode moderated, without bound. Reads don't slow down with the number of pending types, as services are marked as pending when stored, but GET /services/types and GET /approvals list them all. Rate-limit write requests, at least POST /services/types and POST /services, at the ingress or reverse proxy in front of the app (see Rate limits), and remove unwanted types as an admin (DELETE /services/types).

Only admins can remove types, in every mode. Set this option rather than removing the policy add-service-types (in security.access_control.policies), which only applies to POST /services/types: users also add types by registering services, and GET /me (mayAddServiceTypes) only reflects this option. Overlays that give the policies as a list must include that rule, and list-approvals for GET /approvals. For example, in a config overlay (with the Helm chart, in config):

custom:
    access_control:
        service_types: moderated
        max_unused_service_types: 5

Services per user¶

Users other than admins may only have registered custom.access_control.max_services_per_user of the registered services (default: 50; null for no limit; 0 to let only admins register services): further registrations (POST /services) are rejected with 409 Conflict and the code services-limit, until the user deletes services (or admins do). Updates, renames, roles and accepted invitations never fail on the limit, and services registered by admins (in admin mode) count for nobody. A service counts for the user who registered it as long as it is registered, whatever its roles later: giving a service to other users or a group doesn't free a place. The limit holds also for concurrent registrations of a user (only registrations and deletions while an instance of the app starts may make a counter differ by one each, until the next start). Clients read it from the service info (see Registration rules).

The app stores who registered each service with the service (not served), and keeps a counter per user (collection service_counts). On startup, it corrects the counters (e.g., after a crash between storing a service and counting it). For example, in a config overlay (with the Helm chart, in config):

custom:
    access_control:
        max_services_per_user: 20

The limit is per account, so users who can create accounts at will (e.g., with an identity provider open to anyone) can still register more services; rate-limit write requests at the ingress as well (see Rate limits).

Invitations¶

custom.access_control.invitations sets how long invitations to services are valid (expiry_days, default: 7), for how many days expired ones that weren't accepted are kept before they are removed (keep_expired_days, default: 30, 0 removes them when they expire), and how many that haven't expired, open or accepted, a service may have (max_per_service, default: 20; null for no limit). Invitations expire after at most 3650 days and are kept for at most 365 days: the app refuses to start with larger values, and there is no value for keeping them forever. For example, in a config overlay (with the Helm chart, in config):

custom:
    access_control:
        invitations:
            expiry_days: 2
            keep_expired_days: 7
            max_per_service: 5

Changes of keep_expired_days take effect when the app starts, without manual steps in the database: the app changes the TTL index that removes them, and logs a warning with the old and the new value (the chart restarts the app when its config changes). Lowering the value removes invitations that are past the new value soon after the start; raising it can't bring back those removed already. The value is in the database once for all instances of the app, and the last instance to start sets it, so an instance with the previous config that restarts during a rollout sets it back, until the next start; check for that warning if you rely on a higher value.

Expired invitations don't count towards max_per_service, and lists of invitations aren't paginated: a service can have about max_per_service * (1 + keep_expired_days / expiry_days) stored invitations, about 105 by default, so avoid combining a few expiry_days with many keep_expired_days.

Request size limit¶

Request bodies are limited to server.max_content_length bytes (default: 1048576, i.e., 1 MiB), by FOCA. Larger requests are rejected with 413 Content Too Large before the access token is checked and the body is parsed: requests whose Content-Length exceeds the limit are rejected without reading the body (and requests with an invalid Content-Length with 400). Bodies of unknown size (Transfer-Encoding: chunked) are read up to one byte over the limit, and rejected if they are larger. The app refuses to start with null (which would remove the limit in FOCA).

Set the same limit at the ingress or reverse proxy in front of the app, so that it rejects larger requests itself; the Helm chart sets none, as it depends on the proxy (e.g., ingress-nginx limits request bodies to 1 MiB by default, the app's default; see Ingress).

Server¶

The image serves the app with Gunicorn on port 8080, with FOCA's settings for apps in containers (foca.gunicorn_config; see Running in production in FOCA's docs): one worker process with 4 threads, and idle keep-alive connections closed after 100 seconds. Keep the keep-alive timeout longer than that of a reverse proxy in front of the app for its connections to the app (e.g., 60 seconds for ingress-nginx, 90 seconds for Traefik): otherwise, the proxy may reuse a connection just as Gunicorn closes it, and the request fails (502). To change the settings, set GUNICORN_CMD_ARGS to Gunicorn options (e.g., --workers 2 --threads 8); Docker Compose passes it on from the host, and the Helm chart sets it from its cloudRegistry.gunicorn values.

Each worker loads the whole app (about 55 MiB of memory), with its own database client, and runs the startup checks (e.g., of the service info and the admin groups); if they fail, Gunicorn exits. Gunicorn always listens on 0.0.0.0:8080 (its bind setting): server.host and server.port of the config apply only to Flask's development server (python -m cloud_registry.app), which is not for production, and changing them does not move the image's traffic.

Serve the app behind a reverse proxy that buffers requests (e.g., ingress-nginx, or Traefik with its buffering middleware): Gunicorn's threads read requests themselves, so slow clients can tie them up.

Service info¶

GET /service-info serves the service info from the config (service_info.service), without the database; it is served by FOCA. On startup, the service info is checked against the GA4GH Service schema, and the app refuses to start if it is invalid, naming the invalid field. version defaults to the version of the installed app (service_info.package); createdAt and updatedAt are served only if set, and must be RFC 3339 date-times (e.g., 2020-11-08T00:00:00Z). The defaults describe the app, not a deployment: id (org.example.cloud-registry) and organization (Example Organization, https://example.org) are placeholders with reserved example domains, and there is no contactUrl. Deployments set their own id (unique, e.g., in reverse domain name notation), name, description, organization, contactUrl and environment (the default is dev; it must be one of the allowed values, or the app refuses to start); while id or the name or URL of organization are the placeholders, the app logs a warning on startup, naming the fields to set (also with Docker Compose, whose development config keeps them). Fields and list items set to null (e.g., in a config overlay) are not served. The app adds the registration rules (registrationRules), built from the custom config on each request (service_info.runtime_fields): keep that option, and don't set registrationRules in service_info.service (not even to null), so that they are always the rules the registry enforces; the app refuses to start otherwise. To change the service info, change the config and restart the app (the Helm chart restarts it when the config value changes). If admins listed the registry among its services, the app then also brings that entry up to date. Previously, service info was set via the API; see Upgrading.