proTES

A gateway for the GA4GH Task Execution Service (TES) API that federates your TES instances, brought to you by ELIXIR On-Cloud. Clients send their tasks to proTES as to any TES instance; proTES passes each task through a configurable stack of middlewares, which pick the TES instance to run it on (or change the request), forwards it there and tracks its state. Built-in middlewares distribute tasks randomly, for load balancing, or to the TES instance closest to the task's input data, to bring compute to the data; write your own for access control, request processing or validation, or the selection of instances by data use restrictions and client preferences. Run it with Docker Compose or deploy it to Kubernetes with its Helm chart.

Read the documentation for the architecture, the configuration, the API, the middlewares, the deployment with the Helm chart in deployment/, operations, upgrades and development.

Known deployments

None listed yet.

Running proTES yourself? Open a pull request that adds your deployment (URL, operator and notes) to this section of README.md.

Quick start

With Git and Docker with the Compose plugin (version 2) installed, clone the repository and start proTES with its MongoDB database, RabbitMQ broker and Celery worker:

git clone https://github.com/elixir-cloud-aai/proTES.git
cd proTES
docker compose up --build -d

The TES API is then served at http://localhost:8080/ga4gh/tes/v1/, with a Swagger UI at http://localhost:8080/ga4gh/tes/v1/ui:

curl -X GET "http://localhost:8080/ga4gh/tes/v1/service-info" -H "accept: application/json"

Tasks are forwarded to the TES instances listed in custom.tes.service_list of the config, with the credentials of proTES for each; list yours in config.dev.yaml (see Credentials of TES instances). All endpoints except GET /service-info, the specification and the Swagger UI require access tokens of the identity provider set in config.dev.yaml: replace its placeholder issuer and groups with those of your identity provider, or try the API with the Compose setup of the smoke test, which brings a fake one (see Access control).

Contributing

Contributions are welcome! See CONTRIBUTING.md and Contributing in the documentation, and mind our code of conduct.