NMDC Runtime API (0.1.0)

Download OpenAPI specification:Download

This is a draft of the NMDC Runtime API. The resource layout currently covers aspects of workflow execution and automation, and is intended to facilitate discussion as more of the API is developed.

Authentication

OAuth2PasswordOrClientCredentialsBearer

Security Scheme Type OAuth2
password OAuth Flow
Token URL: token
Scopes:
    clientCredentials OAuth Flow
    Token URL: token
    Scopes:

      sites

      A site corresponds to a physical place that may participate in job execution.

      A site may register data objects and capabilties with NMDC. It may claim jobs to execute, and it may update job operations with execution info.

      A site must be able to service requests for any data objects it has registered.

      A site may expose a "put object" custom method for authorized users. This method facilitates an operation to upload an object to the site and have the site register that object with the runtime system.

      List Sites

      query Parameters
      filter
      string (Filter)
      max_page_size
      integer (Max Page Size)
      Default: 20
      page_token
      string (Page Token)

      Responses

      Response samples

      Content type
      application/json
      {
      • "resources": [
        ],
      • "next_page_token": "string"
      }

      Create Site

      Request Body schema: application/json
      id
      required
      string (Id)
      capability_ids
      Array of strings (Capability Ids)
      Default: []

      Responses

      Request samples

      Content type
      application/json
      {
      • "id": "string",
      • "capability_ids": [ ]
      }

      Response samples

      Content type
      application/json
      {
      • "id": "string",
      • "capability_ids": [ ],
      • "clients": [ ]
      }

      Get Site

      path Parameters
      site_id
      required
      string (Site Id)

      Responses

      Response samples

      Content type
      application/json
      {
      • "id": "string",
      • "capability_ids": [ ]
      }

      List Site Capabilities

      path Parameters
      site_id
      required
      string (Site Id)

      Responses

      Response samples

      Content type
      application/json
      [
      • {
        }
      ]

      Replace Site Capabilities

      path Parameters
      site_id
      required
      string (Site Id)
      Request Body schema: application/json
      Array ()
      string

      Responses

      Request samples

      Content type
      application/json
      [
      • "string"
      ]

      Response samples

      Content type
      application/json
      [
      • {
        }
      ]

      Put Object In Site

      path Parameters
      site_id
      required
      string (Site Id)
      Request Body schema: application/json
      aliases
      Array of strings (Aliases)
      description
      string (Description)
      mime_type
      string (Mime Type) ^\w+/[-+.\w]+$
      name
      string (Name) ^[A-Za-z0-9._\-]+$

      Responses

      Request samples

      Content type
      application/json
      {
      • "aliases": [
        ],
      • "description": "string",
      • "mime_type": "string",
      • "name": "string"
      }

      Response samples

      Content type
      application/json
      {
      • "id": "string",
      • "done": false,
      • "expire_time": "2019-08-24T14:15:22Z",
      • "result": {
        },
      • "metadata": {
        }
      }

      Get Site Object Link

      path Parameters
      site_id
      required
      string (Site Id)
      Request Body schema: application/json
      access_id
      string (Access Id) non-empty
      object (AccessURL)
      region
      string (Region)
      type
      string
      Default: "https"
      Enum: "s3" "gs" "ftp" "gsiftp" "globus" "htsget" "https" "file"

      An enumeration.

      Responses

      Request samples

      Content type
      application/json
      {
      • "access_id": "string",
      • "access_url": {},
      • "region": "string",
      • "type": "s3"
      }

      Response samples

      Content type
      application/json
      {}

      Generate Credentials For Site Client

      path Parameters
      site_id
      required
      string (Site Id)

      Responses

      Response samples

      Content type
      application/json
      {
      • "client_id": "string",
      • "client_secret": "string"
      }

      users

      Endpoints for user identification.

      Currently, accounts for use of the runtime API are created manually by system administrators.

      Login For Access Token

      Request Body schema: application/x-www-form-urlencoded
      grant_type
      string (Grant Type) ^password$|^client_credentials$
      username
      string (Username)
      password
      string (Password)
      scope
      string (Scope)
      Default: ""
      client_id
      string (Client Id)
      client_secret
      string (Client Secret)

      Responses

      Response samples

      Content type
      application/json
      {
      • "access_token": "string",
      • "token_type": "string",
      • "expires": {
        }
      }

      Read Users Me

      Responses

      Response samples

      Content type
      application/json
      {
      • "username": "string",
      • "email": "string",
      • "full_name": "string",
      • "site_admin": [ ],
      • "disabled": true
      }

      workflows

      A workflow is a template for creating jobs.

      Workflow jobs are typically created by the system via trigger associations between workflows and object types. A workflow may also require certain capabilities of sites in order for those sites to claim workflow jobs.

      List Workflows

      Responses

      Response samples

      Content type
      application/json
      [
      • {
        }
      ]

      Get Workflow

      path Parameters
      workflow_id
      required
      string (Workflow Id)

      Responses

      Response samples

      Content type
      application/json
      {
      • "name": "string",
      • "description": "string",
      • "capability_ids": [
        ],
      • "id": "string",
      • "created_at": "2019-08-24T14:15:22Z"
      }

      List Workflow Object Types

      path Parameters
      workflow_id
      required
      string (Workflow Id)

      Responses

      Response samples

      Content type
      application/json
      [
      • {
        }
      ]

      List Workflow Capabilities

      path Parameters
      workflow_id
      required
      string (Workflow Id)

      Responses

      Response samples

      Content type
      application/json
      [
      • {
        }
      ]

      capabilities

      A workflow may require an executing site to have particular capabilities.

      These capabilities go beyond the simple ability to access the data object resources registered with the runtime system. Sites register their capabilities, and sites are only able to claim workflow jobs if they are known to have the capabilities required by the workflow.

      List Capabilities

      Responses

      Response samples

      Content type
      application/json
      [
      • {
        }
      ]

      Get Capability

      path Parameters
      capability_id
      required
      string (Capability Id)

      Responses

      Response samples

      Content type
      application/json
      {
      • "name": "string",
      • "description": "string",
      • "id": "string",
      • "created_at": "2019-08-24T14:15:22Z"
      }

      object types

      An object type is an object annotation that is useful for triggering workflows.

      A data object may be annotated with one or more types, which in turn can be associated with workflows through trigger resources.

      The data-object type system may be used to trigger workflow jobs on a subset of data objects when a new version of a workflow is deployed. This could be done by minting a special object type for the occasion, annotating the subset of data objects with that type, and registering the association of object type to workflow via a trigger resource.

      List Object Types

      Responses

      Response samples

      Content type
      application/json
      [
      • {
        }
      ]

      Get Object Type

      path Parameters
      object_type_id
      required
      string (Object Type Id)

      Responses

      Response samples

      Content type
      application/json
      {
      • "name": "string",
      • "description": "string",
      • "id": "string",
      • "created_at": "2019-08-24T14:15:22Z"
      }

      List Object Type Workflows

      path Parameters
      object_type_id
      required
      string (Object Type Id)

      Responses

      Response samples

      Content type
      application/json
      [
      • {
        }
      ]

      triggers

      A trigger is an association between a workflow and a data object type.

      When a data object is annotated with a type, perhaps shortly after object registration, the NMDC Runtime will check, via trigger associations, for potential new jobs to create for any workflows.

      List Triggers

      Responses

      Response samples

      Content type
      application/json
      [
      • {
        }
      ]

      Get Trigger

      path Parameters
      trigger_id
      required
      string (Trigger Id)

      Responses

      Response samples

      Content type
      application/json
      {
      • "object_type_id": "string",
      • "workflow_id": "string",
      • "id": "string",
      • "created_at": "2019-08-24T14:15:22Z"
      }

      jobs

      A job is a resource that isolates workflow configuration from execution.

      Rather than directly creating a workflow operation by supplying a workflow ID along with configuration, NMDC creates a job that pairs a workflow with configuration. Then, a site can claim a job ID, allowing the site to execute the intended workflow without additional configuration.

      A job can have multiple executions, and a workflow's executions are precisely the executions of all jobs created for that workflow.

      A site that already has a compatible job execution result can preempt the unnecessary creation of a job by pre-claiming it. This will return like a claim, and now the site can register known data object inputs for the job without the risk of the runtime system creating a claimable job of the pre-claimed type.

      List Jobs

      query Parameters
      filter
      string (Filter)
      max_page_size
      integer (Max Page Size)
      Default: 20
      page_token
      string (Page Token)

      Responses

      Response samples

      Content type
      application/json
      {
      • "resources": [
        ],
      • "next_page_token": "string"
      }

      Get Job Info

      path Parameters
      job_id
      required
      string (Job Id)

      Responses

      Response samples

      Content type
      application/json
      {
      • "workflow": {
        },
      • "name": "string",
      • "description": "string",
      • "id": "string",
      • "created_at": "2019-08-24T14:15:22Z",
      • "config": { }
      }

      Claim Job

      path Parameters
      job_id
      required
      string (Job Id)

      Responses

      Response samples

      Content type
      application/json
      {
      • "id": "string",
      • "done": false,
      • "expire_time": "2019-08-24T14:15:22Z",
      • "result": null,
      • "metadata": null
      }

      Preclaim Job

      Responses

      Response samples

      Content type
      application/json
      {
      • "id": "string",
      • "done": false,
      • "expire_time": "2019-08-24T14:15:22Z",
      • "result": null,
      • "metadata": null
      }

      List Job Executions

      A sub-resource of a job resource, the result of a successful run of that job. An execution resource may be retrieved by any site; however, it may be created and updated only by the site that ran its job.

      Responses

      Response samples

      Content type
      application/json
      null

      Get Job Execution

      Responses

      Response samples

      Content type
      application/json
      null

      objects

      A Data Repository Service (DRS) object represents content necessary for a workflow job to execute, and/or output from a job execution.

      An object may be a blob, analogous to a file, or a bundle, analogous to a folder. Sites register objects, and sites must ensure that these objects are accessible to the NMDC data broker.

      An object may be associated with one or more object types, useful for triggering workflows.

      List Objects

      query Parameters
      filter
      string (Filter)
      max_page_size
      integer (Max Page Size)
      Default: 20
      page_token
      string (Page Token)

      Responses

      Response samples

      Content type
      application/json
      {
      • "resources": [
        ],
      • "next_page_token": "string"
      }

      Create Object

      Create a new DrsObject.

      You may create a blob or a bundle.

      A blob is like a file - it's a single blob of bytes, so there there is no contents array, only one or more access_methods.

      A bundle is like a folder - it's a gathering of other objects (blobs and/or bundles) in a contents array, and access_methods is optional because a data consumer can fetch each of the bundle contents individually.

      At least one checksum is required. The names of supported checksum types are given by the set of Python 3.8 hashlib.algorithms_guaranteed:

      blake2b | blake2s | md5 | sha1 | sha224 | sha256 | sha384 | sha3_224 | sha3_256 | sha3_384 | sha3_512 | sha512 | shake_128 | shake_256

      Each provided access_method needs either an access_url or an access_id.

      Request Body schema: application/json
      aliases
      Array of strings (Aliases)
      description
      string (Description)
      mime_type
      string (Mime Type) ^\w+/[-+.\w]+$
      name
      string (Name) ^[A-Za-z0-9._\-]+$
      Array of objects (Access Methods)
      required
      Array of objects (Checksums)
      Array of objects (Contents)
      created_time
      required
      string <date-time> (Created Time)
      size
      required
      integer (Size) >= 0
      updated_time
      string <date-time> (Updated Time)
      version
      string (Version)

      Responses

      Request samples

      Content type
      application/json
      {
      • "aliases": [
        ],
      • "description": "string",
      • "mime_type": "string",
      • "name": "string",
      • "access_methods": [
        ],
      • "checksums": [
        ],
      • "contents": [],
      • "created_time": "2019-08-24T14:15:22Z",
      • "size": 0,
      • "updated_time": "2019-08-24T14:15:22Z",
      • "version": "string"
      }

      Response samples

      Content type
      application/json
      {
      • "aliases": [
        ],
      • "description": "string",
      • "mime_type": "string",
      • "name": "string",
      • "access_methods": [
        ],
      • "checksums": [
        ],
      • "contents": [],
      • "created_time": "2019-08-24T14:15:22Z",
      • "size": 0,
      • "updated_time": "2019-08-24T14:15:22Z",
      • "version": "string",
      • "id": "string",
      • "self_uri": "http://example.com"
      }

      Get Object Info

      path Parameters
      object_id
      required
      string (Object Id) ^[A-Za-z0-9._~\-]+$

      Responses

      Response samples

      Content type
      application/json
      {
      • "aliases": [
        ],
      • "description": "string",
      • "mime_type": "string",
      • "name": "string",
      • "access_methods": [
        ],
      • "checksums": [
        ],
      • "contents": [],
      • "created_time": "2019-08-24T14:15:22Z",
      • "size": 0,
      • "updated_time": "2019-08-24T14:15:22Z",
      • "version": "string",
      • "id": "string",
      • "self_uri": "http://example.com"
      }

      Replace Object

      Responses

      Response samples

      Content type
      application/json
      {
      • "aliases": [
        ],
      • "description": "string",
      • "mime_type": "string",
      • "name": "string",
      • "access_methods": [
        ],
      • "checksums": [
        ],
      • "contents": [],
      • "created_time": "2019-08-24T14:15:22Z",
      • "size": 0,
      • "updated_time": "2019-08-24T14:15:22Z",
      • "version": "string",
      • "id": "string",
      • "self_uri": "http://example.com"
      }

      Update Object

      Responses

      Response samples

      Content type
      application/json
      {
      • "aliases": [
        ],
      • "description": "string",
      • "mime_type": "string",
      • "name": "string",
      • "access_methods": [
        ],
      • "checksums": [
        ],
      • "contents": [],
      • "created_time": "2019-08-24T14:15:22Z",
      • "size": 0,
      • "updated_time": "2019-08-24T14:15:22Z",
      • "version": "string",
      • "id": "string",
      • "self_uri": "http://example.com"
      }

      Get Object Info

      Redirect to /objects/{object_id}.

      path Parameters
      object_id
      required
      string (Object Id) ^[A-Za-z0-9._~\-]+$

      Responses

      Response samples

      Content type
      application/json
      {
      • "aliases": [
        ],
      • "description": "string",
      • "mime_type": "string",
      • "name": "string",
      • "access_methods": [
        ],
      • "checksums": [
        ],
      • "contents": [],
      • "created_time": "2019-08-24T14:15:22Z",
      • "size": 0,
      • "updated_time": "2019-08-24T14:15:22Z",
      • "version": "string",
      • "id": "string",
      • "self_uri": "http://example.com"
      }

      List Object Types

      path Parameters
      object_id
      required
      string (Object Id) ^[A-Za-z0-9._~\-]+$

      Responses

      Response samples

      Content type
      application/json
      [
      • {
        }
      ]

      Replace Object Types

      path Parameters
      object_id
      required
      string (Object Id)
      Request Body schema: application/json
      Array ()
      string

      Responses

      Request samples

      Content type
      application/json
      [
      • "string"
      ]

      Response samples

      Content type
      application/json
      {
      • "aliases": [
        ],
      • "description": "string",
      • "mime_type": "string",
      • "name": "string",
      • "access_methods": [
        ],
      • "checksums": [
        ],
      • "contents": [],
      • "created_time": "2019-08-24T14:15:22Z",
      • "size": 0,
      • "updated_time": "2019-08-24T14:15:22Z",
      • "version": "string",
      • "id": "string",
      • "self_uri": "http://example.com",
      • "types": [
        ]
      }

      Get Object Access

      path Parameters
      object_id
      required
      string (Object Id) ^[A-Za-z0-9._~\-]+$
      access_id
      required
      string (Access Id)

      Responses

      Response samples

      Content type
      application/json
      {}

      operations

      An operation is a resource for tracking the execution of a job.

      When a job is claimed by a site for execution, an operation resource is created.

      An operation is akin to a "promise" or "future" in that it should eventually resolve to either a successful result, i.e. an execution resource, or to an error.

      An operation is parameterized to return a result type, and a metadata type for storing progress information, that are both particular to the job type.

      Operations may be paused, resumed, and/or cancelled.

      Operations may expire, i.e. not be stored indefinitely. In this case, it is recommended that execution resources have longer lifetimes / not expire, so that information about successful results of operations are available.

      List Operations

      query Parameters
      filter
      string (Filter)
      max_page_size
      integer (Max Page Size)
      Default: 20
      page_token
      string (Page Token)

      Responses

      Response samples

      Content type
      application/json
      {
      • "resources": [
        ],
      • "next_page_token": "string"
      }

      Get Operation

      path Parameters
      op_id
      required
      string (Op Id)

      Responses

      Response samples

      Content type
      application/json
      {
      • "id": "string",
      • "done": false,
      • "expire_time": "2019-08-24T14:15:22Z",
      • "result": null,
      • "metadata": null
      }

      Update Operation

      path Parameters
      op_id
      required
      string (Op Id)
      Request Body schema: application/json
      done
      boolean (Done)
      Default: false
      any or OperationError (object) (Result)
      metadata
      any (Metadata)

      Responses

      Request samples

      Content type
      application/json
      {
      • "done": false,
      • "result": null,
      • "metadata": null
      }

      Response samples

      Content type
      application/json
      {
      • "id": "string",
      • "done": false,
      • "expire_time": "2019-08-24T14:15:22Z",
      • "result": null,
      • "metadata": null
      }

      Wait Operation

      Wait until the operation is resolved or rejected before returning the result. This is a 'blocking' alternative to client-side polling, and may not be available for operation types know to be particularly long-running.

      Responses

      Response samples

      Content type
      application/json
      null

      Cancel Operation

      Responses

      Response samples

      Content type
      application/json
      null

      Pause Operation

      Responses

      Response samples

      Content type
      application/json
      null

      Resume Operation

      Responses

      Response samples

      Content type
      application/json
      null

      queries

      A query is an operation (find, update, etc.) against the metadata store.

      Metadata -- for studies, biosamples, omics processing, etc. -- is used by sites to execute jobs, as the parameterization of job executions may depend not only on the content of data objects, but also on objects' associated metadata.

      Also, the function of many workflows is to extract or produce new metadata. Such metadata products should be registered as data objects, and they may also be supplied by sites to the runtime system as an update query (if the latter is not done, the runtime system will sense the new metadata and issue an update query).

      Run New Query

      Request Body schema: application/json
      name
      string (Name)
      description
      string (Description)
      save
      boolean (Save)
      Default: false
      query
      any (Query)

      Responses

      Request samples

      Content type
      application/json
      {
      • "name": "string",
      • "description": "string",
      • "save": false,
      • "query": null
      }

      Response samples

      Content type
      application/json
      {
      • "name": "string",
      • "description": "string",
      • "save": false,
      • "query": null,
      • "id": "string",
      • "created_at": "2019-08-24T14:15:22Z",
      • "last_ran": "2019-08-24T14:15:22Z",
      • "result": null,
      • "error": null
      }

      Get Query

      path Parameters
      query_id
      required
      string (Query Id)

      Responses

      Response samples

      Content type
      application/json
      {
      • "name": "string",
      • "description": "string",
      • "save": false,
      • "query": null,
      • "id": "string",
      • "created_at": "2019-08-24T14:15:22Z",
      • "last_ran": "2019-08-24T14:15:22Z",
      • "result": null,
      • "error": null
      }

      List Queries

      Responses

      Response samples

      Content type
      application/json
      [
      • {
        }
      ]

      Save Query

      path Parameters
      query_id
      required
      string (Query Id)

      Responses

      Response samples

      Content type
      application/json
      {
      • "name": "string",
      • "description": "string",
      • "save": false,
      • "query": null,
      • "id": "string",
      • "created_at": "2019-08-24T14:15:22Z",
      • "last_ran": "2019-08-24T14:15:22Z",
      • "result": null,
      • "error": null
      }

      Run Query

      path Parameters
      query_id
      required
      string (Query Id)

      Responses

      Response samples

      Content type
      application/json
      {
      • "name": "string",
      • "description": "string",
      • "save": false,
      • "query": null,
      • "id": "string",
      • "created_at": "2019-08-24T14:15:22Z",
      • "last_ran": "2019-08-24T14:15:22Z",
      • "result": null,
      • "error": null
      }

      identifiers

      Tools for identifier generation and resolution.

      Mint Ids

      Generate one or more identifiers.

      Leaving populator blank will set it to the site ID of the request client.

      Request Body schema: application/json
      populator
      string (Populator)
      Default: ""
      naa
      string (Naa)
      Default: "nmdc"
      Value: "nmdc"
      shoulder
      string (Shoulder) >= 2 characters ^(?P<shoulder>(fk0|fk1|fk2|fk3|fk4|fk5|fk6|fk...
      Default: "fk0"
      number
      integer (Number)
      Default: 1

      Responses

      Request samples

      Content type
      application/json
      {
      • "populator": "",
      • "naa": "nmdc",
      • "shoulder": "fk0",
      • "number": 1
      }

      Response samples

      Content type
      application/json
      [
      • "string"
      ]

      Set Id Bindings

      Request Body schema: application/json
      Array ()
      i
      required
      string (I) (?P<naa>(nmdc)):(?P<shoulder>(fk0|fk1|fk2|fk3...
      o
      string
      Default: "set"
      Enum: "set" "addToSet" "rm" "purge"

      An enumeration.

      a
      string (A)
      v
      any (V)

      Responses

      Request samples

      Content type
      application/json
      [
      • {
        }
      ]

      Response samples

      Content type
      application/json
      [
      • { }
      ]

      Get Id Bindings

      path Parameters
      rest
      required
      string (Rest)

      Responses

      Response samples

      Content type
      application/json
      { }