API

This document describes how to use the Folding@home Application Programming Interface to create and control Folding@home projects. This API is also known as the Adaptive Sampling API because it can be used to implement adaptive sampling on Folding@home.

Definitions

AS:

Folding@home Assignment Server

WS:

Folding@home Work Server

JSON:

JavaScript Object Notation

HTTP:

Hyper-Text Transfer Protocol

CA:

Certificate Authority

Usage

This section describes how to use the API to create and run a project on F@H. Examples will use Python 3 and the “requests” module. It will also use the OpenSSL command line tools. On Debian Linux, you can install these with:

sudo apt-get install -y python3-requests openssl

Authentication

In order to access the API you will need an RSA certificate signed by a Folding@home AS. This section describes how to acquire a certificate.

AS Authorization

The first step is to gain access to the AS. For this you will need a Google email address. You can then request access to the AS by emailing one of the F@H admins. E.g. joseph@cauldrondevelopment.com.

Create a Private Key

Next create an RSA private key with this command:

openssl genrsa -out private.pem 4096

Your private key is now in private.pem. Never share its contents.

Create the CSR

Now, create a Certificate Signing Request from key. If your email address was joe@example.com, you would create the CSR like this:

openssl req -out csr.pem -key private.pem -new -subj "/CN=joe@example.com/"

The CSR is now in csr.pem.

Acquire the Certificate

Next, login to the AS and click on the Certificate tab. Paste the contents of csr.pem in to the form and click Submit CSR. Your browser should automatically download the signed certificate.

You can check your certificate like this:

openssl x509 -text -noout -in fah-cert-chain-joe@example.com.pem

Connecting to the API

In order to connect to the API on either the AS or WS you will need a signed certificate and private key as described in the Authentication section. We will assume the certificate is in the file certificate.pem and the private key in private.pem.

The certificate and key is read automatically by the Python “requests” module. We will use the cert variable in the following examples. Create this variable like so:

cert = ('certificate.pem', 'private.pem')

You can then access the API like this:

import requests
import json

url = 'https://assign1.foldingathome.org/api/ws'
r = requests.get(url, cert = cert)

if r.status_code != 200:
  raise Exception('Request failed with %d: %s' % (r.status_code, r.text))

print(json.dumps(r.json()))

The last line prints the resulting data to the screen.

See Error Handling for more information on possible HTTP status codes.

The remaining examples will omit error checking and assume the following Python imports:

import requests
import json
import socket

The examples will also assume the AS is at assign1.foldingathome.org, the WS is at ws.foldingathome.org and the project number is 1234.

Creating Projects

A project can be created or updated on a WS with an HTTP PUT to /api/projects/:project where :project is the project ID. Here’s an example:

project_data = {
  'core_id':  0x22,
  'gens':     25000,
  'atoms':    288449,
  'credit':   56,
  'timeout':  3600,
  'deadline': 21600
}

url = 'https://ws.foldingathome.org/api/projects/1234'
requests.put(url, json = project_data, cert = cert)

When creating new projects, the contact field will automatically be filled using the email address associated with your certificate.

Note that when creating a project a timeout or deadline less than 300 is interpreted as the number of days but is always report in seconds. This is for backwards compatibility.

Project Status

A project’s status can be queried with an HTTP GET, like this:

url = 'https://ws.foldingathome.org/api/projects/1234'
r = requests.get(url, cert = cert)
print(json.dumps(r.json()))

Deleting Projects

Delete a project with the following:

url = 'https://ws.foldingathome.org/api/projects/1234'
requests.delete(url, cert = cert)

This will delete the project and all its associated data. It cannot be undone but the data can be recovered for up to 90 days with manual intervection.

Managing Files

Files can be queried, uploaded, downloaded and deleted from several file endpoints on the WS. These are:

Project Files:

/api/projects/:project/files

Job Files:

/api/projects/:project/runs/:run/clones/:clone/files

Result Files:

/api/projects/:project/runs/:run/clones/:clone/gens/:gen/files

Project files are used to start jobs. Job files represent the current state of a trajectory. Result files are the trajectory data return by Folding@home clients.

Listing Files

You can list the files at one of the file endpoints with an HTTP GET.

url = 'https://ws.foldingathome.org/api/projects/1234/files'
r = requests.get(url, cert = cert)
print(json.dumps(r.json()))

This will return a list of FileData which contain file path, size and modification times.

Uploading Files

Upload a file with an HTTP PUT by appending the file name to one of the the file endpoints.

filename = 'system.xml.bz2'
url = 'https://ws.foldingathome.org/api/projects/1234/files/' + filename

with open(filename, 'rb') as f:
  requests.put(url, data = f, cert = cert)

Downloading Files

A file can be downloaded like this.

filename = 'system.xml.bz2'
url = 'https://ws.foldingathome.org/api/projects/1234/files/' + filename

r = requests.get(url, cert = cert, stream = True)

with open(filename, 'wb') as f:
  for chunk in r.iter_content(chunk_size = 8192):
    f.write(chunk)

r.close()

Deleting Files

Delete a file with an HTTP DELETE.

filename = 'system.xml.bz2'
url = 'https://ws.foldingathome.org/api/projects/1234/files/' + filename
requests.delete(url, cert = cert)

The file will actually be moved to the WS trash bin but it can not be recovered through this API. Contact a server admin to recover lost data.

Starting Jobs

Jobs are not automatically started when a project is created. This provides a chance to uploaded the necessary files first. You can then start jobs either one at a time or start all of the jobs in a run. For example, start all the clones in run 0:

url = 'https://ws.foldingathome.org/api/projects/1234/runs/0/create'
requests.put(url, cert = cert)

Configuring the AS

In order for a project to get assignments, both the WS and project must be configured on the AS.

Configuring a WS

To set a weight of one and no constraints for a WS on the AS run the following:

ws_config = {
  'weight': 1,
  'constraints': ''
}

ip = socket.gethostbyname('ws.foldingathome.org') # Get the WS IPv4 address
url = 'https://assign1.foldingathome.org/api/ws/' + ip
requests.put(url, json = ws_config, cert = cert)

Configuring a Project

Similarly you can configure a project with a weight and constraints.

project_config = {
  'weight': 1,
  'constraints': 'ProjectKey=1234567890'
}

ip = socket.gethostbyname('ws.foldingathome.org') # Get the WS IPv4 address
url = 'https://assign1.foldingathome.org/api/ws/%s/projects/%d' % (ip, 1234)
requests.put(url, json = project_config, cert = cert)

A project key should be used when testing a new project. This ensures that it only gets assigned to clients with the same project key.

Monitoring Progress

Once the project has been created, its files uploaded and the WS and project have been configured on the AS, the project should be able to get assignments. Initially, you should use a project key and will need to run your own client using that project key to get job assignments.

You can monitor the progress of your project pragmatically by calling the Project Jobs endpoint.

import json
import time
import requests

ws = 'ws.foldingathome.org'
project = 1234

cert = ('certificate.pem', 'private.pem')
url = 'https://%s/api/projects/%d/jobs' % (ws, project)
req = {}

while True:
  r = requests.get(url, data = req, cert = cert)

  if r.status_code != 200:
    raise Exception('Request failed with %d: %s' % (r.status_code, r.text))

  data = r.json()
  req['since'] = data['ts']

  for job in data['jobs']:
    # Do something with the job
    print('Job P:%(project)d R:%(run)d C:%(clone)d G:%(gen)d %(state)s' % job)

  time.sleep(60)

Controlling Jobs

Other actions can be performed on jobs by appending a JobAction to a job endpoint. For example, you could stop clone 0 of run 0 like this:

url = 'https://ws.foldingathome.org/api/projects/1234/runs/0/clones/0/stop'
requests.put(url, cert = cert)

These actions can be performed either on a specific job or on all the jobs in a run. See JobAction for more info.

Protocol

This section describes the API protocol in more detail.

API clients communicate with the WS and AS via HTTP over TLS, also known as https. Requests are made to endpoints on the WS and AS at various paths prefixed with /api/. Data will is set to the servers either as HTTP request parameters or JSON data. The WS will respond with JSON data and HTTP status codes.

Request Methods

The API supports GET, PUT, POST and DELETE HTTP request methods.

Content-Type HTTP Header

HTTP messages utilize the Content-Type header which will normally be application/json except when uploading or downloading file data in which case should be application/octet-stream.

Content-Length HTTP Header

HTTP messages must contain a Content-Length header.

Error Handling

API errors are signaled via HTTP status codes. The following codes are used:

400 Bad Request

If any URL parameter or JSON data is formatted incorrectly or of the wrong type.

401 Unauthorized

If no client certificate was presented or the client certificate does not authorize the requested operation.

404 Not Found

If the requested entity does not exist on the server or if the entity exists but the HTTP request method is not supported.

411 Length Required

When sending files or JSON data, if the Content-Length header is missing.

413 Request Entity Too Large

When sending JSON data if the Content-Length header is greater than 5MiB.

When uploading file data if the Content-Length header is greater than 1GiB.

Security

Folding@home uses x509 certificates to authenticate it’s servers and clients. Each certificate is signed by the F@H Certificate Authority (CA) or one of the AS which hold intermediate certificates signed by the F@H CA. The certificates contain the fahKeyUsage extension which defines what the certificate authorizes the holder to do on the Folding@home network. The fahKeyUsage extension has OID 1.2.3.4.70.64.72.

Clients of the Adaptive Sampling API are issued F@H certificates with fahKeyUsage set to manager:<ip>,...,<ip> where <ip> is an IP address or IP address range in CIDR format. Each WS verifies that the presented certificate’s fahKeyUsage includes it’s IP address before granting access. The AS only allows changes to projects and WS which the client is authorized for.

Certificates contain an expiration date and Certificate Revocation Lists (CRL) are used to revoke certificates before the expiration date. CRLs are stored on the AS and automatically disseminated to the WS.

API certificates are presented by clients to the WS and AS via the TLS protocol.

Certificate Acquisition

The authorization configuration is managed on the AS by users in the “admin” group. To acquire an API certificate a user is first authorized by an admin to login to the AS using a Google email address. The admin will set which WS the user is allowed to modify. A user can modify any projects on the WS on which they are authorized.

To acquire a certificate the user must login to one of the AS and submit a Certificate Signing Request (CSR). The CSR can be generated from an RSA key pair using readily available tools such as openssl or Python library. The AS will respond with a signed certificate with the correct fahKeyUsage. The certificate can then be used to access both the AS and WS.

Certificate Renewal

A certificate will need to be renewed before it expires or if changes are made to the user’s authorization. For example, an admin might grant access to a new WS. The certificate can be renewed automatically by submitting a CSR via the API if the current certificate has not yet expired or been revoked. This can be handled transparently via the API Python library so the user will not need to log into the AS to update the API certificate manually.

Reference

Relevant API endpoints on the WS and AS are described in this section. The endpoint definitions list the HTTP method (GET, PUT or DELETE) followed by the endpoint path. Sections of the path which begin with a colon represent parameters that you should replace with a value. For example, an actual example of the endpoint /api/projects/:project is /api/projects/1234.

WS Endpoints

Projects

GET /api/projects

Get all projects on the WS.

Response:: {uint32: ProjectData, …}

Project

GET /api/projects/:project

Get a specific project.

Parameters:

project (uint32) – Project ID.

Response:

ProjectData

PUT /api/projects/:project

Create a new project or update an existing project’s configuration.

Parameters:

project (uint32) – Project ID.

Send:

ProjectData

DELETE /api/projects/:project

Delete a project.

Parameters:

project (uint32) – Project ID.

Project Files

GET /api/projects/:project/files

Get a list of files associated with a project.

Parameters:

project (uint32) – Project ID.

Response:

[FileData, …]

Project File

GET /api/projects/:project/files/:path

Download a project file.

Parameters:

project (uint32) – Project ID.
path (string) – File path relative to the project directory.

Response:

raw file contents

PUT /api/projects/:project/files/:path

Upload a project file.

Parameters:

project (uint32) – Project ID.
path (string) – File path relative to the project directory.

Send:

raw file contents

DELETE /api/projects/:project/files/:path

Delete a project file.

Parameters:

project (uint32) – Project ID.
path (string) – File path relative to the project directory.

Project Jobs

GET /api/projects/:project/jobs

Get a list of the active jobs for the project. Optionally restrict the list to only those jobs which have been updated since the last time. Only jobs which are active will be returned. Some new jobs that have not yet been created may not have records in the work server.

Parameters:

project (uint32) – Project ID.
since (time) – (optional) Return jobs which have changed after this time.

Response:

JobResults

Project Run Action

PUT /api/projects/:project/runs/:run/:action

Apply a job action to all of the clones in a run.

Parameters:

project (uint32) – Project ID.
run (uint16) – Run ID.
action (JobAction) – JobAction.

Project Clone

GET /api/projects/:project/runs/:run/clones/:clone

Get information about a job.

Parameters:

project (uint32) – Project ID.
run (uint16) – Run ID.
clone (uint16) – Clone ID.

Response:

JobData

Project Clone Action

PUT /api/projects/:project/runs/:run/clones/:clone/:action

Perform an action on a single job.

Parameters:

project (uint32) – Project ID.
run (uint16) – Run ID.
clone (uint16) – Clone ID.
action (JobAction) – JobAction.

Project Clone Files

GET /api/projects/:project/runs/:run/clones/:clone/files

Get a list of job files.

Parameters:

project (uint32) – Project ID.
run (uint16) – Run ID.
clone (uint16) – Clone ID.

Response:

[FileData, …]

Project Clone File

GET /api/projects/:project/runs/:run/clones/:clone/files/:path

Download a file.

Parameters:

project (uint32) – Project ID.
run (uint16) – Run ID.
clone (uint16) – Clone ID.
path (string) – File path relative to the job directory.

Response:

raw file contents

PUT /api/projects/:project/runs/:run/clones/:clone/files/:path

Upload a new or replace an existing file.

Parameters:

project (uint32) – Project ID.
run (uint16) – Run ID.
clone (uint16) – Clone ID.
path (string) – File path relative to the job directory.

Send:

raw file contents

DELETE /api/projects/:project/runs/:run/clones/:clone/files/:path

Delete an existing file.

Parameters:

project (uint32) – Project ID.
run (uint16) – Run ID.
clone (uint16) – Clone ID.
path (string) – File path relative to the job directory.

Project Gen Files

GET /api/projects/:project/runs/:run/clones/:clone/gens/:gen/files

Get a list of result files associated with a job generation.

Parameters:

project (uint32) – Project ID.
run (uint16) – Run ID.
clone (uint16) – Clone ID.
gen (uint16) – Job generation.

Response:

[FileData, …]

Project Gen File

GET /api/projects/:project/runs/:run/clones/:clone/gens/:gen/files/:path

Download a file.

Parameters:

project (uint32) – Project ID.
run (uint16) – Run ID.
clone (uint16) – Clone ID.
gen (uint16) – Job generation.
path (string) – File path relative to the results directory.

Response:

raw file contents

DELETE /api/projects/:project/runs/:run/clones/:clone/gens/:gen/files/:path

Delete an existing file.

Parameters:

project (uint32) – Project ID.
run (uint16) – Run ID.
clone (uint16) – Clone ID.
gen (uint16) – Job generation.
path (string) – File path relative to the results directory.

AS Endpoints

WS Config

GET /api/ws/:ws

Get the current WS configuration.

Parameters:

ws (ip) – The WS IP address.

Response:

ASWorkServerData

PUT /api/ws/:ws

Set the WS configuration. Caller must be authorized to modify this WS.

Parameters:

ws (ip) – The WS IP address.

Send:

ASWorkServerData

Project Config

GET /api/ws/:ws/projects/:project

Get the current project configuration.

Parameters:

ws (ip) – The WS IP address.
project (uint32) – Project ID.

Response:

ASProjectData

PUT /api/ws/:ws/projects/:project

Set the project configuration. Caller must be authorized to modify this WS.

Parameters:

ws (ip) – The WS IP address.
project (uint32) – Project ID.

Send:

ASProjectData

CSR Submission

POST /api/auth/csr

Submit a CSR in PEM format and receive a signed x509 certificate chain also in PEM. The chain will contain not only the certificate but the CA certificate and any intermediate certificates.

The CN or Common Name of the CSR must be the Google mail of the requestor. Access will be granted based on the authorization of this user.

To use this API you must already have a valid certificate. So this is only used for renewing certificates before their expiration. For this reason, the CN of the CSR and requesting certificate must be the same.

Send:: {‘csr’: string}
Response:: string

Data Types

Simple Types

Type	Description
uint8	Unsigned 8-bit integer
uint16	Unsigned 16-bit integer
uint32	Unsigned 32-bit integer
uint64	Unsigned 64-bit integer
float	A JSON number.
string	A JSON string.
time	A time in ISO8601 format.
email	addr-spec defined in RFC 5322.
ip	IPv4 address in <uint8>.<uint8>.<uint8>.<uint8> format.

Enumerations

JobAction

Name	Description
create	Create jobs if they don’t already exist.
fail	Mark the job as failed and stop further execution.
reset	Clear any errors and continue job execution.
stop	Stop further execution of the job.
restart	Restart job execution from the beginning.

CompressionType

Name	Description
none	No compression.
bzip2	Bzip2 compression.
zlib	Zlib compression.
gzip	GZip compression
lz4	LZ4 compression.

JobState

Name	Description
new	Job is not yet created but is ready to run.
ready	Job is ready to run.
assigned	One or more WUs are outstanding.
finished	The maximum number of gens has been completed.
failed	Consecutive failed or faulty WUs exceeded the maximum.
stopped	The job was stopped.
held	The job cannot be assigned.
processing	An operation on the job is in progress.

Compound Types

ProjectData

ProjectData contains a projects configuration and is based on the format currently used by the Folding@home WS.

Field	Type	Description
core_id	uint8	The core ID. E.g. 0xa8.
contact	email	The person responsible for the project.
runs	uint16	The number of runs.
clones	uint16	The number of clones.
gens	uint32	Maximum number of generations per job.
atoms	uint64	Approximate number of atoms in the simulations.
credit	uint64	The base credit awarded for the WU.
timeout	uint64	Seconds before the WU can be reassigned.
deadline	uint64	Seconds in which the WU can be returned for credit.
compression	CompressionType	Enable WU compression.

The format of variable references and the variables available to each argument are described in detail in the WS online and command line help.

Additional data is read-only and will be returned when querying a project.

JobData

Field	Type	Description
project	uint32	The project ID.
run	uint16	The job run.
clone	uint16	The job clone.
gen	uint16	The latest job generation.
state	JobState	The current job state.
last	time	Last time the job state changed.

JobResults

Field	Type	Description
jobs	JobData	List of jobs.
ts	time	Timestamp for these results

FileData

Field	Type	Description
path	string	File path relative to the project, job or gen directory.
size	uint64	The file size in bytes.
modified	time	The file modification time.

ASWorkServerData

Field	Type	Description
max-assign-rate	float	The maximum assigns/sec allowed for this WS.
weight	float	The WS weight.
constraints	string	WS constraints as defined in the AS online help.

ASProjectData

Field	Type	Description
ws	ip	IP Address of the WS.
weight	float	The project weight.
constraints	string	Project constraints as defined in the AS online help.