Providers and Connectors

Providers and connectors are used to integrate applications developed on DigitalSuite with external systems and services in order to carry out specific tasks and activities in the processes and workflows.

A provider represents an external system or organization, like Salesforce or the European Central Bank, that exposes enterprise or web services for usage in one or more applications. A connector is a resource that connects a service offered by a provider with an application, for example, to update customer information in Salesforce or obtain the daily Euro exchange rate from the European Central Bank.

While each connector is offered by a single provider, a provider can group multiple connectors that are available at the same web address. In fact, each process and composite API developed on DigitalSuite technically is a connector offered by the RunMyProcess Server provider.

For easy reuse of providers and their connectors across applications in DigitalSuite, a separate project should be created for each provider. For example, a mail provider defined in a separate project can be imported and reused by all projects that need to send emails.

DigitalSuite comes with an extensive library of pre-built connectors as well as the tools necessary to create custom connectors of any kind.

Providers and connectors and their configuration comprise the following settings and considerations, which are described in detail in the sections below:

Identification of Providers and Connectors

Providers and connectors are identified by their name, also referred to as the title.

A provider name should represent the provider's purpose and/or organization, for example, SMTP Office365 or Google Calendar. If indicated, it should also include specific and useful information, for example:

A login account: SMTP Google Apps - john@runmyprocess.com
A server address and domain, if the provider is used to connect to a local system through DigitalSuite EnterpriseConnect (DSEC): 127.0.0.1:8080 via DSEC mydomain.com
The usage rules specified for the provider: MyCRM (max frequency 500 ms)

A connector name should describe the action performed by the connector. This may be followed by the version and method of the web service invoked, for example:

Add Contact to Account (v20.0)
Get Incident Detail (v4) /getTicketDetail

Important: When a new version of a service becomes available, a new connector should be created for it instead of modifying an existing one in order to avoid dependency conflicts and problems in the applications using it.

Provider and Connector URLs

Providers and connectors are addressed by their URL.

Provider URL: This is the root endpoint of the provider's services, including the protocol (http, smtp ftp, etc.), for example: https://www.ecb.int/ or smtps://mymailprovider:8086/. The URL may be a fixed string, a variable, or a combination thereof. It must always end with a slash (/). For each execution mode of DigitalSuite, a different provider URL can be specified. This is useful, for example, to avoid access to external systems or the manipulation of productive data in Test mode.
Connector URL: In the configuration of a connector, a partial URL (path) can be specified, for example, stats/eurofxref/eurofxref-daily.xml. This is appended to the provider URL to form the complete URL of the service addressed by the connector, like https://www.ecb.int/stats/eurofxref/eurofxref-daily.xml.

Some providers, like Salesforce or Oracle Taleo, return dynamic endpoints for their services after authentication with their platform. The dynamic URL must be used throughout the session for any exchanges with the provider. In this case, a corresponding setting must be enabled in the provider configuration in DigitalSuite. The dynamic URL of the service endpoint must be passed in a corresponding variable (${url}). This variable needs to be specified as the connector URL in the connector configuration and as an output variable of a process step that calls the connector.

With some providers, all services are accessible by the same simple URL, for example, https://sdb.amazonaws.com/. In this case, the provider in DigitalSuite needs to be configured with dynamic endpoints, and the same URL must be specified as the provider and connector URL.

Mail providers using the SMTP or SMTPS protocol only work with the provider URL and do not allow to create and configure connectors. All parameters required for sending emails, such as the recipient, subject, or message, are specified at the places where the provider is used, for example, an Email/Notification activity of a process or specific button widgets of web interfaces.

Supported Protocols

The following protocols are supported for working with providers and connectors:

http:// : for communication with web servers and services based on REST or SOAP
https:// : for secure communication with web servers and services based on REST or SOAP
smtp:// : for email transmission with a mail server
smtps:// : for secure email transmission with a mail server
ftp:// : for file transfer with an FTP server.
ftps:// : for secure file transfer with an FTP server using Transport Layer Security (TLS) and SSL encryption
ftpes:// : for file transfer with an FTP server using mutually agreed security and encryption methods
sftp:// : for file transfer with a server based on the Secure Shell Protocol (SSH)
rdcp:// : for executing remote requests on a database through a JDBC driver. The provider URL addresses the server where the JDBC driver is set up. Connector URLs are not required.
coap:// : for communication with constrained nodes and networks in the Internet of Things. For details on the CoAP protocol, refer to https://tools.ietf.org/html/rfc7252.
coaps:// : for secure communication with constrained nodes and networks in the Internet of Things
mqtttcp:// : for messaging with IoT devices. The provider URL addresses the message broker. Topics are specified in place of connector URLs. For details on the MQTT protocol, refer to https://mqtt.org/mqtt-specification/.
mqttssl:// : for secure messaging with IoT devices using TLS and SSL encryption. The provider URL addresses the message broker. Topics are specified in place of connector URLs.

Examples

Here are some examples of provider and connector URLs

Provider URLs for sending emails through Gmail:

smtps://smtp.gmail.com:465/ to work with an SSL connection.
smtp://smtp.gmail.com:587/ to work without SSL. In this case, Plain with TLS should be used as the authentication scheme.
Providers and connectors addressing AWS need to be defined according to the rules provided in https://docs.aws.amazon.com/general/latest/gr/rande.html.

Example of an AWS RDS provider URL in the US-West-2 region:
https://rds.us-west-2.amazonaws.com/
The action portion is defined in the connector definition, for example:
?Action=DescribeDBInstances&MaxRecords=100
Note: This pattern is not consistent across all AWS services. Refer to the AWS REST documentation for details.

Example of an AWS provider URL for an S3 bucket in the AWS standard region:
https://bucketname.s3.amazonaws.com/
Example of a connector URL for an S3 GET request (download):
the_filename
Note: DigitalSuite does not support multipart file uploads to S3 (files bigger than 5GB).
Connector URLs for FTP providers:

The URLs generally look like this:
${ftp_directory_name}/${filename_on_ftp_server}
The first parameter including the subsequent slash (${ftp_directory_name}/) is optional, if the home directory is addressed. The file transfer type should be Binary.

Authentication at Providers

Provider systems usually require some sort of authentication for the clients which want to work with the services they offer. The authentication schemes and parameters depend on the service and the protocol used to work with it.

DigitalSuite supports various authentication schemes which can be configured for a provider. Like with protocols, a different authentication scheme can be specified for each execution mode.

For robustness reasons, external systems should not require the re-use of sessions. This may cause problems with protocols like FTPES in case of frequent connection requests. RunMyProcess does not keep track of connection-related sessions.

The following authentication schemes are available for the given protocols:

Authentication	Protocol	Description
None	All	No authentication is required.
Login/Password	http, https, ftp, ftps, ftpes, sftp, mqtttcp, mqttssl	Basic authentication with a login (user name or ID) and password. The credentials can be specified as fixed strings and/or variables.
Digest	http, https	Secured authentication with a handshake before sending a login (user name or ID) and password. The credentials can be specified as fixed stings and/or variables.
Custom	http, https	Authentication with a scheme and credentials which are specific to the provider system. The header for authorization requests is built as follows: `<scheme name> <credentials>` The scheme name must be a fixed string, not a variable.
AWS Signature V4 (Header)	http, https	AWS Signature Version 4 authentication based on a given access key ID and secret access key. The derived signature is added to the header of the HTTP request.
AWS Signature V4 (Query)	http, https	AWS Signature Version 4 authentication based on a given access key ID and secret access key. The derived signature is added to the query in the HTTP request. Note: The query-based authentication scheme is limited by the maximum length of URLs. With AWS S3, the (POST) payload will not be part of the signature. For these reasons, use of the header-based authentication scheme is recommended, unless a request is to be expressed entirely via the URL.
AWS Signature V2 (Header)	http, https	AWS Signature Version 2 authentication with a given access key ID and signature. Note: Signature Version 2 is deprecated and no longer supported by all AWS regions.
RunMyProcess Secured Connection	http, https	Authentication with the credentials of the current DigitalSuite user (`P_user`). This scheme is used for providers whose connectors address services internal to DigitalSuite, for example, the prebuilt connectors of the `RunMyProcess - Secured Connection` provider in the connector library.
Windows Azure	http, https	Windows Azure authentication with a given account name and key.
OAuth v2	http, https	OAuth 2.0 authentication with a given access token. By means of composite APIs, OAuth 2.0 can be customized as required by different applications. Refer to the Integration Guide for an example.
Java Keystore	https	Authentication with a certificate, which is obtained from a given Java keystore file and appended to each request. The keystore must have been uploaded to the versioned files of the provider's project. It may be protected by a password, which can be specified in the provider configuration.
None with TLS	smtp, smtps	No authentication is required, but the connection uses TLS.
Plain	smtp, smtps	Authentication at the mail server with a given email address and password.
Plain with TLS	smtp, smtps	Authentication at the mail server with a given email address and password as well as TLS for the connection. This is used, for example, for Gmail.
SSL identity	sftp	Authentication at the server with a user name and a public and private key. The file containing the private key must have been uploaded to the versioned files of the provider's project. It may be protected by a password, which can be specified in the provider configuration.
Identity / Key	coaps	Authentication with a pre-shared identity and key.

A provider configuration option determines whether the DigitalSuite server sends a password or similar credentials to the user interface, or if only modifications are sent from the user interface to the server.

Provider Configuration

In addition to the provider URL and the authentication scheme, the configuration of a provider comprises the following settings:

Use of DigitalSuite EnterpriseConnect (DSEC): DSEC enables secure access from the DigitalSuite platform in the cloud to resources located on-premise on dedicated systems behind firewalls. The use of DSEC requires the installation of specific components in the local environment as well as the configuration of an appropriate domain for the customer account. For details, refer to the DigitalSuite EnterpriseConnect documentation, which also provides the artifacts for installing the on-premise components.
Usage Rules: Usage rules define a minimum delay between two requests to the provider. This avoids overloading the provider, for example, when using one of its connectors in a loop. A provider can launch up to five connectors per second.
Description: A short description of the provider and its parameters.
Documentation: A link to the provider's detailed documentation.

Connector Configuration

In addition to the connector URL, the configuration of a connector specifies all the details of the input to the service at the provider, the actions invoked, and the result returned. The available settings and options depend on the protocol specified at the provider.

For all types of connectors, a description should be provided, which contains the details of the service, its exposed interface, and the input and output data and parameters.

HTTP Connectors

Connectors of providers using the http or https protocol are employed to work with web servers and services based on REST or SOAP. They take the following configuration settings:

Architecture: The protocol used to transmit messages from/to the service addressed by the connector:

REST/XML-RPC: HTTP is used for transport, XML for encoding. REST stands for Representational State Transfer, XML-RPC for XML Remote Procedure Call.
SOAP 1.1: The service conforms to version 1.1 of the SOAP specification.
SOAP 1.2: The service conforms to version 1.2 of the SOAP specification.
Method: The action to be performed by the service: GET, POST, PUT, DELETE, HEAD, or PATCH. All the methods except for PATCH support the TLS SNI extension.
Accept media type: The type of content DigitalSuite accepts from the service (Accept request header). DigitalSuite supports various types, for example, application/json, application/xml, image/*, multipart/soap+xml, text/html, or *.*.
Character set: The character set to be used by the connector. DigitalSuite can try to retrieve the character set from the result received; if this is not possible, UTF-8 is used.
Content and content type: The content and its type (Content-Type header) for POST, PUT, HEAD, and PATCH requests. DigitalSuite supports various content types and provides ways to define custom types in addition.
File: A file, which has been uploaded to DigitalSuite before, can be attached to the content of POST, PUT, and PATCH requests (multipart format). Files which are received with GET requests are stored with the non-versioned files of the connector's project.
Result format: The format of the result sent to DigitalSuite, where it can be parsed. The available formats include XML, TEXT, JSON, CSV/SCSV/TSV with/without header, EDI (electronic data exchange) , Name=Value pairs, URL encoding, HTML, and File.
Result transformation: A FreeMarker script to process the result sent by the service, for example, before loading it into the P_result variable of a process. The transformed result must comply with the given result format.
Custom headers: Any number of custom headers required by the connector and the service addressed by it, for example, SOAPAction. It does not make sense to add an authorization header, since this will be overwritten by the authentication information specified at the provider.
Cookies: Any cookies required, for example, to emulate web browser behavior or work with specific SaaS applications such as NetSuite.

FTP Connectors

Connectors of providers using the ftp, ftps, ftpes, or sftp protocol are employed to work with FTP servers. They take the following configuration settings:

Method: The action to be performed with the FTP server: GET, PUT, LIST, DELETE, RENAME, MKDIR, or RMDIR. RENAME acts as a MOVE; the new URL is specified in the same way as the connector URL, relative to the provider URL.
Result format: The format of the result sent to DigitalSuite, where it can be parsed. The available formats include, for example, XML, TEXT, JSON, and File.
Accept media type: The type of content DigitalSuite accepts with GET requests, for example, application/pdf, application/xml, or text/plain.
Character set: The character set to be used with GET requests. DigitalSuite can try to retrieve the character set from the result received; if this is not possible, UTF-8 is used.
File: The file to be transferred with a PUT request. The file must have been uploaded to DigitalSuite before. Files which are received with GET requests are stored with the non-versioned files of the connector's project.
Result transformation: A FreeMarker script to process the result obtained by a LIST request, for example, before loading it into the P_result variable of a process. The transformed result must comply with the given result format.

RDCP Connectors

Connectors of providers using the rdcp protocol are employed to execute remote requests on databases through a JDBC driver. RDCP connectors do not have a connector URL. Instead, the provider URL addresses the server where the JDBC driver is set up.

RDCP connectors take the following configuration settings:

User and password: The credentials to be used to connect to the database.
Database type: The type of database to connect to, for example, Microsoft SQL Server, MySQL, Oracle, or Informix.
Database address: The host, port, and name of the database to be addressed.
Query string: Any parameter to be sent to the JDBC driver in the JDBC URL.
JDBC class: The JDBC class to be used, depending on the type of database, for example, com.mysql.jdbc.Driver for MySQL.
JDBC URL: The URL to address the database. The URL is built automatically based on the given database type, address, and query string, for example, jdbc:mysql://{HOST}:3306/{DB_NAME}?myquerystring
SQL query: The SQL query to execute on the database, for example, SELECT * FROM 'customers' LIMIT 0 , 30
Result format: The format of the result sent to the DigitalSuite platform, where it can be parsed. This can be XML or JSON.
Custom headers: Any custom headers required to connect to the database. Several custom headers are predefined to hold the connector settings, for example, x-sqlp-url for the JDBC URL or x-sqlp-format for the result format.
Cookies: Any cookies required by the connector.

CoAP Connectors

Connectors of providers using the coap or coaps protocol are employed for communication with constrained nodes and networks in the Internet of Things. For details on the CoAP protocol, refer to https://tools.ietf.org/html/rfc7252. The CoAP User Guide describes how to integrate DigitalSuite with IoT devices using CoAP connectors and composite APIs.

CoAP connectors take the following configuration settings:

Method: The action to be performed with the node or network addressed by the connector: GET, POST, PUT, or DELETE.
Accept media type: The type of content DigitalSuite accepts from the target node or network, for example, application/json, application/xml, or text/plain.
Character set: The character set to be used by the connector. DigitalSuite can try to retrieve the character set from the result received; if this is not possible, UTF-8 is used.
Content and content type: The content and its type for POST and PUT requests. The available types include, for example, application/json, application/xml, and text/plain.
Result format: The format of the result sent to DigitalSuite, where it can be parsed. The available formats include, for example, XML, JSON, Name=Value pairs, and Octet stream.
Result transformation: A FreeMarker script to process the result received, for example, before loading it into the P_result variable of a process. The transformed result must comply with the given result format.
CoAP parameters: Optional CoAP parameters. The following parameters can be defined: message-type, proxi-URI, and timeout.

MQTT Connectors

Connectors of providers using the mqtttcp or mqttssl protocol are employed for messaging with IoT devices in a publish/subscribe fashion. The provider URL addresses the message broker. In place of a connector URL, a topic is specified. For details on the MQTT protocol, refer to https://mqtt.org/mqtt-specification/.

MQTT connectors take the following configuration settings:

Topic: The topic to which the message of the connector is addressed, for example, building/groundfloor/room2/temperature.
Method: The action to be performed. POST is used to publish a message.
Payload: The message to be published.
MQTT parameters: Additional parameters for MQTT. The following parameters are predefined: clientId, QoS (Quality of Service), and retained (flag setting).

Usage and Execution

A connector can be executed in the following ways:

By an activity in a process or composite API. Any parameters needed by the connector are automatically created as input variables of the activity. Variables of the connector result can be injected as output variables to the activity. For details, refer to Process Step Types.
From a listener configured for a web interface. For details, refer to Web Interfaces.
Directly from a project's configuration or the Resources module in DigitalSuite Studio.

A connector can only be launched if it belongs to a project. It is executed synchronously without pause or interruption. The result is the connector's direct response.

Connector Library

DigitalSuite offers an extensive library of pre-built connectors for a wide range of providers, for example, Amazon S3, Google Sheets, Microsoft 365, or Salesforce. They are available for import into any customer project, where they can be used unchanged or customized as required. A connector can be imported into a project together with its provider or be attached to one of the project's existing providers.

Customers can build any number of providers and connectors for their own account but not add them to the connector library by themselves. If specific providers or connectors seem suitable or are required for general use across accounts, customers can contact their DigitalSuite support organization with a request to add them to the connector library.