Introduction
Many entities can be subscribed to so that when triggers occur like updates or new records, call-outs can be made to other API listeners (like that of a client).
Very high level flow (can share with customers)
How to subscribe to, and receive call-outs about an Entity
In this walkthrough we’ll look at a scenario where a third-party wants to receive API calls every time that a new Employee belonging to a particular region is created in the TrackTik Cloud Portal.
1 - Identify the Entity
The entity in this scenario is Employees. In an event subscription, this is written in a domain:entity format like:
entity:employee
2 - Identify the Trigger
In this scenario, we want the trigger to be on created.
The trigger for our scenario will be defined in the event subscription using this convention:
entity:employee:created
The other possible trigger choices are updated or the name of one of the endpoint's actions.
NB: There can only be one entity targeted in the events list, but the number of triggers for that event isn't limited. So you can have employees:created and employees:updated, but not two different entities like employees:created and clients:created. Each entity requires its own subscription.
3 - Identify the Regional Context
TrackTik Cloud Portals have regions and you need to know what region the data for your entity data is referentially linked to, so that you don’t trigger call-outs about too much information.
In this scenario, we’ll assume there is a Region called “Health Care”. We’ll need to get its ID number:
{
...
"name": "Health Care",
...
"status": "ACTIVE",
"id": 3,
...
}
The Region’s ID is 3.
When specifying a Regional Context in an event subscription, there is a naming convention of:
context.region.xx
In our scenario it will therefore be context.region.3
Ignoring the Regional Context
If your situation requires that you ignore all regional contexts for data triggers, you may set the context to context.*
4 - Identify the URL the Events Subscription should call-out to (the listener)
Identify the URL to the external webhook that the Events Subscription should POST a call-out to with a JSON payload after the event trigger has occurred. In other words for our solution, if an Employee gets created, at what address should information about the Employee be sent to, using the POST method?
Let’s assume for the example that the webhook URL is:https://webhook.site/9191f8dd-xxxxxxxxxxx
5 - Determine the values of secretHeaderName and secret
For your external webhook to accept POST call-outs from authenticated sources, must define and use values for secretHeaderName and secret.
Keep the secret confidential and we encourage its use for the verification the web-hook calls. Generate an HMAC(SHA512) of the request body using the secret as the key and compare it with the value of the of the request header “TrackTik-Subscription-Secret” or the secretHeaderName set when creating the subscription.
When defining the secretHeaderName, you can put anything you like as long as there are NoSpaces. The callout will fail and DevOps will receive notifications otherwise.
PHP example:
assert(hash_hmac('sha512', $requestBody, $secret) === $requestSecretHeaderValue);
NB: When receiving call-outs from the TrackTik Events Subscription service, the header will include the HMAC value as header key: X-HMAC-Signature
6 - Build the JSON Payload to POST the Event Subscription record
We have all that we need now. There will be additional detail to provide, but it’s just metadata strings that you’re at liberty to describe as you like. Those will be indicated in the payload sample below:
{
"customId" : "customId of your choosing like ABC-123",
"name" : "A bit more descriptive text about the purpouse of this subscription",
"events" : [
"entity:employees:created"
],
"contextFilters": {
"filters": [
{
"tag": "context.region.3",
"includeChildren": true
}
]
},
"url" : "https://webhook.site/9191f8dd-xxxxxxxxx",
"secretHeaderName" : "TrackTik-subscription-secret-without-spaces",
"secret" : "B2C60113ECB70E7....",
"failureEmail" : "(optional) An email address you'd like to monitor errors at"
}
After the event subscription has been created, you can retrieve it (including the display of secret and secretHeaderName) with:
[GET] /events-subscriptions/<id number>?include=secret,secretHeaderName
"data": {
"customId": "customId of your choosing like ABC-123",
"name": "A bit more descriptive text about the purpouse of this subscription",
"events": [
"entity:employees:created"
],
"entityResponse": null,
"contextFilters": {
"filters": [
{
"tag": "context.region.2",
"includeChildren": true
}
]
},
"userDefinedFilters": null,
"url": "https://webhook.site/9191f8dd-04c7-4d15-91df-4e7e888a2a19",
"failureEmail": "",
"lastTriggeredOn": null,
"lastTriggeredTime": null,
"id": 6,
"secret": "FBBB628016EE289...",
"secretHeaderName": "TrackTik-subscription-secret-without-spaces"
}
Crafting the fields list and adding filters
Any GET call that you build to an /entity endpoint can be replicated in the events subscription. You just need to distribute the parameters across three different arrays.
Includes
Relation lists can be pulled into the entity’s response by adding an include array in the entityResponse section:
"entityResponse": {
"include": [
"employee",
"employee.employmentProfile"
]
}
Fields list
The number of keys and their values returned from the API entity can be limited with a fields array within the entityResponse.
Notice how you can use the dot member naming convention (employee.id) to target fields inside of included relation lists in the example below.
"entityResponse": {
"fields": [
"id",
"locked",
"status",
"vacant"
]
}
Additional (user defined) Filters
The API’s basic filters can be added to a subscription as well as a stand-alone object list of string literals. These are outside the entityResponse because they are part of the API call made to obtain the data first in order to form the entityResponse in the callout:
These filters are the same as URL based parameters like:
?locked=false&vacant=false&payable=true etc.
"userDefinedFilters": {
"locked": false,
"vacant": false,
"payable": true,
"employee.employmentProfile.payRateType": "HOURLY",
"employee.employmentProfile.employeeType": "EMPLOYEE"
},
Filtering Triggers by Changed Fields
For integrations that monitor changes to employee records, these can be needlessly triggered just by the act of the employee logging in because TrackTik records that changed value. This can be avoided with an additional dedicated filter for changed fields.
Integrators can filter out login activity from entity:employees:updated triggers in events subscriptions by adding this JSON object to a subscription:
"ignoreFieldsOnUpdate": [
"lastLoggedInOn"
]
It can be added at the root level, like this:
{
"customId": "EmployeeUpdate",
"name": "Employees Updated",
"events": [
"entity:employees:updated"
],
"contextFilters": {
"filters": [
{
"tag": "context.*",
"includeChildren": true
}
],
"exclusions": []
},
"userDefinedFilters": null,
==> "ignoreFieldsOnUpdate": [
"lastLoggedInOn"
],
"url": "https://etc.",
"failureEmail": "person@email.com",
"lastTriggeredOn": "2023-11-20T18:03:25+00:00",
"lastTriggeredTime": "2023-11-20T18:03:25+00:00",
"id": 10
}
This looks configurable for any number of fields, and we’re working on that, but for now we’re offering just the “lastLoggedInOn” value in the array.
Testing Events Subscriptions
To test if the ES is working, you don’t need to set up a webhook listener, but it will trigger system errors and DevOps will come and ask you about it.
You should therefore set up a listener. The site https://webhook.site/ is handy for this. Visit the site, and a URL and failure email will be generated automatically for you. Put these values into your subscription.
When defining the secretHeaderName, you can put anything you like as long as there are NoSpaces. The callout will fail and DevOps will receive notifications otherwise.
You can also look at the logs of the subscription with the endpoint path:
/events-subscriptions/{subscriptionId}/logs
If your callouts fail too many times, then the subscription will disable itself.
To re-enable the subscription for further testing, you need to:
POST /events-subscriptions/{subscriptionId}/actions/enable
And here is an example of an error notification sent via email:
Retry Mechanism
Active Events Subscriptions will attempt to make their callouts after triggers and retry upon failure following these rules (can vary a bit per client if they negotiate a different configuration with us):
|
Setting |
Value |
|---|---|
|
Max retries |
10 |
|
Time between retries |
120 seconds |
|
Timeout (no response received to callout) |
3 seconds |
|
HTTP Status Codes (that signal OK, otherwise it’s a failure that will lead to a retry) |
2xx |
Logging
The callouts made by Events Subscriptions are logged and can be accessed in two different methods. The first is a view of all logs sorted by most recent, and the other is to view the parent subscription and then include its log entries:
[GET] /events-subscription-logs?sort=-id
or by the ID of the
[GET] /events-subscriptions/7?include=logs
Log entries are routinely archived and then removed from the system for cold storage. Here are the time intervals:
E.g.
/events-subscriptions/9/logs?includeInactive=1
The logs will show you the subscription, the number of attempts made (up to 10 before disabling itself), the status, responseType, and lastResponse. These will help you determine the status and reasons for possible failure.
Statuses (in chronological order)
1 CREATED : The subscription has been been triggered
2 WORKING : The call-out attempts are in progress
3 COMPLETED : A call-out has been successful (200 OK)
Example of a full log entry (with failure indicators, how to troubleshoot)
NB: => prefixes have been added to some lines to help orient you to sections to inspect
{
"meta": {
"request": [
"events-subscription-logs",
"8"
],
"security": {
"granted": true,
"requested": "admin:events-subscription-logs:view",
"grantedBy": "admin:*",
"scope": null
},
"debug": null,
"resource": "events-subscription-logs"
},
"data": {
"id": 8,
"eventsSubscription": 1,
"attempts": 3,
"status": "COMPLETED",
==> "responseType": "FAILED",
==> "lastResponse": {
"code": "ERR_BAD_REQUEST",
"name": "AxiosError",
"stack": "AxiosError: Request failed with status code 404\n at settle (/var/task/node_modules/axios/dist/node/axios.cjs:1913:12)\n at Unzip.handleStreamEnd (/var/task/node_modules/axios/dist/node/axios.cjs:2995:11)\n at Unzip.emit (node:events:529:35)\n at Unzip.emit (node:domain:489:12)\n at endReadableNT (node:internal/streams/readable:1368:12)\n at process.processTicksAndRejections (node:internal/process/task_queues:82:21)",
"config": {
"env": [],
"url": "https://webhook.site/8867c489-1127-4a29-bdce-e5886436c4a5-wrong",
"data": "{\"context\":\"context.region.2\",\"eventName\":\"entity:employees:updated\",\"entity\":{\"jobTitle\":\"Principal Applications Manager\",\"region\":2,\"employmentProfile\":781,\"id\":1857,\"customId\":\"customId1701877306\",\"firstName\":\"Catherine\",\"lastName\":\"K\",\"name\":\"Catherine K\",\"primaryPhone\":\"000-000-0000\",\"secondaryPhone\":\"000-000-0000\",\"email\":\"Catherine.K@api2679.qa.staffr.ca\",\"status\":\"ACTIVE\",\"avatar\":\"https://api2679.qa.staffr.ca/rest/v1/avatar/employees/1857/5306e7759bc04da2d5568d2e7445dc0c\",\"address\":{\"addressLine1\":\"\",\"addressLine2\":\"\",\"city\":\"\",\"country\":\"\",\"state\":\"\",\"postalCode\":\"\",\"latitude\":null,\"longitude\":null,\"id\":1564},\"updatedBy\":{\"jobTitle\":\"Support (HQ)\",\"region\":2,\"employmentProfile\":1,\"address\":2,\"id\":1000,\"customId\":\"0001\",\"firstName\":\"TrackTik\",\"lastName\":\"Support (HQ)\",\"name\":\"TrackTik Support (HQ)\",\"primaryPhone\":\"1-555-454-5606\",\"secondaryPhone\":\"\",\"email\":\"\",\"status\":\"ACTIVE\",\"avatar\":\"https://api2679.qa.staffr.ca/rest/v1/avatar/employees/1000/9b64f53592ca357adc00ef8885bf2fe3\"},\"updatedOn\":\"2023-12-06T15:41:48+00:00\"},\"arguments\":{\"resource\":\"employees\"}}",
"method": "post",
"adapter": [
"xhr",
"http"
],
"headers": {
"Accept": "application/json, text/plain, */*",
"User-Agent": "axios/1.5.1",
"Content-Type": "application/json",
"Content-Length": "1105",
"Accept-Encoding": "gzip, compress, deflate, br",
"TrackTik-Subscription-Secret": "xxxxxxxxxx"
},
"timeout": 3000,
"transitional": {
"forcedJSONParsing": true,
"silentJSONParsing": true,
"clarifyTimeoutError": false
},
"maxBodyLength": -1,
"xsrfCookieName": "XSRF-TOKEN",
"xsrfHeaderName": "X-XSRF-TOKEN",
"maxContentLength": -1,
"transformRequest": [
null
],
"transformResponse": [
null
]
},
==> "status": 404,
==> "message": "Request failed with status code 404"
},
"createdBy": 1000,
"createdOn": "2023-12-06T15:41:48+00:00",
"updatedBy": 1000,
"updatedOn": "2023-12-06T15:45:53+00:00"
}
}
Call-Out Volume / Limits
The cloud infrastructure for TrackTik software is such that the subscription service is separate from the main resources of web portals, and dedicated to its task. Being separate, the subscription service can assume more call-out operations workload than the web portals themselves, which means there is no activity a web portal can generate (like creating/updating reports) that can exceed the events subscriptions mechanism.
There can be a limit to the quantity of operations just the same based on default configurations, like 1000 concurrent executions, but TrackTik’s cloud team monitors this for any approach and can scale it to 10 000 or higher for example should the need presents itself.