2.6 Embed End User Logs
Give your end users more details about the integrations they have enabled.
- 1.Users can click on the details icon of any enabled integration (Flow) to see an overview over all runs:
- 2.From there users can look at details of specific Flow Runs, to see number of successful and unsuccessful records per entity:
- 3.Users can drill down even more by looking at the error details of the unsuccessful records in order to see why there has been an error and thus potentially adjust their data in order to prevent the error in the next run:
In order to make the end user logs useful to a wide variety of use cases, general terms/wordings are used.
Term | Description | Examples |
|---|---|---|
Record | Collection of related fields treated as a unit | Row, one data entry |
Entity | The type of record | Contact, company, ticket, building, … |
Record ID | Unique identifier of a record (end user facing) | Company id, ticket number, … |
Run | Flow Run | |
E.g. if 5 companies are synced between two system during a Flow run, there would 5 records of the entity company. If one of them failed, its id (company id) would be the Record ID.
The UI itself is translated into the available languages, however the values will be displayed as specified (see below).
Depending on your end users, it might make sense to write entity names in their local language instead of using the terms used by the API.
In order to show the end user logs like this, the underlying Flows need to provide the necessary data in JSON format as the last step of the Flow in the following format (filled with examples values as shown in the screenshots above):
{
"overview": [
{
"status": "success",
"number_of_records": 3
},
{
"status": "error",
"number_of_records": 5
}
],
"details": [
{
"entity": "unit",
"status": "success",
"number_of_records": 4
},
{
"entity": "unit",
"status": "error",
"number_of_records": 4
},
{
"entity": "building",
"status": "success",
"number_of_records": 3
},
{
"entity": "building",
"status": "error",
"number_of_records": 1
}
],
"errors": [
{
"id": "1",
"error": "Unit size missing",
"entity": "unit",
"internal_id": "2268dda0-a821-498e-ac7d-fcbef0e29fa9"
},
{
"id": "2",
"error": "Unit size missing",
"entity": "unit",
"internal_id": "8b34849c-b820-4be4-8b59-4407aa80134a"
},
{
"id": "3",
"error": "Unit category is missing",
"entity": "unit",
"internal_id": "798b9817-453a-40fa-abc9-1502814b6824"
},
{
"id": "4",
"error": "Unit size missing",
"entity": "unit",
"internal_id": "6ee52240-8f1a-4052-8bf7-5910064b2978"
},
{
"id": "3",
"error": "Building address is missing",
"entity": "building",
"internal_id": "5c0cb0b4-7bbe-4694-824c-cffe6e8ff4a8"
}
]
}
The overview view (1.) will be automatically generated without having to adjust the underlying Flows.
However, the # of successful and unsuccessful records will only be shown when setting up Flows as described.
The content of the JSON should be generated during the Flow run and then put together as the final step of the Flow.
Each upsert (or create/create) step in a Flow should use manual error handling and the error path should set a dictionary such as:
{
"entity": "CORRESPONDING ENTITY",
"status": "error",
"internal_id": "{{ LOOP.id }}",
"id": "{{ LOOP.reference }}",
"error": "{{ UPSERT_STEP }}"
}
The
id should be an id that's familiar to the end user (e.g. a number shown in the UI of your system, instead of the internal id used by the API. If applicable, both can be the same).In case of success, a similar step (but with a different
status) should be set, so that the results can later be easily be summarized.This part of the Flow would then for example look like this:
For common errors, it might make sense to validate records with a filter action or the Validation Helper and then set error directly, without actually sending a request (in order to reduce the number of requests).
The same setup should be similar for other entities that are upserted.
After all steps of the Flow are finished, the results need to be summarized and transformed into the expected format. For that it's easiest to use a Spreadsheet Helper with the Query Spreadsheet Action, with the following SQL:
SELECT
internal_id,
id,
entity,
error
FROM ENTITY_LOOP
WHERE
status = 'error'
UNION ALL // If there are multiple loops for different entites
SELECT
internal_id,
id,
entity,
error
FROM ENTITY_LOOP
WHERE
status = 'error'
// Add more UNION ALLs in case there're more loops for different entites
SELECT
entity,
status,
COUNT(*) AS number_of_records
FROM ENTITY_LOOP
GROUP BY
status
UNION ALL // If there are multiple loops for different entites
SELECT
entity,
status,
COUNT(*) AS number_of_records
FROM ENTITY_LOOP
GROUP BY
status
// Add more UNION ALLs in case there're more loops for different entites
Similar to the
errors list, the Query Spreadsheet Action can be used. This time, the details output should be the reference of the Action:SELECT
status,
SUM(number_of_records) AS number_of_records
FROM DETAILS_LIST_QUERY_OUTPUT
GROUP BY
status
Finally, all the lists are combined in the final step of the Flow:
{
"overview": {{ OVERVIEW_LIST_QUERY_OUTPUT }},
"details": {{ DETAILS_LIST_QUERY_OUTPUT }},
"errors": {{ ERRORS_LIST_QUERY_OUTPUT }}
}
The end of the Flow would look similar to this:
The logic and format in the Flow can be freely defined (i.e. the examples shown above are just one possibility and might not be applicable to all Flows).
The only part that needs to be in the same structure as shown in the documentation is the final step of the Flow.
The run status is set automatically, based on the following logic:
Status | Condition |
|---|---|
Success | If there's at least 1 success record and 0 error records specified in overview
OR: If the Flow Run has status success and the Flow is not properly setup |
Error | If there's at least 1 error record and 0 success records specified in overview
OR: If the Flow Run has status error |
Finished | In cases where neither the Success nor Error conditions are fulfilled |
Scheduled |
Last modified 1mo ago