1.6 Setting up End User logs

Give your end users more details about the integrations they have enabled.

Flow setup

To show the logs for an end user, the underlying Flows need to provide the necessary data in JSON format as the last step of the Flow in the following format (filled with example values, see the example of the resulting logs):

{
  "overview": [
    {
      "status": "success",
      "number_of_records": 3
    },
    {
      "status": "error",
      "number_of_records": 5
    }
  ],
  "details": [
    {
      "entity": "unit",
      "status": "success",
      "number_of_records": 4
    },
    {
      "entity": "unit",
      "status": "error",
      "number_of_records": 4
    },
    {
      "entity": "building",
      "status": "success",
      "number_of_records": 3
    },
    {
      "entity": "building",
      "status": "error",
      "number_of_records": 1
    }
  ],
  "errors": [
    {
      "id": "1",
      "error": "Unit size missing",
      "entity": "unit",
      "internal_id": "2268dda0-a821-498e-ac7d-fcbef0e29fa9"
    },
    {
      "id": "2",
      "error": "Unit size missing",
      "entity": "unit",
      "internal_id": "8b34849c-b820-4be4-8b59-4407aa80134a"
    },
    {
      "id": "3",
      "error": "Unit category is missing",
      "entity": "unit",
      "internal_id": "798b9817-453a-40fa-abc9-1502814b6824"
    },
    {
      "id": "4",
      "error": "Unit size missing",
      "entity": "unit",
      "internal_id": "6ee52240-8f1a-4052-8bf7-5910064b2978"
    },
    {
      "id": "3",
      "error": "Building address is missing",
      "entity": "building",
      "internal_id": "5c0cb0b4-7bbe-4694-824c-cffe6e8ff4a"
    }
  ]
}

The overview view (1.) will be automatically generated without having to adjust the underlying Flows.

However, the # of successful and unsuccessful records will only be shown when setting up Flows as described.

The values for an end user will be displayed as specified (see above).

Depending on your end users, it might make sense to write entity names in their local language instead of using the terms used by the API.

1. Error handling and status logging

Each upsert (or create/create) step in a Flow should use manual error handling and the error path should set a dictionary such as:

{
  "entity": "CORRESPONDING ENTITY",
  "status": "error",
  "internal_id": "{{ LOOP.id }}",
  "id": "{{ LOOP.reference }}",
  "error": "{{ UPSERT_STEP }}"
}

The id should be an ID that's familiar to the end user (e.g. a number shown in the UI of your system, instead of the internal ID used by the API. If applicable, both can be the same).

In case of success, a similar step (but with a different status) should be set, so that the results can later be easily summarized.

This part of the Flow would then for example look like this:

For common errors, it might make sense to validate records with a filter action

The same setup should be similar for other entities that are upserted.

2. Setting the errors list

After all steps of the Flow are finished, the results need to be summarized and transformed into the expected format. For that, it's easiest to use a Spreadsheet Helper with the Query Spreadsheet Action, with the following SQL:

SELECT
  internal_id,
  id,
  entity,
  error
FROM ENTITY_LOOP
WHERE
  status = 'error'
UNION ALL // If there are multiple loops for different 
SELECT
  internal_id,
  id,
  entity,
  error
FROM ENTITY_LOOP
WHERE
  status = 'error'
// Add more UNION ALLs in case there're more loops for different 

3. Setting the details list

Similar to the errors list, the Query Spreadsheet Action can be used:

SELECT
  entity,
  status,
  COUNT(*) AS number_of_records
FROM ENTITY_LOOP
GROUP BY
  status
UNION ALL // If there are multiple loops for different 
SELECT
  entity,
  status,
  COUNT(*) AS number_of_records
FROM ENTITY_LOOP
GROUP BY
  status
// Add more UNION ALLs in case there're more loops for different 

4. Setting the overview list

Similar to the errors list, the Query Spreadsheet Action can be used. This time, the details output should be the reference of the Action:

SELECT
  status,
  SUM(number_of_records) AS number_of_records
FROM DETAILS_LIST_QUERY_OUTPUT
GROUP BY
  status

5. Setting the combined list

Finally, all the lists are combined in the final step of the Flow:

{
  "overview": {{ OVERVIEW_LIST_QUERY_OUTPUT }},
  "details": {{ DETAILS_LIST_QUERY_OUTPUT }},
  "errors": {{ ERRORS_LIST_QUERY_OUTPUT }}
}

The end of the Flow would look similar to this:

The logic and format in the Flow can be freely defined (i.e. the examples shown above are just one possibility and might not apply to all Flows).

The only part that needs to be in the same structure as shown in the documentation is the final step of the Flow.

Run Status

The run status is set automatically, based on the following logic:

Status	Condition
Success	If there's at least 1 success record and 0 error records specified in overview OR: If the Flow Run has status success and the Flow is not properly setup
Error	If there's at least 1 error record and 0 success records specified in overview OR: If the Flow Run has status error
Finished	In cases where neither the Success nor Error conditions are fulfilled
Scheduled	Future Flow Run (only if Flow has a Scheduler)