Search Automation

This will automatically do the entire pagination logic and will output a nice 'flat' list of records.

Feature Introduction

The remote search feature allows users to search for IDs by a name they're used to, e.g. they can search for a contact ID by the name of a contact:

Remote search demo

To implement this feature, the following configurations are required:

• Connector configuration: Defines how the search is performed.

• Pagination configuration (if needed): Handles cases where results span multiple pages.

• UI form schema: Specifies where and how the search input appears in the user interface.

Connector configuration

The connector's configuration is provided in JSON format, which defines the logic and parameters for the search operation.

Configuration fields on connector

Search configuration

The search configuration defines most of the logic required to execute a search. It specifies the fields to retrieve, query structure, endpoint, and optional parameters like body and conditions. You can use Jinja in all fields to use parameters passed from the action or even have conditional statements

Example configuration

{
  "response_fields": {
    "id": "$[*].id",
    "match": "$[*].name"
  },
  "query": {
    "include": "{{ resource_type }}",
    "q": "{{ q }}",
    "per_page": 50
  },
  "type": "search",
  "endpoint": "search"
}

Key Fields:

• response_fields:

  • A dictionary specifying JSON paths for search results.
  • id: JSON path for the field to insert into the connector (typically an ID). $[*].id retrieves the contact ID field (example above)

  • match: JSON path for the field used in user search (e.g., name). $[*].name retrieves the contact's name (example above)

• endpoint:

  • The endpoint is the endpoint that is used for searching by that connector and will be appended to the base domain, just like the endpoint in actions.

• query (optional)

  • A JSON string defining query parameters for a GET request.

  • q The search input from the UI field where remote search is configured in the action.

• body (optional)

  • A JSON string for request body parameters in a POST request.

• type:

  • Specifies the search type:

    • search: Uses the connector's search endpoint directly.

    • list: Retrieves a list of entities when the connector lacks a search endpoint. In this case, the actual search happens in our backend.

• when (optional)

  • A condition evaluated as True to determine if this configuration should be used.

  • This needs to be used in combination with multiple configurations.

Multiple configurations

To support multiple search scenarios for one Connector, the configuration needs to be a list of configurations, and the parameter when needs to be used, except for the last statement, which can be a 'catch all' configuration.

The when parameter will be checked from top to bottom and the one that matches first will be used. In case none match and a search configuration does not have a when parameter, that configuration will be used (i.e. similar to multiple elif and lastly an else statement)

Multiple configurations example
[
  {
    "when": "{{ resource_type == 'contacts' }}",
    "type": "search",
    "response_fields": {
      "id": "$[*].id",
      "match": "$[*].name"
    },
    "endpoint": "contacts/search"
  },
  {
    "type": "list",
    "response_fields": {
      "id": "$[*].id",
      "match": "$[*].lastName"
    },
    "endpoint": "users/list"
  }
]

Pagination configuration (optional)

This is only needed in case there's any pagination for the endpoint required. Parameters such as limit can be directly configured within the search configuration.

The pagination configuration is also used for Automatic Pagination Configuration

Configuration on action

On the action that uses the search some more configurations are necessary, in order to specify in which field the search should happen and to pass further parameters.

"id": {
  "title": "Account ID",
  "type": "text",
  "required": true,
  "placeholder": "Search by account name...",
  "hasRemoteSearch": true,
  "resourceType": "sales_account",
  "remoteSearchParam": [],
  "count": 10
}

The field used for search needs to be on the first level of JSON nesting (i.e. it can't be nested).

type the type of the field has to be text

We didn't see a use case for any other field type, so we kept it simple. In case you see a use case, let us know.

placeholder is optional, however, this is probably the best way to make it obvious to the user that they can search here.

hasRemoteSearch needs to be set to true.

resourceType is required to be filled and will be accessible in the search configuration. This could e.g. be "customer", "ticket", "user"; so an entity name that exists in the connector.

remoteSearchParam With this field, we can use input from other fields, which are also on the first level of JSON nesting and then access it within the search configuration

E.g. for Asana, we need to pass the workspace_gid to the search configuration, which looks like this:

{
  "type": "search",
  "response_fields": {
    "match": "$.data[*].name",
    "id": "$.data[*].gid"
  },
  "query": "{\"resource_type\": \"{{ resource_type }}\", \"query\": \"{{ q }}\", \"count\": \"50\"}",
  "endpoint": "workspaces/{{ workspace_gid }}/typeahead"
}

The UI Form Schema in this case would look something like this:

{
  "workspace_gid": {
    "title": "Workspace GID",
    "type": "text",
    "placeholder": "Required to enable search",
    "info": "Globally unique identifier for the workspace or organization."
  },
  "project_id": {
    "title": "Project ID",
    "type": "text",
    "required": true,
    "info": "Globally unique identifier for the project.",
    "placeholder": "Search by project name…",
    "hasRemoteSearch": true,
    "resourceType": "project",
    "remoteSearchParam": [
      "workspace_gid"
    ]
  }
}

count is optional. This could e.g. be 5 or 10; and specifies the number of entities that we want to return (in case this is configured in the search configuration).

We haven't seen a use case yet where it makes sense to configure this on the action. If we don't see one in the next month, we can probably remove it entirely

Examples

Freshsales - Search

Search configuration

{
  "type": "search",
  "response_fields": {
    "match": "$[*].name",
    "id": "$[*].id"
  },
  "query": "{\"include\": \"{{ resource_type }}\", \"q\": \"{{ q }}\", \"per_page\": 50}",
  "endpoint": "search"
}

Action search configuration

"id": {
  "title": "Account ID",
  "type": "text",
  "required": true,
  "placeholder": "Search by account name...",
  "hasRemoteSearch": true,
  "resourceType": "sales_account",
  "remoteSearchParam": [],
  "count": 10
}

HubSpot - Search

Search configuration

HubSpot has a search endpoint, which works great for the resource types that could have potentially thousands of entries (i.e. using type list is not an option), however, it does not work for all resource types and for the ones that are not supported get calls need to be made.

Thus, multiple configurations are needed in order to cover all resource types:

1. For the search endpoint HubSpot expects a POST request, thus we need to provide the field body instead of the field query. This configuration will use the when parameter, as HubSpot has a set list of resource types that are supported by this endpoint.

2. For the get endpoints, search type list needs to be used. Here, the when parameter will not be used, as it should always be used if the resource type is not supported by the search endpoint (and thus the first search configuration).

[
  {
    "when": "{{ resource_type == 'companies' or resource_type == 'contacts' or resource_type == 'deals' or resource_type == 'feedback_submissions' or resource_type == 'products' or resource_type == 'tickets' or resource_type == 'line_items' or resource_type == 'quotes' }}",
    "type": "search",
    "body": "{ \"filterGroups\": [{\"filters\": [{\"propertyName\": \"{% if resource_type == 'companies' %}name{% elif resource_type == 'contacts' %}lastname{% elif resource_type == 'deals' %}dealname{% endif %}\", \"operator\": \"CONTAINS_TOKEN\", \"value\": \"{{ q }}\"}]}]}",
    "response_fields": {
      "id": "$.results[*].id",
      "match": "$.results[*].properties.{% if resource_type == 'companies' %}name{% elif resource_type == 'contacts' %}lastname{% elif resource_type == 'deals' %}dealname{% endif %}"
    },
    "endpoint": "crm/v3/objects/{{ resource_type }}/search"
  },
  {
    "type": "list",
    "query": "",
    "response_fields": {
      "id": "$.results[*].id",
      "match": "$.results[*].lastName"
    },
    "endpoint": "crm/v3/owners"
  }
]

Action search configuration

"companyId": {
  "type": "text",
  "title": "Company ID",
  "info": "Company to be updated",
  "required": true,
  "placeholder": "Search by company name…",
  "hasRemoteSearch": true,
  "resourceType": "companies",
  "remoteSearchParam": []
}

Slack - List

Search configuration

This is not dynamic yet, needs to be improved

{
  "type": "list",
  "response_fields": {
    "match": "$.channels[*].name",
    "id": "$.channels[*].name"
  },
  "endpoint": "conversations.list"
}

Action search configuration

"channel": {
  "title": "Channel Name without the # ",
  "type": "text",
  "placeholder": "marketing-team",
  "required": true,
  "info": "Search by name...",
  "hasRemoteSearch": true,
  "resourceType": "channels",
  "remoteSearchParam": []
}

Stripe - List

Search configuration

Here we use conditional Jinja logic in order to reference to the proper match field.

{
  "endpoint": "{{ resource_type }}",
  "type": "list",
  "response_fields": {
    "id": "$.data[*].id",
    "match": "$.data[*].{% if resource_type == 'customers' or 'products' %}name{% elif resource_type == 'invoices' %}customer_name{% endif %}"
  }
}

Action search configuration

"id": {
  "type": "text",
  "title": "ID",
  "required": true,
  "info": "Search by name...",
  "hasRemoteSearch": true,
  "resourceType": "customers",
  "remoteSearchParam": []
}