HTTP REST API source

Use the HTTP REST API source when SQIP should regularly request an API and publish the returned JSON records to a topic.

Before you begin

Ask the API owner for:

The HTTPS endpoint URL.
The authentication method and credentials, if required.
A sample JSON response.
The API's pagination details, if it returns multiple pages.
A stable ID field for each record, such as id or orderId.

Warning

The endpoint must use https://. If it is only reachable on a private company network, ask your SQIP administrator to approve the endpoint.

Example response

The following API response contains records under items, uses id as the record key, and supplies the cursor for the next request in nextCursor.

{
  "items": [
    { "id": "order-42", "status": "paid" },
    { "id": "order-43", "status": "pending" }
  ],
  "nextCursor": "page-002"
}

For this response, enter /items, /id, and /nextCursor in the corresponding JSON Pointer fields.

Install the connector

Select the connector

In Connectors, find HTTP REST API Source and select Select and install.

Choose the output

Enter an installation name, select the tenant, cluster, and namespace, and choose the output topic that will receive the API records.

Configure the API

Complete every field using the table below. Keep the recommended values unless the API owner has given you different requirements.

Add authentication

If authentication is required, select the correct authentication type and enter the credential only in SQIP's credential section.

Install and verify

Review the settings, select Install connector, and wait for Running. Confirm that records appear in the selected output topic.

Field	What to enter	Example	Required
URL	The complete HTTPS endpoint to request	`https://api.example.com/v1/orders`	Yes
Auth Type	`NONE`, `BEARER`, `BASIC`, or `API_KEY`	`BEARER`	No; defaults to `NONE`
Auth Header Name	The header used for `API_KEY` authentication	`X-API-Key`	Only for `API_KEY`
Pagination Mode	`CURSOR` when the API returns a next cursor; otherwise `NONE`	`CURSOR`	No; defaults to `CURSOR`
Cursor Query Parameter	The query parameter used to send the current cursor	`cursor`	For `CURSOR` mode
Initial Cursor	The starting cursor supplied by the API owner; otherwise leave blank	Leave blank	No
Records JSON Pointer	The location of the records in the JSON response; leave blank when the response itself is an array or record	`/items`	No; defaults to `/items`
Key JSON Pointer	A stable identifier inside each record	`/id`	No, but recommended
Next Cursor JSON Pointer	The location of the next cursor in the response	`/nextCursor`	For `CURSOR` mode
Poll Interval Ms	How long SQIP waits between successful requests	`30000`	No
Connect Timeout Ms	How long SQIP waits to establish a connection	`5000`	No
Request Timeout Ms	Maximum time allowed for one API request	`10000`	No
Max Response Bytes	Largest accepted API response, in bytes	`5242880` (5 MiB)	No
Max Records Per Poll	Largest number of records accepted in one response	`1000`	No
Require JSON Content Type	Select Yes to require the API to identify its response as JSON	`Yes`	No
Parallelism	Number of connector instances	`1`	Yes; fixed at `1`

A JSON Pointer starts with / and follows the field names in the response. For example, /data/orders selects orders inside data. JSON Pointer is not the same as JSONPath, so do not enter values such as $.items.

Authentication choices

Auth Type	When to use it	Credential input
`NONE`	The API is public and requires no login	None
`BEARER`	The API expects an access token	Token
`BASIC`	The API expects a username and password	Username and password
`API_KEY`	The API expects a key in a named header	API key and Auth Header Name

Enter only the credential fields required by the selected authentication type. Alternatively, expand Use an existing secret reference instead and enter the approved secret name. Do not use both approaches.

Public test example

You can use this credential-free endpoint to verify a new SQIP environment.

Field	Input
URL	`https://jsonplaceholder.typicode.com/posts`
Auth Type	`NONE`
Auth Header Name	`X-API-Key`
Pagination Mode	`NONE`
Cursor Query Parameter	`cursor`
Initial Cursor	Leave empty
Records JSON Pointer	Leave empty
Key JSON Pointer	`/id`
Next Cursor JSON Pointer	`/nextCursor`
Poll Interval Ms	`30000`
Connect Timeout Ms	`5000`
Request Timeout Ms	`10000`
Max Response Bytes	`5242880`
Max Records Per Poll	`1000`
Require JSON Content Type	`Yes`