GRPC API

FeatureBase supports querying via its gRPC interface.

By default, the interface is available on port 20101. To change to a different port, use either the command-line flag --bind-grpc <address:port> or the yaml configuration key bind-grpc.

The command-line tool grpcurl is used here to show usage examples.

gRPC API with Authentication

When authentication is enabled, gRPC endpoints can be authenticated by passing a valid JWT token in the Authorization header. The token may be obtained by following these instructions.

grpcurl -H 'Authorization: Bearer <token>' localhost:20101 pilosa.Pilosa.GetIndexes

Endpoint details

In the examples below, “None” is used to indicate whenever the request or result is supposed to be an empty message.

The following gRPC calls are available:

CreateIndex

Creates an index (table).

Request:

• name of type string (name of index)
• keys of type bool (optional, to enable string column keys, defaults false)

Response:

• None

Example:

grpcurl -plaintext -d '{"name": "repository", "keys": true}' \
    localhost:20101 pilosa.Pilosa.CreateIndex

{

}

GetIndexes

List the indexes present in FeatureBase.

Request:

• None

Response:

• List of Index messages. Each Index contains a name of type string. Each name identifies the respective index.

Example:

grpcurl -plaintext localhost:20101 pilosa.Pilosa.GetIndexes

{
  "indexes": [
    {
      "name": "customer_segmentation"
    },
    {
      "name": "example"
    },
    {
      "name": "repository"
    }
  ]
}

GetIndex

Use to check if a specific index exists. Returns an Index message if it exists, otherwise, it returns a NotFound error.

Request:

• name as type string (name of the index)

Response:

• Index message, containing a name of type string

Example:

grpcurl -plaintext -d '{"name": "repository"}' localhost:20101 pilosa.Pilosa.GetIndex

{
  "index": {
    "name": "repository"
  }
}

DeleteIndex

Delete a given index. Returns a NotFound error if the given index does not exist.

Request:

• name as string (name of index)

Response:

• None

Example:

grpcurl -plaintext -d '{"name": "example"}' localhost:20101 pilosa.Pilosa.DeleteIndex

{

}

QuerySQL

Query FeatureBase using SQL. This call is non-blocking from the client’s perspective: the server sends the response in ‘chunks’ and the client can handle each response asynchronously. This can be useful when the result set is large in size. Terminating the SQL string with ‘;’ is optional.

Request:

• sql as string (the sql query)

Response:

• headers as list of ColumnInfo. The headers are only present on the first record received. Each ColumnInfo entry consists of the following fields:

  • name as string
  • datatype as string

• columns as list of ColumnResponse. Each ColumnResponse entry consists of a field whose name and type depends on the schema of the table. For example, if the column consists of strings, it will have the name stringVal and type string. Similarly, if it consists of boolean values, it will have the name boolVal and type bool. Note that the number of ColumnResponse entries is equal to the number of ColumnInfo entries.

• duration as an integer containing the number of nanoseconds it took for the server to complete the request.

Example:

grpcurl -plaintext -d '{"sql": "select count(*) from repository"}' \
    localhost:20101 pilosa.Pilosa.QuerySQL

{
  "headers": [
    {
      "name": "count(*)",
      "datatype": "uint64"
    }
  ],
  "columns": [
    {
      "uint64Val": "161"
    }
  ],
  "duration": 534456
}

QuerySQLUnary

Query FeatureBase using SQL. As its name indicates, this is a unary call - it follows a typical synchronous request-response pattern. That is, the client blocks on the call until the entire response arrives.

Request:

• sql as string (the sql query)

Response:

• headers as list of ColumnInfo. Each ColumnInfo entry consists of the following fields:
  • name as string
  • datatype as string

• rows as list of Row. Each Row entry consists of a list of ColumnResponse. The structure of a ColumnResponse is detailed in the QuerySQL call section.

• duration as an integer containing the number of nanoseconds it took for the server to complete the request.

Example:

grpcurl -plaintext -d '{"sql": "select count(*) from repository"}' \
    localhost:20101 pilosa.Pilosa.QuerySQLUnary

{
  "headers": [
    {
      "name": "count(*)",
      "datatype": "uint64"
    }
  ],
  "rows": [
    {
      "columns": [
        {
          "uint64Val": "161"
        }
      ]
    }
  ],
  "duration": 408939
}

QueryPQL

The PQL counterpart to QuerySQL.

Request:

• index as string (name of index)
• pql as string (actual PQL query)

Response:

• headers as list of ColumnInfo. The headers are only present on the first record received. Each ColumnInfo entry consists of the following fields:

  • name as string
  • datatype as string

• columns as list of ColumnResponse. Each ColumnResponse entry consists of a field whose name and type depends on the schema of the table. For example, if the column consists of strings, it will have the name stringVal and type string. Similarly, if it consists of boolean values, it will have the name boolVal and type bool. Note that the number of ColumnResponse entries is equal to the number of ColumnInfo entries.

• duration as an integer containing the number of nanoseconds it took for the server to complete the request.

Example:

grpcurl -plaintext -d '{"pql": "Row(language=1)", "index": "repository"}' \
    localhost:20101 pilosa.Pilosa.QueryPQL

{
  "headers": [
    {
      "name": "_id",
      "datatype": "uint64"
    }   
  ],
  "columns": [
    {
      "uint64Val": "0"
    }
  ],
  "duration": 116070
}
// Truncated

QueryPQLUnary

The PQL counterpart to QuerySQLUnary.

Request:

• index as string (name of index)
• pql as string (actual PQL query)

Response:

• headers as list of ColumnInfo. Each ColumnInfo entry consists of the following fields:
  • name as string
  • datatype as string
• rows as list of Row. Each Row entry consists of a list of ColumnResponse. The structure of a ColumnResponse is detailed in the QuerySQL call section.
• duration as an integer containing the number of nanoseconds it took for the server to complete the request.

Example:

grpcurl -plaintext -d '{"pql": "Row(language=1, limit=1)", "index": "repository"}' \
    localhost:20101 pilosa.Pilosa.QueryPQLUnary

{
  "headers": [
    {
      "name": "_id",
      "datatype": "uint64"
    }   
  ],
  "rows": [
    {
      "columns": [
        {
          "uint64Val": "0"
        }
      ]
    }
  ],
  "duration": 116070
}

Inspect

Explore sections or subsections of the data.

:::caution Deprecation Notice Deprecation Notice! Inspect remains available in Molecula 3.x, but is now deprecated in favor of the Extract PQL Query. :::

Request:

• index as string (name of index)
• columns as list of string. That is, the ids of each entry.
• filterFields as list of string. That is, which fields to keep in the result. If empty, all fields are included.
• query as string. Apply filtering before retrieving entries.
• limit as uint64. Set number of entries to return, by default, it is 100000.
• offset as uint64. Number entries to skip before applying limit.

Response:

• headers as list of ColumnInfo. Each ColumnInfo entry consists of the following fields:

  • name as string
  • datatype as string

• columns as list of ColumnResponse. Each ColumnResponse entry consists of a field whose name and type depends on the schema of the table. For example, if the column consists of strings, it will have the name stringVal and type string. Similarly, if it consists of boolean values, it will have the name boolVal and type bool. Note that the number of ColumnResponse entries is equal to the number of ColumnInfo entries.

grpcurl -plaintext \
    -d '{"index": "repository", "columns": {"ids": {"vals": [3, 378]}}, "filterFields": ["language"], "limit": 1 }' \
    localhost:20101 pilosa.Pilosa.Inspect

{
  "headers": [
    {
      "name": "_id",
      "datatype": "uint64"
    },
    {
      "name": "language",
      "datatype": "[]uint64"
    },
    {
      "name": "stargazer",
      "datatype": "int64"
    }
  ],
  "columns": [
    { "uint64Val": "3"},
    { "uint64ArrayVal": { "vals": ["5"] }},
  ]
}
{
  "columns": [
    { "uint64Val": "378"},
    { "uint64ArrayVal": { "vals": ["5"] }},
  ]
}