Example Impala data source

This example is intended for use with the fbsql loader command and provides:

valid SQL to create a table then insert data
TOML key/values to save as a configuration file

Before you begin

Step 1 - Create data source

Connect to your Impala database.

Create the source table:

CREATE TABLE testdb.impala_table (
  idkey int,
  intf int,
  stringf string,
  idf int,
  stringsetf string,
  idsetf string);

Insert data to the source table:

INSERT INTO testdb.impala_table VALUES
 (0, 0, 'a', 0, 'a', '3'),
 (1, 0, 'a', 0, 'c', '4'),
 (2, 0, 'a', 0, 'd', '5');

Step 2 - Create TOML configuration file

Open a terminal then run nano impala-import.toml
Add the following keys and values and substitute your Impala connection string as directed:

table = "loader-target"
query = "select idkey, intf, stringf, idf, stringsetf, idsetf from <impaladb>.impala_table;"
driver = "impala"
connection-string = "impala://<connection-string>/database=<source-db-name>"
batch-size = 1

The following [[fields]] keys and values are optional

[[fields]]
name = "idkey"
source-type = "id"
primary-key = true

[[fields]]
name = "intf"
source-type = "int"

[[fields]]
name = "stringf"
source-type = "string"

[[fields]]
name = "idf"
source-type = "id"

[[fields]]
name = "stringsetf"
source-type = "stringset"

[[fields]]
name = "idsetf"
source-type = "idset"

Save then exit nano.

Next step

Import data from your data source

Further information