Configuring PBench

This page presents how to create new configuration files for PBench.

See Configuration File Examples for several examples of how to create configuration files for specific goals.

Creating a Stage File

Create a JSON file for each stage of a benchmark.

Always begin the file with a description.

Set the catalog and schema using catalog and schema.

Use queries and query_files to define the SQL queries to run in a stage.

Use next to define the order of stage files in a benchmark.

For more information, see Parameters.

Defining the Order of Stages

Use the next parameter in a stage file to define the order of stage files in a benchmark.

For example, consider the four files stage1.json, stage2.json, stage3.json, and stage4.json:

• stage1.json has the following entry for next:

  "next": [
    "stage2.json",
    "stage3.json"
  ]

  stage2 and stage3 are started in parallel when stage1 completes.

  stage2 and stage3 inherit parameters from stage1. See Inherited Parameters in Stage Files.

• stage2.json and stage3.json have the following entry for next:

  "next": [
    "stage4.json"
  ]

  stage4 inherits its parameters from the stage that starts it:

  • if stage3 completes before stage2, then stage3 passes inheritable parameters to stage4
  • if stage2 completes before stage3, then stage2 passes inheritable parameters to stage4

  See Inherited Parameters in Stage Files.

  When both stage2 and stage3 are finished, stage4 is started.

• stage4.json is the last stage of the benchmark run and has no next parameter entry.

Inherited Parameters in Stage Files

A child stage inherits some parameters from its parent stage if those parameters are not explicitly set in the child file. If a child stage has multiple parents, the child stage inherits those parameters from the first of the parent stages that finishes, which starts the child stage.

The parameters that a child stage inherits from a parent stage are:

• abort_on_error
• catalog
• cold_runs
• no_random_duplicates
• random_execution
• randomly_execute_until
• save_column_metadata
• save_json
• save_output
• schema
• session_params
• timezone
• warm_runs

If a child stage sets a different catalog, schema, or timezone than what the inherited client has, a new client is automatically created. You can also use start_on_new_client = true to force a new client (e.g., to get fresh session params).

For more information, see Parameters.

Creating a Query File

A query file is a file containing one or more - usually several - SQL queries that are called by the query_files parameter of a stage file. For a query file example, see query_01.sql.

Setting Session Parameters in Query Files

In addition to the session_params stage parameter, you can use SET SESSION statements directly in query files. The Presto client handles the server's X-Presto-Set-Session response header, so session parameters are applied to all subsequent queries in the same client session.

SET SESSION join_reordering_strategy = 'NONE';
SET SESSION hive.parquet_pushdown_filter_enabled = true;
SELECT l_orderkey, l_partkey FROM lineitem WHERE l_extendedprice <= 950;

This is useful when you need specific session parameters for a particular query without restructuring your stage files. The SET SESSION statement is executed as a normal query — no special syntax is required.

Note: Session parameters set this way persist for the duration of the client session. If a stage uses start_on_new_client: true, a fresh session is created and previously set parameters will not carry over.

Configuring InfluxDB and MySQL Connections

Save the connection information for InfluxDB and MySQL databases in configuration files, then use those configuration files with pbench run with the --influx and --mysql command options.

InfluxDB

NOTE:
An example configuration template for InfluxDB named influxdb.template.json can be found in the PBench repository. Copy influxdb.template.json to the directory local to PBench and edit it as appropriate.

Create a file named influxdb.json similar to the following example and edited as appropriate in the directory local to PBench.

Use --influx influxdb.json with pbench run to call the new file.

{
  "url": "https://example.com",
  "org": "myorg",
  "bucket": "benchmark",
  "token": "*******=="
}

MySQL

NOTE:
An example configuration template for MySQL named mysql.template.json can be found in the PBench repository. Copy mysql.template.json to the directory local to PBench and edit it as appropriate.

Create a file named mysql.json similar to the following example and edited as appropriate in the directory local to PBench.

Use --mysql mysql.json with pbench run to call the new file.

{
  "username": "presto",
  "password": "*******",
  "server": "localhost",
  "database": "test"
}

For more information, see The Run Command.