List of Properties
The following list of commonly used properties can be set to configure the cluster. These properties can be set in the conf/servers, conf/leads or conf/locators configuration files.
| Property | Description | Components |
|---|---|---|
| -bind-address | IP address on which the member is bound. The default behavior is to bind to all local addresses. | ServerLeadLocator |
| -classpath | Location of user classes required by the TIBCO ComputeDB Server.This path is appended to the current classpath. | ServerLeadLocator |
| -client-port | The port that the network controller listens for client connections in the range of 1 to 65535. The default value is 1527. | LocatorServer |
| -critical-heap-percentage | Sets the Resource Manager's critical heap threshold in percentage of the old generation heap, 0-100. If you set -heap-size, the default value for critical-heap-percentage is set to 95% of the heap size. Use this switch to override the default.When this limit is breached, the system starts canceling memory-intensive queries, throws low memory exceptions for new SQL statements, and so forth, to avoid running out of memory. |
ServerLead |
| -critical-off-heap-percentage | Sets the critical threshold for off-heap memory usage in percentage, 0-100. When this limit is breached, the system starts canceling memory-intensive queries, throws low memory exceptions for new SQL statements, and so forth, to avoid running out of off-heap memory. | Server |
| -dir | Working directory of the member that contains the TIBCO ComputeDB Server status file and the default location for the log file, persistent files, data dictionary, and so forth (defaults to the current directory). | ServerLeadLocator |
| -eviction-heap-percentage | Sets the memory usage percentage threshold (0-100) that the Resource Manager will use to start evicting data from the heap. By default, the eviction threshold is 85.5% of whatever is set for -critical-heap-percentage.Use this switch to override the default. |
ServerLead |
| -eviction-off-heap-percentage | Sets the off-heap memory usage percentage threshold, 0-100, that the Resource Manager uses to start evicting data from off-heap memory. By default, the eviction threshold is 85.5% of whatever is set for -critical-off-heap-percentage. Use this switch to override the default. |
Server |
| -heap-size | Sets the maximum heap size for the Java VM, using TIBCO ComputeDB default resource manager settings. For example, -heap-size=1GB. If you use the -heap-size option, by default TIBCO ComputeDB sets the critical-heap-percentage to 95% of the heap size, and the eviction-heap-percentage to 85.5% of the critical-heap-percentage. TIBCO ComputeDB also sets resource management properties for eviction and garbage collection if they are supported by the JVM. |
ServerLeadLocator |
| -J | JVM option passed to the spawned TIBCO ComputeDB server JVM. For example, use -J-Xmx1GB to set the JVM heap to 1GB. | ServerLeadLocator |
| -J-Dgemfirexd.hostname-for-clients | Set the IP address or host name that this server/locator sends to JDBC/ODBC/thrift clients to use for connection. The default value causes the client-bind-address to be given to clients. This value can be different from client-bind-address for cases where locators, servers are behind a NAT firewall (AWS for example) where client-bind-address needs to be a private one that gets exposed to clients outside the firewall as a different public address specified by this property. In many cases this is handled by hostname translation itself, i.e. hostname used in client-bind-address resolves to internal IP address from inside but to public IP address from outside, but for other cases this property will be required. | Server |
| -J-Dsnappydata.enable-rls | Enables the system for row level security when set to true. By default this is off. If this property is set to true, then the Smart Connector access to TIBCO ComputeDB fails. | ServerLeadLocator |
| -J-Dsnappydata.RESTRICT_TABLE_CREATION | Applicable when security is enabled in the cluster. If true, users cannot execute queries (including DDLs and DMLs) even in their default or own schema unless cluster admin explicitly grants them the required permissions using GRANT command. The default is false. | ServerLeadLocator |
| jobserver.waitForInitialization | When this property is set to true, the cluster startup waits for the Spark jobserver to be fully initialized before marking the lead node as RUNNING. The default is false. | Lead |
| -locators | List of locators as comma-separated host:port values used to communicate with running locators in the system and thus discover other peers of the distributed system. The list must include all locators in use and must be configured consistently for every member of the distributed system. This property should be configured for all the nodes in the respective configuration files, if there are multiple locators. | ServerLeadLocator |
| -log-file | Path of the file to which this member writes log messages (default is snappy[member].log in the working directory. For example, snappylocator.log, snappyleader.log,snappyserver.log. In case logging is set via log4j, the default log file is snappydata.log for each of the TIBCO ComputeDB member.) | ServerLeadLocator |
| -memory-size | Specifies the total memory that can be used by the node for column storage and execution in off-heap. The default value is either 0 or it gets auto-configured in specific scenarios. | ServerLead |
| -member-timeout | Uses the member-timeout server configuration, specified in milliseconds, to detect the abnormal termination of members. The configuration setting is used in two ways: 1) First, it is used during the UDP heartbeat detection process. When a member detects that a heartbeat datagram is missing from the member that it is monitoring after the time interval of 2 * the value of member-timeout, the detecting member attempts to form a TCP/IP stream-socket connection with the monitored member as described in the next case. 2) The property is then used again during the TCP/IP stream-socket connection. If the suspected process does not respond to the are you alive datagram within the time period specified in member-timeout, the membership coordinator sends out a new membership view that notes the member's failure. Valid values are in the range 1000-600000 milliseconds. For more information, refer to Best Practices | ServerLeadLocator |
| -peer-discovery-address | Use this as value for the port in the "host:port" value of "-locators" property | Locator |
| -peer-discovery-port | Port on which the locator listens for peer discovery (includes servers as well as other locators). Valid values are in the range 1-65535, with a default of 10334. | Locator |
| -spark.driver.maxResultSize | Limit of the total size of serialized results of all partitions for each action (e.g. collect). The value should be at least 1MB or 0 for unlimited. Jobs will be aborted if the total size of results is above this limit. Having a high limit may cause out-of-memory errors in the lead. The default max size is 1GB. | Lead |
| -spark.executor.cores | The number of cores to use on each server. | Lead |
| -spark.context-settings.num-cpu-cores | The number of cores that can be allocated. The default is 4. | Lead |
| -spark.context-settings.memory-per-node | The executor memory per node (-Xmx style. For example: 512m, 1G). The default is 512m. | Lead |
| -spark.context-settings.streaming.batch_interval | The batch interval for Spark Streaming contexts in milliseconds. The default is 1000. | Lead |
| -spark.context-settings.streaming.stopGracefully | If set to true, the streaming stops gracefully by waiting for the completion of processing of all the received data. The default is true. | Lead |
| -spark.context-settings.streaming.stopSparkContext | if set to true, the SparkContext is stopped along with the StreamingContext. The default is true. | Lead |
| -spark.jobserver.bind-address | The address on which the jobserver listens. Default address is 0.0.0. | Lead |
| -spark.jobserver.job-result-cache-size | The number of job results to keep per JobResultActor/context. The default is 5000. | Lead |
| -spark.jobserver.max-jobs-per-context | The number of jobs that can be run simultaneously in the context. The default is 8. | Lead |
| -spark.jobserver.port | The port on which to run the jobserver. Default port is 8090. | Lead |
| -spark.local.dir | Directory to use for scratch space in TIBCO ComputeDB, including map output files and RDDs that get stored on disk. This should be on a fast, local disk in your system. It can also be a comma-separated list of multiple directories on different disks. For more information, refer to Best Practices. | LeadServer |
| -spark.network.timeout | The default timeout for all network interactions while running queries. | Lead |
| -spark.sql.codegen.cacheSize | Size of the generated code cache. This effectively controls the maximum number of query plans whose generated code (Classes) is cached. The default is 2000. | Lead |
| spark.sql.files.maxPartitionBytes | Maximum number of bytes to pack into a single partition when reading files. You can modify this setting in conf/leads file. This can be used to tune performance and memory requirements for data ingestion tasks when the data is read from a file based source for ingestion into a TIBCO ComputeDB column/row table. In TIBCO ComputeDB, default value for this setting is 33554432 bytes (32 MB). | Lead |
| -spark.ssl.enabled | Enables or disables Spark layer encryption. The default is false. | Lead |
| -spark.ssl.keyPassword | The password to the private key in the key store. | Lead |
| -spark.ssl.keyStore | Path to the key store file. The path can be absolute or relative to the directory in which the process is started. | Lead |
| -spark.ssl.keyStorePassword | The password used to access the keystore. | Lead |
| -spark.ssl.trustStore | Path to the trust store file. The path can be absolute or relative to the directory in which the process is started. | Lead |
| -spark.ssl.trustStorePassword | The password used to access the truststore. | Lead |
| -spark.ssl.protocol | The protocol that must be supported by JVM. For example, TLS. | Lead |
| -spark.ui.port | Port for your TIBCO ComputeDB Monitoring Console, which shows tables, memory and workload data. The default is 5050 | Lead |
| Properties for SSL Encryption | ssl-enabled, ssl-ciphers, ssl-protocols, ssl-require-authentication. These properties need not be added to the Lead members in case of a client-server connection. | ServerLeadLocator |
| -thrift-ssl | Specifies if you want to enable or disable SSL. Values are true or false. | ServerLeadLocator |
| -thrift-ssl-properties | Comma-separated SSL properties including:protocol: default "TLS",enabled-protocols: enabled protocols separated by ":"cipher-suites: enabled cipher suites separated by ":"client-auth=(true or false): if client also needs to be authenticated keystore: Path to key store file keystore-type: The type of key-store (default "JKS") keystore-password: Password for the key store filekeymanager-type: The type of key manager factory truststore: Path to trust store filetruststore-type: The type of trust-store (default "JKS")truststore-password: Password for the trust store file trustmanager-type: The type of trust manager factory |
Server |
Other than the above properties, you can also refer the Configuration Parameters section for properties that are used in special cases.
SQL Properties
These properties can be set using a SET SQL command or using the configuration properties in the conf/leads file. The SET SQL command sets the property for the current SnappySession while setting it in conf/leads file sets the property for all SnappySession.
For example: Set in the Snappy shell
snappy> connect client 'localhost:1527';
snappy> set snappydata.column.batchSize=100k;
This sets the property for the Snappy shell's session.
Set in the conf/leads file
$ cat conf/leads
node-l -heap-size=4096m -spark.ui.port=9090 -locators=node-b:8888,node-a:9999 -spark.executor.cores=10 -snappydata.column.batchSize=100k
| Property | Description |
|---|---|
| -snappydata.column.batchSize | The default size of blocks to use for storage in TIBCO ComputeDB column and store. When inserting data into the column storage, this is the unit (in bytes or k/m/g suffixes for unit) that is used to split the data into chunks for efficient storage and retrieval. This property can also be set for each table in the create table DDL. Maximum allowed size is 2GB. The default is 24m. |
| -snappydata.column.maxDeltaRows | The maximum number of rows that can be in the delta buffer of a column table. The size of the delta buffer is already limited by ColumnBatchSize property, but this allows a lower limit on the number of rows for better scan performance. So the delta buffer is rolled into the column store whichever of ColumnBatchSize and this property is hit first. It can also be set for each table in the create table DDL, else this setting is used for the create table |
| -snappydata.hiveServer.enabled | Enables the Hive Thrift server for TIBCO ComputeDB.This is enabled by default when you start the cluster. Thus it adds an additional 10 seconds to the cluster startup time. To avoid this additional time, you can set the property to false. |
| -snappydata.sql.hashJoinSize | The join would be converted into a hash join if the table is of size less than the hashJoinSize. The limit specifies an estimate on the input data size (in bytes or k/m/g/t suffixes for unit). The default value is 100MB. |
| -snappydata.sql.hashAggregateSize | Aggregation uses optimized hash aggregation plan but one that does not overflow to disk and can cause OOME if the result of aggregation is large. The limit specifies the input data size (in bytes or k/m/g/t suffixes for unit) and not the output size. Set this only if there are queries that can return a large number of rows in aggregation results. The default value is set to 0 which means, no limit is set on the size, so the optimized hash aggregation is always used. |
| -snappydata.sql.planCacheSize | Number of query plans that will be cached. |
| -spark.sql.autoBroadcastJoinThreshold | Configures the maximum size in bytes for a table that is broadcast to all server nodes when performing a join. By setting this value to -1 broadcasting can be disabled. |
| -snappydata.linkPartitionsToBuckets | When this property is set to true, each bucket is always treated as a separate partition in column/row table scans. When this is set to false, TIBCO ComputeDB creates only as many partitions as executor cores by clubbing multiple buckets into each partition when possible. The default is false. |
| -snappydata.preferPrimaries | Use this property to configure your preference to use primary buckets in queries. This reduces the scalability of queries in the interest of reduced memory usage for secondary buckets. The default is false. |
| -snappydata.sql.partitionPruning | Use this property to set/unset the partition pruning of queries. |
| -snappydata.sql.tokenize | Use this property to enable/disable tokenization. |
| snappydata.cache.putIntoInnerJoinResultSize | Use this property with extreme limits such as 1K and 10GB. The default is 100 MB. |
| -snappydata.scheduler.pool | Use this property to define scheduler pool to either default or low latency. You can also assign queries to different pools. |
| -snappydata.enable-experimental-features | Use this property to enable and disable experimental features. You can call out in case some features are completely broken and need to be removed from the product. |
SDE Properties
The SDE properties can be set using a Snappy shell (snappy-sql) command or using the configuration properties in the conf/leads file. The command sets the property for the current SnappySession while setting it in conf/leads file sets the property for all SnappySession.
For example: Set in the Snappy shell (snappy-sql)
snappy> connect client 'localhost:1527';
snappy> set snappydata.flushReservoirThreshold=20000;
Set in the conf/leads file
$ cat conf/leads
node-l -heap-size=4096m -spark.ui.port=9090 -locators=node-b:8888,node-a:9999 -spark.executor.cores=10 -snappydata.column.batchSize=100k -spark.sql.aqp.error=0.5
This sets the property for the Snappy shell's session.
| Properties | Description |
|---|---|
| -snappydata.flushReservoirThreshold | Reservoirs of sample table will be flushed and stored in columnar format if sampling is done on the base table of size more than flushReservoirThreshold. The default value is10,000. This property must be set in the conf/servers and conf/leads file. |
| -spark.sql.aqp.numBootStrapTrials | Number of bootstrap trials to do for calculating error bounds. The default value is100. This property must be set in the conf/leads file. |
| -spark.sql.aqp.error | Maximum relative error tolerable in the approximate value calculation. It should be a fractional value not exceeding 1. The default value is0.2. This property can be set as connection property in the Snappy shell. |
| -spark.sql.aqp.confidence | Confidence with which the error bounds are calculated for the approximate value. It should be a fractional value not exceeding 1. The default value is0.95. This property can be set as connection property in the Snappy shell. |
| -sparksql.aqp.behavior | The action to be taken if the error computed goes outside the error tolerance limit. The default value isDO_NOTHING. This property can be set as connection property in the Snappy shell. |