Per Session Settings Author: Vikram Agrawal Date: 10 June, 2014
Motivation It would be useful to have a per session settings mechanism that can be used to control optimizations and other functionalities. These settings can be query specific configurations, metadata properties or flags to enable/disable any functionality/optimization. Such a control will help to group a set of queries in one sessions and execute them as per desired configuration.
Requirements ● ● ●
The variables are available to optimization, execution and metadata components System to set, unset and show these session variables Provide a way to use existing ConnectorSession Object so that the same object can be used to figure out session properties.
User Surface Settings can configured using SQL syntax ● SET var = literal It sets the value to the variable. Literal can be string, integer, double, duration, memory (mb/gb), boolean. Presto Server returns True on successful execution of this command. Else it returns False. ● UNSET var; It unsets a variable. i.e. change the value of the variable to its default value, if any. Presto Server returns True on successful execution of this command. ● SET var; Print the current value of the variable. It result in failed query if var neither has any default value nor any value has been set in this session. ● SET; Display current value for all existing session variables
Background: ●
There is per query session object (ConnectorSession).
●
ConnectorSession object is not shared across queries. So even if we set some session settings in one query, we cannot use in subsequent queries.
●
There are global settings which are loaded from config files when we start Presto Server
●
There is no way to override these global settings using Presto Queries for some
queries.
Architecture and Design Details Key elements in the implementation are illustrated in the following diagram
Presto Sessions Each Presto query would be attached to one Presto Session. Presto CLI will send a sessionId in its request to create a query. Multple queries with same sessionId will be part of same session. If no session identifier is specified for a query, then the query would attached itself to a "default" session. SET operation would error out in ‘default’ session since there is no semantics to set if there is no session context.
Session Configuration Manager Create a singleton object for PrestoSessionConfigManager in Presto Coordinator. PrestoSessionConfigManager has a map between all alive sessions and their associated configurations. It supports following functionality ● Creates a PrestoSessionConfig object for a session if it does not exist. ● Fetch the PrestoSessionConfig object for given session Id ● Removes the PrestoSessionConfig object for given session Id (This functionality will be used to clean up inactive sessions) PrestoSessionConfig Object Each Presto Session will have a unique SessionConfig Object. We extend Presto SQL parser grammar to support SET/UNSET/SHOW commands for session variables. These commands will appropriately update its associated session configurations. Life cycle of this object is as follow. Creation
createquery function in StatementResource and QueryResource handles the POST requests to create queries. We would extend it ● to find the sessionId of the query, ● check if there is already a sessionconfig object for the given sessionId ● create a new sessionconfig object if it does not exist. Usage
ConnectorSession is available everywhere in a query execution workflow. We extend this object to include a new field "sessionId". Wherever we want to use session variable during query planning and optimization, we use this sessionId to find sessionConfig object from PrestoSessionConfigManager, which is globally available in Presto Coordinator Limitation to this approach is that PrestoSessionConfigManager is not accessible in Task Workers. One of the ways to overcome this problem might be to pass SessionConfig object to TaskWorker. Cleanup
Add a REST endpoint in StatementResouce to delete a session. Presto Client can explicity make a rest call to delete an inactive session which in turn delete the associated SessionConfig object.
Motivation Requirements User Surface -
i.e. change the value of the variable to its default value, if any. Presto. Server returns True on successful ... PrestoSessionConfigManager has a map between all alive sessions and their associated configurations. It supports ... will appropriately update its associated session configurations. Life cycle of this object is as follow ...