Deploying in Google Cloud

Introduction

ScaleOut StateServer supports deployment to the Google Cloud Platform (GCP), providing you with cost-effective, pay-as-you-go pricing. You pay only for the resources you need while having the flexibility to adjust your deployment to meet the demands of your load.

ScaleOut users can use one of two ways to deploy their store:

• The ScaleOut Management Console has been enhanced to simplify the deployment and management of Google Cloud virtual machine instances in a cloud-based cluster. Using 3-page wizard, a ScaleOut store can be configured to run in an existing Google Cloud environment.

• ScaleOut Software also provides a set of Terraform deployment files to deploy a store using Google’s Infrastructure Manager as part of their existing infrastructure-as-code solution.

Terminology

The following terms defined by GCP are used in this guide and are defined here for your convenience:

• Instance: An instance represents a single running copy of a Google Cloud Image, and can be accessed through the Google Cloud console in the Compute Engine section.

• Configuration file: The Terraform file describing resources to be created and parameters to configure your store.

• IAM: Identity and Access Management (IAM) lets you manage granular access to specific Google Cloud resources and prevents unwanted access to other resources.

• Service account: A service account is a special kind of account typically used by an application or compute workload, such as a Compute Engine instance, rather than a person. A service account is identified by its email address, which is unique to the account.

• Project and deployment: Google Cloud projects form the basis for creating, enabling, and using all Google Cloud services including managing APIs, enabling billing, managing collaborators and permissions for Google Cloud resources.

• Machine type: When you create a VM, you select a machine type from a machine family that determines the resources available to that VM. See Machine families for more details.

• Zone: Compute Engine resources are hosted in multiple locations worldwide. These locations are composed of regions and zones.

• VPC: A Virtual Private Cloud (VPC) network is a virtual version of a physical network that is implemented inside of Google’s production network.

• Secret manager: Use Secret Manager to securely store API keys, passwords, and other sensitive information.

• Startup script: A script that will be run at startup of any new instance. In the case of ScaleOut, the startup script will install ScaleOut StateServer and configure all of its required settings.

• Metadata: Every virtual machine (VM) instance stores its metadata on a metadata server. Your VM automatically has access to the metadata server API without any additional authorization. It is also used to store values that can be queried by scripts.

• SOSS license key: The license key issued by ScaleOut Software for your store.

Note

The current support of ScaleOut StateServer deployments in Google Cloud is limited to the Debian 11 images. This means a new deployment will always use the latest Debian 11 image.

Prerequisites

To deploy ScaleOut StateServer in Google Cloud, begin by visiting https://cloud.google.com/ and signing up for a GCP account.

Service Accounts

ScaleOut’s management tools require a set of access credentials to perform actions in GCP on your behalf. The credentials that a ScaleOut deployment use in Google Cloud are linked to a Service Account. To retrieve or configure service accounts for your project, visit the IAM & Admin page in the GCP console. You will need to provide information about the service account and its associated key to run ScaleOut stores in GCP.

Note

Service accounts provide an identity for unattended applications, such as batch jobs, worker processes, or resource-monitoring agents. By using a service account, you allow these applications to run without user interaction. For production environments (or anywhere security is a concern), use the GCP Identity and Access Management (IAM) feature to create security-limited credentials.

To simplify your experience, you can use the following Terraform configuration to create a role that can be assigned to the service account you decide to use for your deployments. This configuration will create a ScaleOut StateServer Service Role in your project including all the necessary permissions.

terraform {
   required_providers {
      google = {
         source  = "hashicorp/google"
         version = "~> 5.0"
      }
   }
}

provider "google" {
   project = var.project_id
}

variable "project_id" {
   type = string
}

resource "google_project_iam_custom_role" "scaleout_stateserver_role" {
   role_id     = "scaleOutServiceRole"
   title       = "ScaleOut StateServer Service Role"
   description = "Permissions required for Google Cloud deployments of ScaleOut StateServer clusters"
   stage       = "GA"

   permissions = [
      "cloudbuild.connections.list",
      "cloudbuild.repositories.accessReadToken",
      "cloudbuild.repositories.list",
      "cloudquotas.quotas.get",
      "compute.autoscalers.list",
      "compute.instanceGroupManagers.create",
      "compute.instanceGroupManagers.get",
      "compute.instanceGroupManagers.list",
      "compute.instanceGroupManagers.update",
      "compute.instanceGroupManagers.delete",
      "compute.instanceGroups.create",
      "compute.instanceGroups.delete",
      "compute.instanceGroups.get",
      "compute.instanceGroups.list",
      "compute.instanceGroups.update",
      "compute.instanceGroups.use",
      "compute.instanceTemplates.create",
      "compute.instanceTemplates.get",
      "compute.instanceTemplates.delete",
      "compute.instanceTemplates.useReadOnly",
      "compute.disks.create",
      "compute.subnetworks.use",
      "compute.subnetworks.useExternalIp",
      "compute.instances.setLabels",
      "compute.networks.get",
      "compute.instances.create",
      "compute.instances.get",
      "compute.instances.list",
      "compute.instances.setMetadata",
      "compute.instances.start",
      "compute.instances.stop",
      "compute.machineTypes.list",
      "compute.networks.list",
      "compute.regions.list",
      "config.artifacts.import",
      "config.deployments.deleteState",
      "config.deployments.getLock",
      "config.deployments.getState",
      "config.deployments.updateState",
      "config.previews.upload",
      "iam.serviceAccounts.actAs",
      "iam.serviceAccounts.get",
      "iam.serviceAccounts.list",
      "resourcemanager.projects.get",
      "secretmanager.versions.access",
      "storage.buckets.create",
      "storage.buckets.delete",
      "storage.buckets.get",
      "storage.buckets.list",
      "storage.buckets.update",
      "storage.folders.create",
      "storage.folders.delete",
      "storage.folders.get",
      "storage.folders.list",
      "storage.objects.create",
      "storage.objects.createContext",
      "storage.objects.delete",
      "storage.objects.get",
      "storage.objects.list",
      "storage.objects.move"
   ]
}

output "role_name" {
   value = google_project_iam_custom_role.scaleout_stateserver_role.name
}

output "role_id" {
   value = google_project_iam_custom_role.scaleout_stateserver_role.id
}

Note

All permissions listed above are required for some aspect of management or creation of resources in Google Cloud. Failure to add the required permissions will result in some issues with deploying or managing your cluster. While ScaleOut management tools make every effort to notify you of missing permissions when detected, double-checking that your account has the right permissions will make the process smoother.

Once a service account is created with the proper permissions, you will also need to generate a service account file that contains the private key that will be used by the deployment scripts to authenticate with the Google Platform and use the GCP APIs to deploy your stores.

Note

The private key associated with a service account can be used to access GCP resources. It should be handled safely. For best practices related to Service accounts, please refer to https://cloud.google.com/iam/docs/best-practices-service-accounts.

Networking

Configure the following inbound rules on your VPC network:

• TCP 720-724: the default ports used by ScaleOut StateServer hosts to communicate with each other. If you use different, custom values for the ports used by the ScaleOut service, you will also need to open those ports in your VPC instead.

• TCP 4000: the web management console requires port 4000 (TCP) to be open. For security, the port should only be open to source IP addresses that are expected to use the management console.

• TCP 22: the port used for SSH connections to the GCP Compute Engine instances.

Secret manager

Google Secret Manager lets you store, manage, and secure access to your application secrets. For security reasons, the configuration scripts do not handle the ScaleOut license key directly but instead read the license key from your Google Secret Manager. In order for ScaleOut software to access that secret, the Secret Manager permission must be added to the role defined for the service account used to deploy the store.

You can create a secret to store your license key by opening the Security section in the Google Cloud console. Secrets are versioned, so if you need to update your license key at any point, you will be able to specify which version of the secret to use in your next deployment. This also allows you to have a different secret value for different environments (e.g. production or test).

Deploying using the ScaleOut Management Console

Note

Google Cloud is deprecating the Google Deployment Manager API effective March 31, 2026. Versions of the ScaleOut Management Console prior to 6.2.2.405 relied on this API to deploy ScaleOut StateServer clusters in Google Cloud.

Starting with version 6.2.2.405, the ScaleOut Management Console uses Google Infrastructure Manager to automate deployments of ScaleOut StateServer clusters in Google Cloud. While the user experience remains unchanged, the underlying implementation now leverages Terraform configurations in conjunction with the Infrastructure Manager API.

As a result of this change, prior versions of the ScaleOut Management Console no longer support deployments in Google Cloud.

Enable the Google Cloud Management View

Enable Google Cloud Management by clicking on the main menu View->Show Google Cloud Management:

This will bring up the Google Cloud Management node in the left pane. Once you select this node, the Google Cloud Management pane will open on the right.

Establish a connection

Click on the ‘Select a connection’ box to bring up the connection manager.

If you have not previously established a connection to Google Cloud, pick the ‘Manage connections’ link.

In the pop-up window, you will need to provide two parameters:

• the zone the store will get deployed into.

• the service account file holding the credentials ScaleOut will use to create resources in GCP on your behalf.

Note

The credentials file is securely encrypted in your Windows account using Windows Data Protection (DPAPI).

You can validate the connection by clicking on the ‘Test’ button or by closing the window.

Launch a new store

Once you have selected a connection, you can start the creation of a new store by clicking on ‘Launch New Store…’. This will open a 3-page wizard which will guide you through the available parameters.

In the first page of the wizard, you will select the machine type for your instances, the VPC, and the name and version of the secret holding your ScaleOut license key.

Note

The secret must be created prior to the execution of this wizard, as ScaleOut will attempt to read the license key from the Secret manager while deploying the store.

In the second page of the wizard, you can customize the ScaleOut store parameters such as version, ports, or optional REST services.

In the final page of the wizard, pick a name for your store and the number of instances to deploy. A summary view will help you validate your parameters.

Once you are satisfied with the parameters, click on ‘Launch’ to initiate the deployment of your store.

Configure the remote client

Once the new store is deployed, a pop-up dialog will offer you to configure the remote client parameters to connect to this new store. If you accept, the client configuration will get automatically updated, allowing the Console to manage the ScaleOut hosts in your new deployment.

If you want to change that selection or did not elect to configure the client after the creation of the store, you can do so by opening the ‘Client configuration’ view in the left pane.

There you can select the Cloud in the location drop-down box. This will reveal the Cloud provider options. Pick ‘Google Cloud Platform’ in the Cloud Provider drop-down, and the active Google Cloud connection you have created your store in.

If you have multiple stores in that zone, you will be given a chance to pick which one your client will be configured to use in the Store drop-down. You can then press ‘Apply’ to add this store to the remote client configuration. Click ‘Select as Active’ if you want to make this store the active remote client configuration.

You will see the update in the left pane as the store is now appearing:

Managing the store

In the Google Cloud Management view, you can see details about the existing stores for a given connection. You can see details of each instance, or reboot them.

You can also perform the following management operations: - resize the store by updating the size to its new value and clicking ‘Resize Store’. The Google Cloud Platform will automatically pick instances to add or remove to match the new size. - delete the entire store.

Deploying using Terraform

ScaleOut publishes templates and scripts to be used with Google Infrastructure Manager. The scripts are packed into a zip file available for download here.

The Google Infrastructure Manager relies on Terraform configuration to create and manage deployments. To deploy a ScaleOut StateServer cluster in Google Cloud, you will essentially modify the parameters provided in the zip file to meet your needs, upload the customized zip to a bucket on your Google Storage, and launch a new deployment using the Infrastructure Manager in the console.

Configuration files

The zip file you will download from our product download page contains:

• terraform.tfvars: the parameters used by Terraform to configure the deployment.

• main.tf: the Terraform configuration containing the logic to deploy instances.

• startup_install.sh.tftpl: the script which will be executed on each new instance at startup, used to configure the StateServer cluster.

• shutdown_script.sh: the script that runs on each terminating instance to perform cleanup tasks.

When deploying a ScaleOut store with a configuration template, you will need to customize the parameters to the configuration to meet your needs. Terraform will read the parameters provided in terraform.tfvars and use these in the deployment. You will need to provide the following required parameters:

• project: the Google Cloud project ID where you want the ScaleOut StateServer cluster to be hosted.

• service_account: the email for the Google Service Account you will be using to deploy. Make sure this account has all the roles and permissions listed above.

• soss_license_key_secret_name: as mentioned above, you will store the ScaleOut StateServer license key in a secret so that Terraform can configure your cluster. This is the name of the secret.

• soss_license_key_secret_version: the version of the secret that contains the current ScaleOut StateServer license key.

• machine_type: the type of Compute Engine instance to use for each ScaleOut StateServer instance.

• zone: the zone in which the ScaleOut StateServer cluster will be deployed (e.g. ‘us-central1-a’)

• vpc_name: the name of the VPC network where the cluster will be deployed. Make sure you define the proper inbound rules described earlier.

All the other parameters can also be customized to control the network settings, the size of the cluster, or ScaleOut-specific parameters.

Here is what the parameters file looks like:

project  = "<YOUR_GOOGLE_PROJECT_ID>"
region   = "us-central1"
zone     = "us-central1-a"

service_account = "<YOUR_GOOGLE_SERVICE_ACCOUNT_EMAIL"
config_id       = "soss-config"

vpc_name       = "default"
instance_count = 3
machine_type   = "e2-standard-4"

store_name     = "my-soss-store"
soss_version   = "6.1.3-402"

management_port          = 720
server_port              = 721
interconnect_port        = 722
secure_management_port   = 723
secure_server_port       = 724

enable_client_rest_service     = "True"
enable_management_rest_service = "True"
service_network_binding        = "internal"

soss_license_key_secret_name    = "<SECRET_NAME_HOLDING_SOSS_LICENSE>"
soss_license_key_secret_version = "<SECRET_VERSION>"

Note

terraform.tfvars is the only file you need to edit in the provided zip.

Use Infrastructure manager

Once you have edited the parameters file, update the zip with the customized parameters and upload it to Google Storage. In the Google Cloud console, open the Infrastructure Manager and click on Create new deployment.

On that new page, provide the requested information about deployment ID, service account, and the path to the Terraform you just uploaded.

Click on Create deployment.

The Terraform configuration will be applied to your project, VMs will get created and the ScaleOut StateServer will be configured automatically through the startup scripts.