AWS OpenSearch server setup#

AWS OpenSearch Service allows you to search large volumes of data quickly, in near real-time, by creating and managing an index of post data. The indexing process can be managed from the System Console after setting up and connecting an OpenSearch server. The post index is stored on the OpenSearch server and updated constantly after new posts are made. In order to index existing posts, a bulk index of the entire post database must be generated.

Deploying AWS OpenSearch includes the following two steps: setting up AWS OpenSearch, and configuring Mattermost.

Set up AWS OpenSearch#

From Mattermost v9.11, beta support is available for AWS OpenSearch v1.x and v2.x. This document covers both on‑premises and AWS OpenSearch setup, including manual steps and Terraform examples.

We highly recommend that you set up an AWS OpenSearch server on a separate machine from the Mattermost server.

On-Premises OpenSearch

To install on-premise OpenSearch, provision a dedicated server (e.g. Ubuntu 22.04 LTS).
Install Java (OpenSearch requires Java 11+):

sudo apt update
sudo apt install -y openjdk-11-jdk
java -version

Download & extract OpenSearch 2.x:

wget https://artifacts.opensearch.org/releases/bundle/opensearch/2.9.0/opensearch-2.9.0-linux-x64.tar.gz
tar -xzf opensearch-2.9.0-linux-x64.tar.gz
sudo mv opensearch-2.9.0 /usr/share/opensearch

Create a dedicated user & set permissions:

sudo useradd --no-create-home --shell /bin/false opensearch
sudo chown -R opensearch:opensearch /usr/share/opensearch

Configure systemd:

[Unit]
  Description=OpenSearch
  Wants=network-online.target
  After=network-online.target

[Service]
  Type=notify
  User=opensearch
  Group=opensearch
  ExecStart=/usr/share/opensearch/bin/opensearch
  Restart=on-failure
  LimitNOFILE=65536
  LimitNPROC=4096

[Install]
  WantedBy=multi-user.target

Edit opensearch.yml to include the following:

cluster.name: mattermost-cluster node.name:
node-1
path.data: /var/lib/opensearch
path.logs: /var/log/opensearch
network.host: 0.0.0.0
discovery.seed_hosts: ["<other-node-ip>"]
cluster.initial_master_nodes: ["node-1", "node-2"]

Enable & start OpenSearch:

sudo systemctl daemon-reload
sudo systemctl enable opensearch
sudo systemctl start opensearch
sudo systemctl status opensearch

Terraform (Docker) Example

provider "docker" {
  host = "unix:///var/run/docker.sock"
}

resource "docker_image" "opensearch" {
  name = "opensearchproject/opensearch:2.9.0"
}

resource "docker_container" "opensearch" {
  name = "opensearch"
  image = docker_image.opensearch.latest

  ports {
    internal = 9200
    external = 9200
  }
  ports {
    internal = 9600
    external = 9600
  }

  env = [
    "cluster.name=mattermost-cluster",
    "network.host=0.0.0.0",
    "discovery.type=single-node", # remove for multi-node
  ]

  restart = "unless-stopped"
}

AWS OpenSearch Console Setup

To install AWS OpenSearch, open the AWS Console > OpenSearch Service.
Create a domain, where:

Domain name: mattermost-os

Engine version: OpenSearch 2.x.

Configure the cluster, where:

instance type: r6g.xlarge.search

data nodes: 2

master nodes: 2

storage: EBS gp3 (1536 GiB, 4608 IOPS, 250 MiB/s)

Specify the network for: VPC with 2 subnets, and a security group allowing Mattermost IPs on port 9200.
Configure the access policy (JSON):

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": { "AWS":
"arn:aws:iam::123456789012:role/MattermostAppRole" },
    "Action": "es:*",
    "Resource": "arn:aws:es:us-east-1:123456789012:domain/mattermost-os/*" }]
}

Configure the following advanced settings (JSON):

{
  "action.destructive_requires_name": "false",
  "rest.action.multi.allow_explicit_index": "true",
  "indices.query.bool.max_clause_count": "1024",
  "indices.fielddata.cache.size": "20"
}

Configure the automated snapshot start hour as 23 (UTC), enforce HTTPS, then review & create.
To test, run the following command:

curl https://mattermost-os-xxxxxxxxxxx.us-east-1.es.amazonaws.com

AWS Terraform Example

provider "aws" {
  region = "us-east-1"
}

resource "aws_iam_role" "os_service_role" {
  name = "OSServiceRole"
  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [{
    "Action": "sts:AssumeRole",
    "Effect": "Allow",
    "Principal": { "Service": "es.amazonaws.com" }
  }]
}
EOF
}

resource "aws_opensearch_domain" "mattermost" {
  domain_name = "mattermost-os"
  engine_version = "OpenSearch_2.9"
  cluster_config {
    instance_type = "r6g.xlarge.search"
    instance_count = 2
    dedicated_master_enabled = true
    dedicated_master_type = "r6g.xlarge.search"
    dedicated_master_count = 2
    zone_awareness_enabled = true
  }

  ebs_options {
    ebs_enabled = true
      volume_type = "gp3"
      volume_size = 1536
      iops = 4608
  }

  vpc_options {
    subnet_ids = ["subnet-blah1", "subnet-blah2"]
    security_group_ids = ["sg-1234567890"]
  }

  advanced_options = {
    "rest.action.multi.allow_explicit_index" = "true"
    "indices.query.bool.max_clause_count" = "1024"
    "indices.fielddata.cache.size" = "20"
    "action.destructive_requires_name" = "false"
  }

  access_policies = <<POLICY
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {
      "AWS": "arn:aws:iam::123456789012:role/MattermostAppRole"
    },
      "Action": "es:*",
      "Resource": "arn:aws:es:us-east-1:123456789012:domain/mattermost-os/*"
    }]
  }
  POLICY
    service_software_options {
      automated_snapshot_start_hour = 23
    }

    domain_endpoint_options {
      enforce_https = true
    }
}

Configure Mattermost#

Follow these steps to configure Mattermost to use your AWS OpenSearch server and to generate the post index:

Go to System Console > Environment > Elasticsearch.
Set Enable Elasticsearch Indexing to true to enable the other the settings on the page.
Ensure Backend type is set to opensearch.
Set the Server Connection Address to your Elasticsearch or OpenSearch cluster endpoint.
Monitor cluster health: curl https://mattermost-os-xxxxx.us-east-1.es.amazonaws.com/_cluster/health

Set server connection details#

(Optional) Enter Server Username used to access the enterprise search server.
(Optional) Enter Server Password associated with the username.
Set Enable Cluster Sniffing (Optional). Sniffing finds and connects to all data nodes in your cluster automatically.
Optional CA and client certificate configuration settings are available for use with basic authentication credentials or to replace them. See the Enterprise search configuration settings documentation for details.
Select Test Connection and then select Save. If the server connection is unsuccessful you won’t be able to save the configuration or enable searching with Elasticsearch or AWS OpenSearch.

Build the post index of existing messages#

Select Index Now. This process can take up to a few hours depending on the size of the post database and number of messages. The progress percentage can be seen as the index is created. To avoid downtime, set Enable Elasticsearch for search queries to false so that database search is available during the indexing process.

Enable enterprise search#

Ensure bulk indexing is complete before enabling enterprise search, otherwise search results will be incomplete.

Set Enable Elasticsearch for search queries to true, and setting Enable Elasticsearch for autocomplete to true. Save your configuration updates and restart the Mattermost server.

Note

For high post volume deployments, we strongly encourage you to read and properly configure the Mattermost LiveIndexingBatchSize configuration setting.

Once the configuration is saved, new posts made to the database are automatically indexed on the Elasticsearch or AWS OpenSearch server.

Enterprise search limitations#

Elasticsearch and AWS OpenSearch uses a standard selection of “stop words” to keep search results relevant. Results for the following words will not be returned: “a”, “an”, “and”, “are”, “as”, “at”, “be”, “but”, “by”, “for”, “if”, “in”, “into”, “is”, “it”, “no”, “not”, “of”, “on”, “or”, “such”, “that”, “the”, “their”, “then”, “there”, “these”, “they”, “this”, “to”, “was”, “will”, and “with”.
Searching stop words in quotes returns more results than just the searched terms (ticket).
Search results are limited to a user’s team and channel membership. This is enforced by the Mattermost server. The entities are indexed in Elasticsearch or AWS OpenSearch in a way that allows Mattermost to filter them when querying, so the Mattermost server narrows down the results on every Elasticsearch or AWS OpenSearch request applying those filters.