CloudWiki
Resource

OpenSearch

Amazon Web Services
Analytics
Amazon OpenSearch is the successor to Elasticsearch service. It is an open-source, distributed search and analytics suite based on Elasticsearch, commonly used for log analytics, full-text search, security intelligence, business analytics, and operational intelligence use cases.
Terraform Name
terraform
aws_opensearch_domain
OpenSearch
attributes:

The following arguments are required:

  • domain_name - (Required) Name of the domain.

The following arguments are optional:

  • access_policies - (Optional) IAM policy document specifying the access policies for the domain.
  • advanced_options - (Optional) Key-value string pairs to specify advanced configuration options. Note that the values for these configuration options must be strings (wrapped in quotes) or they may be wrong and cause a perpetual diff, causing Terraform to want to recreate your OpenSearch domain on every apply.
  • advanced_security_options - (Optional) Configuration block for fine-grained access control. Detailed below.
  • auto_tune_options - (Optional) Configuration block for the Auto-Tune options of the domain. Detailed below.
  • cluster_config - (Optional) Configuration block for the cluster of the domain. Detailed below.
  • cognito_options - (Optional) Configuration block for authenticating Kibana with Cognito. Detailed below.
  • domain_endpoint_options - (Optional) Configuration block for domain endpoint HTTP(S) related options. Detailed below.
  • ebs_options - (Optional) Configuration block for EBS related options, may be required based on chosen instance size. Detailed below.
  • engine_version - (Optional) Either Elasticsearch_X.Y or OpenSearch_X.Y to specify the engine version for the Amazon OpenSearch Service domain. For example, OpenSearch_1.0 or Elasticsearch_7.9. See Creating and managing Amazon OpenSearch Service domains. Defaults to OpenSearch_1.1.
  • encrypt_at_rest - (Optional) Configuration block for encrypt at rest options. Only available for certain instance types. Detailed below.
  • log_publishing_options - (Optional) Configuration block for publishing slow and application logs to CloudWatch Logs. This block can be declared multiple times, for each log_type, within the same resource. Detailed below.
  • node_to_node_encryption - (Optional) Configuration block for node-to-node encryption options. Detailed below.
  • snapshot_options - (Optional) Configuration block for snapshot related options. Detailed below. DEPRECATED. For domains running OpenSearch 5.3 and later, Amazon OpenSearch takes hourly automated snapshots, making this setting irrelevant. For domains running earlier versions, OpenSearch takes daily automated snapshots.
  • tags - (Optional) Map of tags to assign to the resource. If configured with a provider default_tags configuration block present, tags with matching keys will overwrite those defined at the provider-level.
  • vpc_options - (Optional) Configuration block for VPC related options. Adding or removing this configuration forces a new resource (documentation). Detailed below.

advanced_security_options

  • anonymous_auth_enabled - (Optional) Whether Anonymous auth is enabled. Enables fine-grained access control on an existing domain. Ignored unless advanced_security_options are enabled. Can only be enabled on an existing domain.
  • enabled - (Required, Forces new resource when changing from true to false) Whether advanced security is enabled.
  • internal_user_database_enabled - (Optional) Whether the internal user database is enabled. Default is false.
  • master_user_options - (Optional) Configuration block for the main user. Detailed below.

master_user_options

  • master_user_arn - (Optional) ARN for the main user. Only specify if internal_user_database_enabled is not set or set to false.
  • master_user_name - (Optional) Main user's username, which is stored in the Amazon OpenSearch Service domain's internal database. Only specify if internal_user_database_enabled is set to true.
  • master_user_password - (Optional) Main user's password, which is stored in the Amazon OpenSearch Service domain's internal database. Only specify if internal_user_database_enabled is set to true.

auto_tune_options

  • desired_state - (Required) Auto-Tune desired state for the domain. Valid values: ENABLED or DISABLED.
  • maintenance_schedule - (Required if rollback_on_disable is set to DEFAULT_ROLLBACK) Configuration block for Auto-Tune maintenance windows. Can be specified multiple times for each maintenance window. Detailed below.
  • rollback_on_disable - (Optional) Whether to roll back to default Auto-Tune settings when disabling Auto-Tune. Valid values: DEFAULT_ROLLBACK or NO_ROLLBACK.

maintenance_schedule

  • start_at - (Required) Date and time at which to start the Auto-Tune maintenance schedule in RFC3339 format.
  • duration - (Required) Configuration block for the duration of the Auto-Tune maintenance window. Detailed below.
  • cron_expression_for_recurrence - (Required) A cron expression specifying the recurrence pattern for an Auto-Tune maintenance schedule.

duration

  • value - (Required) An integer specifying the value of the duration of an Auto-Tune maintenance window.
  • unit - (Required) Unit of time specifying the duration of an Auto-Tune maintenance window. Valid values: HOURS.

cluster_config

  • cold_storage_options - (Optional) Configuration block containing cold storage configuration. Detailed below.
  • dedicated_master_count - (Optional) Number of dedicated main nodes in the cluster.
  • dedicated_master_enabled - (Optional) Whether dedicated main nodes are enabled for the cluster.
  • dedicated_master_type - (Optional) Instance type of the dedicated main nodes in the cluster.
  • instance_count - (Optional) Number of instances in the cluster.
  • instance_type - (Optional) Instance type of data nodes in the cluster.
  • warm_count - (Optional) Number of warm nodes in the cluster. Valid values are between 2 and 150. warm_count can be only and must be set when warm_enabled is set to true.
  • warm_enabled - (Optional) Whether to enable warm storage.
  • warm_type - (Optional) Instance type for the OpenSearch cluster's warm nodes. Valid values are ultrawarm1.medium.search, ultrawarm1.large.search and ultrawarm1.xlarge.search. warm_type can be only and must be set when warm_enabled is set to true.
  • zone_awareness_config - (Optional) Configuration block containing zone awareness settings. Detailed below.
  • zone_awareness_enabled - (Optional) Whether zone awareness is enabled, set to true for multi-az deployment. To enable awareness with three Availability Zones, the availability_zone_count within the zone_awareness_config must be set to 3.

cold_storage_options

  • enabled - (Optional) Boolean to enable cold storage for an OpenSearch domain. Defaults to false. Master and ultrawarm nodes must be enabled for cold storage.

zone_awareness_config

  • availability_zone_count - (Optional) Number of Availability Zones for the domain to use with zone_awareness_enabled. Defaults to 2. Valid values: 2 or 3.

cognito_options

AWS documentation: Amazon Cognito Authentication for Kibana

  • enabled - (Optional) Whether Amazon Cognito authentication with Kibana is enabled or not. Default is false.
  • identity_pool_id - (Required) ID of the Cognito Identity Pool to use.
  • role_arn - (Required) ARN of the IAM role that has the AmazonOpenSearchServiceCognitoAccess policy attached.
  • user_pool_id - (Required) ID of the Cognito User Pool to use.

domain_endpoint_options

  • custom_endpoint_certificate_arn - (Optional) ACM certificate ARN for your custom endpoint.
  • custom_endpoint_enabled - (Optional) Whether to enable custom endpoint for the OpenSearch domain.
  • custom_endpoint - (Optional) Fully qualified domain for your custom endpoint.
  • enforce_https - (Optional) Whether or not to require HTTPS. Defaults to true.
  • tls_security_policy - (Optional) Name of the TLS security policy that needs to be applied to the HTTPS endpoint. Valid values: Policy-Min-TLS-1-0-2019-07 and Policy-Min-TLS-1-2-2019-07. Terraform will only perform drift detection if a configuration value is provided.

ebs_options

  • ebs_enabled - (Required) Whether EBS volumes are attached to data nodes in the domain.
  • iops - (Optional) Baseline input/output (I/O) performance of EBS volumes attached to data nodes. Applicable only for the GP3 and Provisioned IOPS EBS volume types.
  • throughput - (Required if volume_type is set to gp3) Specifies the throughput (in MiB/s) of the EBS volumes attached to data nodes. Applicable only for the gp3 volume type. Valid values are between 125 and 1000.
  • volume_size - (Required if ebs_enabled is set to true.) Size of EBS volumes attached to data nodes (in GiB).
  • volume_type - (Optional) Type of EBS volumes attached to data nodes.

encrypt_at_rest

  • enabled - (Required) Whether to enable encryption at rest. If the encrypt_at_rest block is not provided then this defaults to false. Enabling encryption on new domains requires an engine_version of OpenSearch_X.Y or Elasticsearch_5.1 or greater.
  • kms_key_id - (Optional) KMS key ARN to encrypt the Elasticsearch domain with. If not specified then it defaults to using the aws/es service KMS key. Note that KMS will accept a KMS key ID but will return the key ARN. To prevent Terraform detecting unwanted changes, use the key ARN instead.

log_publishing_options

  • cloudwatch_log_group_arn - (Required) ARN of the Cloudwatch log group to which log needs to be published.
  • enabled - (Optional, Default: true) Whether given log publishing option is enabled or not.
  • log_type - (Required) Type of OpenSearch log. Valid values: INDEX_SLOW_LOGS, SEARCH_SLOW_LOGS, ES_APPLICATION_LOGS, AUDIT_LOGS.

node_to_node_encryption

  • enabled - (Required) Whether to enable node-to-node encryption. If the node_to_node_encryption block is not provided then this defaults to false. Enabling node-to-node encryption of a new domain requires an engine_version of OpenSearch_X.Y or Elasticsearch_6.0 or greater.

snapshot_options

  • automated_snapshot_start_hour - (Required) Hour during which the service takes an automated daily snapshot of the indices in the domain.

vpc_options

AWS documentation: VPC Support for Amazon OpenSearch Service Domains

  • security_group_ids - (Optional) List of VPC Security Group IDs to be applied to the OpenSearch domain endpoints. If omitted, the default Security Group for the VPC will be used.
  • subnet_ids - (Required) List of VPC Subnet IDs for the OpenSearch domain endpoints to be created in.

Associating resources with a
OpenSearch
Resources do not "belong" to a
OpenSearch
Rather, one or more Security Groups are associated to a resource.
Create
OpenSearch
via Terraform:
The following HCL creates an OpenSearch domain
Syntax:

resource "aws_opensearch_domain" "example" {
 domain_name    = "example"
 engine_version = "Elasticsearch_7.10"

 cluster_config {
   instance_type = "r4.large.search"
 }

 tags = {
   Domain = "TestDomain"
 }
}

Create
OpenSearch
via CLI:
Parameters:

create-domain
--domain-name <value>
[--engine-version <value>]
[--cluster-config <value>]
[--ebs-options <value>]
[--access-policies <value>]
[--snapshot-options <value>]
[--vpc-options <value>]
[--cognito-options <value>]
[--encryption-at-rest-options <value>]
[--node-to-node-encryption-options <value>]
[--advanced-options <value>]
[--log-publishing-options <value>]
[--domain-endpoint-options <value>]
[--advanced-security-options <value>]
[--tag-list <value>]
[--auto-tune-options <value>]
[--cli-input-json | --cli-input-yaml]
[--generate-cli-skeleton <value>]
[--debug]
[--endpoint-url <value>]
[--no-verify-ssl]
[--no-paginate]
[--output <value>]
[--query <value>]
[--profile <value>]
[--region <value>]
[--version <value>]
[--color <value>]
[--no-sign-request]
[--ca-bundle <value>]
[--cli-read-timeout <value>]
[--cli-connect-timeout <value>]
[--cli-binary-format <value>]
[--no-cli-pager]
[--cli-auto-prompt]
[--no-cli-auto-prompt]

Example:

aws opensearch create-domain --domain-name my-domain --engine-version "OpenSearch_1.0" --node-to-node-encryption-options Enabled=true --encryption-at-rest-options Enabled=true --node-type data --instance-count 3 --master-node-type r5.large.elasticsearch --master-node-count 3 --ebs-options EBSEnabled=true,VolumeType=gp2,VolumeSize=10 --access-policies file://access-policy.json --advanced-security-options Enabled=true --tag-specifications "ResourceType=opensearch-domain,Tags=[{Key=Environment,Value=Development},{Key=Owner,Value=MyTeam}]"

aws cost
Costs
The cost of using OpenSearch depends on several factors, including the number of search queries, the amount of storage used for indexing data, and the amount of data stored in S3. For search queries, you are charged based on the number of queries made per month. The cost of search queries varies depending on the region you are using. For data storage for indexing, you are charged for the amount of data stored and the number of read and write requests made to the data. The cost of data storage varies depending on the region you are using. For data storage in S3, you are charged for the amount of data stored and the number of requests made to access the data. The cost of data storage varies depending on the region you are using.
Direct Cost

ESInstance:<Instance_Type>

ES:GP2-Storage

ES:GP3-Storage

DataTransfer-Regional-Bytes

AWS-Out-Bytes

Indirect Cost
No items found.
Best Practices for
OpenSearch

Categorized by Availability, Security & Compliance and Cost

Low
Access allowed from VPN
No items found.
Low
Auto Scaling Group not in use
No items found.
Medium
Connections towards DynamoDB should be via VPC endpoints
No items found.
Medium
Container in CrashLoopBackOff state
No items found.
Low
EC2 with GPU capabilities
No items found.
Medium
EC2 with high privileged policies
No items found.
Medium
ECS cluster delete alarm
No items found.
Critical
ECS task with Admin access (*:*)
Medium
ECS task with high privileged policies
No items found.
Critical
EKS cluster delete alarm
No items found.
Medium
ElastiCache cluster delete alarm
No items found.
Medium
Ensure Container liveness probe is configured
No items found.
Medium
Ensure ECS task definition has memory limit
No items found.
Critical
Ensure EMR cluster master nodes are not publicly accessible
No items found.
More from
Amazon Web Services