star-rocks-module

A basic setup of a StarRocks cluster using an S3 backend on AWS EC2. You can scale up compute nodes as needed and they will join the front end servers. New front end servers need to join the cluster manually. New instances are automatically added to Grafana and Prometheus, although deleted instances are not removed from Prometheus for a while.

This currently only deploys an internal NLB to a single subnet, uses private DNS, and only allows traffic from within the VPC to encourage security by default. It does not create an authenticated mySQL user, instead only using creating the user root with all privileges and no password.

You can connect to your Star Rocks instance by using the SSM proxy command:

aws ssm start-session --debug --target "$INSTANCE_ID" \
  --document-name AWS-StartPortForwardingSession \
  --parameters "{\"portNumber\":[\"$PORT\"],\"localPortNumber\":[\"$PORT\"]}"

where the default for PORT is 9030. After that, you can access Star Rocks using a mysql client:

mysql -h 127.0.0.1 -P 9030 -u root

Upgrade Process

We do a modified canary process for upgrades.

1. Ensure the SSM parameter for the leader IP is up to date.
2. Change the value for starrocks_upgrade_version in variables.tfvars for your environment. Create a PR, get reviews, and merge.
3. This will add 2 new nodes to the cluster, both upgraded. They will join the cluster and syncronize their data.
4. Make sure the new FE nodes join the old cleanly. You can query SHOW FRONTENDS; on the old FE.
5. Verify new CNs joined the old FE correctly.
6. Verify new FEs have data in their databases.
7. Make a new PR with starrocks_version matching the upgraded version, get reviews, and merge.
8. This will delete the old leader, forcing a leader election. Watch for the new leader and update the SSM parameter accordingly.
9. Depending on where in the new instance initialization you updated the SSM parameter, you may need to restart the systemd service on the new instance.
10. Go through the database and delete any terminated CNs and FE nodes.
11. You may now scale down the FEs to 1 if desired. If you delete FEs, be sure to remove them from the MySQL database first (ALTER SYSTEM DROP FOLLOWER '{IP}:9010') before terminating the instances.

Requirements

No requirements.

Providers

Name	Version
aws	n/a
random	n/a

Modules

No modules.

Resources

Name	Type
aws_dlm_lifecycle_policy.ebs_snapshots	resource
aws_iam_instance_profile.monitoring_instance_profile	resource
aws_iam_instance_profile.star_rocks_instance_profile	resource
aws_iam_policy.monitoring_policy	resource
aws_iam_policy.s3_policy	resource
aws_iam_role.dlm_role	resource
aws_iam_role.monitoring_role	resource
aws_iam_role.star_rocks_role	resource
aws_iam_role_policy_attachment.dlm_attach	resource
aws_iam_role_policy_attachment.monitoring_attach	resource
aws_iam_role_policy_attachment.monitoring_ssm_attach	resource
aws_iam_role_policy_attachment.s3_attach	resource
aws_iam_role_policy_attachment.ssm_attach	resource
aws_instance.star_rocks_compute_nodes	resource
aws_instance.star_rocks_frontend	resource
aws_instance.star_rocks_grafana	resource
aws_instance.star_rocks_prometheus	resource
aws_lb.star_rocks_nlb	resource
aws_lb_listener.frontend_listener	resource
aws_lb_target_group.frontend_tg	resource
aws_lb_target_group_attachment.frontend_attachment	resource
aws_route53_record.private_star_rocks_dns	resource
aws_s3_object.overview_dashboard	resource
aws_secretsmanager_secret.grafana_admin_pw	resource
aws_secretsmanager_secret_version.grafana_admin_pw_version	resource
aws_security_group.grafana_sg	resource
aws_security_group.prometheus_sg	resource
aws_security_group.star_rocks_sg	resource
random_password.initial_password	resource
aws_ami.al2023	data source
aws_caller_identity.current	data source
aws_kms_key.ebs_kms_key	data source
aws_route53_zone.private_dns_zone	data source
aws_s3_bucket.star_rocks_bucket	data source
aws_vpc.target_vpc	data source

Inputs

Name	Description	Type	Default	Required
cn_volume_size_gb	n/a	string	"950"	no
compute_node_heap_size	n/a	string	"124000"	no
compute_node_instance_count	n/a	string	"3"	no
compute_node_instance_type	n/a	string	"r6id.4xlarge"	no
domain_name	n/a	any	n/a	yes
environment	n/a	any	n/a	yes
frontend_heap_size	n/a	string	"28000"	no
frontend_instance_count	n/a	string	"1"	no
frontend_instance_type	n/a	string	"m6i.2xlarge"	no
frontend_volume_size_gb	n/a	string	"150"	no
monitoring_instance_type	n/a	string	"m6i.large"	no
project	n/a	string	"star-rocks"	no
region	n/a	string	"us-east-1"	no
root_volume_size_gb	n/a	string	"30"	no
ssh_key_name	n/a	string	"devops"	no
star_rocks_version	n/a	string	"3.3.11"	no
starrocks_bucket	n/a	any	n/a	yes
starrocks_data_path	n/a	string	"/opt/starrocks/"	no
subnet_id	n/a	any	n/a	yes
vpc_id	n/a	any	n/a	yes

Outputs

Name	Description
fe_dns_name	n/a
grafana_address	n/a