Cross-Region Read Replicas

One of the questions that came up during my discussion with our data center admins was durability and availability of an RDS instance. I knew about Multi-AZ Read Replicas but Cross-Region was new to me.

With Amazon Relational Database Service (Amazon RDS), you can create a MySQL, PostgreSQL, or MariaDB Read Replica in a different AWS Region than the source DB instance. You create a Read Replica to do the following:

  • Improve your disaster recovery capabilities.
  • Scale read operations into a region closer to your users.
  • Make it easier to migrate from a data center in one region to a data center in another region.

I know that I have a lot to learn about cloud deployment so during a meeting with our internal data center admins I asked what they thought about managning releases with VPC deployments. They kindly explained that since some of the resource we would need would have to have a VPN connection to our data center creating new VPCs wouldn’t be ideal because each VPC would require reconfiguration of the corporate firewall. They suggested exploring deployment via subnets.

This seems to make sense, I think my problem was that I was coming from the thought of building everything out, but the operations people control the VPC so I need to rethink things with that in mind.

While doing my investigation into CloudFormation I ran into the problem of bootstrapping the environment versus upgrading it. My CloudFormation template had everything I needed to spin up the entire environment, a VPC, public and private subnets, bastion host, RDS instance in the private subnet, a DynamoDB instance, etc. While this would nicely spin up a brand new environment it didn’t address how to update an existing environment. My programmer’s mindset thought about conditional checks, but CloudFormation templates don’t have any such logic.

The approach I plan to investigate is to have a template that spins up the long-lived services like the RDS and DynamoDB instances into a VPC and then have another template that spins up a VPC for the data and event processing environment that will have continuous enhancements and fixes. The two VPCs can then communicate with VPC Peering. This setup allows for Blue/Green deployment and the ability to minimize downtime in the processing environment.

This is theoretical of course, but seems like a reasonable approach. Anybody have a similar challenge they solved in this way?

The past two years I have been struggling to build a system that would help me make progress. I failed over and over to outline and stick to a process. If I’m honest with myself, there are many reasons for this but the overarching one is that I don’t have any long-term, meaningful goals.

Oddly enough, I’m pretty confident in my purpose in life which is informed by my Christian faith and that is God has placed me where He has to love and serve my neighbors. This guides my immediate actions in that I know what to do in a particular moment. If my co-worker is struggling with a bug I help her as I can. If my daughter needs her homework checked I do that. If there are dishes in the sink I help my wife by washing them.

Where I stuggle is those free moments. The time when I could/should be working on the side-project, learning something new, or producing content that could help someone. I don’t know what I should be doing so I endup watching something on Netflix or playing some iOS game. Then later I’ll look back angrily and lament the time I’ve wasted. Occassionaly and stupidly, this frustration bleeds over into causing me to avoid the immediate and obvious items (washing dishes, etc).

Last year I read a bunch of books and listened to a ton of podcasts trying to find a direction or theme for what I’m doing to go beyond the basic yet important day-to-day activities that serve my family, friends, co-workers, and church and find some bigger goal(s) that integrates those activities and adds other meaningful ones. I can’t say that I was successful. In fact, it frustrated me so much that I’ve taken a break from direction and goal setting books and trimmed my podcast subscriptions to cut out shows that focus on those topics.

Where does that leave me? The title of the post. I’m not sure where it comes from; if lighting will strike, but I desire something to happen because this stagnation is not where I want to be.

Last week, as part of learning to process data was Lambda functions, I automated the Lambda deployment process by writing a Python script that creates an IAM role, attaches some policies, and deploys the Lambda function. I thought this was a huge improvement over doing this through the UI and a necessary step in making our product cloud native.

After some discussion about where we needed to go our team decided that while the script was a useful learning point, we needed to investigate a more comprehensive deployment technology and one of my teammates mentioned that during several conversations at re:Invent people cited CloudFormation as a deployment technology they wish they had started with from the beginning. So that’s my task this week: investigate CloudFormation and see how we can leverage it to deploy our product.