From 5976bc736ba6ebce721a7866ba612dfa1794a049 Mon Sep 17 00:00:00 2001 From: David Thor Date: Mon, 27 Mar 2023 11:55:27 -0400 Subject: [PATCH 1/4] Added full-stack app guide --- architecture/full-stack-apps.mdx | 251 +++++++++++++++++++++++++++++++ backend/architect.yml | 25 +++ mint.json | 6 + 3 files changed, 282 insertions(+) create mode 100644 architecture/full-stack-apps.mdx create mode 100644 backend/architect.yml diff --git a/architecture/full-stack-apps.mdx b/architecture/full-stack-apps.mdx new file mode 100644 index 0000000..0923437 --- /dev/null +++ b/architecture/full-stack-apps.mdx @@ -0,0 +1,251 @@ +--- +title: Creating full-stack apps +sidebarTitle: Full-stack apps +--- + +Full-stack applications are the bread and butter of Architect. Our component framework has been +designed specifically to make it as easy as possible to package up modern applications into +portable, deployable units. + +In this guide, you'll learn how to create and integrate two components: one for your frontend +webapp, and another for your backend API and its database. + +## The backend + +Even though users will primarily work with the frontend application, we're going to begin by describing +the backend API. The frontend needs to connect to the backend to run correctly, so we'll need the +backend to exist before the frontend can be completed. + +To kick things off, let's create a new folder to house our project and put an empty architect.yml file in +it: + +```sh +$ mkdir ./backend +$ touch ./backend/architect.yml +``` + +Next, let's open up `./backend/architect.yml` and give the component a name, description, and possibly +some keywords if I want my team to find the application more easily: + +```yml backend/architect.yml +name: backend +description: My apps API backend +keywords: + - backend + - my-app + - REST +``` + +### The database + +Now that we have the skeleton for our backend component, let's add our first cloud resource - a database! +Databases are a key building block for any full-stack application. Without the ability to persist data, +your app will be extremely limited in what features it can deliver for users. + +Adding a database is really easy with Architect. Let's open up that `backend/architect.yml` file again +and add the following: + +```yaml backend/architect.yml +databases: + main: + type: postgres:13 + description: Stores user data +``` + +The above will ensure that a new database is created for your component that uses `postgres` version `13`. +It also includes a small description of what the database is used for so that production engineering teams +have some visibility into what the cloud resources in the environment are used for. + +[Learn more about Architect's support for databases](/components/databases) + +### The API + +Now that we've created our `main` database, let's go ahead and add our API service. In this example we'll +be using a docker image our team created in advance, but you can check out the [create a component](/guides/create-a-component) +guide in our docs to learn more about building services from your own source code. + +```yaml backend/architect.yml +services: + api: + image: registry.gitlab.com/architect-io/docker-files/sample-backend:latest + interfaces: + main: + port: 8080 + ingress: + subdomain: api + environment: + PORT: 8080 + DB_ADDR: ${{ databases.main.url }} + DB_USER: ${{ databases.main.username }} + DB_PASS: ${{ databases.main.password }} +``` + +The `api` service above has a few key elements: 1) The reference to the Docker image powering the service, +2) the `main` interface the service listens for traffic on, and 3) the `environment` variables injected +into the service at deploy-time. + +Take special note of the `DB_ADDR`, `DB_USER`, and `DB_PASS` values in the environment variables. You'll +notice that they use Architect's [expression syntax](/components/architect-yml) to automatically fill the +values with our database address and credentials every time we deploy. This helps reduce the amount of +configuration required dramatically. + +### Running the backend + +Great! Your backend component is complete! Before we move on to the frontend, lets quickly test it out by +running it locally: + +```sh +$ architect dev ./backend/ +``` + +The command above will detect the `architect.yml` file in the backend directly and spin up a fresh environment +containing our new backend component. You'll also see in the logs that our API service is exposed at +`https://api.localhost.architect.sh/`. If you see an empty json array when you run +`curl https://api.localhost.architect.sh/items` then you know its alive and running! + +### Registering the backend + +Now that we know the backend component works, we'll need to publish it to Architect's cloud registry so that it +can be found when other components cite it as a dependency. + +```sh +$ architect register ./backend/ --account my-account +``` + +The command above will publish the backend component to the `my-account` account in Architect's registry. You'll +be able to see it by going to https://cloud.architect.io/my-account/components. + +Don't want to sign up for Architect just yet? No problem. You can simulate the way the registry works for local +deployments using the `architect link` command as follows: + +```sh +$ architect link ./backend +``` + +The command above will ensure that every time you run a component locally that depends on the `backend`, it will +find the dependency on the local filesystem without having to call out to the registry. + +## The frontend + +Finally, we're ready to create our frontend! Let's create another sibling folder to the `backend` we created earlier +and put another `architect.yml` file inside: + +```sh +$ mkdir ./frontend +$ touch ./frontend/architect.yml +``` + +Next, we'll add similar metadata to give our frontend and name and description: + +```yml frontend/architect.yml +name: frontend +description: My apps frontend webapp +keywords: + - frontend + - my-app + - REST +``` + +### Using dependencies + +Before we introduce our frontend webapp to the new component, let's cite the `backend` as a dependency. +In doing so, we guarantee that the backend will always exist every time we deploy the frontend. + +```yaml frontend/architect.yml +dependencies: + backend: latest +``` + +Guaranteeing the existance of the backend isn't everything we can do though. We can also inject the service +addresses from the backend component into our frontend's environment variables. This automates configuration +for us the same way we did in the backend when connecting to the database. + +```yaml frontend/architect.yml +services: + frontend: + image: registry.gitlab.com/architect-io/docker-files/sample-frontend:latest + interfaces: + main: + port: 3000 + ingress: + subdomain: app + environment: + REACT_APP_API_ADDR: ${{ dependencies.backend.services.api.interfaces.main.ingress.url }} +``` + +Take a look at the `REACT_APP_API_ADDR` and notice how we referred to the `api` service inside the `backend` +component using Architect's [expression syntax](/components/architect-yml) again. Specifically, we referred +to the `ingress` URL of the `main` interface which will inject a publicly accessible address. This is important +so that our react app, which runs in the browser, can connect to the API from outside the cloud environment. + +### Running the frontend + +Great! Now you're frontend component is complete as well! Let's try running it to make sure it works. + +```sh +$ architect dev ./backend/ +``` + +The command above will detect the `architect.yml` file in the frontend directly and spin up a fresh environment +containing our new frontend component AND the backend component from the registry that we created earlier. +You'll also see in the logs that our frontend service is exposed at `https://app.localhost.architect.sh/`. Once +its ready, the app will open in the browser automatically. + +## Putting it all together + +Congratulations! You've successfully created and run your first full-stack web app using Architect. If you come +back to this guide as a reference point, you can easily see the two component specs for your frontend and backend +below: + + +```yml frontend/architect.yml +name: frontend +description: My apps frontend webapp +keywords: + - frontend + - my-app + - REST + +dependencies: + backend: latest + +services: + frontend: + image: registry.gitlab.com/architect-io/docker-files/sample-frontend:latest + interfaces: + main: + port: 3000 + ingress: + subdomain: app + environment: + REACT_APP_API_ADDR: ${{ dependencies.backend.services.api.interfaces.main.ingress.url }} +``` + +```yml backend/architect.yml +name: backend +description: My apps API backend +keywords: + - backend + - my-app + - REST + +databases: + main: + type: postgres:13 + description: Stores user data + +services: + api: + image: registry.gitlab.com/architect-io/docker-files/sample-backend:latest + interfaces: + main: + port: 8080 + ingress: + subdomain: api + environment: + PORT: 8080 + DB_ADDR: ${{ databases.main.url }} + DB_USER: ${{ databases.main.username }} + DB_PASS: ${{ databases.main.password }} +``` + \ No newline at end of file diff --git a/backend/architect.yml b/backend/architect.yml new file mode 100644 index 0000000..a6134d0 --- /dev/null +++ b/backend/architect.yml @@ -0,0 +1,25 @@ +name: backend +description: My apps API backend +keywords: + - backend + - my-app + - REST + +databases: + main: + type: postgres:13 + description: Stores user data + +services: + api: + image: registry.gitlab.com/architect-io/docker-files/sample-backend:latest + interfaces: + main: + port: 8080 + ingress: + subdomain: api + environment: + PORT: 8080 + DB_ADDR: ${{ databases.main.url }} + DB_USER: ${{ databases.main.username }} + DB_PASS: ${{ databases.main.password }} \ No newline at end of file diff --git a/mint.json b/mint.json index 13598f8..ea9c1f0 100644 --- a/mint.json +++ b/mint.json @@ -58,6 +58,12 @@ "reference/templates" ] }, + { + "group": "Architecture", + "pages": [ + "architecture/full-stack-apps" + ] + }, { "group": "Guides", "pages": [ From e4afd7d8b6142b808a86af681390a253d379d434 Mon Sep 17 00:00:00 2001 From: David Thor Date: Mon, 27 Mar 2023 11:56:15 -0400 Subject: [PATCH 2/4] Removed backend example accidentally committed --- backend/architect.yml | 25 ------------------------- 1 file changed, 25 deletions(-) delete mode 100644 backend/architect.yml diff --git a/backend/architect.yml b/backend/architect.yml deleted file mode 100644 index a6134d0..0000000 --- a/backend/architect.yml +++ /dev/null @@ -1,25 +0,0 @@ -name: backend -description: My apps API backend -keywords: - - backend - - my-app - - REST - -databases: - main: - type: postgres:13 - description: Stores user data - -services: - api: - image: registry.gitlab.com/architect-io/docker-files/sample-backend:latest - interfaces: - main: - port: 8080 - ingress: - subdomain: api - environment: - PORT: 8080 - DB_ADDR: ${{ databases.main.url }} - DB_USER: ${{ databases.main.username }} - DB_PASS: ${{ databases.main.password }} \ No newline at end of file From 7cd1cbab8766b64857e3b335ed5701bd2015e178 Mon Sep 17 00:00:00 2001 From: David Thor Date: Mon, 27 Mar 2023 15:26:03 -0400 Subject: [PATCH 3/4] Added event driven architecture guide --- architecture/event-driven-architecture.mdx | 201 +++++++++++++++++++++ 1 file changed, 201 insertions(+) create mode 100644 architecture/event-driven-architecture.mdx diff --git a/architecture/event-driven-architecture.mdx b/architecture/event-driven-architecture.mdx new file mode 100644 index 0000000..c556d9d --- /dev/null +++ b/architecture/event-driven-architecture.mdx @@ -0,0 +1,201 @@ +--- +title: Event-driven architecture +--- + +Event-driven architecture is a powerful design pattern that allows cloud applications to make +calls to one another asyncronously and without coupling. This has been a pattern rapidly adopted +by microservice applications to increase resiliance of inter-process communication. + + + Check out this great write-up on event-driven architecture from Architect's founder + + +Integrations may be more resiliant with event-driven architecture compared to syncronous API calls, +but that doesn't mean your application doesn't have dependencies. If you subscribe to an event you +can safely run your application even if the publisher doesn't exist, but why would you want to? Your +application would be listening, but nothing would be talking to it. It'd just be burning compute. + +By using Architect's dependency management features, developers can still take advantage of the +asyncronous nature of event-driven architecture, but are empowered to create end-to-end environments +that automatically deploy the event-publishers to aid in testing. + +## Brokers + +The first part of any event-driven application is a broker - a database or persistence layer that +oftentimes includes features to mediate the relationship between event publishers and subscribers. +A few examples include [RabbitMQ](https://www.rabbitmq.com/), [Celery](https://docs.celeryq.dev/), +[Kafka](https://kafka.apache.org/), or managed service providers like [GCPs PubSub](https://cloud.google.com/pubsub) +or AWSs [SNS](https://aws.amazon.com/sns/)/[SQS](https://aws.amazon.com/sqs/). + +In this guide, we'll be using GCP PubSub. The first step is to define our pubsub Architect Component. The GCP +team has gone above and beyond to help developers test their integrations by providing local emulators for +pub/sub. This component uses the emulator to make it easier to test the integration without needing a cloud account. + +```yaml architect.yml +name: gcp-pubsub + +secrets: + gcp_project_id: + description: Name of the GCP project to connect to + default: test + service_endpoint: + description: If you want to connect to the remote pub/sub service, use the value 'pubsub.googleapis.com'. Otherwise, the local emulator will be used. + required: false + service_protocol: + description: If you want to connect to the remote pub/sub service, use the value 'https'. Otherwise, the local emulator will be used. + default: http + service_port: + description: If you want to connect to the remote pub/sub service, use the value '443'. Otherwise, the local emulator will be used. + default: 8085 + +outputs: + gcp_project_id: + description: Project ID the pubsub is configured in + value: ${{ secrets.gcp_project_id }} + +services: + pubsub: + image: gcr.io/google.com/cloudsdktool/cloud-sdk:emulators + command: gcloud beta emulators pubsub start --host-port=0.0.0.0:${{ secrets.service_port }} --project=${{ secrets.gcp_project_id }} + interfaces: + main: + host: ${{ secrets.service_endpoint }} + protocol: ${{ secrets.service_protocol }} + port: ${{ secrets.service_port }} +``` + +There are three core resources in use in the component describe above: `secrets`, `outputs`, and `services`. +If you've [created a component](/guides/create-a-component) before, you'll already be familiar with [services](/components/services). +But `secrets` represent inputs that can be passed into components at deploy time uniquely for each environment. +They allow your application to behave differently or use different credentials in each environment. In this case +they're used to configure the GCP project ID where the event topic and subscribers will be hosted when deploying +to the cloud. + +The outputs on the other hand are helpful for broadcasting values to upstream consumers that use this component +as a dependency. By using the outputs from the broker instead of separate inputs, you can guarantee that all +the components in your environment use the same project ID. + +Now that we know what we're looking at, lets setup the directory to house this broker component: + +```sh +$ mkdir ./broker +$ touch ./broker/architect.yml +``` + +Now just update the contents of `./broker/architect.yml` to match the contents above, and then we can +register it with Archtitect's cloud registry: + +```sh +$ architect register ./pubsub --account my-account +``` + +## Publishers + +Next up is an event publisher - a service that will write events to our broker for others to +subscribe to. The only thing a publisher depends on is the broker itself, and doesn't really +care who the subscribers are. + +```yaml publisher/architect.yml +name: publisher + +secrets: + publish_topic: + description: Topic name the service will publish events to + default: topic + +outputs: + publish_topic: + description: Topic name events will be published to + value: ${{ secrets.publish_topic }} + +dependencies: + gcp-pubsub: latest + +services: + api: + build: + context: ./ + environment: + GCP_PROJECT_ID: ${{ dependencies['gcp-pubsub'].outputs.gcp_project_id }} + PUBSUB_ENDPOINT: ${{ dependencies['gcp-pubsub'].services.pubsub.interfaces.main.url }} + PUBSUB_TOPIC: ${{ secrets.publish_topic }} +``` + +The component above shows how event publishers can coordinate with upstream subscribers +by advertising the event topic using architect `outputs`. This will allow subscribers +to get the correct topic name without hardcoding to something that the publisher may want +to change later. + +Let's go ahead and create a new directory to house the publisher component: + +```sh +$ mkdir ./publisher +$ touch ./publisher/architect.yml +``` + +Then we can simply register the new component with Architect's cloud registry: + +```sh +$ architect register ./publisher/architect.yml --tag latest --account my-account +``` + +## Subscribers + +Finally, we're ready to setup our subscriber. Even though we're not making a call to the +event publisher, we still want to cite it as a dependency because our application won't +do anything of substance with those events being published. + +```yaml architect.yml +name: subscriber + +secrets: + subscription_key: + description: Key used to define this subscriber + default: subscriber + +dependencies: + gcp-pubsub: latest + publisher: latest + +services: + api: + build: + context: ./ + environment: + PUBSUB_ENDPOINT: ${{ dependencies['gcp-pubsub'].services.pubsub.interfaces.main.url }} + GCP_PROJECT_ID: ${{ dependencies['gcp-pubsub'].outputs.gcp_project_id }} + PUBSUB_TOPIC: ${{ dependencies['publisher'].outputs.publish_topic }} + PUBSUB_SUBSCRIPTION: ${{ secrets.subscription_key }} +``` + +Notice how the subscribe depends on both the `gcp-pubsub` component and the `publisher` component. +PubSub is used as the broker so that the service can read the events, but the topic name is +collected from the publisher so that we don't hardcode to something that may change. + +Let's create one more folder to house this subscriber component: + +```sh +$ mkdir ./subscriber +$ touch ./subscriber/architect.yml +``` + +The other two components were dependencies of this one which is why they had to be registered with Architect, +but this subscriber we can run directly: + +```sh +$ architect dev ./subscriber + +# ... + +subscriber--api-1 | 0s +subscriber--api-1 | 1s +subscriber--api-1 | 2s +subscriber--api-1 | 3s +subscriber--api-1 | 4s +subscriber--api-1 | 5s +# ... +``` + +Congratulations! You've written your first event-driven application using Architect! Check out some of our +other architectural guides to learn other cool design patterns! + From 27fdb7e39c610ca05a1b295414bf8cf71362a9b5 Mon Sep 17 00:00:00 2001 From: David Thor Date: Mon, 27 Mar 2023 15:30:00 -0400 Subject: [PATCH 4/4] Added event-driven architecture to guides --- mint.json | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mint.json b/mint.json index ea9c1f0..f241b1f 100644 --- a/mint.json +++ b/mint.json @@ -61,7 +61,8 @@ { "group": "Architecture", "pages": [ - "architecture/full-stack-apps" + "architecture/full-stack-apps", + "architecture/event-driven-architecture" ] }, {