vhs · lf- · Apr 29, 2023 · Apr 29, 2023 · Apr 29, 2023 · Apr 29, 2023
diff --git a/README.md b/README.md
@@ -8,7 +8,103 @@ This system in a way acts as the rule set for how things are governed, via membe
 ## Development
 
 See here for complete setup, API, and philosophy:
-https://github.com/vhs/membership-manager-pro/wiki
+https://github.com/vhs/nomos/wiki
 
-First steps should be installing your test environment:
-https://github.com/vhs/membership-manager-pro/wiki/Contributing
+For the old development guide, see:
+https://github.com/vhs/nomos/wiki/Contributing
+
+### docker-compose
+
+#### Site configuration files
+
+Two configuration files are used, which will vary based on the installation:
+`docker/nomos.env`, and `docker-compose.conf`. These should be set up before
+running the `docker-compose` setup.
+
+There is a development configuration at `docker-compose.dev.conf`, which can be
+used for development.
+
+#### Usage
+
+- Copy `docker-compose.template.conf`, `docker-compose.sample.conf`, or `docker-compose.dev.conf` to `docker-compose.conf`
+- Edit/uncomment the relevant lines in `docker-compose.conf` to enable - or even add - specific functionality
+- Run `./docker-compose.sh` as a 1:1 wrapper for docker-compose, or generate a
+  local `docker-compose.yml` file for direct usage with `docker-compose` with
+  `./docker-compose.sh config > docker-compose.yml`
+
+#### Development setup guide
+
+On Linux, first install `docker` and `docker-compose` from your distribution
+package manager. On Mac/Windows, you probably want [Docker Desktop], which is a
+fancy app that makes and manages a Linux virtual machine that it runs Docker in.
+
+[Docker Desktop]: https://docs.docker.com/get-docker/
+
+Copy `docker/nomos.env.template` to `docker/nomos.env`.
+
+Copy `docker-compose.dev.conf` to `docker-compose.conf`.
+
+Grant write permission to all users on the log directory: `chmod a+w logs`. The
+reason this is needed is because the back-end PHP code runs as a non-root user
+inside the container, and by default permissions don't grant write access to
+non-owners of directories.
+
+Start the service with `./docker-compose.sh up`. This should bring everything up,
+but the webhook service will still be failing, which is expected.
+
+To get the webhook service working, run `tools/make-webhook-key.sh`, which will
+provide the correct value of `NOMOS_RABBITMQ_NOMOS_TOKEN`. Then, edit that into
+`docker/nomos.env`.
+
+Once you have done this, press Ctrl-C in the terminal with `./docker-compose.sh up`,
+then run `./docker-compose.sh up` again.
+
+You're all set! You can get the address to access the Nomos service locally by
+running the following in a separate terminal as `docker-compose.sh`:
+
+```
+$ docker inspect nomos-frontend | jq -r '.[0].NetworkSettings.Networks | to_entries | .[0].value.IPAddress'
+```
+
+The username is `vhs` and the password is `password`.
+
+### OpenTelemetry
+
+Nomos optionally supports [OpenTelemetry](https://opentelemetry.io/docs/), a
+system for collecting traces, logs, and metrics (we have not yet implemented
+metrics and logs) from production and dev systems to greatly ease debugging and
+performance work. It supports distributed tracing (we have not implemented this
+yet), allowing tracing operations through their whole lifecycle through multiple
+systems.
+
+One feature we've implemented is the `vhs-trace-link` HTTP header, which allows
+you to look at dev tools in your browser, then click the link to open the trace
+for any problematic request and immediately see all the database queries
+performed by that request.
+
+To set it up, you need to choose some kind of aggregation service:
+- [Jaeger](https://jaegertracing.io/) is an open source trace aggregation
+  service that you can self host.
+- [Honeycomb](https://honeycomb.io/) is a proprietary cloud tracing service
+  which does metrics, traces, and logs, as well as dashboards and alerting.
+  They have a generous free tier.
+
+Once you've chosen and set up one of these services, enable
+[opentelemetry-collector.yml](./docker-compose/opentelemetry-collector.yml) in
+`docker-compose.conf`. Next, configure `docker/nomos.env`: set
+`OT_COLLECTOR_OTLP_UPSTREAM` to point to your aggregation service's ingest
+endpoint and optionally set `OT_COLLECTOR_HONEYCOMB_API_KEY` if using Honeycomb
+(make it blank otherwise).
+
+Then, optionally set up the `vhs-trace-link` system by setting
+`NOMOS_TRACE_URL_FORMAT` to the appropriate value for your trace aggregator.
+There is an example given for Honeycomb.
+
+You should start seeing traces in your trace aggregator of choice when you
+start the service and make some backend requests.
+
+#### Caveats
+
+Currently the database traces show full statements because the Nomos database
+library escapes strings instead of using placeholders. This may expose PII if
+used in production. Fixing this is planned.
diff --git a/app/app.php b/app/app.php
@@ -18,6 +18,8 @@
 
 $serverLog = new \vhs\loggers\FileLogger(dirname(__FILE__) . "/../logs/server.log");
 
+\vhs\observability\OpenTelemetry::Init($serverLog);
+
 \vhs\web\HttpContext::Init(new \vhs\web\HttpServer(new \vhs\web\modules\HttpServerInfoModule("Nomos"), $serverLog));
 
 \vhs\web\HttpContext::Server()->register(new \app\security\HttpApiAuthModule(\app\security\Authenticate::getInstance()));
@@ -27,6 +29,7 @@
 \vhs\web\HttpContext::Server()->register(\app\modules\HttpPaymentGatewayHandlerModule::getInstance());
 \vhs\web\HttpContext::Server()->register(\app\security\oauth\modules\OAuthHandlerModule::getInstance());
 \vhs\web\HttpContext::Server()->register(new \vhs\web\modules\HttpJsonServiceHandlerModule("web"));
+\vhs\web\HttpContext::Server()->register(new \vhs\web\modules\HttpTraceLinkModule());
 
 
 \app\modules\HttpPaymentGatewayHandlerModule::register(new \app\gateways\PaypalGateway());
@@ -40,4 +43,21 @@
 \app\security\oauth\modules\OAuthHandlerModule::register(new \app\security\oauth\modules\GoogleOAuthHandler());
 \app\security\oauth\modules\OAuthHandlerModule::register(new \app\security\oauth\modules\SlackOAuthHandler());
 
-\vhs\web\HttpContext::Server()->handle();
+try {
+    \vhs\web\HttpContext::Server()->handle();
+} catch (\vhs\RequestFinished $e) {
+} finally {
+    /*
+     * This is here to fix a problem where exit-handlers need to finish dealing
+     * with a request e.g. by sending traces to an OpenTelemetry service, and
+     * that may take between a few ms and a while, which we should never make
+     * the client wait for.
+     *
+     * https://stackoverflow.com/questions/15273570/continue-processing-php-after-sending-http-response
+     */
+    session_write_close();
+    fastcgi_finish_request();
+    // Ensure that the root span of the server is always closed cleanly.
+    \vhs\web\HttpContext::Server()->endRootSpan();
+    exit();
+}
diff --git a/composer.json b/composer.json
@@ -3,9 +3,25 @@
     "aws/aws-sdk-php": "2.*",
     "nicmart/string-template": "~0.1",
     "league/oauth2-client": "0.10.*",
-    "php-amqplib/php-amqplib": "2.5.*"
+    "php-amqplib/php-amqplib": "2.5.*",
+    "open-telemetry/api": "^0.0.17",
+    "open-telemetry/sdk": "^0.0.17",
+    "symfony/http-client": "^5.4",
+    "nyholm/psr7": "^1.8",
+    "guzzlehttp/promises": "^1.5",
+    "php-http/message-factory": "^1.1",
+    "open-telemetry/exporter-otlp": "^0.0.17",
+    "psr/http-client": "^1.0"
   },
   "require-dev": {
     "phpunit/phpunit": "^6"
+  },
+  "config": {
+    "platform": {
+      "php": "7.4.15"
+    },
+    "allow-plugins": {
+      "php-http/discovery": true
+    }
   }
 }
diff --git a/conf/config.ini.php.docker b/conf/config.ini.php.docker
@@ -39,6 +39,14 @@ require_once('env.php');
      define('STRIPE_WEBHOOK_SECRET', NOMOS_STRIPE_WEBHOOK_SECRET);
      define('STRIPE_PRODUCTS', json_decode(NOMOS_STRIPE_PRODUCTS, true));
 
+     // Format for URLs to link to traces (see HttpTraceLinkModule)
+     // For Honeycomb, see: https://docs.honeycomb.io/api/direct-trace-links/
+     //    https://ui.honeycomb.io/<team>/environments/<environment>/datasets/nomos/trace?trace_id=%TRACE_ID%&trace_start_ts=%TRACE_START_TS%
+     // The following parameters will be replaced:
+     // %TRACE_ID% - trace ID in use
+     // %TRACE_START_TS% - timestamp of the start of the root span
+     define('TRACE_URL_FORMAT', defined('NOMOS_TRACE_URL_FORMAT') ? NOMOS_TRACE_URL_FORMAT : '(trace url format not configured)');
+
 	/** 
 	* Show MySql Errors. 
 	* Not recomended for live site. true/false 

diff --git a/conf/config.ini.php.template b/conf/config.ini.php.template
@@ -35,6 +35,14 @@
      define('RABBITMQ_PASSWORD', 'password');
      define('RABBITMQ_VHOST', 'nomos');
 
+     // Format for URLs to link to traces (see HttpTraceLinkModule)
+     // For Honeycomb, see: https://docs.honeycomb.io/api/direct-trace-links/
+     //    https://ui.honeycomb.io/<team>/environments/<environment>/datasets/nomos/trace?trace_id=%TRACE_ID%&trace_start_ts=%TRACE_START_TS%
+     // The following parameters will be replaced:
+     // %TRACE_ID% - trace ID in use
+     // %TRACE_START_TS% - timestamp of the start of the root span
+     define('TRACE_URL_FORMAT', '(trace url format not configured)');
+
 	/** 
 	* Show MySql Errors. 
 	* Not recomended for live site. true/false 

diff --git a/conf/otel-collector-config.yaml b/conf/otel-collector-config.yaml
@@ -0,0 +1,25 @@
+receivers:
+  otlp:
+    protocols:
+      grpc: # port 4317
+      http: # port 4318
+
+processors:
+  batch:
+
+exporters:
+  otlp:
+    endpoint: "${env:OTLP_UPSTREAM}"
+    headers:
+      "x-honeycomb-team": "${env:HONEYCOMB_API_KEY}"
+
+service:
+  pipelines:
+    traces:
+      receivers: [otlp]
+      processors: [batch]
+      exporters: [otlp]
+    logs:
+      receivers: [otlp]
+      processors: [batch]
+      exporters: [otlp]
diff --git a/docker-compose.dev.conf b/docker-compose.dev.conf
@@ -0,0 +1,167 @@
+###################################################################################################
+##                                                                                               ##
+## Self-contained development configuration using docker-compose.                                ##
+## This configuration provides an internal mysql and RabbitMQ server.                            ##
+##                                                                                               ##
+###################################################################################################
+
+## Always use the core
+docker-compose/core.yml
+
+##
+## Expose port
+##
+# docker-compose/core.ports.yml
+
+##
+## Lift environments from nomos.env file
+##
+
+docker-compose/files-local-nomos-env.yml
+
+##
+## Build core
+##
+
+docker-compose/build-backend.yml
+docker-compose/build-frontend.yml
+
+##
+## Enable bridge network_mode for core services
+##
+#docker-compose/core.network-bridge.yml
+
+##
+## Enable proxy network for core services
+##
+#docker-compose/core.network-proxy.yml
+
+##
+## Enable proxy network for core services, managed by docker-compose
+##
+docker-compose/core.network-proxy-internal.yml
+
+##
+## MySQL
+##
+
+##
+## Local MySQL instance
+##
+
+docker-compose/mysql-local.yml
+#docker-compose/mysql-local-network-bridge.yml
+docker-compose/mysql-local-network-mysql.yml
+
+##
+## External MySQL instance
+##
+
+# docker-compose/mysql-external-mysqld.yml
+
+##
+## External MySQL network (mysql)
+##
+
+# docker-compose/mysql-external-network-mysql.yml
+
+##
+## Inject the MySQL container through EXTERNAL_MYSQL_HOST
+##
+
+# docker-compose/mysql-external-variable.yml
+
+##
+## Webhooker
+##
+## Webhooker depends on RabbitMQ, so always enable a RabbitMQ
+##
+
+docker-compose/webhooker.yml
+
+##
+## Webhooker logs
+##
+
+docker-compose/webhooker-logs-local.yml
+
+##
+## Build webhooker
+##
+
+docker-compose/webhooker-build.yml
+
+##
+## Enable bridge network_mode for webhooker
+##
+
+# docker-compose/webhooker-network-bridge.yml
+
+##
+## Enable proxy network for webhooker
+##
+
+# docker-compose/webhooker-network-proxy.yml
+
+##
+## Enable rabbitmq network for webhooker
+##
+
+# docker-compose/webhooker-network-rabbitmq.yml
+
+##
+## Local RabbitMQ instance
+##
+
+docker-compose/rabbitmq-local.yml
+docker-compose/rabbitmq-local-management.yml
+# docker-compose/rabbitmq-local-network-bridge.yml
+
+# Network not managed with docker-compose
+#docker-compose/rabbitmq-local-network-rabbitmq.yml
+
+# Network managed with docker-compose
+docker-compose/rabbitmq-local-network-internal.yml
+
+##
+## External RabbitMQ instance
+##
+
+# docker-compose/rabbitmq-external.yml
+
+##
+## OpenTelemetry collector process (realistically required for using
+## OpenTelemetry)
+##
+
+# docker-compose/opentelemetry-collector.yml
+
+##
+## Vhosts
+##
+## TODO: Example Traefik config
+##
+
+##
+## Enable membership.vanhack.ca for nginx-proxy
+##
+
+# docker-compose/vhost-membership-vanhack-ca.yml
+
+##
+## Enable membership.test.vanhack.ca for nginx-proxy
+##
+
+# docker-compose/vhost-membership-test-vanhack-ca.yml
+
+##
+## Override backend files from local filesystem
+##
+
+docker-compose/files-local-backend.yml
+
+##
+## Override frontend files from local filesystem
+##
+
+docker-compose/files-local-frontend.yml