ansible: add role to send journald to CloudWatch (
bug 1364231); r?dividehex
https://github.com/saymedia/journald-cloudwatch-logs is a Go program
for directly monitoring journald and writing events to Amazon CloudWatch
in near real time. It seems to be the most popular of the "send journald
to CloudWatch" tools I could find. Its compelling advantage over other
"send to CloudWatch" tools in this space is that it watches journald
directly: other tools poll files on disk. Because it is watching the
journal directly, it has access to the raw, unified stream of all
events. Not only does this provider greater assurances of handling
all the events, but it also doesn't result in data loss: the CloudWatch
events are JSON and contain everything from the raw journal message.
This facilitates easier analysis, including creating CloudWatch filters
to react to specific events.
This commit adds an Ansible role for configuring this tool. Things
are pretty straightforward: we have a user/group for running the
process via systemd. A minimal config file specifies which CloudWatch
log group to write to. The log name is the EC2 instance name by default.
I'm fine with this, at least to begin with. The documentation on GitHub
describes other configuration options.
To prove it works, we hook it up to the vcssync deployment.
MozReview-Commit-ID: KfMC7KdgFLu
new file mode 100644
--- /dev/null
+++ b/ansible/roles/journald-cloudwatch/files/journald-cloudwatch-logs.service
@@ -0,0 +1,12 @@
+[Unit]
+Description=journald-cloudwatch-logs
+Wants=basic.target
+After=basic.target network.target
+
+[Service]
+User=journald-cloudwatch
+Group=journald-cloudwatch
+ExecStart=/usr/local/bin/journald-cloudwatch-logs /etc/journald-cloudwatch
+KillMode=process
+Restart=on-failure
+RestartSec=1s
new file mode 100644
--- /dev/null
+++ b/ansible/roles/journald-cloudwatch/tasks/main.yml
@@ -0,0 +1,49 @@
+---
+
+- name: create journald-cloudwatch group
+ group: name=journald-cloudwatch
+
+- name: create journald-cloudwatch user
+ user: name=journald-cloudwatch
+ group=journald-cloudwatch
+ groups=systemd-journal
+ shell=/bin/false
+ createhome=no
+ comment="Runs journald-cloudwatch service"
+
+- name: download journald-cloudwatch-logs
+ get_url: url=https://s3-us-west-2.amazonaws.com/moz-packages/CentOS7/journald-cloudwatch-logs-df8eb69f89f8
+ dest=/usr/local/bin/journald-cloudwatch-logs
+ sha256sum=2ac6b55fa9f1d1b4e1163f4526c4a396a0fa2146c00df029c439db6e5a94d863
+ owner=root
+ group=root
+ mode=0755
+
+- name: directory for journald-cloudwatch-logs state
+ file: path=/var/lib/journald-cloudwatch-logs
+ state=directory
+ owner=journald-cloudwatch
+ group=journald-cloudwatch
+ mode=0750
+
+- name: install journald-cloudwatch-logs config file
+ template: src=journald-cloudwatch.j2
+ dest=/etc/journald-cloudwatch
+ owner=root
+ group=root
+ mode=0644
+
+- name: install journald-cloudwatch-logs systemd unit
+ copy: src=journald-cloudwatch-logs.service
+ dest=/etc/systemd/system/journald-cloudwatch-logs.service
+ owner=root
+ group=root
+ mode=0644
+
+- name: reload systemd units
+ command: /usr/bin/systemctl daemon-reload
+
+- name: journald-cloudwatch-logs systemd service enabled and running
+ service: name=journald-cloudwatch-logs.service
+ enabled=yes
+ state=started
new file mode 100644
--- /dev/null
+++ b/ansible/roles/journald-cloudwatch/templates/journald-cloudwatch.j2
@@ -0,0 +1,2 @@
+log_group = "{{ journald_cloudwatch_group | mandatory }}"
+state_file = "/var/lib/journald-cloudwatch-logs/state"
--- a/ansible/vcssync-deploy.yml
+++ b/ansible/vcssync-deploy.yml
@@ -24,9 +24,10 @@
pulse_password: "{{ (secrets.stdout | from_yaml).pulse_password | mandatory }}"
servo_github_token: "{{ (secrets.stdout | from_yaml).servo_github_token | mandatory }}"
servo_github_ssh_key: "{{ (secrets.stdout | from_yaml).servo_github_ssh_key | mandatory }}"
servo_hgmo_ssh_key: "{{ (secrets.stdout | from_yaml).servo_hgmo_ssh_key | mandatory }}"
servo_hgmo_ssh_user: "{{ (secrets.stdout | from_yaml).servo_hgmo_ssh_user | mandatory }}"
roles:
- common
+ - { role: journald-cloudwatch, journald_cloudwatch_group: /vcssync }
- vcs-sync