Troubleshooting an AWS SES Link Rewriting Issue

What Happened

Earlier today, some teachers were unable to activate their accounts because Amazon SES rewrote the HTTPS activation links generated by our teacher application into insecure HTTP tracking links (awstrack.me). This mixed-content redirect was blocked by modern browsers, leading to blank or error pages when click the link.

Account level enabled “open” and "click" tracking in SES silently turned on link rewriting, and—because no secure, custom redirect domain was configured—the service defaulted to an insecure generic domain.

I had to disable engagement tracking in Virtual Deliverability Manager for a quick fix, moving forward, we will configure a branded HTTPS redirect domain in a dedicated SES configuration set before re-enabling tracking.


Investigation

  1. Initial Report — Several teachers reported that the activation link in their email was not working.
  2. Verifying the Application — The teacher application generated a correct, functional link: https://xxxx.nsw.edu.au/…, confirming the application itself was not at fault.
  3. Analyzing the Email — In the email received, the link had been rewritten to http://cs8fdnj.r.ap-southeast-2.awstrack.me/….
  4. Identifying the Change — awstrack.me is Amazon SES’s default click-tracking domain, used to monitor link clicks.
  5. Pinpointing the Configuration — In configuration set xxxx-ses-config-set, “open” tracking was not enabled, which should not automatically activated link rewriting.

[Image Placeholder 01: Introduction Graphic]


Root Cause Analysis

  • Immediate Cause: Amazon SES automatically rewrote links in the invitation email which proved when saw awstrack.me.
  • Contributing Cause: The rewritten link was insecure (http://) while the destination was secure (https://), triggering mixed-content blocks.
  • Underlying Cause: Engagement tracking was enabled in the account level under AWS SES Virtual Deliverability Manager, even we have a configuration set with event destination for unchecked `open` and `click`. needing a custom, company-branded, secure redirect domain, to prevent SES to fall back to an insecure generic domain.

Resolution (Immediate Fix)

  1. Opened xxxx-ses-config-set in the SES console.
  2. Located the event destination sespoc that enabled tracking.
  3. Verified that tracking for email opens and clicks was not active in the existing event destination.
  4. Turned off engagement tracking in Virtual Deliverability Manager (account level).
  5. Sent a new invitation to confirm the link now pointed directly to the teacher application; activation succeeded.

Status: The system is fully functional.

[Image Placeholder 01: Introduction Graphic]


Custom Redirect Domains (Long-Term)

Why?

  • Brand-aligned URLs instead of awstrack.me.
  • End-to-end HTTPS (no mixed-content errors).
  • Isolation of our reputation from other SES customers.

How?

  1. Verify a sub-domain in SES (e.g., click.example.com).
  2. Point a CDN (commonly CloudFront) at r.<region>.awstrack.me and attach an ACM certificate.
  3. In the configuration set, select “Use a custom redirect domain” and choose the verified domain.
  4. From 30 September 2024, enforce HTTPS for opens and clicks via the new HTTPS-policy option.

[Image Placeholder 01: Introduction Graphic]


Reflections

LayerWhere to turn it on/offWhat it doesScope & Precedence
VDM (account-wide) SES console → Virtual Deliverability Manager → Settings → Engagement tracking • Inserts open-tracking pixel
• Rewrites every <a> into …awstrack.me… or a custom domain
• Emits “Open” & “Click” CloudWatch metrics
Applies to every message unless a configuration set overrides it.
Configuration set SES console → Configuration sets → [set] → Tracking options → “Override account-level settings” Enable or disable engagement tracking for this set only; optionally specify a custom HTTPS redirect domain. When an email is sent with this configuration set, its settings override the account default.

Note: Clearing the “Open & Click” event destination only stops events flowing into CloudWatch; it does not stop SES from rewriting links. Use the two switches above.


Future Design Patterns (Multiple Apps → SES)

ScenarioRecommended Pattern
App needs metrics (marketing, newsletters) Keep VDM engagement tracking enabled. Create a configuration set (e.g., marketing) that inherits the account default and specifies a custom HTTPS redirect domain.
App must not rewrite links (transactional) Create xxx-no-track; enable “Override account-level settings” → Engagement tracking = Disabled. Send mail with X-SES-CONFIGURATION-SET: xxx-no-track (SMTP) or ConfigurationSetName='xxx-no-track' (API).
Only one link should bypass tracking Leave tracking on but add ses:no-track to that <a> tag: <a ses:no-track href="https://xxx.nsw.edu.au/…">Activate</a>.
Staged migration / A/B test Duplicate the existing configuration set, disable tracking in the clone, and direct a subset of traffic to the new set.

Lesson learned: always verify the end-to-end user experience after infrastructure changes.

Welcome to Zack's Blog

Join me for fun journey about ##AWS ##DevOps ##Kubenetes ##MLOps

  • Latest Posts