A/B test social posts programmatically and pick the winner by engagement

Most social media advice is a guess about someone else’s audience. An A/B test tells you about yours: publish two or more versions of a post, measure them the same way, and keep the one that worked. SocialCannon runs the whole test from one API call, from publishing the variants to choosing a winner. This guide shows how to set one up and how to read the result honestly.

How a test works

1. You send the variants, the account, and the metric you care about.
2. SocialCannon publishes every variant to that account, now or at a time you choose.
3. It collects analytics for each variant while the test runs.
4. After the minimum duration, and once analytics are in for every eligible variant, the test completes and records the winner with a confidence level. If no variant has a non-zero score, it completes without a winner.

Every variant is a real post on the account, published like any other post. Plan the test as you would plan a set of real posts.

Create a test

Tests run on Instagram, X and YouTube accounts (see the plan and platform notes below). Replace ACCOUNT_ID, TOKEN and the example image URL below with your own, and set scheduledAt to a time within the next 30 days or leave it out. Media must be publicly reachable, and a YouTube account needs a video URL instead of an image.

curl -X POST https://socialcannon.app/api/v1/ab-tests \
  -H "Authorization: Bearer TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: cta-test-2026-11-05" \
  -d '{
    "accountId": "ACCOUNT_ID",
    "name": "Launch CTA",
    "metric": "engagementRate",
    "minDurationHours": 48,
    "scheduledAt": "2026-11-05T12:00:00Z",
    "variants": [
      { "content": "Dark mode is here. Turn it on in Settings.", "mediaUrls": ["https://example.com/dark.png"] },
      { "content": "Late-night editing just got easier on the eyes. Dark mode is live.", "mediaUrls": ["https://example.com/dark.png"] }
    ]
  }'

• metric decides the winner: engagementRate, likes or impressions. The API also accepts clicks, but Instagram, X and YouTube do not report clicks to SocialCannon, so a clicks test cannot pick a winner today.
• minDurationHours runs from 1 to 168, with a default of 24. The clock starts when the last eligible variant is published.
• scheduledAt is optional. Leave it out to publish the variants immediately, or set a UTC time up to 30 days ahead to publish them all together.
• Idempotency-Key is required. If the response is lost, send the same request with the same key: you get the original test back, or an error that names it if it is still being set up or failed, never a second test. The same key with a different body is refused.

When every variant is accepted, a new test returns 201 with the test ID and one post ID per variant. If a variant fails, or a network’s outcome is uncertain, the error response usually includes the test and its post IDs; check it with GET /api/v1/ab-tests/TEST_ID before doing anything else. A 503 without an ID means the test may not have been set up. Repeat the identical request with the same key, which returns the test if it was created and is safe if it was not. In either case, do not start a new test while the outcome is uncertain. Failed variants are left out of the comparison, and at least two published variants are needed for the test to run. If fewer than two publish, the test settles as failed; to try again, create a replacement test with a new Idempotency-Key.

Read the results

curl https://socialcannon.app/api/v1/ab-tests/TEST_ID \
  -H "Authorization: Bearer TOKEN"

The response includes each variant’s latest collected metrics, the current leader as currentWinner, and a confidence value. SocialCannon collects analytics about every four hours; for fresher numbers on one variant, call GET /api/v1/posts/POST_ID/analytics, which fetches them from the network. Confidence describes how far the leader is ahead of the runner-up on your chosen metric:

Designing a test you can trust

• Change one thing. Test the opening line, or the call to action, or the image. If everything differs, you cannot tell what caused the result.
• Publish together. Use one scheduledAt for all variants, so time of day does not decide the winner.
• Pick the metric before you look. Engagement rate suits most posts; impressions show which version gets shown more.
• Give it time. 24 to 48 hours suits most accounts. Very small audiences may need longer.

Plans and platform support

A/B testing is not part of the Free plan. Tests run on Instagram, X and YouTube. Facebook and LinkedIn post analytics are not available through SocialCannon yet, so tests on those accounts are refused with PLATFORM_UNSUPPORTED. TikTok is not supported either, because every TikTok post needs its own privacy setting (TikTok post analytics are also not available yet). Prices are on the pricing page.

Let an agent run the experiment

The MCP server offers a simpler version as create_ab_test and get_ab_test_results: text-only variants that publish immediately, so an assistant can draft variants, run the test and report back. Because the variants carry no media, this suits X; Instagram needs an image or video in every post and YouTube a video file, so run tests there, or any test with a scheduled start, with the REST request above. See the MCP guide. To generate variants from one idea, combine this with AI repurposing, and the API reference documents the endpoints.