Parsing the search response as HTML finds no jobs.

The /V2/Vacancy/ApplySearchFilter endpoint returns a JSON envelope. Read the searchResults property first, then feed that string into your HTML parser.

The vacancy id is wrong and every detail fetch fails validation.

The path is /Vacancies/{channel}/{org}/{sequence}/{id}/{location}/{slug}; the id is the fourth path segment, not the first number. Extract it with the six-part regex and confirm it against #btn-save-vacancy[data-vacancyid] on the detail page.

Internal-only roles appear that you cannot publicly link to.

Always send searchControlViewModel.Type=External in the form body so the board returns only externally advertised vacancies.

A board with zero cards is indistinguishable from a broken parser.

On page 1 with no cards, look for an explicit 'no vacancies' message before declaring the board empty; otherwise treat it as a parse error so real breakage surfaces.

Posted and closing dates come out with day and month swapped.

Engage renders en-GB dates (DD/MM/YYYY). Parse with a UK locale and treat closing dates as end-of-day.

Engage ATS Jobs API.

Pull public-sector and higher-education vacancies from white-label council and university boards on engageats.co.uk, complete with directorate, closing dates, and tenant-owned apply links.

Get API access

Data fields

Full Job Descriptions
Posted & Closing Dates
Directorate & Service Fields
Job Reference & Type
Tenant-Owned Apply URLs
Structured Location Data

Use cases

01Public Sector Job Aggregation
02Council Vacancy Monitoring
03Higher Education Recruitment Feeds
04Closing Date Alerts

Trusted by

London School of Economics
Barnsley Council
North Yorkshire Council

Fetch a page of vacancy cards

Engage boards live on a tenant subdomain (e.g. barnsley.engageats.co.uk). POST the search filter form to the fragment endpoint with Type=External so you only collect publicly linkable roles. The response is a JSON envelope, not raw HTML.

import requests

def fetch_page(origin: str, page: int = 1) -> str:
    body = (
        "searchControlViewModel.Criteria="
        "&searchControlViewModel.PostCode="
        "&searchControlViewModel.TravelDistance="
        "&searchControlViewModel.SortBy="
        "&searchControlViewModel.Type=External"
        f"&searchControlViewModel.PageNo={page}"
    )
    resp = requests.post(
        f"{origin}/V2/Vacancy/ApplySearchFilter",
        data=body,
        headers={
            "Accept": "application/json",
            "Content-Type": "application/x-www-form-urlencoded",
            "Referer": f"{origin}/V2/Vacancy",
        },
        timeout=30,
    )
    resp.raise_for_status()
    # The vacancy markup is a string nested inside the JSON response.
    return resp.json()["searchResults"]

origin = "https://barnsley.engageats.co.uk"
fragment = fetch_page(origin)

Parse vacancy cards from the fragment

Re-parse the searchResults string as HTML. Each result is a button.btn-search-results-view whose data-param1 holds the detail URL. The vacancy id is the fourth path segment (after channel, organisation, and sequence) — extract it with the six-part regex, not by grabbing the first number.

import re
from urllib.parse import urljoin
from bs4 import BeautifulSoup

# /Vacancies/{channel}/{organisationId}/{sequence}/{id}/{locationId}/{slug}
CARD_URL = re.compile(
    r"/Vacanc(?:y|ies)/([^/?#]+)/(\d+)/(\d+)/(\d+)/(\d+)/[^/?#]+",
    re.IGNORECASE,
)

def parse_cards(fragment: str, origin: str) -> list[dict]:
    soup = BeautifulSoup(fragment, "html.parser")
    jobs = []
    for button in soup.select("button.btn-search-results-view[data-param1]"):
        url = urljoin(origin, (button.get("data-param1") or "").strip())
        match = CARD_URL.search(url)
        if not match:
            continue
        channel, org_id, _seq, vacancy_id, location_id = match.groups()
        card = button.find_parent(class_=["ats-gray-panel", "ats-white-panel"])
        title_el = card.select_one(".ats-heading-font") if card else None
        jobs.append({
            "external_id": vacancy_id,
            "listing_url": url,
            "title": title_el.get_text(strip=True) if title_el else None,
            "channel_id": channel,
            "organisation_id": org_id,
            "location_id": location_id,
        })
    return jobs

Walk pagination via the page marker

The fragment text includes a 'Page N of M' marker. Loop pages defensively: stop when a page returns no cards or when the current page reaches the reported total, so a markup change can never spin an infinite loop.

PAGE_COUNT = re.compile(r"Page\s+(\d+)\s+of\s+(\d+)", re.IGNORECASE)

def total_pages(fragment: str, current_page: int) -> int:
    text = BeautifulSoup(fragment, "html.parser").get_text(" ")
    match = PAGE_COUNT.search(text)
    return int(match.group(2)) if match else current_page

def scrape_all(origin: str) -> list[dict]:
    page, results = 1, []
    while True:
        fragment = fetch_page(origin, page)
        cards = parse_cards(fragment, origin)
        if not cards:
            break
        results.extend(cards)
        if page >= total_pages(fragment, page):
            break
        page += 1
    return results

Fetch and validate the detail page

Detail pages are server-rendered HTML. GET the listing URL, then confirm #btn-save-vacancy[data-vacancyid] matches the id parsed from the card before trusting the page. Read the title, description, and tenant-owned ApplicationForm.aspx apply URL from the .view-vacancy-container root.

def fetch_detail(listing: dict) -> dict | None:
    html = requests.get(listing["listing_url"], timeout=30).text
    soup = BeautifulSoup(html, "html.parser")
    root = soup.select_one(".view-vacancy-container")
    save_btn = soup.select_one("#btn-save-vacancy[data-vacancyid]")
    native_id = save_btn.get("data-vacancyid") if save_btn else None
    # Guard against a mismatched or stale detail page.
    if root is None or native_id != listing["external_id"]:
        return None
    title_el = root.select_one("[role='heading'][aria-level='1'].ats-title-font")
    desc_el = root.select_one("div.order-md-1 > div.row > div.ats-normal-font")
    apply_btn = root.select_one("button.apply-application-btn[data-vacancyapplyurl]")
    apply_url = (
        urljoin(listing["listing_url"], apply_btn["data-vacancyapplyurl"])
        if apply_btn else None
    )
    return {
        **listing,
        "title": title_el.get_text(strip=True) if title_el else listing["title"],
        "description": desc_el.get_text(" ", strip=True) if desc_el else None,
        "apply_url": apply_url,
    }

Common issues

highParsing the search response as HTML finds no jobs.: The /V2/Vacancy/ApplySearchFilter endpoint returns a JSON envelope. Read the searchResults property first, then feed that string into your HTML parser.
highThe vacancy id is wrong and every detail fetch fails validation.: The path is /Vacancies/{channel}/{org}/{sequence}/{id}/{location}/{slug}; the id is the fourth path segment, not the first number. Extract it with the six-part regex and confirm it against #btn-save-vacancy[data-vacancyid] on the detail page.
mediumInternal-only roles appear that you cannot publicly link to.: Always send searchControlViewModel.Type=External in the form body so the board returns only externally advertised vacancies.
mediumA board with zero cards is indistinguishable from a broken parser.: On page 1 with no cards, look for an explicit 'no vacancies' message before declaring the board empty; otherwise treat it as a parse error so real breakage surfaces.
lowPosted and closing dates come out with day and month swapped.: Engage renders en-GB dates (DD/MM/YYYY). Parse with a UK locale and treat closing dates as end-of-day.

Best practices

1Send searchControlViewModel.Type=External so you only collect publicly linkable vacancies.
2Re-parse the searchResults string as HTML — the outer response is a JSON envelope.
3Validate each detail page's data-vacancyid against the id from the card URL before trusting it.
4Paginate off the 'Page N of M' marker and stop once the current page reaches the reported total.
5Throttle requests with a short delay and modest concurrency, and treat HTTP 429 as a retryable rate-limit signal.
6Parse dates as en-GB and keep closing dates as end-of-day.

Or skip the complexity

One endpoint. All Engage ATS jobs. No scraping, no sessions, no maintenance.

Get API access

curl "https://connect.jobo.world/api/jobs?sources=engage ats" \
  -H "X-Api-Key: YOUR_KEY"

Ready to integrate

Access Engage ATS
job data today.

One API call. Structured data. No scraping infrastructure to build or maintain — start with the $5 free starting balance and scale as you grow.

Get API access View all platforms

99.9%API uptime

<200msAvg response

50M+Jobs processed

Engage ATS Jobs API.

What's in every response.

Data fields

Use cases

Trusted by

How to scrape Engage ATS.

Fetch a page of vacancy cards

Parse vacancy cards from the fragment

Walk pagination via the page marker

Fetch and validate the detail page

Try the API.

Access Engage ATS
job data today.

Engage ATS Jobs API.

Data fields

Use cases

Trusted by

Fetch a page of vacancy cards

Parse vacancy cards from the fragment

Walk pagination via the page marker

Fetch and validate the detail page

Access Engage ATSjob data today.

Access Engage ATS
job data today.