Skip to content

Proposal: CI/CD Pipeline for ELTMAESTRO Workflow Version Tracking & Promotion

Audience: Engineering / platform stakeholders Status: Proposal for review Scope: Automate version tracking and controlled promotion of Maestro workflow XML definitions from a lower environment to production, using Jenkins, GitHub, and the existing ELTMAESTRO meta-service.

Executive Summary

Today an ELTMAESTRO workflow definition exists only inside the environment it was authored in. There is no version history outside the live database and no controlled path to move a workflow from a lower environment to production — so the only way to change production is to edit it directly, which is precisely the practice we want to eliminate.

This proposal introduces a Git-backed, Jenkins-automated promotion pipeline. Developers continue to author and test workflows in a lower (DEV) environment exactly as they do today. A Jenkins job automatically captures each workflow's XML into a GitHub repository (giving us full version history, diffs, and rollback), and a separate gated promotion job carries an approved version into production after a dry-run preview and a manual approval.

Crucially, this reuses machinery that already exists in the product — the meta-service already exposes SOAP operations to export, validate, and deploy workflow definitions. The only genuinely new element is placing the workflow .xml artifacts under Git version control and orchestrating the existing operations from Jenkins.

Problem Statement

A workflow definition is a single <job> XML document. In each running environment it lives:

  • On disk, one .xml file per job, under $MAESTRO_DEPLOYED_FILES_DIR (named after the job), and
  • In Postgres, with metadata in t_job_meta / t_job_meta_history.

The gaps this creates:

  1. No version history of a workflow outside the live database — no diffs, no blame, no easy rollback to a known-good version.
  2. No controlled promotion path from a lower environment to production. Moving a workflow means either re-authoring it in the target or a manual, error-prone export/import by hand.
  3. Direct production editing becomes the path of least resistance — the opposite of safe change control.

Goals

  • Every workflow .xml is version-tracked in Git: full history, diffs, blame, rollback.
  • Developers never edit production directly; promotion is a controlled pipeline action.
  • Promotion is previewable (dry-run) and gated by human approval before it touches production.
  • Clear visibility into which version of each workflow is live in each environment.

Non-Goals (this phase)

  • Migrating connections/credentials between environments. Connections are pre-created per environment with identical names (see Connection Strategy).
  • Promoting non-workflow assets (integrator system.cfg, JDBC drivers, engine images) — these already have their own delivery paths and are out of scope here.
  • Replacing the WPF authoring experience — developers keep authoring in DEV as they do now.

What Already Exists (reused, not rebuilt)

The product already provides every primitive operation this pipeline needs. We orchestrate them; we do not reimplement them.

Capability Where it lives Role in the pipeline
Workflow XML on disk $MAESTRO_DEPLOYED_FILES_DIR/<job> The artifact we version and promote
Workflow metadata Postgres t_job_meta, t_job_meta_history Job type/platform, audit, version history
Export workflow XML SOAP getDeployedJobFileContent(token, user, fileName) (meta-service MaestroMeta) Pull a workflow's XML out of an environment
Deploy / import workflow XML SOAP deploy(...) (meta-service MaestroMeta) — writes XML into the target environment Push a workflow's XML into an environment
Dry-run validation SOAP runMigrationCheck(...) (MigrationUtility / MigrationObject) Preview an import without writing
Enumerate workflows The same job list the WPF Migration Export screen uses Know which workflows to export
Jenkins automation source/Jenkinsfile (ACTION-driven) + a PowerShell trigger helper Run the capture and promote jobs

Key insight: export + import + dry-run already exist as SOAP operations. The new work is Git versioning and two Jenkins jobs that string these together.

Proposed Solution

1. Git as the source of truth

A dedicated GitHub repository (working name atni-maestro-workflows) holds one .xml file per workflow, mirroring how they sit on disk in the lower environment:

workflows/
  <JOB_NAME_1>.xml
  <JOB_NAME_2>.xml
  ...
manifest.json          # job name -> { type, platform, last-exported-from, checksum }

Each export becomes a commit — a point-in-time snapshot. Promotion is then expressed as a Git operation (a tag or a merge), which is exactly what delivers "version tracking from lower environment to prod."

2. Capture job (Jenkins: DEV → Git) — automated

A Jenkins job (scheduled and/or on-demand) connects to the DEV meta-service over SOAP and:

  1. Enumerates the workflows (the same list the WPF Migration Export screen uses).
  2. Calls getDeployedJobFileContent for each to pull its XML.
  3. Writes each into the Git working copy and updates manifest.json.
  4. Commits and pushes — only if something actually changed.

Result: every change a developer makes in DEV becomes a reviewable commit and diff, with no new Git discipline required of developers — their day-to-day authoring is unchanged.

3. Promote job (Jenkins: Git → PROD) — gated

A separate Jenkins "promote" job takes an approved set of workflow XML from Git (a chosen tag or commit) and, against the PROD meta-service:

  1. Preview — for each workflow, run runMigrationCheck (plus a connection-existence check) and show a Git diff of the XML versus what is currently live in production.
  2. Manual approval gate — a Jenkins input step pauses the pipeline; a human reviews the preview and approves (or aborts).
  3. Apply — call deploy(...) per workflow to write the XML into production, then record the promoted Git commit/tag for audit.

Developers never touch production; promotion is an explicit, logged, approved action.

4. Connection Strategy (identical names per environment)

Connection names (for example WAREHOUSE_CONN, SYSTEM_SSH) exist in every environment, each pointing at that environment's own host and credentials. Because a workflow's XML references connections by name, the same XML promotes unchanged — only the target environment's connection configuration differs.

This is the lowest-risk option: it matches the importer's existing skip-if-connection-exists behavior, so a promotion never overwrites production connection settings. The one-time setup cost is ensuring the matching named connections exist in each environment.

Pipeline Flow (end to end)

Developer authors / tests in DEV (WPF client)  ── unchanged workflow ──┐
                                                                       v
[Jenkins: capture]   export DEV workflows   →   commit/push .xml to GitHub
                                                                       │
                              (history · diffs · review · tag a release)│
                                                                       v
[Jenkins: promote]   read approved tag from Git
        →  runMigrationCheck against PROD   (preview + diff)
        →  ──  MANUAL APPROVAL GATE  ──
        →  deploy() each workflow into PROD   →   record promoted version

Components to Build (after approval)

  1. Workflow repository — new GitHub repo, directory layout, and manifest.json schema.
  2. Capture script — a small client (PowerShell or a thin Java/CLI) that calls getDeployedJobFileContent, reusing the existing SOAP authentication approach (service user, TLS endpoint on port 8181). Driven by a Jenkins job.
  3. Promote script — calls runMigrationCheck then deploy, with a Jenkins input approval gate between preview and apply.
  4. Jenkins jobs — two new jobs/ACTIONs (capture, promote) wired into the existing source/Jenkinsfile ACTION pattern and PowerShell trigger helper.
  5. Per-environment connection setup (one-time) — ensure identically named connections exist in DEV and PROD.

Service account: Use a dedicated automation user (not a developer's interactive WPF login) for the SOAP calls. Reusing a logged-in user's token can invalidate that user's live client session.

Phased Roadmap

Phase Outcome
0 — Foundations Create the workflow Git repo; stand up identically named connections in DEV and PROD; create the automation user.
1 — Capture Jenkins job exports DEV workflows and commits them to Git. Delivers version tracking immediately, before any automated promotion exists.
2 — Promote (gated) Jenkins promote job: dry-run preview + manual approval + deploy to PROD.
3 — Hardening Promotion audit trail (who promoted which tag, when); rollback recipe (re-promote a prior tag); notifications.

Phase 1 has standalone value: even on its own it gives us a complete version history of every workflow, which addresses the most acute gap today.

Risks & Mitigations

Risk Mitigation
Connection name drift between environments breaks a promoted workflow Identical-names convention + a pre-promote check that every connection referenced by the XML exists in PROD (part of the dry-run)
A bad workflow reaches production Mandatory dry-run preview + diff + manual approval gate; Git history enables a one-tag rollback
SOAP session collision invalidating a live WPF session Dedicated automation user for all pipeline SOAP calls
Raw-XML import semantics Use the lighter deploy() (raw XML) path deliberately — connections are not migrated; confirm in a short Phase-1 spike before finalizing the promote design
Secrets (SOAP credentials, keystore) in CI Use the Jenkins credentials store (already the established pattern for the Jenkins API token)

Open Items to Confirm with Stakeholders

  • DEV and PROD meta-service endpoints (hosts/ports) and network reachability from the Jenkins agent.
  • Whether a third TEST/QA tier is wanted later. The design extends to N tiers — it is the same promote job pointed at a different target environment.
  • Who owns the approval gate (which role approves a production promotion).

Why This Approach

  • Low build cost / high leverage — it composes existing, proven SOAP operations rather than building a new migration engine.
  • No disruption to developers — authoring in DEV is unchanged; capture is automatic.
  • Safe by construction — version history, dry-run preview, and a human approval gate stand between a change and production.
  • Incrementally valuable — Phase 1 alone delivers full workflow version tracking.